WO2024253431A1 - Dispositif de visiocasque comprenant un capteur de suivi oculaire et procédé de fonctionnement pour dispositif de visiocasque - Google Patents

Dispositif de visiocasque comprenant un capteur de suivi oculaire et procédé de fonctionnement pour dispositif de visiocasque Download PDF

Info

Publication number
WO2024253431A1
WO2024253431A1 PCT/KR2024/007730 KR2024007730W WO2024253431A1 WO 2024253431 A1 WO2024253431 A1 WO 2024253431A1 KR 2024007730 W KR2024007730 W KR 2024007730W WO 2024253431 A1 WO2024253431 A1 WO 2024253431A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
region
interest
image
display device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2024/007730
Other languages
English (en)
Korean (ko)
Inventor
정지원
김덕호
이건일
이원우
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of WO2024253431A1 publication Critical patent/WO2024253431A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements

Definitions

  • the present disclosure relates to a head mounted display device and an operating method of the head mounted display device. Specifically, the present disclosure relates to a head mounted display device including a gaze tracking sensor and an operating method of the head mounted display device.
  • head mounted display devices are being developed that provide images to users through a display placed close to the user's eyes while worn on the user's head.
  • a display included in a head-mounted display device may include an optical see-through display for overlaying a virtual image on a physical environment space of the real world or a real world object, or may include a video see-through display for providing a user with an image captured by a camera or the like of a physical environment space of the real world.
  • the head mounted display device may include a camera that acquires an image in front.
  • the head mounted display device may include a gaze tracking sensor that tracks a user's gaze direction.
  • the head mounted display device may include a memory that stores at least one instruction and at least one processor that executes at least one instruction stored in the memory.
  • the at least one processor may acquire a user's gaze based on the gaze direction by executing the at least one instruction.
  • the at least one processor may divide an acquired image into a region of interest and a region of non-interest based on the acquired user's gaze by executing the at least one instruction.
  • the at least one processor may determine whether a preset object is included in the region of non-interest by executing the at least one instruction.
  • the at least one processor may provide a notification to the user if the preset object is included in the region of non-interest by executing the at least one instruction.
  • the method for operating the head-mounted display device may include a step of acquiring a front image through a camera.
  • the method for operating the head-mounted display device may include a step of acquiring a user's viewpoint based on a gaze direction acquired through a gaze tracking sensor that tracks the user's gaze direction.
  • the method for operating the head-mounted display device may include a step of dividing the acquired image into a region of interest and a region of non-interest based on the acquired viewpoint of the user.
  • the method for operating the head-mounted display device may include a step of determining whether a preset object is included in the region of non-interest.
  • the method for operating the head-mounted display device may include a step of providing a notification to the user if the preset object is included in the region of non-interest.
  • a computer-readable recording medium having recorded thereon a program for performing at least one method of the disclosed operating method on a computer can be provided.
  • FIG. 1 is a conceptual diagram illustrating a head mounted display device according to one embodiment of the present disclosure.
  • FIG. 2 is a drawing for explaining the configuration of a head mounted display device according to one embodiment of the present disclosure.
  • FIG. 3 is a block diagram illustrating a configuration of a head mounted display device according to one embodiment of the present disclosure.
  • FIG. 4 is a flowchart for explaining an operating method of a head mounted display device according to one embodiment of the present disclosure.
  • FIG. 5 is a flowchart illustrating a method of operating a head mounted display device for generating an attention map according to one embodiment of the present disclosure.
  • FIG. 6 is a block diagram illustrating a head mounted display device and an operating method of the head mounted display device according to one embodiment of the present disclosure.
  • FIG. 7 is a flowchart illustrating an operation of dividing an image into a plurality of grids according to one embodiment of the present disclosure.
  • FIG. 8a is a diagram illustrating a plurality of grids for distinguishing a high complexity image according to one embodiment of the present disclosure.
  • FIG. 8b is a diagram illustrating a plurality of grids for distinguishing low complexity images according to one embodiment of the present disclosure.
  • FIG. 9A is a diagram for explaining an attention map according to a detected user's viewpoint according to one embodiment of the present disclosure.
  • FIG. 9b is a diagram illustrating an attention map divided into a plurality of grids according to a detected user's viewpoint according to one embodiment of the present disclosure.
  • FIG. 9c is a diagram for explaining an attention map according to a movement path of a user's viewpoint according to one embodiment of the present disclosure.
  • FIG. 10A is a diagram illustrating an operation of dividing an attention map into a region of interest and a region of non-interest according to one embodiment of the present disclosure.
  • FIG. 10b is a diagram illustrating non-interest regions separated from each other within an attention map according to one embodiment of the present disclosure.
  • FIG. 10c is a diagram for explaining how a non-interest region that is distinguished according to a preset condition in an attention map according to one embodiment of the present disclosure changes.
  • FIG. 11a is a diagram for explaining an operation of dividing an image into a region of interest and a region of non-interest according to one embodiment of the present disclosure.
  • FIG. 11b is a diagram for explaining an operation of dividing an image into a region of interest and a region of non-interest according to one embodiment of the present disclosure.
  • FIG. 12 is a diagram illustrating an operation of detecting a final position of an object included in a non-interest region according to one embodiment of the present disclosure.
  • FIG. 13A is a diagram for explaining an operation of providing a notification in a head mounted display device of an optical see-through type according to one embodiment of the present disclosure.
  • FIG. 13b is a diagram for explaining an operation of providing a notification in a head mounted display device of a video see-through type according to one embodiment of the present disclosure.
  • FIG. 13c is a diagram for explaining an operation of providing a notification in a head mounted display device of a video see-through type according to one embodiment of the present disclosure.
  • a system configured to can mean that the system is “capable of” doing something together with other devices or components.
  • a processor configured (or set) to perform A, B, and C can mean a dedicated processor (e.g., an embedded processor) to perform those operations, or a generic-purpose processor (e.g., a CPU or an application processor) that can perform those operations by executing one or more software programs stored in memory.
  • the 'electronic device' may be a head mounted display (HMD) device.
  • HMD head mounted display
  • the present disclosure is not limited thereto, and the 'electronic device' may be implemented as various types of electronic devices, such as a TV, a mobile device, a smart phone, a laptop computer, a desktop, a tablet PC, an e-book reader, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a wearable device, etc.
  • PDA Personal Digital Assistants
  • PMP Portable Multimedia Player
  • FIG. 1 is a conceptual diagram illustrating a head mounted display device according to one embodiment of the present disclosure.
  • an electronic device can provide content to a user wearing the electronic device.
  • the electronic device can be a head mounted display (HMD) device (100) as a device that can provide content to a user.
  • HMD head mounted display
  • FIG. 1 illustrates an electronic device having a shape similar to glasses, including a support portion that is hung over a user's face.
  • the electronic device may also include a support portion that is hung over a user's face and head.
  • the electronic device is described as a head mounted display device (100).
  • the left-eye configuration and the right-eye configuration included in the head mounted display device (100) only the left-eye configuration is illustrated in FIG. 1.
  • the head mounted display device (100) may be an optical see-through display device that provides a user with a virtual image by overlaying it on a physical environment space of the real world or a real world object.
  • the head mounted display device (100) may provide the user with data provided from the outside or a virtual image generated on the head mounted display device (100).
  • the head mounted display device (100) may be a video see-through electronic device that captures an image of a physical environment space in the direction of the user's gaze with a camera, and then provides the acquired image to the user.
  • the head mounted display device (100) of the present disclosure is described as an optical see-through display device, but the present disclosure is not limited thereto.
  • the head mounted display device (100) may include a display (110), an optical lens (120), and an eye tracking sensor (130). Only components for explaining the structure of the head mounted display device (100) are illustrated in FIG. 1, and the components included in the head mounted display device (100) are not limited to those illustrated in FIG. 1.
  • the optical lens (120) may be composed of a material that allows a user to view a physical environment space of the real world or a real world object through the optical lens (120). In one embodiment, the user may gaze at a specific space or a specific object of the real world through the optical lens (120).
  • the optical lens (120) is illustrated as having a circular shape in FIG. 1, but the present disclosure is not limited thereto. The optical lens (120) may also have a square shape, etc.
  • the display (110) may be positioned on the left side of the optical lens (120). In one embodiment, the display (110) may be positioned on the upper left side of the optical lens (120). In addition, the display (110) may be positioned at a location farther away from the user than the optical lens (120). However, the present disclosure is not limited thereto, and the display (110) may be positioned at various locations and may have various shapes. In addition, the optical lens (120) and the display (110) may be formed as a single configuration. In one embodiment, the head mounted display device (100) may provide a virtual image to the user through the display (110). In one embodiment, the virtual image provided through the display (110) may be provided to the user through the optical lens (120).
  • the head mounted display device (100) can obtain the gaze direction of a user wearing the head mounted display device (100) through a gaze tracking sensor (130) and obtain the user's viewpoint.
  • the obtained user's viewpoint may be a point where the user gazes at a space or object in the real world observed through an optical lens (120).
  • the head mounted display device (100) may include a camera that captures an image in front of the head mounted display device (100).
  • the image in front may mean an image of the real world, i.e., the user's surrounding environment.
  • the head mounted display device (100) may obtain an image through the camera.
  • the head mounted display device (100) may obtain a viewpoint of the user, which is a point at which the user gazes at the surrounding environment, based on the user's gaze direction obtained through the gaze tracking sensor (130) and the obtained image.
  • the head mounted display device (100) can divide the acquired image into a region of interest (200) and a region of non-interest (210) based on the acquired user's viewpoint.
  • the "region of interest (200)" may be a region in the image where the user's concentration is relatively high.
  • the "region of non-interest (210)" may be a region in the image where the user's concentration is relatively low.
  • the user's concentration may be determined based on the acquired user's viewpoint.
  • the region in the image where the acquired user's viewpoint is determined to be located may be a region where the user's concentration is high.
  • the region in the image where the acquired user's viewpoint is determined not to be located may be a region where the user's concentration is low.
  • an area in the image where the number of user viewpoints per unit area is determined to be large based on the acquired multiple user viewpoints may be a region of interest (200).
  • An area in the image where the number of user viewpoints per unit area is determined to be small based on the acquired multiple user viewpoints may be a non-region of interest (210).
  • the user's degree of attention to the region of interest (200) may be higher than the user's degree of attention to the non-region of interest (210).
  • the user may gaze at the region of interest (200) in the surrounding environment and may not be aware of objects located in the non-region of interest (210) or unexpected situations occurring in the non-region of interest (210).
  • the head mounted display device (100) may determine whether a preset object (230) is included in a non-interest region (210).
  • the preset object (230) may be a part of a plurality of objects included in an object list set by the user to receive a notification (240) through the head mounted display device (100).
  • the head mounted display device (100) may include a user interface.
  • the head mounted display device (100) may add or delete a plurality of objects included in an object list according to a user input obtained through the user interface.
  • the present disclosure is not limited thereto, and the head mounted display device (100) may also receive an object list from an external server or a peripheral electronic device.
  • the head mounted display device (100) may provide a notification (240) to the user when a preset object (230) is included in a non-interest region (210).
  • the notification (240) may be information including that the preset object (230) is located in the non-interest region (210).
  • a user can obtain information that a preset object (230) has been detected in a non-interest area (210) to which the user is not paying attention, through a notification (240) provided by the head mounted display device (100).
  • the user can recognize a preset object (230) located in a non-interest area (210), through the notification (240) provided by the head mounted display device (100).
  • the head mounted display device (100) can determine whether a preset object (230) is included in an image corresponding to a non-interest region (210) in the image, thereby increasing the computation speed and reducing power consumption compared to determining whether the preset object (230) is included in the entire image.
  • the head mounted display device (100) may provide the user with a notification (240) that includes information about a preset object (230) included in a non-interest region (210), thereby avoiding providing the user with a notification that includes unnecessary information about an interest region (200) that the user is paying attention to. This may prevent the user from being confused by unnecessary notifications.
  • the head mounted display device (100) may provide a user with a notification (240) regarding a preset object (230) included in an object list based on an object list set for a purpose.
  • the object list may include an object that the user wants to purchase, an object that the user is looking for, or an object that the user is interested in.
  • the object list may include objects that are accessible to the user, such as cars, bicycles, people, etc., or objects related to an emergency situation, such as a hole in the road, an object falling from the air, etc.
  • the head mounted display device (100) may provide a notification (240) to the user when the above-mentioned object is detected in a non-interest area (210), thereby increasing the convenience of the user.
  • the head mounted display device (100) may generate an attention map representing the distribution of the user's viewpoints based on the acquired image and the acquired user's viewpoints.
  • the "attention map” may be a map representing the degree of the user's concentration by corresponding the user's viewpoints to the image.
  • the "attention map” may include a focus area and a non-focus area.
  • the "focus area” may be an area corresponding to an area in the image where the user's viewpoint is located.
  • the “non-focus area” may be an area corresponding to an area in the image where the user's viewpoint is not located.
  • the focus area may be an area where the user gazes and is determined to have a high degree of focus by the user.
  • the non-focus area may be an area where the user does not gaze and is determined to have a low degree of focus by the user.
  • the head mounted display device (100) when the head mounted display device (100) acquires multiple user viewpoints for multiple frames, the head mounted display device (100) can generate an attention map based on the acquired image and the multiple user viewpoints acquired.
  • the focused area may be an area corresponding to an area in the image where the number of user viewpoints per unit area is large.
  • the non-focused area may be an area corresponding to an area in the image where the number of user viewpoints per unit area is small.
  • the head mounted display device (100) may divide the acquired image into a region of interest (200) and a region of non-interest (210) based on the generated attention map. In one embodiment, the head mounted display device (100) may divide an area corresponding to a focused area of the attention map in the image as a region of interest (200). The head mounted display device (100) may divide an area corresponding to a non-focused area of the attention map in the image as a non-interest area (210). Accordingly, an area in the image where the user's concentration is relatively high may be divided as a region of interest, and an area where the user's concentration is relatively low may be divided as a region of non-interest.
  • the head mounted display device (100) and an operating method thereof will be described in detail.
  • FIG. 2 is a drawing for explaining the configuration of a head mounted display device according to one embodiment of the present disclosure.
  • the head mounted display device (100) may include a display (110), an optical lens (120), an eye tracking sensor (130), a frame (140), a nose bridge (150), a camera (160), a memory (170), at least one processor (180), a nose support (190), a display engine (191), a battery (194), and a temple of glasses (195).
  • a display 110
  • an optical lens 120
  • an eye tracking sensor 130
  • a frame 140
  • a nose bridge 150
  • a camera 160
  • a memory a memory
  • FIG. 2 only components for explaining the structure of the head mounted display device (100) are disclosed in FIG. 2, and the components included in the head mounted display device (100) are not limited as illustrated in FIG. 2.
  • the display (110) and the optical lens (120) may each be placed in a frame (140).
  • the frame (140) may have a shape similar to a frame of a typical eyeglass structure.
  • the frame (140) may have a rim shape surrounding the display (110) and the optical lens (120).
  • the display (110) may include a first display (111) corresponding to the user's left eye (301) and a second display (112) corresponding to the user's right eye (302).
  • the optical lens (120) may include a first optical lens (121) corresponding to the user's left eye (301) and a second optical lens (122) corresponding to the user's right eye (302).
  • the distance between the optical lens (120) and the user's eye (300) in the first direction (10) may be closer than the distance between the display (110) and the user's eye (300) in the first direction (10).
  • the display (110) and the optical lens (120) may be arranged on planes parallel to the second direction (20) and the third direction (30), which intersect the first direction (10), respectively.
  • the first display (111) may correspond to the first optical lens (121), and the second display (112) may correspond to the second optical lens (122).
  • the first optical lens (121) may be positioned closer to the user's left eye (301) than the first display (111).
  • the second optical lens (122) may be positioned closer to the user's right eye (302) than the second display (112).
  • the gaze tracking sensor (131) may include a first gaze tracking sensor (131) corresponding to the user's left eye (301) and a second gaze tracking sensor (132) corresponding to the user's right eye (302).
  • the frame (140) may include a first frame (141) corresponding to the user's left eye (301) and a second frame (142) corresponding to the user's right eye (302).
  • the first frame (141) may include a rim surrounding the first display (111) and the first optical lens (121).
  • the second frame (142) may include a rim surrounding the second display (112) and the second optical lens (122).
  • the camera (160) may include a first camera (161) corresponding to the first frame (141) and a second camera (162) corresponding to the second frame (142).
  • the first camera (161) may capture an image by photographing the front of the first frame (141).
  • the second camera (162) may capture an image by photographing the front of the second frame (142).
  • the camera (160) may capture an image by photographing the surrounding environment of the user in a direction opposite to the direction in which the display (110) faces the user.
  • the present disclosure is not limited thereto, and an image may be captured by photographing the side or the back of the first frame (141) and the second frame (142), depending on the arrangement or performance of each of the first camera (161) and the second camera (162).
  • a first display (111), a first optical lens (121), a first gaze tracking sensor (131), and a first camera (161) may be arranged in a first frame (141).
  • a second display (112), a second optical lens (122), a second gaze tracking sensor (132), and a second camera (162) may be arranged in a second frame (142).
  • the first display (111) and the second display (112) may be formed integrally and placed in the frame (140).
  • the first optical lens (121) and the second optical lens (122) may be formed integrally and placed in the frame (140).
  • the first display (111) and the first optical lens (121) may be formed integrally and placed in the frame (140).
  • the second display (112) and the first optical lens (122) may be formed integrally and placed in the frame (140).
  • the glasses leg (195) may be connected to the frame (140).
  • the glasses leg (195) is a portion that hangs over the user's ear and supports the head mounted display device (100) when the user wears the head mounted display device (100).
  • the glasses leg (195) may include a first glasses leg (196) connected to the first frame (141) and a second glasses leg (197) connected to the second frame (142).
  • the display engine (191) may include a first display engine (192) that projects a virtual image onto the first display (111) and a second display engine (193) that projects a virtual image onto the second display (112).
  • the display engine (191) may perform functions such as a projector.
  • the display engine (191) may further include an illumination optics system, a light path converter, an image panel, a beam splitter, and a projection optics system.
  • the virtual image may include a notification (230, see FIG. 1), which will be described later.
  • the memory (170), at least one processor (180), a display engine (191), and a battery (194) included in the head mounted display device (100) may be built into a temple of glasses (195).
  • FIG. 2 illustrates that the memory (170), the at least one processor (180), and the battery (194) are built into a first temple of glasses (196), this is an exemplary structure, and the present disclosure is not limited to the illustrated structure.
  • the first display engine (192) may be built into the first temple of glasses (196).
  • the second display engine (193) may be built into the second temple of glasses (197).
  • the nose support (190) may be connected to the frame (140).
  • the nose support (190) is a portion that hangs over the nose of the user and supports the head mounted display device (100) when the user wears the head mounted display device (100).
  • the nose support (190) may include a nose bridge and a glasses nose.
  • the nose bridge and the glasses nose may be formed as an integral part, but are not limited thereto.
  • the nose support (190) and the frame (140) may be formed as an integral part.
  • the nose bridge (150) is a support connecting the first frame (141) and the second frame (142).
  • the nose bridge (150) may be connected to the nose support (190).
  • the head mounted display device (100) may not include the nose bridge (150).
  • the gaze tracking sensor (130) can track the gaze direction of a user by irradiating light to the eyes (300) of a user wearing a head mounted display device (100) and receiving reflected light reflected from the user's eyes (300).
  • the configuration, operation, and function of the gaze tracking sensor (130) will be described in detail with reference to FIG. 3.
  • the battery (194) may be electrically and/or physically connected to the gaze tracking sensor (130), the camera (160), the memory (170), and at least one processor (180).
  • the battery (194) may supply driving power to the gaze tracking sensor (130) and the camera (160) under the control of the at least one processor (180).
  • the battery (194) may include at least one battery module comprised of a rechargeable secondary battery.
  • the battery (194) may be comprised of, but is not limited to, a lithium ion battery (Li-ion Battery), a lithium ion polymer battery (Li-Ion Polymer Battery; LIPB), a nickel cadmium battery (Ni-Cd Battery), or a nickel metal hydride battery (Ni-MH Battery), for example.
  • FIG. 3 is a block diagram illustrating a configuration of a head mounted display device according to one embodiment of the present disclosure.
  • the head mounted display device (100) may include a display (110), an optical lens (120), an eye tracking sensor (130), a camera (160), a memory (170), at least one processor (180), and a user interface (198).
  • the head mounted display device (100) may be implemented with more components than the components illustrated in FIG. 3, or may be implemented with fewer components.
  • the configuration of the frame (140), the nose bridge (150), the nose support (190), the display engine (191), the battery (194), and the glasses temple (195) is not illustrated in FIG. 3 compared to FIG. 2.
  • the display (110), the optical lens (120), the gaze tracking sensor (130), the camera (160), the memory (170), at least one processor (180), and the user interface (198) may be electrically and/or physically connected to each other, respectively.
  • the same reference numerals are given to the same configurations as those described in FIGS. 1 and 2, and redundant descriptions are omitted.
  • the display (110) is an optical element formed of a transparent material.
  • the display (110) may be formed of a transparent material such that a portion of the back surface is visible when the user wears the head mounted display device (100).
  • the display (110) may include a waveguide.
  • the waveguide may be formed of a single-layer or multi-layer flat plate made of a transparent material through which light can be propagated while being reflected inside.
  • the present disclosure is not limited thereto, and the display (110) may include another type of display capable of providing a virtual image to the user.
  • the display (110) can face the emission surface of the display engine (191, see FIG. 2) and receive light of a virtual image projected from the display engine (191).
  • the light of the virtual image projected onto the display (110) can be propagated within the display (110) by the principle of total reflection.
  • the display (110) can change the path of the light and ultimately provide the virtual image to the user.
  • the optical lens (120) may include a material that is capable of transmitting light provided from the real world toward the head mounted display device (100). In one embodiment, the optical lens (120) may include a material that is capable of transmitting light that constitutes an image provided through the display (110).
  • the gaze tracking sensor (130) can obtain the gaze direction of the user. In one embodiment, the gaze tracking sensor (130) can obtain the gaze direction of the user by detecting an image of the user's pupil or pupil, or detecting the direction or amount of reflected light such as near-infrared light reflected from the cornea. In one embodiment, the gaze tracking sensor can include a left-eye gaze tracking sensor and a right-eye gaze tracking sensor. The gaze tracking sensor can detect the gaze direction of the user's left eye and the gaze direction of the user's right eye, respectively.
  • the gaze tracking sensor (130) may include a light source that irradiates light toward the user's eyes and a sensor that receives reflected light reflected from the user's eyes.
  • the light irradiated toward the user's eyes and the reflected light reflected from the user's eyes may each be infrared rays (IR).
  • the camera (160) may include an RGB camera that captures a front view of the head mounted display device (100) to obtain an image including RGB information. Additionally, the camera (160) may include an RGB-Depth camera that captures an image including RGB information and depth information about the front view of the head mounted display device (100).
  • the memory (170) may include at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a Mask ROM, a Flash ROM, etc.), a hard disk drive (HDD), or a solid state drive (SSD).
  • the memory (170) may store at least one command or program code for performing functions or operations of the head mounted display device (100). At least one instruction, algorithm, data structure, program code and application program stored in the memory (170) may be implemented in a programming or scripting language such as, for example, C, C++, Java, assembler, etc.
  • the memory (170) may store an image acquisition module (171), a gaze direction acquisition module (172), a viewpoint acquisition module (173), an attention map generation module (174), a region segmentation module (175), an object detection module (176), and a notification provision module (177).
  • an image acquisition module (171) may store an image acquisition module (171), a gaze direction acquisition module (172), a viewpoint acquisition module (173), an attention map generation module (174), a region segmentation module (175), an object detection module (176), and a notification provision module (177).
  • the memory (170) may store more modules than the modules illustrated in FIG. 2, or may store fewer modules.
  • a 'module' included in the memory (170) may mean a unit that processes a function or operation performed by at least one processor (180).
  • a 'module' included in the memory (170) may be implemented as software such as at least one instruction, algorithm, data structure, or program code.
  • the memory (170) may store an object list (178, see FIG. 4) containing a plurality of objects.
  • At least one processor (180) may be configured as, but is not limited to, at least one of a Central Processing Unit, a microprocessor, a Graphic Processing Unit, an Application Processor (AP), an Application Specific Integrated Circuits (ASICs), a Digital Signal Processors (DSPs), a Digital Signal Processing Devices (DSPDs), a Programmable Logic Devices (PLDs), a Field Programmable Gate Arrays (FPGAs), and a Neural Processing Unit or an Artificial Intelligence (AI) dedicated processor designed with a hardware structure specialized for learning and processing an artificial intelligence (AI) model.
  • AP Application Processor
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processors
  • DSPDs Digital Signal Processing Devices
  • PLDs Programmable Logic Devices
  • FPGAs Field Programmable Gate Arrays
  • AI Artificial Intelligence
  • At least one processor (180) may be configured as a circuit (circuitry) such as a System on Chip (SoC) or an Integrated Circuit (IC).
  • a circuit such as a System on Chip (SoC) or an Integrated Circuit (IC).
  • SoC System on Chip
  • IC Integrated Circuit
  • At least one processor (180) can execute various types of modules stored in the memory (170). At least one processor (180) can execute at least one command constituting various types of modules stored in the memory (170). By executing the program or at least one command stored in the memory (170), at least one processor (180) can process data according to a predefined operation rule. In one embodiment of the present disclosure, at least one processor (180) can control the operation of the head mounted display device (100) by executing at least one command stored in the memory (170).
  • At least one processor (180) may execute at least one module among an image acquisition module (171), a gaze direction acquisition module (172), a viewpoint acquisition module (173), an attention map generation module (174), a region segmentation module (175), an object detection module (176), or a notification provision module (177) stored in a memory (170).
  • At least one processor (180) may include a plurality of processors.
  • At least one module among the image acquisition module (171), the gaze direction acquisition module (172), the viewpoint acquisition module (173), the attention map generation module (174), the area segmentation module (175), the object detection module (176), or the notification provision module (177) stored in the memory (170) may be executed by any one of the plurality of processors.
  • the user interface (198) may include a touch unit, a push button, a voice recognition unit, a motion recognition unit, etc.
  • the head mounted display device (100) may obtain a user input provided by a user through the user interface (198).
  • the head mounted display device (100) may obtain a user input from a user who touches, presses, provides a voice, or performs a gesture, etc. on the user interface (198).
  • the user may add a new object to the object list or delete an object included in the object list through the user interface (198).
  • the user may provide an input to modify the type of notification (240), the size of the notification, the sound of the notification, the color of the notification, etc.
  • the head mounted display device (100) may also modify the type of notification (240), the size of the notification, the sound of the notification, the color of the notification, etc. based on user input provided through the user interface (198).
  • the head mounted display device (100) may further include a communication interface.
  • the communication interface may perform data communication with an external server under the control of at least one processor (180).
  • the communication interface may perform data communication not only with the external server but also with other peripheral electronic devices.
  • the communication interface may perform data communication with servers or other peripheral electronic devices using at least one of data communication methods including, for example, wired LAN, wireless LAN, Wi-Fi, Bluetooth, zigbee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliance (WiGig), and RF communication.
  • data communication methods including, for example, wired LAN, wireless LAN, Wi-Fi, Bluetooth, zigbee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliance (WiGig), and RF communication.
  • Wi-Fi Direct WFD
  • IrDA
  • the communication interface can receive preset criteria for dividing the attention map into regions of interest and regions of non-interest from external servers or peripheral electronic devices.
  • the communication interface can receive an object list including preset objects from external servers or peripheral electronic devices.
  • FIG. 4 is a flowchart for explaining a method of operating a head mounted display device according to an embodiment of the present disclosure.
  • FIG. 5 is a flowchart for explaining a method of operating a head mounted display device for generating an attention map according to an embodiment of the present disclosure.
  • FIG. 6 is a block diagram for explaining a head mounted display device and a method of operating the head mounted display device according to an embodiment of the present disclosure.
  • a method of operating a head mounted display device (100) may include a step (S100) of acquiring a front image through a camera (160).
  • the image acquisition module (171) may be configured with commands or program codes related to an operation or function of the head mounted display device (100) acquiring a front image using the camera (160).
  • at least one processor (180) may capture the front of the head mounted display device (100) through the camera (160) by executing the commands or program codes of the image acquisition module (171), thereby acquiring an image.
  • the present disclosure is not limited thereto, and at least one processor (180) may acquire an image from an external server or peripheral electronic device by executing the commands or program codes of the image acquisition module (171).
  • a method of operating a head mounted display device (100) may include a step (S200) of obtaining a user's gaze direction through a gaze tracking sensor (130) that tracks the user's gaze direction.
  • the gaze direction acquisition module (172) may be configured with commands or program codes related to an operation or function of the head mounted display device (100) to acquire the user's gaze direction by tracking the user's gaze direction using the gaze tracking sensor (130).
  • at least one processor (180) may execute commands or program codes of the gaze direction acquisition module (172) to track the user's gaze direction using the gaze tracking sensor (130) and acquire the user's gaze direction.
  • the gaze direction acquisition module (172) may be configured with instructions or program codes related to an operation or function of tracking a gaze direction of a user using the gaze tracking sensor (130) over a plurality of frames, and acquiring a plurality of gaze directions of the user corresponding to each of the plurality of frames.
  • at least one processor (180) may execute instructions or program codes of the gaze direction acquisition module (172), thereby tracking the gaze direction of the user for a plurality of frames using the gaze tracking sensor (130), and acquiring a plurality of gaze directions of the user corresponding to each of the plurality of frames.
  • a method of operating a head mounted display device (100) may include a step (S300) of acquiring a user's viewpoint based on an acquired gaze direction.
  • the viewpoint acquisition module (173) may be configured with commands or program codes related to an operation or function of acquiring the user's viewpoint based on the acquired direction of the user's gaze. In one embodiment, the viewpoint acquisition module (173) may calculate a point where the gazes of the user's two eyes intersect each other based on the acquired direction of the user's gaze, and acquire the corresponding point as a viewpoint. In one embodiment, in the step (S300) of acquiring the user's viewpoint, at least one processor (180) may acquire the user's viewpoint based on the user's gaze direction by executing commands or program codes of the viewpoint acquisition module (173).
  • the viewpoint acquisition module (173) may be configured with commands or program codes related to an operation or function of acquiring the user's viewpoint based on the distance between the head mounted display device (100) and the surrounding environment and the user's gaze direction.
  • the gaze acquisition module (173) may be configured with instructions or program codes related to an operation or function of acquiring a plurality of viewpoints corresponding to the plurality of frames, respectively, based on a plurality of gaze directions corresponding to the plurality of frames, respectively.
  • at least one processor (180) may execute instructions or program codes of the gaze acquisition module (173), thereby acquiring a plurality of viewpoints corresponding to the plurality of frames, respectively, based on a plurality of gaze directions corresponding to the plurality of frames, respectively.
  • step of obtaining the user's gaze direction (S200) and the step of obtaining the user's viewpoint (S300) are illustrated separately in FIG. 4, the present disclosure is not limited thereto.
  • the operation of obtaining the user's gaze direction and the operation of obtaining the user's viewpoint may be performed in one step.
  • a method of operating a head mounted display device (100) may include a step (S400) of dividing an acquired image into a region of interest and a region of non-interest based on an acquired user's viewpoint.
  • the region segmentation module (175) may be configured with instructions or program codes regarding an operation or function of segmenting an acquired image into a region of interest and a region of non-interest based on the acquired time point. In one embodiment, the region segmentation module (175) may be configured with instructions or program codes regarding an operation or function of segmenting an area in the image where the user's concentration is relatively high as a region of interest, and segmenting an area where the user's concentration is relatively low as a region of non-interest.
  • At least one processor (180) may segment the acquired image into a region of interest and a region of non-interest by executing instructions or program codes of the region segmentation module (175) in the step (S400) of segmenting the acquired image into a region of interest and a region of non-interest.
  • a method of operating a head mounted display device (100) may include a step (S500) of determining whether a preset object is included within a non-interest region.
  • the object detection module (176) may be configured with instructions or program codes related to an operation or function of detecting a preset object within a non-interest region. In one embodiment, the object detection module (176) may be configured with instructions or program codes related to an operation or function of determining whether some of a preset plurality of objects included in an object list (178) are included in a non-interest region. In one embodiment, the object detection module (176) may include an object detection algorithm such as a Recurrent Neural Network (RNN), a You Only Look Once (YOLO), a Region-based Convolutional Neural Networks (R-CNN), or a Single Shot MultiBox Detector (SSD).
  • RNN Recurrent Neural Network
  • YOLO You Only Look Once
  • R-CNN Region-based Convolutional Neural Networks
  • SSD Single Shot MultiBox Detector
  • At least one processor (180) may execute instructions or program codes of an object detection module (176) to determine whether a preset object is included within the non-interest area.
  • a method of operating a head mounted display device (100) may include a step (S600) of providing a notification to a user when a preset object is included within a non-interest area.
  • the notification providing module (177) may be configured with commands or program codes related to an operation or function of providing a notification including information that a preset object is included in a non-interest area.
  • the notification may include a notification image, a notification sound, or a notification vibration.
  • the notification image may include an instruction image indicating a location of a preset object, an emphasis image emphasizing a preset object, and the like.
  • the shape, size, color, and the like of the notification image may be set by the user interface (198) or may be preset.
  • the notification sound may include a guide sound guiding the location of a preset object, and the like.
  • the notification vibration may provide vibration to the user through the head mounted display device (100) when a preset object is detected, thereby drawing the user's attention.
  • a method of operating a head mounted display device (100) may include a step (S350) of generating an attention map representing a distribution of a user's viewpoint based on an acquired user's viewpoint.
  • the attention map generation module (174) may be configured with instructions or program codes related to operations or functions for generating an attention map based on the acquired image and the acquired user's viewpoint. In one embodiment, the attention map generation module (174) may generate an attention map indicating a point where the user gazes in the real world by corresponding the acquired viewpoint to the image.
  • an area corresponding to an area where the user's viewpoint is located in the image and is determined to be a point where the user is looking at, and an area corresponding to an area where the user's viewpoint is not located in the image and is determined to be a point where the user is not looking at, may be displayed differently.
  • at least one processor (180) may generate the attention map by executing instructions or program codes of the attention map generation module (174).
  • the attention map generation module (174) may be configured with instructions or program codes regarding an operation or function of generating an attention map based on a plurality of viewpoints and images corresponding to a plurality of frames, respectively. In one embodiment, the attention map generation module (174) may be configured with instructions or program codes regarding an operation or function of generating an attention map based on a distribution of a plurality of viewpoints corresponding to an image during a plurality of frames, a duration of a viewpoint at a specific location, a movement path of the viewpoints, a distance between adjacent viewpoints, etc. In one embodiment, the attention map may be a map indicating a degree of a user's attention toward the front based on a plurality of viewpoints of the user looking ahead during a plurality of frames.
  • At least one processor (180) may generate an attention map based on a plurality of viewpoints and images corresponding to each of a plurality of frames by executing instructions or program codes of the attention map generation module (174). The operation of generating an attention map will be described below with reference to FIGS. 9A to 9C.
  • the acquired image in the step (S400) of dividing the acquired image into a region of interest and a region of non-interest, can be divided into a region of interest and a region of non-interest based on an attention map (S410).
  • the region division module (175) may be configured with commands or program codes regarding an operation or function of dividing the generated attention map into a focused area and a non-focused area according to a preset criterion. In one embodiment, the region division module (175) may be configured with commands or program codes regarding an operation or function of dividing an area of up to the lower 30% of the user's attention index in the attention map into a non-focused area, and the remaining areas into focused areas.
  • the preset criterion includes the lower 30% criterion.
  • the lower 30% is only an example and is not limited thereto. In one embodiment, if it is desired to increase an area divided into a non-focused area in the attention map, the lower 30% may be increased to the lower 50%, etc. If it is desired to reduce an area divided into a non-focused area in the attention map, the lower 30% may be increased to the lower 20%, etc.
  • the "user's attention index” may indicate the degree of the user's attention to the front.
  • the user's attention index may be high when the number of viewpoints corresponding to the image acquired by shooting the front is large, the duration of the viewpoints is long, or the distance between adjacent viewpoints is close.
  • the user's attention index may be low when the number of viewpoints corresponding to the image is small, the duration of the viewpoints is short, or the distance between adjacent viewpoints is far.
  • the area division module (175) may be set to divide N grids, in order of increasing user attention indices, among the K grids, into focused areas, and divide the remaining M grids into non-focused areas.
  • the preset criterion may include a ratio between N and M.
  • the preset criterion when it is desired to increase an area divided into a non-focused area in the attention map, the preset criterion may be set such that the ratio occupied by M out of K becomes larger.
  • the preset criterion may be set such that the ratio occupied by N out of K becomes larger.
  • K may be a natural number
  • N and M may each be a positive integer
  • the sum of N and M may be K.
  • the region division module (175) may be configured with commands or program codes regarding an operation or function of dividing some of the regions located at a distance as non-focus regions based on the region with the highest user attention index in the attention map.
  • the preset criterion may be set to sequentially divide the regions from the furthest distance into non-focus regions based on the region with the highest user attention index.
  • the preset criterion may be set to divide the regions spaced apart from the region with the highest user attention index by a specific distance or more based on the region with the highest user attention index into non-focus regions.
  • the specific distance may be set to be large.
  • the region segmentation module (175) may be configured with commands or program codes related to an operation or function of segmenting a region corresponding to a focused region among the acquired images as a region of interest and segmenting a region corresponding to a non-focused region as a non-interested region.
  • At least one processor (180) may divide the attention map into a region of focus and a region of non-focus according to preset criteria by executing instructions or program codes of the region division module (175). At least one processor (180) may divide an area corresponding to a region of focus among the acquired image into a region of interest and a region corresponding to a region of non-focus as a region of non-interest by executing instructions or program codes of the region division module (175).
  • the present disclosure is not limited thereto, and the operation of dividing the attention map into a region of focus and a region of non-focus and the operation of dividing the acquired image into a region of interest and a region of non-interest may not be separated and may be performed as a single operation.
  • the operation of dividing the attention map into a region of interest and a region of non-interest will be described later with reference to FIGS. 10A to 10C.
  • the operation of dividing the image into areas of interest and areas of non-interest based on the attention map will be described later in Figs. 11a and 11.
  • the operating method of the head mounted display device (100) may include a step of cropping the divided non-interest region after the step (S400) of dividing an image into a region of interest and a non-interest region.
  • At least one processor (180) may crop the region divided into a non-interest region in the image.
  • At least one processor (180) may determine whether a preset object is detected in a cropped non-interest region through a region segmentation module (175) in the image. Through this, compared to a case where a preset object is detected for the entire image, when a preset object is detected for an image corresponding to a cropped non-interest region, the computation speed of object detection can be increased and power consumption can be reduced.
  • FIG. 7 is a flowchart for explaining an operation of dividing an image into a plurality of grids according to one embodiment of the present disclosure.
  • steps identical to those described in FIG. 4 are given the same drawing reference numerals, and redundant descriptions are omitted.
  • the operating method of the head-mounted display device (100) may further include a step (S310) of determining the complexity of an image.
  • at least one processor (180) may determine the complexity of an acquired image.
  • the complexity of an image may be determined based on a feature included in the image.
  • at least one processor (180) may determine the complexity of an image through an edge detection algorithm. The greater the amount of detected edges, the greater the complexity of the image may be determined to be.
  • at least one processor (180) may determine the complexity of an image by measuring a change in gray included in the image, a change in color of the image, a change in brightness of the image, and the like.
  • the operating method of the head mounted display device (100) may include a step (S320) of dividing an image into a plurality of grids.
  • at least one processor (180) may divide the image into a plurality of grids.
  • a “grid” means dividing an image at a regular interval, and at least one processor (180) may perform an operation of detecting a preset object for each of the divided plurality of grids.
  • At least one processor (180) may divide the image into a plurality of grids by varying the size of each of the plurality of grids according to the complexity of the image. In one embodiment, as the complexity of the image increases, at least one processor (180) may determine a smaller size for each of the plurality of grids.
  • the operating method of the head mounted display device (100) may further include a step of determining the resolution of the image.
  • at least one processor (180) may determine the resolution of the acquired image in the step of determining the resolution of the image. At this time, at least one processor (180) may also determine the resolution of the image based on the specifications of the camera (160).
  • At least one processor (180) may divide the image into a plurality of grids by making each of the plurality of grids different in size according to the resolution of the image. In one embodiment, as the resolution of the image increases, at least one processor (180) may determine that each of the plurality of grids becomes larger.
  • At least one processor (180) may divide an image into a plurality of grids without performing an operation of determining the resolution or complexity of the image.
  • an attention map divided into a plurality of grids can be generated.
  • the image in the step of dividing an image into a region of interest and a region of non-interest based on the attention map (S410), the image can be divided into a region of interest and a region of non-interest based on the attention map divided into a plurality of grids.
  • FIG. 8a is a diagram for explaining a plurality of grids for distinguishing an image with high complexity according to one embodiment of the present disclosure.
  • FIG. 8b is a diagram for explaining a plurality of grids for distinguishing an image with low complexity according to one embodiment of the present disclosure.
  • FIG. 8A illustrates an image with high complexity
  • FIG. 8B illustrates an image with low complexity
  • an image with high complexity may mean an image that includes a large number of objects, or an image with large changes in gradation, color, brightness, etc. within the image.
  • an image with low complexity may mean an image that includes a small number of objects, or an image with small changes in gradation, color, brightness, etc. within the image.
  • the detection resolution required for images with high complexity may be greater than the detection resolution required for images with low complexity.
  • the size of each of a plurality of grids (800) is divided into smaller sizes, and detection resolution is increased by determining whether a preset object is detected for each divided grid, thereby improving the detection capability.
  • the size of each of a plurality of grids (810) is divided into large sizes, and whether or not a preset object is detected is determined for each divided grid, thereby improving the speed of the detection operation.
  • FIG. 9A is a diagram for explaining an attention map according to a detected user's viewpoint according to one embodiment of the present disclosure.
  • FIG. 9B is a diagram for explaining an attention map divided into a plurality of grids according to a detected user's viewpoint according to one embodiment of the present disclosure.
  • FIG. 9C is a diagram for explaining an attention map according to a movement path of a user's viewpoint according to one embodiment of the present disclosure.
  • FIG. 9A illustrates a plurality of user viewpoints (900) acquired during an image and a plurality of frames corresponding to the image.
  • At least one processor (180) may generate an attention map representing a distribution of a plurality of user viewpoints (900) obtained by executing instructions or program codes of an attention map generation module (174).
  • the attention map may include a first region (910), a second region (920), a third region (930), and a fourth region (940).
  • the first region (910), the second region (920), the third region (930), and the fourth region (940) may be distinguished by the distribution of the plurality of user viewpoints (900) within the image.
  • At least one processor (180) can generate an attention map including regions that are distinguished based on the number of user viewpoints per unit area within the image.
  • the number of user viewpoints per unit area within the first region (910) can be greater than the number of user viewpoints per unit area within the second region (920).
  • the number of user viewpoints per unit area within the second region (920) can be greater than the number of user viewpoints per unit area within the third region (930).
  • the number of user viewpoints per unit area within the third region (930) can be greater than the number of user viewpoints per unit area within the fourth region (940).
  • the fourth region (940) can be a region where no user viewpoint is located.
  • an area with a large number of user viewpoints per unit area may be an area with high user concentration.
  • An area with a small number of user viewpoints per unit area may be an area with low user concentration.
  • the first area (910) may be an area with higher user concentration than the second area (920).
  • the second area (920) may be an area with higher user concentration than the third area (930).
  • the third area (930) may be an area with higher user concentration than the fourth area (940).
  • At least one processor (180) may distinguish an area in which a time period in which a viewpoint at a specific location in the attention map is maintained for a long time as an area in which the user's concentration is high. In addition, at least one processor (180) may distinguish an area in which the user's concentration is high as the distance between adjacent viewpoints in the attention map is close.
  • FIG. 9B illustrates an image divided into a plurality of grids (800) and a plurality of user viewpoints (900) detected during a plurality of frames corresponding to the image.
  • At least one processor (180) may generate an attention map divided into a plurality of grids (800) based on a plurality of acquired user viewpoints (900) and an image divided into a plurality of grids by executing instructions or program codes of an attention map generation module (174).
  • At least one processor (180) may generate an attention map based on a user's viewpoint included in each of the plurality of grids (800).
  • the attention map may include a first region (911), a second region (921), a third region (931), and a fourth region (941).
  • the number of user viewpoints included in each of at least one grid included in the first region (911) may be greater than the number of user viewpoints included in each of at least one grid included in the second region (921).
  • the number of user viewpoints included in each of at least one grid included in the second region (921) may be greater than the number of user viewpoints included in each of at least one grid included in the third region (931).
  • the number of user viewpoints included in each of at least one grid included in the third region (931) may be greater than the number of user viewpoints included in each of at least one grid included in the fourth region (941).
  • the user viewpoint may not be located in at least one grid included in the fourth region (940).
  • the focus area and the non-focus area can be divided on a grid basis.
  • the focus area and the non-focus area can be divided on a grid basis. Based on this, the computational process of at least one processor (180) can be simplified, and the processing speed can be improved.
  • the image and the attention map are divided into a plurality of grids (800).
  • FIG. 9c illustrates an image divided into a plurality of grids (800) and a plurality of user viewpoints (900) detected during a plurality of frames corresponding to the image.
  • FIG. 9b the same configuration as that described in FIG. 9b is given the same drawing reference numerals, and redundant descriptions are omitted.
  • FIG. 9C illustrates an image divided into a plurality of grids (800) and a plurality of user viewpoints (900) detected during a plurality of frames corresponding to the image.
  • At least one processor (180) may execute instructions or program codes of an attention map generation module (174) to generate an attention map based on a plurality of user viewpoints (900) and movement paths (950) of the plurality of user viewpoints (900) across a plurality of frames.
  • a user can move the viewpoint across multiple frames to observe a background or object in front of them.
  • the user gazes at a specific area in front of them and then gazes at another area, the user can perceive a background or object in an intermediate area between the specific area and the other area.
  • At least one processor (180) may distinguish an area where a movement path (950) is located as an area of high user concentration based on a plurality of user viewpoints (900) and a movement path (950) of the plurality of user viewpoints (900) across a plurality of frames.
  • the attention map may include a first region (911), a second region (921), a third region (931), and a fourth region (941).
  • at least one processor (180) may distinguish a region located on the movement path (950) as a first region (911) or a second region (921) with a high user concentration.
  • FIG. 10A is a diagram for explaining an operation of dividing an attention map into an area of interest and an area of non-interest according to one embodiment of the present disclosure.
  • FIG. 10B is a diagram for explaining non-interest areas separated from each other in an attention map according to one embodiment of the present disclosure.
  • FIG. 10C is a diagram for explaining that a non-interest area divided according to a preset condition in an attention map according to one embodiment of the present disclosure changes.
  • At least one processor (180) can divide an attention map into a focused area (1000) and a non-focused area (1010) by executing instructions or program codes of a region division module (175).
  • At least one processor (180) may divide the attention map into a focus area (1000) where the user's focus is high and a non-focus area (1010) where the user's focus is low, based on preset criteria.
  • the preset criterion may be a criterion set to distinguish M grids, which are the lower 32% of K multiple grids included in the attention map, as non-focused areas (1010).
  • K when the attention map includes a total of 50 multiple grids, K may be 50, M may be 16, and the remaining N of K may be 34.
  • At least one processor (180) may distinguish an area including 34 grids among the 50 multiple grids included in the attention map as a focus area (1000), and may distinguish an area including the remaining 16 grids (1020) as a non-focus area (1010).
  • the 16 grids (1020) included in the non-focus area (1010) may be grids classified in descending order of the user's focus level among the 50 grids included in the attention map.
  • the 16 grids (1020) included in the non-focus area (1010) may be grids included in the fourth area (941).
  • a plurality of grids included in the first to third areas (911, 921, 931) may be divided into a focus area (1000).
  • a grid that is farther away from the first area (911) where the user's focus level is the highest may be divided into a non-focus area (1010).
  • a grid having a high image complexity may be classified into a non-focused region (1010). Accordingly, referring to FIG. 10a, among the plurality of grids included in the fourth region (941), a grid corresponding to the ground in the front background included in the lower right of the attention map having a low image complexity may not be classified as a non-focused region (1010). Through this, an area in the fourth region (941) that is highly likely to include a preset object can be classified as a non-focused region (1010).
  • the present disclosure is not limited thereto, and when the preset criteria are set to distinguish grids with a higher ratio than the lower 32% as non-focused areas, or are set to distinguish grids with a number greater than M as non-focused areas, the area of the area distinguished as a non-focused area (1010) in the focus map may increase.
  • At least one processor (180) may determine a second position of a non-focused region (210, see FIG. 11a) within an image. At this time, the second position of the non-focused region (210) within the image may correspond to a position of the non-focused region (1010) within an attention map. In one embodiment, at least one processor (180) may determine the second position of the non-focused region (210) within the image based on the positions of each of at least one grid (1020) included within the non-focused region (1010). Additionally, at least one processor (180) may also determine the second position of the non-focused region (210) based on the position of the entire non-focused region (1010) including at least one grid (1020).
  • the second position when at least one processor (180) determines the second position based on the position of each of at least one grid (1020), the second position can be determined by considering the positions of each of at least one grid (1020) included in the non-focused area (1010), thereby increasing the accuracy of the determination.
  • the computational speed for determining the second position can be reduced.
  • At least one processor (180) can divide an attention map into a focused area (1000) and two or more non-focused areas (1010) spaced apart from each other by executing instructions or program codes of a region division module (175).
  • a region division module 175
  • the non-focused area (1010) may include a plurality of sub non-focused areas that are spaced apart from each other.
  • the non-focused area (1010) may include a first sub non-focused area (1011) and a second sub non-focused area (1012).
  • at least one processor (180) may distinguish the first sub non-focused area (1011) and the second sub non-focused area (1012) that are spaced apart from each other as the non-focused area (1010).
  • at least one processor (180) may determine the second position by considering the positions of at least one grid (1020) included in each of the first sub non-focused area (1011) and the second sub non-focused area (1012).
  • At least one processor (180) can divide an attention map into a focused area (1000) and a non-focused area (1010) according to preset criteria by executing instructions or program codes of a region division module (175).
  • a region division module 175
  • the preset criterion may be a criterion set to distinguish M grids, which are the bottom 20% of K multiple grids included in the attention map, as non-focused areas (1010).
  • K when the attention map includes a total of 50 multiple grids, K may be 50, M may be 10, and the remaining N out of K may be 40.
  • At least one processor (180) may distinguish an area including 40 grids among the 50 multiple grids included in the attention map as a focus area (1000), and distinguish an area including the remaining 10 grids (1020) as a non-focus area (1010).
  • the ratios of M and N to K included in the preset criteria can be set differently.
  • FIG. 11a is a diagram for explaining an operation of dividing an image into a region of interest and a region of non-interest according to one embodiment of the present disclosure.
  • FIG. 11b is a diagram for explaining an operation of dividing an image into a region of interest and a region of non-interest according to one embodiment of the present disclosure.
  • At least one processor (180) may identify a region corresponding to a non-focused region (1010) of an attention map in an image as a non-interested region (210).
  • At least one processor (180) may distinguish an area corresponding to a non-focused area (210) of an attention map in an image as a non-interested area (210) and crop the distinguished non-interested area (210). The at least one processor (180) may determine whether a preset object is included in the cropped non-interested area (210).
  • the computational speed of the at least one processor (180) increases and power consumption is reduced, thereby increasing the usage time of the battery (194, see FIG. 2).
  • the cropped non-interest region (210) of FIG. 11A may be divided into 16 grids.
  • At least one processor (180) may perform an operation of detecting a preset object for each of the 16 grids. In one embodiment, since it may be determined whether a preset object is included for each grid having a small size, the accuracy of the determination operation of at least one processor (180) may be increased.
  • FIG. 11b illustrates a cropped non-interest region (211) in an image.
  • the complexity of the image of the cropped non-interest region (211) illustrated in FIG. 11b may be less than the complexity of the image of the cropped non-interest region (210) illustrated in FIG. 11a.
  • the cropped non-interest region (211) of FIG. 11b may be divided into four grids.
  • At least one processor (180) may perform an operation of detecting a preset object for each of the four grids. In one embodiment, since it is possible to determine whether a preset object is included in each large grid, the speed of the determination operation of at least one processor (180) may be increased.
  • FIG. 12 is a diagram illustrating an operation of detecting a final position of an object included in a non-interest region according to one embodiment of the present disclosure.
  • At least one processor (180) can execute instructions or program codes of an object detection module (176) to distinguish a non-interest region (1210) from an image (1200) and determine whether a preset object (1220) is included in the distinguished non-interest region (1210).
  • the method of operating the head mounted display device (100) may further include a step of determining a final position of a preset object (1220) included in an image.
  • At least one processor (180) determines whether a preset object (1220) is included within the cropped non-interest region (1210), and thus determines a first location of the preset object (1220) within the non-interest region (1210). In one embodiment, at least one processor (180) can determine a first location that the preset object (1220) is included at the bottom within the non-interest region (1210).
  • At least one processor (180) can determine a second location of a cropped non-interest region (1210) within the image based on a location of the non-interest region (1010) within the attention map. In one embodiment, at least one processor (180) can determine that the second location is included on the right side of the image.
  • At least one processor (180) can determine a final position of a preset object (1220) included in an image based on the first position and the second position. In one embodiment, at least one processor (180) can determine that the final position of the preset object (1220) is located at the lower right within the image.
  • FIG. 13A is a diagram for explaining an operation of providing a notification in a head mounted display device of an optical see-through type according to one embodiment of the present disclosure.
  • At least one processor (180) may control the display (110) to provide a notification (240) indicating that a preset object (1220) is located at a final location of the image by executing instructions or program codes of a notification providing module (177).
  • the notification (240) may include a notification image indicating the location of the determined preset object.
  • the user when the head mounted display device (100) is of an optical see-through type, the user can observe a background or an object of the real world through the optical lens (120) and receive a virtual image through the display (110).
  • the virtual image provided through the display (110) may include content corresponding to a region of interest in the image that the user is gazing at for the convenience of the user.
  • the notification (240) for providing the user with information about the final location of the preset object (1220) may include content indicating a direction in which the preset object (1220) is facing based on the display (110).
  • the notification (240) provided on the display (110) may include directional content pointing to the lower right, allowing the user to move the viewpoint in that direction to recognize the preset object (1220).
  • FIG. 13b is a diagram for explaining an operation of providing a notification in a head mounted display device of a video see-through type according to an embodiment of the present disclosure.
  • FIG. 13c is a diagram for explaining an operation of providing a notification in a head mounted display device of a video see-through type according to an embodiment of the present disclosure.
  • At least one processor (180) may control the display (110) to provide a notification (240) indicating that a preset object (1220) is located at a final location in the image by executing instructions or program codes of a notification providing module (177).
  • the user when the head mounted display device (100) is of a video see-through type, the user can be provided with an image acquired by a camera through the display (1300).
  • the display (1300) can display an image captured in front of the head mounted display device (100).
  • the user can clearly recognize an image of an area (1310) corresponding to the user's viewpoint among the images displayed on the display (1300), and can blur an image of an area (1320) not corresponding to the user's viewpoint.
  • a notification (240) for providing the user with information on the final location of a preset object (1220) includes content indicating a direction in which the preset object (1220) is facing based on the area (1310) corresponding to the user's viewpoint, and can be displayed within the area (1310) corresponding to the user's viewpoint.
  • a notification (1330) for providing information on the final location of a preset object (1220) to a user may be displayed in an area (1320) that does not correspond to the user's viewpoint, so as to correspond to the final location of the preset object (1220).
  • the notification (1330) may include content having various shapes, icons, colors, etc., in order to focus the user's attention.
  • the user may delete a preset object (1220) included in the notification (1330) from the object list via the user interface (198).
  • the present disclosure is not limited thereto, and even if the user is provided with a notification (1330), the user may not move to the final location where the preset object (1220) is located, or may maintain the preset object (1220) in the object list if the user wishes to continue paying attention.
  • a head mounted display device may include a camera that acquires an image in front.
  • the head mounted display device may include a gaze tracking sensor that tracks a user's gaze direction.
  • the head mounted display device may include a memory that stores at least one instruction and at least one processor that executes at least one instruction stored in the memory.
  • the at least one processor may acquire a user's gaze based on the gaze direction by executing the at least one instruction.
  • the at least one processor may divide the acquired image into a region of interest and a region of non-interest based on the acquired user's gaze by executing the at least one instruction.
  • the at least one processor may determine whether a preset object is included in the region of non-interest by executing the at least one instruction.
  • the at least one processor may provide a notification to the user if the preset object is included in the region of non-interest by executing the at least one instruction.
  • At least one processor can generate an attention map representing a distribution of the user's viewpoints based on the acquired image and the acquired user's viewpoints by executing at least one command. At least one processor can divide the acquired image into a region of interest and a region of non-interest based on the attention map generated by executing at least one command.
  • At least one processor can divide an attention map into a focused region and a non-focused region according to a preset criterion by executing at least one command. At least one processor can divide an acquired image into a region of interest corresponding to the focused region and a non-interest region corresponding to the non-focused region by executing at least one command.
  • the number of user viewpoints per unit area included in the focused region can be greater than the number of user viewpoints per unit area included in the non-focused region.
  • At least one processor can track a gaze direction of a user through an eye tracking sensor for a plurality of frames by executing at least one command. At least one processor can acquire a plurality of viewpoints of users corresponding to each of the plurality of frames based on the gaze direction by executing at least one command. At least one processor can generate an attention map based on the acquired plurality of viewpoints of the users within the acquired image by executing at least one command.
  • At least one processor can divide an image into a plurality of grids by executing at least one command. At least one processor can generate an attention map divided into a plurality of grids based on a distribution of a plurality of user viewpoints included in each of the plurality of grids by executing at least one command. As the resolution of the image increases, the size of each of the plurality of grids can also increase.
  • At least one processor can determine the complexity of an image by executing at least one instruction. As the complexity of the image increases, the size of each of the plurality of grids can decrease.
  • At least one processor can crop a non-interest region from an acquired image by executing at least one command. At least one processor can determine whether a preset object is included within the cropped non-interest region by executing at least one command.
  • At least one processor can determine a final position of a preset object included in an image based on a first position of a preset object included in a non-interest region and a second position of the non-interest region within the image by executing at least one command.
  • the at least one processor can provide a notification to a user based on the determined final position by executing at least one command.
  • the head mounted display device may further include a display.
  • the notification may include a notification image indicating a location of the determined object.
  • At least one processor may control the display to display the notification image by executing at least one command.
  • the head mounted display device may further include a user interface.
  • the preset object may be a part of a plurality of objects included in an object list.
  • the plurality of objects included in the object list may be added or deleted based on user input obtained through the user interface.
  • the method for operating a head-mounted display device may include a step of acquiring an image in front through a camera.
  • the method for operating a head-mounted display device may include a step of acquiring a user's viewpoint based on a gaze direction acquired through a gaze tracking sensor that tracks the user's gaze direction.
  • the method for operating a head-mounted display device may include a step of dividing the acquired image into a region of interest and a region of non-interest based on the acquired viewpoint of the user.
  • the method for operating a head-mounted display device may include a step of determining whether a preset object is included in the region of non-interest.
  • the method for operating a head-mounted display device may include a step of providing a notification to the user if the preset object is included in the region of non-interest.
  • a method of operating a head-mounted display device may include a step of generating an attention map representing a distribution of the user's viewpoints based on the acquired image and the acquired user's viewpoints.
  • the acquired image may be divided into a region of interest and a region of non-interest based on the generated attention map.
  • a method of operating a head-mounted display device may include a step of dividing an attention map into a focus region and a non-focus region according to preset criteria.
  • the acquired image may be divided into a region of interest corresponding to the focus region and a non-interest region corresponding to the non-focus region.
  • the number per unit area of a user's viewpoint included in the focus region may be greater than the number per unit area of a user's viewpoint included in the non-focus region.
  • a plurality of user's viewpoints corresponding to each of the plurality of frames can be acquired based on the user's gaze direction acquired through the gaze tracking sensor during a plurality of frames.
  • an attention map can be generated based on the acquired multiple user's viewpoints within the acquired image.
  • a method of operating a head-mounted display device may include a step of dividing an image into a plurality of grids.
  • an attention map divided into a plurality of grids may be generated based on a distribution of a plurality of user viewpoints included in each of the plurality of grids. As the resolution of the image increases, the size of each of the plurality of grids may also increase.
  • the method of operating the head-mounted display device may further include a step of cropping a non-interest region from the acquired image.
  • the step of determining whether a preset object is detected it may be determined whether the preset object is included within the cropped non-interest region.
  • a method of operating a head-mounted display device may include a step of determining a final position of a preset object included in an image based on a first position of a preset object included in a non-interest region and a second position of the non-interest region within the image.
  • a notification may be provided to the user based on the determined final position.
  • the notification may include a notification image indicating the location of the determined object.
  • the method of operating the head mounted display device may further include the step of controlling the display to display the notification image.
  • the preset object may be a part of a plurality of objects included in an object list.
  • the method of operating the head mounted display device may include a step of adding or deleting a plurality of objects included in the object list based on a user input obtained through a user interface.
  • a computer-readable recording medium having recorded thereon a program for performing at least one method of the method of operating a head-mounted display device disclosed in one embodiment can be provided.
  • the program executed by the head mounted display device described in the present disclosure may be implemented as hardware components, software components, and/or a combination of hardware components and software components.
  • the program may be executed by any system capable of executing computer-readable instructions.
  • Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device.
  • the software may be implemented as a computer program including instructions stored on a computer-readable storage medium.
  • Examples of the computer-readable storage medium include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disks, hard disks, etc.) and optical readable media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)).
  • the computer-readable storage medium may be distributed across network-connected computer systems so that the computer-readable code may be stored and executed in a distributed manner.
  • the storage medium may be readable by a computer, stored in a memory, and executed by a processor.
  • a computer-readable storage medium may be provided in the form of a non-transitory storage medium.
  • a 'non-transitory storage medium' means only that it is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is stored temporarily.
  • signals e.g., electromagnetic waves
  • a 'non-transitory storage medium' may include a buffer in which data is temporarily stored.
  • a program according to the embodiments disclosed in this specification may be provided as a computer program product.
  • the computer program product may be traded between sellers and buyers as a commodity.
  • the computer program product may include a software program, a computer-readable storage medium having the software program stored thereon.
  • the computer program product may include a product in the form of a software program (e.g., a downloadable application) that is distributed electronically by the manufacturer of the head-mounted display device (100) or through an electronic market (e.g., Samsung Galaxy Store).
  • a software program e.g., a downloadable application
  • the storage medium may be a storage medium of a server of the manufacturer of the head-mounted display device (100), a server of the electronic market, or a relay server that temporarily stores the software program.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

La présente invention comprend un dispositif de visiocasque et son procédé de fonctionnement, le dispositif de visiocasque comprenant : une caméra pour acquérir une image à partir du côté avant ; un capteur de suivi du regard pour suivre une direction du regard d'un utilisateur ; une mémoire ; et au moins un processeur pour exécuter au moins une instruction stockée dans la mémoire, le ou les processeurs exécutant la ou les instructions pour : acquérir un point de vue de l'utilisateur sur la base de la direction du regard ; diviser l'image acquise en une zone d'intérêt et une zone de non-intérêt sur la base du point de vue acquis de l'utilisateur ; déterminer si un objet préconfiguré est inclus dans la zone de non-intérêt ; et fournir une notification à l'utilisateur lorsque l'objet préconfiguré est inclus dans la zone de non-intérêt.
PCT/KR2024/007730 2023-06-07 2024-06-05 Dispositif de visiocasque comprenant un capteur de suivi oculaire et procédé de fonctionnement pour dispositif de visiocasque Ceased WO2024253431A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020230073131A KR20240173959A (ko) 2023-06-07 2023-06-07 시선 추적 센서를 포함하는 헤드 마운티드 디스플레이 장치 및 헤드 마운티드 디스플레이 장치의 동작 방법
KR10-2023-0073131 2023-06-07

Publications (1)

Publication Number Publication Date
WO2024253431A1 true WO2024253431A1 (fr) 2024-12-12

Family

ID=93796156

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2024/007730 Ceased WO2024253431A1 (fr) 2023-06-07 2024-06-05 Dispositif de visiocasque comprenant un capteur de suivi oculaire et procédé de fonctionnement pour dispositif de visiocasque

Country Status (2)

Country Link
KR (1) KR20240173959A (fr)
WO (1) WO2024253431A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130048299A (ko) * 2011-11-01 2013-05-10 삼성전자주식회사 영상 처리 장치 및 방법
KR20160044528A (ko) * 2013-08-22 2016-04-25 엠파이어 테크놀로지 디벨롭먼트 엘엘씨 운전자 안전을 위한 시선의 영향
KR20170042164A (ko) * 2015-10-08 2017-04-18 엘지전자 주식회사 이동단말기 및 그 제어방법
KR102005528B1 (ko) * 2017-11-20 2019-07-30 에스케이텔레콤 주식회사 사용자 시선 유도 방법 및 상기 방법이 적용된 가상 현실 장치
KR20210062962A (ko) * 2019-11-22 2021-06-01 유엔젤주식회사 동공 추적 기반 가상 콘텐츠 표시 장치 및 그 제어 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130048299A (ko) * 2011-11-01 2013-05-10 삼성전자주식회사 영상 처리 장치 및 방법
KR20160044528A (ko) * 2013-08-22 2016-04-25 엠파이어 테크놀로지 디벨롭먼트 엘엘씨 운전자 안전을 위한 시선의 영향
KR20170042164A (ko) * 2015-10-08 2017-04-18 엘지전자 주식회사 이동단말기 및 그 제어방법
KR102005528B1 (ko) * 2017-11-20 2019-07-30 에스케이텔레콤 주식회사 사용자 시선 유도 방법 및 상기 방법이 적용된 가상 현실 장치
KR20210062962A (ko) * 2019-11-22 2021-06-01 유엔젤주식회사 동공 추적 기반 가상 콘텐츠 표시 장치 및 그 제어 방법

Also Published As

Publication number Publication date
KR20240173959A (ko) 2024-12-16

Similar Documents

Publication Publication Date Title
WO2022139375A1 (fr) Dispositif de réalité augmentée et son procédé d'opération
WO2019172678A1 (fr) Système et procédé d'interaction de réalité augmentée
WO2019147021A1 (fr) Dispositif de fourniture de service de réalité augmentée et son procédé de fonctionnement
WO2020111594A1 (fr) Dispositif électronique, dispositif de réalité augmentée destiné à fournir un service de réalité augmentée, et son procédé de fonctionnement
WO2015105347A1 (fr) Appareil d'affichage vestimentaire
EP3911992A1 (fr) Dispositif de visiocasque et son procédé de fonctionnement
WO2019039771A1 (fr) Dispositif électronique pour mémoriser des informations de profondeur en relation avec une image en fonction des propriétés d'informations de profondeur obtenues à l'aide d'une image, et son procédé de commande
WO2016143983A1 (fr) Procédé et dispositif pour émettre une lumière utilisée pour capturer un iris
WO2021025509A1 (fr) Appareil et procédé d'affichage d'éléments graphiques selon un objet
EP3092523A1 (fr) Appareil d'affichage vestimentaire
WO2022092517A1 (fr) Dispositif électronique pouvant être porté comprenant une unité d'affichage, procédé de commande d'affichage, système comprenant un dispositif électronique pouvant être porté et boîtier
WO2016182090A1 (fr) Terminal de type lunettes et son procédé de commande
EP3259658A1 (fr) Procédé et appareil photographique destinés à commander une fonction sur la base d'un geste d'utilisateur
WO2015064935A1 (fr) Dispositif électronique et son procédé de commande
WO2016126083A1 (fr) Procédé, dispositif électronique et support d'enregistrement pour notifier des informations de situation environnante
WO2022108001A1 (fr) Procédé de commande de dispositif électronique par reconnaissance d'un mouvement au niveau du bord d'un champ de vision (fov) d'une caméra, et dispositif électronique associé
WO2021242008A1 (fr) Dispositif électronique et son procédé de fonctionnement
WO2024253431A1 (fr) Dispositif de visiocasque comprenant un capteur de suivi oculaire et procédé de fonctionnement pour dispositif de visiocasque
WO2022146018A1 (fr) Dispositif électronique et son procédé de commande
WO2025037863A1 (fr) Dispositif de visiocasque pour afficher une interface, et procédé de fonctionnement du dispositif de visiocasque
WO2024144353A1 (fr) Dispositif de visiocasque comprenant un capteur de suivi oculaire et procédé de fonctionnement pour dispositif de visiocasque
WO2024205017A1 (fr) Dispositif électronique et procédé de fourniture d'image d'espace virtuel
WO2024248486A1 (fr) Dispositif de réalité augmentée et son procédé de fonctionnement
WO2026063609A1 (fr) Appareil visiocasque et son procédé de fonctionnement
WO2024248559A1 (fr) Dispositif de réalité augmentée pour effectuer un paiement à l'aide d'informations de regard d'un utilisateur et son procédé de fonctionnement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24819582

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE