WO2015102464A1 - Dispositif électronique et procédé d'affichage d'événement en mode de réalité virtuelle - Google Patents

Dispositif électronique et procédé d'affichage d'événement en mode de réalité virtuelle Download PDF

Info

Publication number
WO2015102464A1
WO2015102464A1 PCT/KR2015/000097 KR2015000097W WO2015102464A1 WO 2015102464 A1 WO2015102464 A1 WO 2015102464A1 KR 2015000097 W KR2015000097 W KR 2015000097W WO 2015102464 A1 WO2015102464 A1 WO 2015102464A1
Authority
WO
WIPO (PCT)
Prior art keywords
event
virtual reality
electronic device
reality mode
screen
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2015/000097
Other languages
English (en)
Korean (ko)
Inventor
윤영성
강상원
이진구
최성훈
최우석
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020150000892A external-priority patent/KR102355118B1/ko
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Priority to US15/109,558 priority Critical patent/US10431004B2/en
Priority to EP15733319.6A priority patent/EP3093743B1/fr
Priority to CN201580003868.XA priority patent/CN105900051B/zh
Publication of WO2015102464A1 publication Critical patent/WO2015102464A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0132—Head-up displays characterised by optical features comprising binocular systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0132—Head-up displays characterised by optical features comprising binocular systems
    • G02B2027/0136—Head-up displays characterised by optical features comprising binocular systems with a single image source for both eyes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems

Definitions

  • Various embodiments of the present disclosure relate to an event display method in an electronic device and a virtual reality mode.
  • VR virtual reality
  • the user can interact with the virtual reality in real time through the operation of the device, and can experience a sensory experience similar to the reality.
  • Augmented reality is a field of virtual reality that is a computer graphics technique for synthesizing virtual objects or information in a real environment to look like objects existing in the original environment.
  • Such augmented reality is a technology of superimposing a virtual object on the real world that the user sees, and is also referred to as mixed reality (MR) because the real world shows additional information and the virtual world in a single image.
  • MR mixed reality
  • virtual reality technology has become common and easily accessible to various services such as education, games, navigation, advertisements, or blogs as mobile devices (eg, smartphones, tablet PCs, etc.) are becoming more common.
  • mobile devices eg, smartphones, tablet PCs, etc.
  • wearable devices that can be worn on a body are commercialized, research on virtual reality technology has become more active.
  • the wearable device is provided in various forms detachable to the human body and clothing, such as head-mounted, eyeglasses, watches, bands, contact lenses, rings, shoes, clothing.
  • the wearable device may increase portability and accessibility by allowing the electronic device to be worn on a body such as clothes or glasses.
  • HMDs head mounted displays
  • AR augmented reality
  • VR virtual reality
  • the see-through form can reinforce additional information that is difficult to obtain only in the real world by synthesizing and combining virtual objects or objects on the basis of the real world using the characteristics of the semi-transparent lens.
  • two displays are placed in front of the eyes, and the content provided through external input (game, movie, streaming, broadcast, etc.) can be viewed alone on a separate screen, thereby providing excellent immersion.
  • an event when an event occurs while operating in the virtual reality mode, an event is displayed in the electronic device and the virtual reality mode which may process the generated event and display it according to the virtual reality mode. It may provide a method.
  • a method of displaying an event in the real mode may be provided.
  • the present invention when an event occurs while operating in the virtual reality mode, the execution of an application related to the event executed by an existing operating system is blocked, and the generated event is processed.
  • the present invention can provide an electronic device capable of displaying in accordance with the virtual reality mode and an event display method in the virtual reality mode.
  • an electronic device including: a display unit configured to display a screen corresponding to a virtual reality mode; And detecting an interrupt according to at least one event occurrence, changing event related information related to the generated event into a form corresponding to the virtual reality mode, and wherein the changed event related information is executed corresponding to the virtual reality mode. It may include a control unit for controlling to be displayed on the image.
  • the event display method in the virtual reality mode according to any one of the various embodiments, the operation of executing the virtual reality mode in the electronic device; Detecting an interrupt due to at least one event occurrence; Changing event related information related to the generated event into a form corresponding to the virtual reality mode; And displaying the changed event related information on a screen executed corresponding to the virtual reality mode.
  • an event display method in an electronic device and a virtual reality mode may include executing an application related to the event in a virtual reality mode when an interrupt occurs due to a specific event while operating in the virtual reality mode.
  • an application related to the event in a virtual reality mode when an interrupt occurs due to a specific event while operating in the virtual reality mode.
  • the event when an interrupt occurs due to a specific event while operating in the virtual reality mode, the event may be detected in the virtual reality mode while minimizing the performance of the electronic device. You can notify efficiently.
  • FIG. 1 is a diagram illustrating a network environment according to an embodiment of the present invention.
  • FIG. 2 is a diagram illustrating a configuration example of an electronic device according to various embodiments of the present disclosure.
  • FIG. 3 is a block diagram of a program module according to various embodiments of the present disclosure.
  • FIG. 4 is a flowchart illustrating an event display procedure in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 5 is a diagram illustrating an example in which an electronic device is mounted on a wearable device according to various embodiments of the present disclosure.
  • FIG. 6 is a diagram illustrating an example in which a user wears a wearable device equipped with an electronic device, according to various embodiments of the present disclosure.
  • FIG. 7 is a diagram illustrating a screen mode of an electronic device according to various embodiments of the present disclosure.
  • FIG. 8 is a screen example illustrating an event display method in a normal mode.
  • FIG. 9 is an example of a screen illustrating an event display method in a virtual reality mode.
  • FIG. 10 is a screen example illustrating an event display method in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 11 is a block diagram illustrating a detailed structure of a controller according to various embodiments of the present disclosure.
  • FIG. 12 is a flowchart illustrating an event display procedure in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 13 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 14 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 15 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 16 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 17 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 18 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 19 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 20 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 21 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 22 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • FIG. 23 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 24 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 25 is a diagram illustrating an example of a screen displaying an event in a virtual reality mode according to various embodiments of the present disclosure.
  • 26 is a block diagram illustrating a detailed structure of an electronic device according to an embodiment of the present disclosure.
  • 27 is a block diagram of a program module according to various embodiments of the present disclosure.
  • FIG. 28 is a block diagram illustrating a configuration example of a wearable device according to various embodiments of the present disclosure.
  • expressions such as “have”, “may have”, “include”, or “may contain” include the presence of a corresponding feature (e.g., numerical, functional, operational, or component such as a component). Does not exclude the presence of additional features.
  • expressions such as “A or B”, “at least one of A or / and B”, or “one or more of A or / and B” may include all possible combinations of items listed together.
  • “A or B”, “at least one of A and B”, or “at least one of A or B” includes (1) at least one A, (2) at least one B, Or (3) both of cases including at least one A and at least one B.
  • first,” “second,” “first,” or “second,” as used in various embodiments may modify various elements in any order and / or importance, and the elements may be modified. It is not limited. The above expressions may be used to distinguish one component from another.
  • the first user device and the second user device may represent different user devices regardless of the order or importance.
  • the first component may be referred to as a second component, and similarly, the second component may be renamed to the first component.
  • One component (such as a first component) is "(functionally or communicatively) coupled with / to" to another component (such as a second component) or " When referred to as “connected to”, it should be understood that any component may be directly connected to the other component or may be connected through another component (eg, a third component).
  • a component e.g., a first component
  • another component e.g., a second component
  • no other component e.g., a third component
  • the expression “configured to” used in this document is, for example, “suitable for”, “having the capacity to” depending on the situation. It may be used interchangeably with “designed to”, “adapted to”, “made to”, or “capable of”.
  • the term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “device configured to” may mean that the device “can” along with other devices or components.
  • the phrase “processor configured (or set up) to perform A, B, and C” may execute a dedicated processor (eg, an embedded processor) to perform the operation, or one or more software programs stored in a memory device. By doing so, it may mean a general-purpose processor (for example, a CPU or an application processor) capable of performing the corresponding operations.
  • the electronic device may be a smartphone, a tablet personal computer, a mobile phone, a video phone, or an e-book reader. reader, desktop personal computer (PC), laptop personal computer (PC), netbook computer, workstation, server, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player , Mobile medical devices, cameras, or wearable devices (e.g. smart glasses, head-mounted-device (HMD)), electronic clothing, electronic bracelets, electronic necklaces, electronic accessories (appcessory), an electronic tattoo, a smart mirror, or a smart watch.
  • PC personal computer
  • PC personal computer
  • netbook computer workstation
  • server personal digital assistant
  • PMP portable multimedia player
  • MP3 player MP3 player
  • Mobile medical devices cameras, or wearable devices (e.g. smart glasses, head-mounted-device (HMD)), electronic clothing, electronic bracelets, electronic necklaces, electronic accessories (appcessory), an electronic tattoo, a smart mirror, or a smart watch.
  • HMD head-mounted-device
  • electronic clothing
  • the electronic device may be a smart home appliance.
  • Smart home appliances are, for example, televisions, digital video disk (DVD) players, audio, refrigerators, air conditioners, vacuum cleaners, ovens, microwave ovens, washing machines, air purifiers, set-top boxes, home automation Home automation control panel, security control panel, TV box (e.g. Samsung HomeSync TM, Apple TV TM, or Google TV TM), game console (e.g. Xbox TM, PlayStation TM), electronics It may include at least one of a dictionary, an electronic key, a camcorder, or an electronic picture frame.
  • DVD digital video disk
  • the electronic device may include various medical devices (eg, various portable medical measuring devices (such as blood glucose meters, heart rate monitors, blood pressure monitors, or body temperature meters), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), Such as CT (computed tomography, imaging or ultrasound), navigation devices, global positioning system receivers, event data recorders (EDRs), flight data recorders (FDRs), automotive infotainment devices, ships Electronic equipment (e.g. ship navigation systems, gyro compasses, etc.), avionics, security devices, vehicle head units, industrial or home robots, automatic teller's machines (financial institutions), shop POS (point of sales), or the Internet of things (e.g. light bulbs, sensors, electrical or gas meters, sprinkler devices, fire alarms, thermostats, street lights, toasters) ), Exercise equipment, hot water tank, heater, boiler, and the like.
  • various portable medical measuring devices such as blood glucose meters, heart rate monitors, blood pressure monitors, or body temperature meters
  • MRA magnetic
  • an electronic device may be a furniture or part of a building / structure, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (eg, Water, electricity, gas, or radio wave measuring instrument).
  • the electronic device may be one or a combination of the aforementioned various devices.
  • An electronic device according to an embodiment may be a flexible electronic device.
  • the electronic device according to an embodiment of the present disclosure is not limited to the above-described devices, and may include a new electronic device according to technology development.
  • the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) that uses an electronic device.
  • an electronic device 101 in a network environment 100 is described.
  • the electronic device 101 includes at least one of the bus 110, the processor 120, the memory 130, the input / output interface 150, the display 160, the communication interface 170, or the event processing module 180. can do.
  • the electronic device 101 may omit at least one of the above components or additionally include other components.
  • the bus 110 may include, for example, circuits that connect the components 120 to 180 to each other and transmit communication (eg, control messages and / or data) between the components. .
  • the processor 120 may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP).
  • the processor 120 may execute, for example, an operation or data processing related to control and / or communication of at least one other element of the electronic device 101.
  • the memory 130 may include a volatile and / or nonvolatile memory.
  • the memory 130 may store, for example, commands or data related to at least one other element of the electronic device 101.
  • the memory 130 may store software and / or a program 140.
  • the program 140 may be, for example, a kernel 141, middleware 143, an application programming interface (API) 145, and / or an application program (or “application”) 147. And the like.
  • At least a portion of the kernel 141, the middleware 143, or the API 145 may be referred to as an operating system (OS).
  • OS operating system
  • the kernel 141 may be, for example, system resources used to execute an operation or function implemented in other programs (eg, middleware 143, API 145, or application program 147).
  • the bus 110, the processor 120, or the memory 130 may be controlled or managed.
  • the kernel 141 may control or manage system resources by accessing individual components of the electronic device 101 from the middleware 143, the API 145, or the application program 147. Can provide an interface.
  • the middleware 143 may serve as an intermediary for allowing the API 145 or the application program 147 to communicate with the kernel 141 to exchange data.
  • the middleware 143 may be connected to at least one application of the application program 147 in connection with the work requests received from the application program 147. For example, assigning a priority to use the bus 110, the processor 120, or the memory 130, or the like, to control work requests (e.g., scheduling or load balancing). Can be.
  • the API 145 is, for example, an interface for the application 147 to control a function provided by the kernel 141 or the middleware 143.
  • a function provided by the kernel 141 or the middleware 143.
  • the file control, window control, and image may include at least one interface or function (eg, a command) for processing or character control.
  • the input / output interface 150 may serve as, for example, an interface capable of transferring a command or data input from a user or another external device to other component (s) of the electronic device 101.
  • the input / output interface 150 may output a command or data received from other component (s) of the electronic device 101 to a user or another external device.
  • the display 160 may be, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or a microelectromechanical systems (MEMS) display, or an electronic paper ( electronic paper) display.
  • the display 160 may display, for example, various contents (for example, text, an image, a video, an icon, a symbol, etc.) to a user.
  • the display 160 may include a touch screen.
  • the display 160 may receive a touch, gesture, proximity, or hovering input using an electronic pen or a part of a user's body.
  • the communication interface 170 may, for example, communicate between the electronic device 101 and an external device (eg, the first external electronic device 102, the second external electronic device 104, or the server 106). Can be set.
  • the communication interface 170 may be connected to the networks 162 and 164 through wireless or wired communication to communicate with the external device (eg, the second external electronic device 104 or the server 106). Can be.
  • the first external electronic device 102 or the second external electronic device 104 may be a wearable device or a wearable device (eg, a head mounted display (HMD)) on which the electronic device 101 may be mounted.
  • HMD head mounted display
  • the electronic device 101 detects that the electronic device 101 is mounted on the HMD device, and enters a virtual reality mode. It can work.
  • the electronic device 101 is mounted on the electronic device 102 (eg, an HMD device)
  • the electronic device 101 can communicate with the electronic device 102 through the communication interface 170.
  • the electronic device 101 may communicate directly with the electronic device 102 by directly connecting to the electronic device 102 through a separate network.
  • the wireless communication may use, for example, at least one of LTE, LTE-A, CDMA, WCDMA, UMTS, WiBro, or GSM as a cellular communication protocol.
  • the wired communication may include, for example, at least one of a universal serial bus (USB), a high definition multimedia interface (HDMI), a reduced standard 232 (RS-232), a plain old telephone service (POTS), and the like.
  • the network 162 may include a telecommunications network, for example, at least one of a computer network (for example, a LAN or a WAN), the Internet, and a telephone network.
  • Each of the first and second external electronic devices 102 and 104 may be a device that is the same as or different from the electronic device 101.
  • the server 106 may include a group of one or more servers.
  • all or part of operations executed in the electronic device 101 may be executed in another or a plurality of electronic devices (for example, the electronic devices 102 and 104 or the server 106).
  • the electronic device 101 when the electronic device 101 needs to perform a function or service automatically or by request, the electronic device 101 instead of executing the function or the service by itself or Additionally, at least some functions associated with the request may be requested to another device (eg, the electronic device 102 or 104 or the server 106).
  • the other electronic device may execute the requested function or additional function and transmit the result to the electronic device 101.
  • the electronic device 101 may process the received result as it is or additionally to provide the requested function or service.
  • cloud computing, distributed computing, or client-server computing technology may be used.
  • the electronic device 101 is provided with a communication interface 170 to communicate with an external electronic device 104 or a server 106 through a network 162, but according to various embodiments of the present disclosure, Accordingly, the electronic device 101 may be implemented to operate independently in the electronic device 101 without a separate communication function.
  • the server 106 drives the electronic device 101 by performing at least one operation (or function) among operations (or functions) implemented in the electronic device 101.
  • the server 106 may include an event processing server module (not shown) capable of supporting the event processing module 180 implemented in the electronic device 101.
  • the event processing server module may include at least one component of the event processing module 180 to perform (or perform) at least one of the operations (or functions) performed by the event processing module 180. You can also act.
  • the event processing module 180 may include at least some of information obtained from other components (eg, the processor 120, the memory 130, the input / output interface 150, the communication interface 170, etc.). Can be processed and provided to the user in various ways.
  • the event processing module 180 may be mounted on the wearable device (eg, the electronic device 102) to function as a display device according to various embodiments of the present disclosure, and may be a virtual reality mode.
  • the information related to the event occurring during the operation may be processed to display in accordance with the virtual reality mode.
  • the event processing module 180 may block the execution of the application or operate in the background when the event generated during the operation in the virtual reality mode is an event related to the execution of the application. Additional information related to the event processing module 180 may be provided through FIG. 2 to be described later.
  • the event processing module 180 is illustrated as a separate module from the processor 120, but at least a part of the event processing module 180 is included in the processor 120 or at least one other module to be implemented. In addition, the entire function of the event processing module 180 may be implemented by being included in the illustrated processor 120 or another processor. In addition, the event processing module 180 may perform operations according to various embodiments of the present disclosure in association with at least one program 140 stored in the memory 130.
  • the electronic device 220 may be an electronic device 220 having at least one display means.
  • the electronic device 220 may be a device having a display function as a main function or may refer to a general electronic device including at least one display means.
  • the electronic device 220 may be an electronic device (eg, a smartphone) having a touch screen 230.
  • the electronic device 220 may include at least one of the touch screen 230, the controller 240, the storage 250, or the communicator 260.
  • the touch screen 230 may include a display panel 231 and / or a touch panel 232.
  • the controller 240 may include at least one of a virtual reality mode processor 241, an event determiner 242, an event information processor 243, or an application controller 244.
  • the electronic device 220 when the electronic device 220 is mounted on the wearable device 210, the electronic device 220 may operate as an HMD device and execute a virtual reality mode.
  • the virtual reality mode may be executed according to a user's setting or execution of an application related to the virtual reality mode.
  • the electronic device 220 is set to be mounted on the wearable device 210 to execute the virtual reality mode, but various embodiments of the present disclosure are not limited thereto.
  • the electronic device 220 when the electronic device 220 operates in a virtual reality mode (for example, the electronic device 220 may be mounted on the wearable device 210 to operate in a head mounted theater (HMT) mode).
  • the screen displayed through the display panel 231 may display two screens (eg, the screen illustrated in FIG. 7) corresponding to the left and right eyes of the user.
  • the controller 240 when the electronic device 220 operates in the virtual reality mode, processes information related to an event generated during the virtual reality mode in accordance with the virtual reality mode and displays the same on the screen. Can be controlled. According to various embodiments of the present disclosure, the controller 240 may block the execution of the application or operate in the background when the event occurring during the virtual reality mode operation is an event related to the execution of the application.
  • the controller 240 may include at least one of a virtual reality mode processor 241, an event determiner 242, an event information processor 243, or an application controller 244. Including one, functions according to various embodiments of the present disclosure may be performed. In addition, various operations related to FIGS. 3 to 25 to be described below are described using at least one configuration (eg, the touch screen 230, the controller 240, the storage 250, etc.) constituting the electronic device 220. Or may be implemented to perform functions.
  • the virtual reality when the electronic device 220 is mounted on the wearable device 210 or the virtual reality mode is executed according to a user's setting or execution of an application related to the virtual reality mode, the virtual reality is executed.
  • the mode processor 241 may process various functions related to the operation of the virtual reality mode.
  • the virtual reality mode processor 241 may load at least one virtual reality program 251 stored in the storage 250 to perform various functions.
  • the event determiner 242 may determine an event occurring while operating in the virtual reality mode by the virtual reality mode processor 241. In addition, the event determination unit 242 may determine whether there is information to be displayed on the screen in relation to an event generated while operating in the virtual reality mode. In addition, the event determination unit 242 may determine an application to be executed in relation to an event generated while operating in the virtual reality mode. Various embodiments of the application related to the type of event will be described later with reference to FIGS. 13 to 25.
  • the event information processor 243 is related to the event to be displayed on the screen according to the determination result of the event determination unit 242. It is possible to process the information to fit the virtual reality mode.
  • Various methods may be used as the processing method of the event related information.
  • the event related information may be converted to match the 3D image.
  • the event-related information displayed in a planar form may be converted into information corresponding to the left eye and the right eye of the 3D image, and the combined information may be displayed on the screen of the virtual reality mode currently being executed. have.
  • the application controller 244 controls to block execution of an application related to the event when there is an application to be executed in relation to an event generated while operating in the virtual reality mode according to the determination result of the event determination unit 242. can do.
  • the application controller 244 determines, according to the determination result of the event determiner 242, the application controller 244.
  • the application may be controlled to be executed in the background so as not to affect the execution or the screen display of the application corresponding to the virtual reality mode.
  • the virtual reality program 251 may be stored in the storage 250, and the virtual reality program 251 may be an application related to a virtual reality mode operation of the electronic device 220.
  • event storage information 252 may be stored in the storage unit 250, and the event determination unit 242 may refer to event-related information 252 stored in the storage unit 250 to determine an event. You can check information about whether the screen is displayed or the application to be executed with respect to the event.
  • the wearable device 210 may be any electronic device including at least one function of the electronic device 101 shown in FIG. 1, or may be a wearable holder that can mount the electronic device 220.
  • the wearable device 210 is an electronic device
  • various functions may be provided through the communication unit 260 of the electronic device 220.
  • the electronic device 220 detects whether the electronic device 220 is mounted with the wearable device 210 through communication with the wearable device 210, and the virtual reality. It may be determined whether to operate in a mode (or HMT mode).
  • the user when it is not possible to automatically determine whether the electronic device 220 is mounted, the user may select a virtual reality program ( Various embodiments of the present disclosure may be applied by executing 251 or selecting a virtual reality mode (or HMT mode).
  • a virtual reality mode or HMT mode
  • the wearable device 210 when the wearable device 210 includes a function as the electronic device 101, it is automatically determined whether the electronic device 220 is mounted on the wearable device 210.
  • the execution mode of the electronic device 220 may be automatically switched to a virtual reality mode (or HMT mode).
  • At least some functions of the controller 240 illustrated in FIG. 2 may be included in the event processing module 180 or the processor 120 of the electronic device 101 illustrated in FIG. 1.
  • the touch screen 230 or the display panel 231 illustrated in FIG. 2 may correspond to the display 160 of FIG. 1.
  • the storage unit 250 illustrated in FIG. 2 may correspond to the memory 130 illustrated in FIG. 1.
  • the touch screen 230 including the display panel 231 and the touch panel 232 is illustrated.
  • the display panel 231 or the display panel 231 is not in the form of one touch screen 230.
  • the touch panel 232 may be provided in a separate panel form.
  • the electronic device 220 may include only the display panel 231 instead of the touch panel 232.
  • the electronic device 220 is referred to as a first device (or first electronic device), and the wearable device 210 is referred to as a second device (or second electronic device). Device).
  • An electronic device may include a display unit configured to display a screen corresponding to a virtual reality mode; And detecting an interrupt according to at least one event occurrence, changing event related information related to the generated event into a form corresponding to the virtual reality mode, and wherein the changed event related information is executed corresponding to the virtual reality mode. It may include a control unit for controlling to be displayed on the image.
  • the event may include a call reception event, a message reception event, an alarm notification, a scheduler notification, a Wi-Fi connection, a Wi-Fi disabling, a low battery notification, a data allowance or usage restriction notification, an application no response notification, or At least one of the application abnormal termination notifications may be selected.
  • the electronic device may further include a storage unit configured to store information related to the event when the event is not an event to be displayed in the virtual reality mode.
  • the electronic device may control to display information related to the event stored in the storage unit on the screen.
  • the electronic device may further include a storage unit that stores information on at least one event to be displayed in the virtual reality mode.
  • the event may be an instant message reception notification event.
  • the controller may control to block execution of the application according to the occurrence of the event.
  • the controller may control the blocked application to be executed when the screen mode of the electronic device is switched from the virtual reality mode to the see-through mode.
  • the controller may control the execution of the application according to the occurrence of the event to be executed in the background of the virtual reality mode screen. Can be.
  • the controller may control the virtual reality mode to be executed when the electronic device is connected to the wearable device.
  • the controller may arrange and process information related to the event to be displayed on a three-dimensional space of the virtual reality mode screen currently displayed on the screen.
  • a program module may include a system operating system (eg, an operating system (OS) 310), a framework 320, an application 330, and the like.
  • OS operating system
  • framework 320 framework 320
  • application 330 application 330
  • the system operating system 310 may include at least one system resource manager or at least one device driver.
  • the system resource manager may perform control, allocation, or retrieval of system resources, and may include at least one manager such as a process manager, a memory manager, or a file system manager.
  • the device driver may include, for example, at least one driver such as a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WIFI driver, an audio driver, or an IPC driver. It may include.
  • the framework 320 may provide, for example, a function commonly required by the application, or the application may efficiently manage limited system resources inside the electronic device.
  • Various functions may be provided to the application through the API for use.
  • the VR framework included in the framework 320 may control a function related to virtual reality mode operation in the electronic device. For example, according to execution of a virtual reality mode operation, the VR framework 320 may control at least one VR application 351 related to virtual reality among the applications 330 to provide a virtual reality mode in the electronic device. have.
  • the application 330 may include a plurality of applications, and may include at least one VR application 351 running in the virtual reality mode and at least one general application 352 running in the normal mode other than the virtual reality mode. Can be.
  • the application 330 may further include a VR control application 340, and at least one VR application 351 and / or under the control of the VR control application 340.
  • the operation of at least one general application 352 may be controlled.
  • the system operating system 310 when at least one event occurs while the electronic device operates in the virtual reality mode, notifies the framework 320 (eg, the VR framework) of the event occurrence. can do.
  • the framework 320 eg, the VR framework
  • the framework 320 may control execution of the corresponding general application 352 so that information related to the event may be displayed on the screen with respect to an event occurring in the general mode other than the virtual reality mode.
  • the framework 320 may control to execute at least one corresponding general application 352.
  • the framework 320 when an event occurs while operating in the virtual reality mode, the framework 320 (eg, the VR framework) may include at least one general application for displaying information related to the generated event. Operation of 352 may be blocked. The framework 320 may provide an event generated while operating in the virtual reality mode to the VR control application 340.
  • the VR control application 340 may process information related to an event generated during the operation of the virtual reality mode according to the virtual reality mode. For example, two-dimensional plane-related event-related information may be processed into three-dimensional information.
  • the VR control application 340 may control at least one VR application 351 currently being executed, and control to display the processed event related information on a screen executed by the VR application 351 and display it. can do.
  • the framework 320 controls to block execution of at least one general application 352 related to the generated event. can do.
  • the framework 320 when an event occurs while operating in the virtual reality mode, temporarily blocks execution of at least one general application 352 related to the generated event.
  • the blocked general application 352 may be controlled to run.
  • the framework 320 controls execution of at least one general application 352 related to the generated event.
  • at least one general application 352 related to the event may be controlled to operate in the background.
  • FIG. 3 is an example for implementing various embodiments of the present invention in a program form, and various embodiments of the present invention are not limited to the embodiment of FIG. 3, and according to various embodiments of the present disclosure. It may also be implemented in other forms.
  • FIG. 4 is a flowchart illustrating an event display procedure in a virtual reality mode according to various embodiments of the present disclosure.
  • the electronic device when a virtual reality mode is executed in the electronic device, the electronic device is mounted or coupled to the wearable device as shown in FIGS. 5 and 6, or interlocked with the wearable device, the electronic device in operation 402. May be operated by switching from the normal mode to the virtual reality mode.
  • the electronic device after the virtual reality mode is executed in the electronic device (for example, by a user's manipulation), the electronic device may be connected to the wearable device so that the virtual reality mode of the electronic device may be displayed on the wearable device.
  • processing related to the event generation may be performed according to various embodiments of the present disclosure.
  • the generated event is an event related to a screen display in operation 406, in operation 412, the information related to the generated event is processed according to various embodiments of the present disclosure. It can be synthesized and displayed on the screen of the mode.
  • the execution of the application may be blocked in operation 410.
  • the execution of the application may be controlled to be executed in the background so as not to affect the screen according to the operation of the virtual reality mode.
  • the blocked application may be controlled to be executed when the running virtual reality mode is terminated or stopped.
  • At least one of the operations illustrated in FIG. 4 may be omitted, and at least one other operation may be added between the operations.
  • the operations of FIG. 4 may be processed in the order shown, and the execution order of at least one operation may be changed from the execution order of another operation.
  • the operations illustrated in FIG. 4 may be performed in an electronic device or may be performed in a server.
  • at least one of the operations illustrated in FIG. 4 may be performed in the electronic device, and the remaining operations may be implemented in the server.
  • a method of operating an electronic device may include: executing a virtual reality mode in the electronic device; Detecting an interrupt due to at least one event occurrence; Changing event related information related to the generated event into a form corresponding to the virtual reality mode; And displaying the changed event related information on a screen executed corresponding to the virtual reality mode.
  • the event may include a call reception event, a message reception event, an alarm notification, a scheduler notification, a Wi-Fi connection, a Wi-Fi disabling, a low battery notification, a data allowance or usage restriction notification, an application no response notification, or At least one of the application abnormal termination notifications may be selected.
  • the event when the event is not an event to be displayed in the virtual reality mode, storing information related to the event; And when the screen mode of the electronic device is switched from the virtual reality mode to the see-through mode, displaying information related to the stored event on the screen.
  • the method may further include setting at least one event to be displayed in the virtual reality mode.
  • the stored event to be displayed may be an instant message reception notification event.
  • the method may further include controlling to block execution of the application according to the occurrence of the event.
  • the screen mode of the electronic device when the screen mode of the electronic device is switched from the virtual reality mode to the see-through mode, executing the blocked application.
  • controlling the execution of the application according to the occurrence of the event to be executed in the background of the virtual reality mode screen may include.
  • the virtual reality mode may be executed when the electronic device is connected to the wearable device.
  • the processing of the information related to the event may include arranging the information related to the event to be displayed on a three-dimensional space of the virtual reality mode screen currently displayed on the screen. can do.
  • FIG. 5 is a diagram illustrating an example of a wearable device 510 according to various embodiments of the present disclosure.
  • the wearable device 510 may be, for example, the electronic device 102 of FIG. 1.
  • the wearable device 510 may provide only a function of a holder having no communication function with the electronic device 520.
  • the wearable device 510 may include a main body and a cover.
  • the wearable device 510 may include a support that may be used by a user to wear the wearable device 510 on the head.
  • the wearable device 510 may include lenses at positions corresponding to each of both eyes of the wearer.
  • the wearer may view a screen of a display (not shown) of the electronic device 520 through the lenses while the electronic device 520 is mounted on the wearable device 510.
  • the wearable device 510 may have a mechanical structure capable of detachably mounting the electronic device 520 as shown.
  • the wearable device 510 may include at least one of the components of the electronic device 101 of FIG. 1.
  • the wearable device 510 may include at least one of a touch panel, a button, a wheel key, a touch pad, and the like.
  • the touch panel may receive a user's touch input.
  • the touch input may be an input of a user directly touching the touch panel or a hovering input close to the touch panel.
  • the wearable device 510 may be connected through an interface such as a USB to communicate with the electronic device 520.
  • the electronic device 520 may control a function corresponding to the input in response to the input received from the wearable device 510. For example, the electronic device 520 may adjust a volume or control a screen (eg, an image playback screen in a virtual reality mode) in response to the received input. For example, when the electronic device 520 is mounted on the wearable device 510, the external device connection part 521 of the electronic device 520 may be electrically connected to the connection part 511 of the wearable device 510. Can communicate with each other.
  • a volume or control a screen eg, an image playback screen in a virtual reality mode
  • the information related to the generated event is converted into information corresponding to the virtual reality mode. It can be processed and displayed.
  • FIG. 6 is a diagram illustrating an example in which a user wears a wearable device 610 equipped with an electronic device 620 according to various embodiments of the present disclosure.
  • a user may wear a wearable device 610 on which the electronic device 620 is mounted.
  • the wearer may view a screen of the display of the electronic device 620 mounted through the lenses of the wearable device 610.
  • the function of the wearable device 630 may be controlled or the function of the electronic device 620 may be controlled through the touch pad 630 provided on the side of the wearable device 630. have.
  • the event display in the virtual reality mode may be implemented in a smartphone or a mobile phone, or may be implemented in a wearable device (eg, an eyeglass type device).
  • a wearable device eg, an eyeglass type device
  • an electronic device eg, a mobile phone
  • FIGS. 5 and 6 an electronic device (eg, a mobile phone) may be implemented in a form of being docked or coupled to the wearable device.
  • the screen of the mobile phone may be converted into a virtual reality mode, and the screen of the converted virtual reality mode may be displayed on the display unit of the wearable device. Can be displayed.
  • the mobile phone mounted on the wearable device supports the virtual reality mode, but operates according to an operating system of a general mobile phone, so when an event occurs in the virtual reality mode, a heterogeneous notification is displayed or a separate application. This execution can affect the display of the virtual reality mode.
  • a virtual reality mode when a virtual reality mode is provided by an electronic device in which an operating system operating in a normal mode is installed as described above, information related to an event generated while operating in the virtual reality mode is included in the virtual reality mode. It can provide methods that can be effectively processed by processing according to the.
  • FIG. 7 is a diagram illustrating a screen mode of an electronic device according to various embodiments of the present disclosure.
  • the electronic device 700 may be the electronic devices 520 and 620 of FIGS. 5 and 6.
  • the electronic device 700 may display one operation screen 710 as shown in the upper portion of FIG. 7.
  • the electronic device 700 when the electronic device 700 is mounted on the wearable device, the electronic device 700 may operate in a virtual reality mode (eg, an HMT mode) as shown in the lower part of FIG. 7.
  • a virtual reality mode eg, an HMT mode
  • the electronic device 700 may be divided into a screen 720a corresponding to the left eye and a screen 720b corresponding to the right eye of the user.
  • one image On the screen of the virtual reality mode, one image may be divided into two images 720a and 720b and displayed.
  • a screen corresponding to the virtual reality mode (the screen 720a corresponding to the left eye and the right eye corresponding to the left eye) may be used.
  • Information related to the generated event may be processed and displayed on the screen 720b).
  • FIG. 8 is a screen example illustrating an event display method in a normal mode. Referring to FIG. 8, when the electronic device 800 operates in the normal mode, as shown in the upper portion of FIG. 8, an application running in the normal mode may be displayed as one operation screen 810.
  • event related information 820 related to the event May be displayed at a set position in the operation screen 810 of the application.
  • FIG. 9 is an example of a screen illustrating an event display method in a virtual reality mode.
  • an application running in the virtual reality mode may correspond to a screen 920a and a right eye corresponding to the left eye of the user.
  • the screen 920b may be divided and displayed.
  • one image may be divided into two images 920a and 920b and displayed.
  • event related information related to the generated event 930 may be displayed at a set position in the operation screen 910 of the application.
  • the event related information 930 is information displayed in the normal mode, when the information displayed in the normal mode is displayed as it is without processing as shown in the lower part of FIG. 9, in the virtual reality mode currently being executed, The event related information 930 may not be displayed normally.
  • FIG. 10 is a screen example illustrating an event display method in a virtual reality mode according to various embodiments of the present disclosure.
  • an application running in the virtual reality mode may correspond to the screen 1020a and the right eye of the user's left eye.
  • the screen 1020b may be divided and displayed.
  • one image may be divided into two images 1020a and 1020b and displayed.
  • Event related information 1030a and 1030b related to the generated event may be processed and displayed according to the virtual reality mode.
  • a control unit may include a system manager 1110, a virtual reality service processor 1120, an application program processor 1130, and the like.
  • the system manager 1110 corresponds to a system layer and may be an operating system (OS) of the electronic device or at least a part of the operating system.
  • the VR service processor 1120 may perform various processes according to the virtual reality mode according to an embodiment of the present invention.
  • the VR service processor 1120 may be implemented as middleware corresponding to a framework layer.
  • the application program processor 1130 may correspond to an application layer and perform a function of processing various applications (or apps) installed and executed in an electronic device.
  • the VR service processor 1120 of the present invention switches from the normal mode to the virtual reality mode and displays a screen according to the virtual reality mode through the display unit. do.
  • the VR service processor 1120 may further perform a function of monitoring an interrupt occurrence according to an event occurrence of the system manager 1110 according to various embodiments of the present disclosure. For example, when an interrupt occurs in the system manager 1110 due to a specific event occurring in the electronic device, the VR service processor 1120 detects the interrupt occurrence according to various embodiments of the present disclosure and virtualizes information related to the corresponding event. It can be provided on the screen in the real mode.
  • the system manager 1110 when there is an application to be executed in relation to the generated event, the system manager 1110 according to various embodiments of the present disclosure, in the normal mode other than the virtual reality mode, the application is executed by the application processor 1130. You can process it to run.
  • the system manager 1110 may block processing of an application related to the generated event through the application program processor 1130.
  • the blocking function of the execution of the related application when the event occurs in the virtual reality mode may be implemented by modifying the function of the system manager 1110 or by system control of the VR service processor 1120.
  • the immersion of the screen in the virtual reality mode may be prevented from being lowered by blocking the execution of the application related to the event generated in the virtual reality mode.
  • each functional unit and module may mean a functional and structural combination of hardware for performing the technical idea of the embodiment of the present invention and software for driving the hardware.
  • each functional unit may mean a logical unit of a predetermined code and a hardware resource for performing the predetermined code, and does not necessarily mean a physically connected code or a kind of hardware. It can be easily inferred by the average expert in the art of the embodiments of the invention.
  • FIGS. 5 and 6 are views of the electronic device.
  • the electronic device runs the virtual reality mode, mounts or combines the electronic device with the wearable device as shown in FIGS. 5 and 6, or interlocks with the wearable device, in operation 1202, the mode of the electronic device is changed to the normal mode. Switch to the virtual reality mode to operate.
  • the electronic device may be connected to the wearable device so that the virtual reality mode of the electronic device may be displayed on the wearable device.
  • the VR service processor 1120 may perform the processing related to the event generation in accordance with an embodiment of the present invention. have.
  • the generated event in operation 1206 is an event related to a screen display in a currently operating virtual reality mode
  • the event related information related to the generated event is processed according to various embodiments of the present disclosure.
  • the processed event related information may be displayed on the screen of the currently operating virtual reality mode.
  • the system manager eg, the operating system
  • the VR service processor 1120 in operation 1210.
  • the execution of the application can be controlled to be blocked.
  • the event generated in operation 1216 is an event to be displayed in a see-through mode (eg, a messenger notification event)
  • information related to the event generated in operation 1218 is stored.
  • the virtual reality mode currently in operation may be implemented to not display. Subsequently, when the electronic device operates in the see-through mode in the virtual reality mode in operation 1220, event-related information related to the generated event or information related to an accumulated event may be displayed on the screen.
  • the see-through mode refers to a mode in which the user can see the front through the screen of the display unit.
  • the front image captured in real time may be displayed on a screen through a camera separately provided in the electronic device or the wearable device, and the implementation method of the see-through mode of the present invention is not limited to the above method.
  • information related to an event displayed in the virtual reality mode may be continuously displayed even after switching to the see-through mode.
  • the information related to the event displayed in the virtual reality mode may be converted into a different form so as to match the see-through mode when displayed in the see-through mode.
  • all the events that are not displayed in the virtual reality mode may be displayed, or at least some of the events that are not displayed in the virtual reality mode may be displayed. For example, among events displayed in the virtual reality mode, only notification events may be displayed in the see-through mode.
  • the generated event when the generated event is an event that can be directly displayed on the screen of the virtual reality mode currently displayed without any processing, information related to the generated event on the screen of the current virtual reality mode. Can be displayed. However, if the generated event is not an event that can be directly displayed on the screen of the virtual reality mode currently being displayed, the application for executing the virtual reality mode again or displaying information related to the event in the virtual reality mode You can run additional new virtual reality applications.
  • an event to be displayed in the virtual reality mode may be variously set.
  • an event eg, a call reception event, etc.
  • an event that must be executed in the virtual reality mode may be set as an event to be displayed in the virtual reality mode.
  • information related to the generated event is displayed even if the corresponding event occurs in the virtual reality mode.
  • it can be implemented to not notify.
  • the event display method in the virtual reality mode may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium.
  • the computer readable medium may include program instructions, data files, data structures, etc. alone or in combination.
  • Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts.
  • Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks.
  • Magneto-optical media and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.
  • program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.
  • the hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.
  • the notification method in the virtual reality mode for the various events described above may be set as shown in Table 1 below.
  • the embodiment of Table 1 is set as an example to aid in understanding various embodiments of the present invention, and various embodiments of the present invention are not limited to the following examples.
  • the notification method in the virtual reality mode may be determined according to the notification method setting of the user. At least one event-related information to be displayed in the virtual reality mode may be set through a separate setting function on the electronic device.
  • a notification is set for an event (eg, an alarm / scheduler app, a received data call, etc.) accompanying the execution of an application
  • the operation related to the event is blocked while operating in the virtual reality mode, and the virtual reality Notification can be made in the form of a pop-up (VR popup) on the mode screen.
  • an event eg, an alarm / scheduler app, a received data call, etc.
  • the virtual reality Notification can be made in the form of a pop-up (VR popup) on the mode screen.
  • the application associated with the event may be executed in the background so as not to affect the currently executed virtual reality mode screen.
  • the virtual reality mode may be notified that the application related to the background is executed in a pop-up form on the screen.
  • a notification is set for an event (for example, a voice or video call related event) accompanied by execution of an application and a user's selection
  • the user of the user associated with the event on the virtual reality mode screen A popup screen for selecting (for example, calling or rejecting) may be displayed.
  • an event such as Wi-Fi release or connection may be displayed in the form of a toast notification.
  • the toast notification may be included and displayed on the virtual reality mode screen.
  • the toast notification refers to a notification to disappear after a predetermined time elapses after displaying a popup on the screen.
  • a separate application blocking procedure is not required, and information related to the event may be displayed on a virtual reality screen. It can be processed and displayed in the form of a popup.
  • the notification in the form of a sound such as an alarm or a notification sound may be implemented to be allowed or blocked in the virtual reality mode according to the notification setting.
  • the vibration type notification accompanying the application execution or notification may be set to be blocked during the virtual reality mode.
  • FIGS. 13 to 25 are diagrams illustrating a screen displaying information related to an event in a virtual reality mode according to an exemplary embodiment of the present invention.
  • the execution of the call receiving application is blocked according to an exemplary embodiment of the present invention, and as shown in the pop-up form, the virtual reality mode screen is included. I can display it.
  • the pop-up screen may be a screen included in the virtual reality.
  • the pop-up screen is not displayed as a screen superimposed on the same plane as the virtual reality mode screen, but according to an embodiment of the present invention, the pop-up screen and the virtual reality mode screen are rearranged in 3D space as one virtual reality mode screen. Can be displayed.
  • a separate call reception application is blocked from being executed by the system manager or the VR service processor of the electronic device, and the generated event
  • the related information may be processed and included in the currently displayed virtual reality mode screen and displayed as one virtual reality screen.
  • the user can receive or reject the call by selecting the button displayed in the popup.
  • reception is selected in FIG. 13
  • information related to a phone call state may be processed and displayed on a virtual reality mode screen as shown in FIG. 14.
  • corresponding event related information may be implemented in the virtual reality screen according to a call reception or rejection on the event display screen of FIG. 13.
  • the event related information may be again disappeared from the virtual reality screen.
  • 17, 18, 19, 20 and 21 is a variety of notification forms (e.g., pop-up or toast notifications) on the virtual reality screen information (e.g., messages, applications, alarms, Internet connection, etc.) related to the occurrence of the event 22, 23, 24, and 25 show an example of information related to an event occurring in the virtual reality mode after switching to the see-through mode.
  • notification forms e.g., pop-up or toast notifications
  • the virtual reality screen information e.g., messages, applications, alarms, Internet connection, etc.
  • FIGS. 26, 27, and 28 illustrate examples of electronic devices that function as display devices
  • FIG. 28 illustrates examples of wearable devices.
  • the electronic device 2601 may include, for example, all or part of the electronic device 101 shown in FIG. 1.
  • the electronic device 2601 may include one or more application processor (AP) 2610, a communication module 2620, a subscriber identification module (SIM) card 2624, a memory 2630, a sensor module 2640, and an input.
  • AP application processor
  • SIM subscriber identification module
  • Device 2650 display 2660, interface 2670, audio module 2680, camera module 2691, power management module 2695, battery 2696, indicator 2697, and motor 2698. It may include.
  • the AP 2610 may control, for example, a plurality of hardware or software components connected to the AP 2610 by running an operating system or an application program, and may perform various data processing and operations.
  • the AP 2610 may be implemented with, for example, a system on chip (SoC).
  • SoC system on chip
  • the AP 2610 may further include a graphic processing unit (GPU) and / or an image signal processor.
  • the AP 2610 may include at least some of the components shown in FIG. 26 (eg, the cellular module 2621).
  • the AP 2610 may load and process a command or data received from at least one of other components (eg, nonvolatile memory) into volatile memory, and store various data in the nonvolatile memory. Can be.
  • the communication module 2620 may have a configuration that is the same as or similar to that of the communication interface 170 of FIG. 1.
  • the communication module 2620 may be, for example, a cellular module 2621, a WIFI module 2623, a BT module 2625, a GPS module 2627, an NFC module 2628, and a radio frequency (RF) module 2627. ) May be included.
  • RF radio frequency
  • the cellular module 2621 may provide, for example, a voice call, a video call, a text service, or an internet service through a communication network. According to an embodiment of the present disclosure, the cellular module 2621 may perform identification and authentication of the electronic device 2601 in a communication network using a subscriber identification module (eg, the SIM card 2624). According to an embodiment of the present disclosure, the cellular module 2621 may perform at least some of the functions that the AP 2610 may provide. According to an embodiment of the present disclosure, the cellular module 2621 may include a communication processor (CP).
  • CP communication processor
  • Each of the WIFI module 2623, the BT module 2625, the GPS module 2627, or the NFC module 2628 may include, for example, a processor for processing data transmitted and received through a corresponding module. Can be. According to some embodiments, at least some (eg, two or more) of the cellular module 2621, the WIFI module 2623, the BT module 2625, the GPS module 2627, or the NFC module 2628 may be one integrated chip. It may be included in an integrated chip (IC) or an IC package.
  • IC integrated chip
  • the RF module 2629 may transmit / receive, for example, a communication signal (for example, an RF signal).
  • the RF module 2629 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), an antenna, or the like.
  • PAM power amp module
  • LNA low noise amplifier
  • at least one of the cellular module 2621, the WIFI module 2623, the BT module 2625, the GPS module 2627, or the NFC module 2628 may transmit and receive an RF signal through a separate RF module. Can be.
  • the SIM card 2624 may include, for example, a card including a subscriber identification module and / or an embedded SIM, and may include unique identification information (eg, an integrated circuit card identifier (ICCID)) or It may include subscriber information (eg, international mobile subscriber identity).
  • ICCID integrated circuit card identifier
  • the memory 2630 may include, for example, an internal memory 2632 or an external memory 2634.
  • the internal memory 2632 may be, for example, volatile memory (for example, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (for example, nonvolatile memory).
  • OTPROM one time programmable ROM
  • PROM programmable ROM
  • EPROM erasable and programmable ROM
  • EEPROM electrically erasable and programmable ROM
  • mask ROM mask ROM
  • flash ROM flash memory
  • flash memory e.g. NAND flash or NOR flash, etc.
  • a hard drive or a solid state drive (SSD).
  • the external memory 2634 may be a flash drive, for example, a compact flash (CF), a secure digital (SD), a micro secure digital (Micro-SD), a mini secure digital (X-), or an xD ( extreme digital), or a memory stick.
  • the external memory 2634 may be functionally and / or physically connected to the electronic device 2601 through various interfaces.
  • the sensor module 2640 may measure a physical quantity or detect an operation state of the electronic device 2601, and may convert the measured or detected information into an electrical signal.
  • the sensor module 2640 may include, for example, a gesture sensor 2640A, a gyro sensor 2640B, an air pressure sensor 2640C, a magnetic sensor 2640D, an acceleration sensor 2640E, a grip sensor 2640F, and a proximity sensor.
  • (2640G) color sensor (e.g. red, green, blue) sensor (2640H), biometric sensor (2640I), temperature / humidity sensor (2640J), light sensor (2640K), or UV (ultra) violet) at least one of the sensors 2640M.
  • the sensor module 2640 may include, for example, an olfactory sensor, an electromyography sensor, an electroencephalogram sensor, an electrocardiogram sensor, an IR (eg infrared) sensor, iris sensor and / or fingerprint sensor.
  • the sensor module 2640 may further include a control circuit for controlling at least one or more sensors belonging therein.
  • the electronic device 2601 further includes a processor configured to control the sensor module 2640 as part of or separately from the AP 2610, while the AP 2610 is in a sleep state. The sensor module 2640 may be controlled.
  • the input device 2650 may be, for example, a touch panel 2652, a (digital) pen sensor 2654, a key 2656, or an ultrasonic input device. (2658).
  • the touch panel 2652 may use, for example, at least one of capacitive, resistive, infrared, or ultrasonic methods.
  • the touch panel 2652 may further include a control circuit.
  • the touch panel 2652 may further include a tactile layer to provide a tactile response to the user.
  • the (digital) pen sensor 2654 may be, for example, part of a touch panel or may include a separate sheet for recognition.
  • the key 2656 may include, for example, a physical button, an optical key, or a keypad.
  • the ultrasonic input device 2658 may check data by detecting sound waves with a microphone (for example, the microphone 2688) in the electronic device 2601 through an input tool for generating an ultrasonic signal.
  • the display 2660 may include a panel 2662, a hologram device 2664, or a projector 2666.
  • the panel 2662 may include a configuration that is the same as or similar to that of the display 160 of FIG. 1.
  • the panel 2662 may be implemented to be, for example, flexible, transparent, or wearable.
  • the panel 2662 may be configured as one module together with the touch panel 2652.
  • the hologram device 2664 may show a stereoscopic image in the air by using interference of light.
  • the projector 2666 may display an image by projecting light onto a screen.
  • the screen may be located inside or outside the electronic device 2601.
  • the display 2660 may further include a control circuit for controlling the panel 2662, the hologram device 2664, or the projector 2666.
  • the interface 2670 may be, for example, a high-definition multimedia interface (HDMI) 2672, a universal serial bus (USB) 2674, an optical interface 2676, or a D-sub (D-D). subminiature) 2678.
  • the interface 2670 may be included in, for example, the communication interface 170 illustrated in FIG. 1. Additionally or alternatively, the interface 2670 may be, for example, a mobile high-definition link (MHL) interface, a secure digital (SD) card / multi-media card (MMC) interface, or an infrared data association (IrDA). It may include a specification interface.
  • MHL mobile high-definition link
  • SD secure digital
  • MMC multi-media card
  • IrDA infrared data association
  • the audio module 2680 may bidirectionally convert, for example, a sound and an electrical signal. At least some components of the audio module 2680 may be included in, for example, the input / output interface 150 illustrated in FIG. 1.
  • the audio module 2680 may process sound information input or output through, for example, a speaker 2262, a receiver 2684, an earphone 2686, a microphone 2688, or the like.
  • the camera module 2691 is, for example, a device capable of capturing still images and moving images.
  • the camera module 2691 may include one or more image sensors (eg, a front sensor or a rear sensor), a lens, and an image signal processor (ISP). ), Or flash (eg, LED or xenon lamp).
  • image sensors eg, a front sensor or a rear sensor
  • ISP image signal processor
  • flash eg, LED or xenon lamp
  • the power management module 2695 may manage, for example, power of the electronic device 2601.
  • the power management module 2695 may include a power management integrated circuit (PMIC), a charger integrated circuit (ICC), or a battery or fuel gauge.
  • the PMIC may have a wired and / or wireless charging scheme.
  • the wireless charging method may include, for example, a magnetic resonance method, a magnetic induction method, an electromagnetic wave method, or the like, and may further include additional circuits for wireless charging, such as a coil loop, a resonance circuit, a rectifier, and the like. have.
  • the battery gauge may measure, for example, the remaining amount of the battery 2696, a voltage, a current, or a temperature during charging.
  • the battery 2696 may include, for example, a rechargeable battery and / or a solar battery.
  • the indicator 2697 may display a specific state of the electronic device 2601 or a portion thereof (for example, the AP 2610), for example, a booting state, a message state, or a charging state.
  • the motor 2698 may convert an electrical signal into mechanical vibration, and may generate a vibration or haptic effect.
  • the electronic device 2601 may include a processing device (eg, a GPU) for supporting mobile TV.
  • the processing apparatus for supporting the mobile TV may process media data according to a standard such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or media flow.
  • DMB digital multimedia broadcasting
  • DVD digital video broadcasting
  • Each of the above-described components of the electronic device may be composed of one or more components, and the name of the corresponding component may vary according to the type of the electronic device.
  • the electronic device may include at least one of the above-described components, and some components may be omitted or further include other additional components.
  • some of the components of the electronic device according to various embodiments of the present disclosure may be combined to form one entity, and thus may perform the same functions of the corresponding components before being combined.
  • the program module 2710 may include an operating system (OS) that controls resources related to the electronic device (eg, the electronic device 101) and / or It may include various applications (eg, an application program 147) running on an operating system.
  • OS operating system
  • the operating system may be, for example, android, ios, windows, symbian, tizen, or bada.
  • the program module 2710 may include a kernel 2720, middleware 2730, an application programming interface (API) 2760, and / or an application 2770. At least a part of the program module 2710 may be preloaded on the electronic device or may be downloaded from a server (for example, the server 106).
  • a server for example, the server 106
  • the kernel 2720 may include, for example, a system resource manager 2721 or a device driver 2723.
  • the system resource manager 2721 may perform control, allocation, or retrieval of system resources.
  • the system resource manager 2721 may include a process manager, a memory manager, or a file system manager.
  • the device driver 2723 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WIFI driver, an audio driver, or an inter-process communication (IPC) driver. have.
  • IPC inter-process communication
  • the middleware 2730 may provide, for example, a function commonly required by the application 2770, or allow the application 2770 to efficiently use limited system resources inside the electronic device. ) May provide various functions to the application 2770.
  • the middleware 2730 (eg, the middleware 143) may include a runtime library 2735, an application manager 2741, a window manager 2742, and a multimedia manager.
  • the runtime library 2735 may include, for example, a library module that the compiler uses to add new functionality through a programming language while the application 2770 is running.
  • the runtime library 2735 may perform input / output management, memory management, or a function for an arithmetic function.
  • the application manager 2741 may manage, for example, a life cycle of at least one of the applications 2770.
  • the window manager 2742 may manage GUI resources used on the screen.
  • the multimedia manager 2743 may identify a format necessary for playing various media files and may encode or decode the media file using a codec suitable for the format.
  • the resource manager 2744 may manage resources such as source code, memory, or storage space of at least one of the applications 2770.
  • the power manager 2745 may operate together with, for example, a basic input / output system (BIOS) to manage a battery or power, and provide power information necessary for the operation of the electronic device. .
  • the database manager 2746 may create, search for, or change a database to be used by at least one of the applications 2770.
  • the package manager 2747 may manage installation or update of an application distributed in the form of a package file.
  • the connection manager 2758 may manage a wireless connection such as WIFI or Bluetooth.
  • the notification manager 2749 may display or notify an event such as an arrival message, an appointment, a proximity notification, or the like in a manner that does not disturb the user.
  • the location manager 2750 may manage location information of the electronic device.
  • the graphic manager 2751 may manage graphic effects to be provided to the user or a user interface related thereto.
  • the security manager 2752 may provide various security functions required for system security or user authentication. According to an embodiment of the present disclosure, when the electronic device (for example, the electronic device 101) includes a telephone function, the middleware 2730 may include a telephone manager for managing a voice or video call function of the electronic device. It may further include.
  • the middleware 2730 may include a middleware module forming a combination of various functions of the above-described components.
  • the middleware 2730 may provide modules specialized according to types of operating systems in order to provide differentiated functions.
  • the middleware 2730 may dynamically delete some of the existing components or add new components.
  • the API 2760 (eg, the API 145) is, for example, a set of API programming functions, and may be provided in a different configuration according to an operating system. For example, in the case of Android or iOS, one API set may be provided for each platform, and in Tizen, two or more API sets may be provided for each platform.
  • the application 2770 may be, for example, a home 2771, a dialer 2772, an SMS / MMS 2773, an instant message (2774), a browser (2775).
  • health May include one or more applications that may provide functions such as health care (eg, to measure exercise or blood sugar), or provide environmental information (eg, to provide barometric pressure, humidity, or temperature information). .
  • the application 2770 may be an application that supports information exchange between the electronic device (eg, the electronic device 101) and an external electronic device (eg, the electronic devices 102 and 104).
  • the "information exchange application" may be included.
  • the information exchange application may include, for example, a notification relay application for delivering specific information to the external electronic device, or a device management application for managing the external electronic device. .
  • the notification delivery application may include notification information generated by another application of the electronic device (eg, an SMS / MMS application, an email application, a health care application, or an environmental information application). (102, 104).
  • the notification delivery application may receive notification information from an external electronic device and provide the notification information to a user, for example.
  • the device management application may, for example, turn-on at least one function of an external electronic device (eg, the electronic device 104) in communication with the electronic device (eg, the external electronic device itself (or some component part). Adjust the brightness (or resolution) of the on / turn-off or display, manage applications (eg, call services or message services) provided on the external electronic device or applications running on the external electronic device (eg, install, Delete or update).
  • the application 2770 may be configured according to an attribute of the external electronic device (eg, the electronic devices 102 and 104) (eg, an attribute of the electronic device, and the type of the electronic device is a mobile medical device). Applications (eg, healthcare applications).
  • the application 2770 may include an application received from an external electronic device (for example, the server 106 or the electronic devices 102 and 104).
  • the application 2770 may include a preloaded application or a third party application downloadable from a server.
  • the names of the components of the program module 2710 according to the shown embodiment may vary depending on the type of operating system.
  • At least a part of the program module 2710 may be implemented in software, firmware, hardware, or a combination of two or more thereof. At least a part of the program module 2710 may be implemented (for example, executed) by, for example, a processor (for example, the AP 2610). At least a part of the program module 2710 may include, for example, a module, a program, a routine, sets of instructions, or a process for performing one or more functions.
  • FIG. 28 is a block diagram illustrating another configuration example of a wearable device 2800 according to various embodiments of the present disclosure.
  • the wearable device 2800 may provide the wearer with the same or similar operation as the above-described electronic device independently of the display device to be mounted.
  • the wearable device 2800 includes an MCU 2810, a communication module 2820, a sensor module 2830, an input module 2840, a gaze tracking module 2850, a vibrator 2852, a focus control module 2854, and power management.
  • the module 2860 may include a battery 2862, a display 2870, and a camera 2880.
  • the communication module 2820 may include, for example, a USB module 2821, a WiFi module 2822, a BT module 2823, an NFC module 2824, and a GPS module 2825.
  • the sensor module 2830 may be, for example, an acceleration sensor 2831, a gyro sensor 2832, a magnetic field sensor 2833, a magnetic sensor 2834, a proximity sensor 2835, a gesture sensor 2836, a grip sensor ( 2837, biometric sensor 2838, and proximity sensor 2839.
  • the input module 2840 may include a touch pad 2841 and a button 2842.
  • the wearable device 2800 may include a display 2870 that can be fixedly installed in the main body instead of having a structure in which the electronic device (eg, the display device) is detachable.
  • the display 2870 may be installed in the main body to provide a screen to the user through the lens units illustrated in FIGS. 5 and 6 instead of the display of the mounted electronic device.
  • the display 2870 may display a screen according to an augmented reality operation or a see-through operation according to various embodiments of the present disclosure.
  • the camera 2880 may be installed at the front of the main body, for example, to photograph the front of the wearer.
  • the wearable device 2800 may provide the wearer with the same or similar augmented reality motion and the see-through motion as the electronic device, independently of the electronic device to which the wearable device 2800 is mounted.
  • module or “functional unit” may refer to a unit including one or a combination of two or more of hardware, software, or firmware.
  • Module or “functional part” is used interchangeably with terms such as, for example, unit, logic, logical block, component, or circuit.
  • Can be The "module” or “functional part” may be a minimum unit or part of an integrally formed part.
  • the module may be a minimum unit or part of performing one or more functions.
  • the “module” or “functional part” can be implemented mechanically or electronically.
  • a “module” or “function unit” may be an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or programmable logic devices that perform certain operations, known or developed in the future. -logic device).
  • ASIC application-specific integrated circuit
  • FPGAs field-programmable gate arrays
  • programmable logic devices that perform certain operations, known or developed in the future. -logic device).
  • an apparatus eg, modules or functions thereof
  • a method eg, operations
  • computer-readable storage media in the form of a program module. It can be implemented as a command stored in.
  • the command is executed by a processor (eg, the processor 120)
  • the one or more processors may perform a function corresponding to the command.
  • the computer-readable storage medium may be, for example, the memory 130.
  • the computer-readable recording medium may include a hard disk, a floppy disk, a magnetic medium (for example, magnetic tape), an optical media (for example, a compact disc read only memory (CD-ROM), a DVD). (digital versatile disc), magneto-optical media (such as floptical disk), hardware devices (such as read only memory, random access memory (RAM), or flash memory Etc.
  • the program instructions may include not only machine code generated by a compiler, but also high-level language code executable by a computer using an interpreter, etc.
  • the above-described hardware device may include It may be configured to operate as one or more software modules to perform the operations of the various embodiments, and vice versa.
  • Modules or program modules according to various embodiments of the present disclosure may include at least one or more of the above components, some of them may be omitted, or may further include other additional components.
  • Operations performed by a module, program module, or other component according to various embodiments of the present disclosure may be executed in a sequential, parallel, repetitive, or heuristic manner. In addition, some operations may be executed in a different order, may be omitted, or other operations may be added.
  • the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Divers modes de réalisation de la présente invention peuvent comporter : une unité d'affichage permettant d'afficher un écran correspondant à un mode de réalité virtuelle; et une unité de commande permettant de détecter une interruption due à la survenue d'au moins un événement, de modifier les informations relatives à l'événement survenu en une forme correspondant au mode de réalité virtuelle et de commander les informations relatives à l'événement modifiées devant être affichées sur l'écran et activées en correspondance avec le mode de réalité virtuelle. D'autres modes de réalisation se situent également dans la portée de l'invention.
PCT/KR2015/000097 2014-01-06 2015-01-06 Dispositif électronique et procédé d'affichage d'événement en mode de réalité virtuelle Ceased WO2015102464A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US15/109,558 US10431004B2 (en) 2014-01-06 2015-01-06 Electronic device and method for displaying event in virtual reality mode
EP15733319.6A EP3093743B1 (fr) 2014-01-06 2015-01-06 Dispositif électronique et procédé d'affichage d'événement en mode de réalité virtuelle
CN201580003868.XA CN105900051B (zh) 2014-01-06 2015-01-06 电子装置以及在虚拟现实模式下的事件显示方法

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2014-0001552 2014-01-06
KR20140001552 2014-01-06
KR1020150000892A KR102355118B1 (ko) 2014-01-06 2015-01-05 전자 장치 및 가상 현실 모드에서의 이벤트 표시 방법
KR10-2015-0000892 2015-01-05

Publications (1)

Publication Number Publication Date
WO2015102464A1 true WO2015102464A1 (fr) 2015-07-09

Family

ID=53493725

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2015/000097 Ceased WO2015102464A1 (fr) 2014-01-06 2015-01-06 Dispositif électronique et procédé d'affichage d'événement en mode de réalité virtuelle

Country Status (1)

Country Link
WO (1) WO2015102464A1 (fr)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105589562A (zh) * 2016-03-22 2016-05-18 京东方科技集团股份有限公司 一种虚拟现实设备和虚拟现实系统
CN105892667A (zh) * 2016-03-31 2016-08-24 联想(北京)有限公司 一种虚拟现实场景下的信息处理方法和电子设备
CN105929541A (zh) * 2016-06-05 2016-09-07 深圳市中幼国际教育科技有限公司 一种虚拟现实参观育儿教育装置
CN106293346A (zh) * 2016-08-11 2017-01-04 深圳市金立通信设备有限公司 一种虚拟现实视景的切换方法及终端
CN106327580A (zh) * 2016-08-12 2017-01-11 深圳市金立通信设备有限公司 一种基于虚拟现实视景的信息处理方法及终端
WO2017086533A1 (fr) * 2015-11-20 2017-05-26 동국대학교 산학협력단 Procédé et système pour simuler une commande de jeu d'environnement vr à base de nui/nux
CN106997241A (zh) * 2016-01-22 2017-08-01 宏达国际电子股份有限公司 虚拟现实环境中与真实世界互动的方法与虚拟现实系统
CN108064445A (zh) * 2016-12-30 2018-05-22 深圳市柔宇科技有限公司 虚拟现实设备及其来电管理方法
WO2018102122A1 (fr) * 2016-12-02 2018-06-07 Google Llc Représentations de notifications d'événements en réalité virtuelle
CN109598797A (zh) * 2017-10-02 2019-04-09 宏碁股份有限公司 可支持虚拟现实应用程序的混合现实系统及其显示方法
WO2019098609A1 (fr) * 2017-11-14 2019-05-23 Samsung Electronics Co., Ltd. Procédé et appareil de gestion de contenu à vue large dans un environnement de réalité virtuelle
WO2019136450A1 (fr) * 2018-01-08 2019-07-11 Facebook Technologies, Llc Procédés, dispositifs et systèmes d'affichage d'interface utilisateur sur un utilisateur et de détection de gestes tactiles
WO2022010660A1 (fr) * 2020-07-08 2022-01-13 Facebook Technologies, Llc Bloc caméra amovible pour dispositif portable
US11467670B2 (en) 2018-03-23 2022-10-11 Meta Platforms Technologies, Llc Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures
CN115576421A (zh) * 2022-10-13 2023-01-06 立讯精密科技(南京)有限公司 信息交互方法、装置、电子设备及存储介质
CN117784918A (zh) * 2022-09-21 2024-03-29 北京字跳网络技术有限公司 信息交互方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120302289A1 (en) * 2011-05-27 2012-11-29 Kang Heejoon Mobile terminal and method of controlling operation thereof
US20130050432A1 (en) * 2011-08-30 2013-02-28 Kathryn Stone Perez Enhancing an object of interest in a see-through, mixed reality display device
WO2013052855A2 (fr) * 2011-10-07 2013-04-11 Google Inc. Ordinateur vêtement à réponse aux objets proches
US20130117377A1 (en) * 2011-10-28 2013-05-09 Samuel A. Miller System and Method for Augmented and Virtual Reality
US20130163089A1 (en) * 2011-12-23 2013-06-27 David D. Bohn Reflective array waveguide

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120302289A1 (en) * 2011-05-27 2012-11-29 Kang Heejoon Mobile terminal and method of controlling operation thereof
US20130050432A1 (en) * 2011-08-30 2013-02-28 Kathryn Stone Perez Enhancing an object of interest in a see-through, mixed reality display device
WO2013052855A2 (fr) * 2011-10-07 2013-04-11 Google Inc. Ordinateur vêtement à réponse aux objets proches
US20130117377A1 (en) * 2011-10-28 2013-05-09 Samuel A. Miller System and Method for Augmented and Virtual Reality
US20130163089A1 (en) * 2011-12-23 2013-06-27 David D. Bohn Reflective array waveguide

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017086533A1 (fr) * 2015-11-20 2017-05-26 동국대학교 산학협력단 Procédé et système pour simuler une commande de jeu d'environnement vr à base de nui/nux
US10477006B2 (en) 2016-01-22 2019-11-12 Htc Corporation Method, virtual reality system, and computer-readable recording medium for real-world interaction in virtual reality environment
CN106997241A (zh) * 2016-01-22 2017-08-01 宏达国际电子股份有限公司 虚拟现实环境中与真实世界互动的方法与虚拟现实系统
WO2017161726A1 (fr) * 2016-03-22 2017-09-28 京东方科技集团股份有限公司 Dispositif de réalité virtuelle et système de réalité virtuelle
CN105589562A (zh) * 2016-03-22 2016-05-18 京东方科技集团股份有限公司 一种虚拟现实设备和虚拟现实系统
CN105892667B (zh) * 2016-03-31 2019-03-29 联想(北京)有限公司 一种虚拟现实场景下的信息处理方法和电子设备
CN105892667A (zh) * 2016-03-31 2016-08-24 联想(北京)有限公司 一种虚拟现实场景下的信息处理方法和电子设备
CN105929541A (zh) * 2016-06-05 2016-09-07 深圳市中幼国际教育科技有限公司 一种虚拟现实参观育儿教育装置
CN106293346A (zh) * 2016-08-11 2017-01-04 深圳市金立通信设备有限公司 一种虚拟现实视景的切换方法及终端
CN106327580A (zh) * 2016-08-12 2017-01-11 深圳市金立通信设备有限公司 一种基于虚拟现实视景的信息处理方法及终端
WO2018102122A1 (fr) * 2016-12-02 2018-06-07 Google Llc Représentations de notifications d'événements en réalité virtuelle
US10595012B2 (en) 2016-12-02 2020-03-17 Google Llc Representations of event notifications in virtual reality
CN108064445A (zh) * 2016-12-30 2018-05-22 深圳市柔宇科技有限公司 虚拟现实设备及其来电管理方法
CN109598797A (zh) * 2017-10-02 2019-04-09 宏碁股份有限公司 可支持虚拟现实应用程序的混合现实系统及其显示方法
CN109598797B (zh) * 2017-10-02 2023-05-30 宏碁股份有限公司 可支持虚拟现实应用程序的混合现实系统及其显示方法
US10706632B2 (en) 2017-11-14 2020-07-07 Samsung Electronics Co., Ltd. Method and apparatus for managing a wide view content in a virtual reality environment
WO2019098609A1 (fr) * 2017-11-14 2019-05-23 Samsung Electronics Co., Ltd. Procédé et appareil de gestion de contenu à vue large dans un environnement de réalité virtuelle
US10678335B2 (en) 2018-01-08 2020-06-09 Facebook Technologies, Llc Methods, devices, and systems for creating haptic stimulations and tracking motion of a user
US11422628B2 (en) 2018-01-08 2022-08-23 Meta Platforms Technologies, Llc Methods, devices, and systems for modifying perceived haptic stimulations on a user
US10795445B2 (en) 2018-01-08 2020-10-06 Facebook Technologies, Llc Methods, devices, and systems for determining contact on a user of a virtual reality and/or augmented reality device
US10824235B2 (en) 2018-01-08 2020-11-03 Facebook Technologies, Llc Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures
US11042221B2 (en) 2018-01-08 2021-06-22 Facebook Technologies, Llc Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures
WO2019136450A1 (fr) * 2018-01-08 2019-07-11 Facebook Technologies, Llc Procédés, dispositifs et systèmes d'affichage d'interface utilisateur sur un utilisateur et de détection de gestes tactiles
US10684690B2 (en) 2018-01-08 2020-06-16 Facebook Technologies, Llc Methods, devices, and systems for creating localized haptic stimulations on a user
US11467670B2 (en) 2018-03-23 2022-10-11 Meta Platforms Technologies, Llc Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures
US11303816B2 (en) 2020-07-08 2022-04-12 Facebook Technologies, Llc Detachable camera block for a wearable device
CN115803676A (zh) * 2020-07-08 2023-03-14 元平台技术有限公司 用于可穿戴设备的可拆卸摄像头模块
WO2022010660A1 (fr) * 2020-07-08 2022-01-13 Facebook Technologies, Llc Bloc caméra amovible pour dispositif portable
US11765462B1 (en) 2020-07-08 2023-09-19 Meta Platforms Technologies, Llc Detachable camera block for a wearable device
CN117784918A (zh) * 2022-09-21 2024-03-29 北京字跳网络技术有限公司 信息交互方法、装置、电子设备及存储介质
CN115576421A (zh) * 2022-10-13 2023-01-06 立讯精密科技(南京)有限公司 信息交互方法、装置、电子设备及存储介质

Similar Documents

Publication Publication Date Title
WO2015102464A1 (fr) Dispositif électronique et procédé d'affichage d'événement en mode de réalité virtuelle
WO2016080792A1 (fr) Procédé et appareil de contrôle d'énergie permettant de réduire la consommation d'énergie
WO2016137187A1 (fr) Appareil et procédé permettant de fournir un service de duplication d'écran
WO2018182292A1 (fr) Dispositif électronique et procédé de commande associé
WO2018070716A1 (fr) Dispositif électronique ayant une pluralité de modes de détection d'empreinte digitale et son procédé de commande
WO2018105952A1 (fr) Module d'appareil de prise de vues comprenant un diaphragme et dispositif électronique comprenant ce module
WO2017171498A1 (fr) Dispositif électronique comprenant un affichage
WO2018182197A1 (fr) Dispositif pour fournir des informations relatives à un objet dans une image
WO2018128421A1 (fr) Procédé de capture d'image et dispositif électronique
WO2018101774A1 (fr) Dispositif électronique et procédé d'affichage d'image permettant la reconnaissance d'iris dans un dispositif électronique
WO2016085253A1 (fr) Procédé de configuration d'écran, dispositif électronique, et support d'informations
WO2017082652A1 (fr) Dispositif d'antenne et dispositif électronique le comportant
WO2017018722A1 (fr) Procédé de fonctionnement d'écran et dispositif électronique le prenant en charge
WO2019039871A1 (fr) Dispositif électronique et procédé permettant de faire fonctionner des applications
WO2017078480A1 (fr) Dispositif électronique, dispositif pouvant être porté et procédé pour commander un objet affiché par l'intermédiaire d'un dispositif électronique
WO2016036074A1 (fr) Dispositif électronique, son procédé de commande et support d'enregistrement
WO2018143624A1 (fr) Procédé de commande d'affichage, support de mémoire et dispositif électronique
WO2017026709A1 (fr) Procédé et dispositif d'ajustement de la résolution d'un dispositif électronique
WO2018155893A1 (fr) Procédé de fourniture d'interface de fonctionnement multitâche et dispositif électronique mettant en œuvre ledit procédé
WO2016144098A1 (fr) Procédé de traitement d'informations d'utilisateur et dispositif électronique prenant en charge ce procédé
EP3542498A1 (fr) Procédé de génération de message et dispositif électronique vestimentaire pour sa prise en charge
WO2018008924A2 (fr) Procédé de fourniture d'interface utilisateur et dispositif électronique le prenant en charge
WO2018016704A1 (fr) Procédé et appareil destinés au fonctionnement d'un dispositif électronique
WO2017052145A1 (fr) Procédé de partage de contenus et dispositif électronique le prenant en charge
WO2017131469A1 (fr) Dispositif électronique de commande d'application et son procédé de mise en œuvre

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15733319

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15109558

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015733319

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015733319

Country of ref document: EP