WO2020230892A1 - Dispositif de traitement - Google Patents

Dispositif de traitement Download PDF

Info

Publication number
WO2020230892A1
WO2020230892A1 PCT/JP2020/019460 JP2020019460W WO2020230892A1 WO 2020230892 A1 WO2020230892 A1 WO 2020230892A1 JP 2020019460 W JP2020019460 W JP 2020019460W WO 2020230892 A1 WO2020230892 A1 WO 2020230892A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
user
display
location
processing device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/019460
Other languages
English (en)
Japanese (ja)
Inventor
祐樹 松永
智仁 山▲崎▼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NTT Docomo Inc
Original Assignee
NTT Docomo Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NTT Docomo Inc filed Critical NTT Docomo Inc
Priority to JP2021519504A priority Critical patent/JP7316354B2/ja
Publication of WO2020230892A1 publication Critical patent/WO2020230892A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/02—Viewing or reading apparatus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/64—Constructional details of receivers, e.g. cabinets or dust covers

Definitions

  • the present invention relates to a processing device.
  • Patent Document 1 discloses a see-through type wearable display. This wearable display hides the display for the display area corresponding to the wearer's stable gaze field depending on the presence or absence of the wearer's autonomous movement. Further, Patent Document 2 discloses a see-through type information processing device that controls the visibility of an image according to environmental information that a user is placed in a dangerous place or situation.
  • the degree of danger of the user represents the degree of indicating that the action that the user wearing the wearable display is going to perform is dangerous.
  • the safety of the user may be impaired by displaying the image. Conventional technology simply switches between showing and hiding the image depending on the user's behavior or the user's environment, so that the image is hidden even though the user is in a safe place or situation. Or, there is a problem that the image is displayed even though the user is placed in a dangerous place or situation.
  • the processing apparatus includes environmental data including at least one of location data regarding the location where the user exists and status data regarding the situation around the user, and user's operation.
  • environmental data including at least one of location data regarding the location where the user exists and status data regarding the situation around the user, and user's operation.
  • the second generation data indicating the degree of danger related to the user is generated. It includes a generation unit and a display control unit that controls the display of an image on the display device included in the head mount display mounted on the user's head based on the risk level data.
  • the display of the image on the display device is controlled in consideration of both the operation of the user and the environment in which the user is placed, it is possible to achieve both the safety of the user and the provision of information to the user.
  • FIG. 1 is a perspective view showing the appearance of the image processing device 1 according to the first embodiment. In the following description, smart glasses are assumed as the image processing device 1.
  • the image processing device 1 has temples 91 and 92, bridges 93, body portions 94 and 95, and lenses 41L and 41R, like general eyeglasses.
  • An imaging device 70 is provided on the bridge 93.
  • the image pickup apparatus 70 takes an image of the outside world and outputs an image pickup data indicating the captured image.
  • each of the temples 91 and 92 is provided with a sound output device 50 for outputting sound.
  • the sound output device 50 is located near the ear when the user wears the image processing device 1 on the head.
  • Each of the lenses 41L and 41R is equipped with a half mirror.
  • the body 94 is provided with a liquid crystal panel or an organic EL panel for the left eye (hereinafter collectively referred to as a display panel) and an optical member that guides the light emitted from the display panel for the left eye to the lens 41L. ..
  • the half mirror provided on the lens 41L transmits the light of the outside world and guides it to the left eye, and reflects the light guided by the optical member to enter the left eye.
  • the body portion 95 is provided with a display panel for the right eye and an optical member that guides the light emitted from the display panel for the right eye to the lens 41R.
  • the half mirror provided on the lens 41R transmits the light of the outside world and guides it to the right eye, and reflects the light guided by the optical member to enter the right eye.
  • the display device 40 described later includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, and a lens 41R, a display panel for the right eye, and an optical member for the right eye.
  • the image displayed on the display panel appears in the user's field of view together with the surroundings of the user. Further, the image processing device 1 causes the user to display the image for the left eye on the display panel for the left eye and the image for the right eye on the display panel for the right eye among the binocular images with disparity. On the other hand, it is possible to make the displayed image perceive as if it has depth and a three-dimensional effect.
  • the image processing device 1 in this example is an optical see-through type smart glass.
  • FIG. 2 is an explanatory diagram showing an example of the field of view W observed when the user wears the image processing device 1 on the head.
  • the image X displayed on the display panel appears in the field of view W.
  • the image X is translucent.
  • various information is provided to the user. In the example shown in FIG. 2, it can be seen that the user has received three emails and has one appointment for today. On the other hand, the presence of the image X reduces the degree to which the user can recognize the surrounding situation.
  • FIG. 3 is a block diagram showing the overall configuration of the image processing device 1.
  • the image processing device 1 includes a processing device 10, a storage device 20, a communication device 30, a display device 40, a sound output device 50, a motion detection device 60, an image pickup device 70, a line-of-sight detection device 80, and a GPS device 90.
  • Each element of the image processing device 1 is connected to each other by a single bus or a plurality of buses for communicating information.
  • the term "device” in the present specification may be read as another term such as a circuit, a device, or a unit. Further, each element of the image processing device 1 may be composed of a single or a plurality of devices, and some elements of the image processing device 1 may be omitted.
  • the processing device 10 is a processor that controls the entire image processing device 1, and is composed of, for example, a single chip or a plurality of chips.
  • the processing device 10 is composed of, for example, a central processing unit (CPU: Central Processing Unit) including an interface with peripheral devices, an arithmetic unit, registers, and the like. Some or all of the functions of the processing device 10 are realized by hardware such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). You may.
  • the processing device 10 executes various processes in parallel or sequentially.
  • the storage device 20 is a recording medium that can be read by the processing device 10, and is a plurality of programs including the control program P1 executed by the processing device 10, the first table TBL1, the second table TBL2, the third table TBL3, and the processing device 10. Stores various data used by.
  • the storage device 20 may be composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory).
  • the storage device 20 may be referred to as a register, a cache, a main memory (main storage device), or the like.
  • the communication device 30 is hardware (transmission / reception device) for communicating with another device.
  • the communication device 30 is also called, for example, a network device, a network controller, a network card, a communication module, or the like.
  • the communication device 30 communicates with another device via the mobile communication network.
  • the communication device 30 communicates with another device using short-range radio.
  • Examples of short-range wireless communication include Bluetooth (registered trademark), ZigBee, WiFi (registered trademark) and the like.
  • the communication device 30 receives a beacon signal transmitted from another device by short-range radio, and outputs an identifier included in the received beacon signal.
  • the identifier can identify the other device at the source.
  • the identification address MAC (Media Access Control) address
  • the ID information included in the advertisement packet conforming to the BLE (Bluetooth Low Energy) standard corresponds to the identifier.
  • the display device 40 is a device that displays an image.
  • the display device 40 displays various images under the control of the processing device 10.
  • the display device 40 includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, and a lens 41R, a display panel for the right eye, and an optical member for the right eye.
  • the display panel for example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are preferably used.
  • the sound output device 50 includes, for example, a DA converter, an amplifier, and a speaker. Sound data is supplied to the sound output device 50 from the processing device 10.
  • the DA converter converts sound data, which is a digital signal, into a sound signal, which is an analog signal.
  • the amplifier amplifies the sound signal and supplies the amplified sound signal to the speaker.
  • the speaker outputs a sound corresponding to the amplified sound signal. For example, when the processing device 10 decodes the moving image data of the movie acquired via the communication device 30, the sound data of the movie is supplied to the sound output device 50. The user can hear the sound of the movie output from the sound output device 50.
  • the motion detection device 60 detects the motion of the image processing device 1 and outputs motion data.
  • the motion detection device 60 corresponds to an inertial sensor such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration.
  • Acceleration sensors detect orthogonal X-axis, Y-axis, and Z-axis accelerations.
  • the gyro sensor detects angular acceleration with each of the X-axis, Y-axis, and Z-axis as the central axis of rotation. Based on the output information of the gyro sensor, it is possible to generate posture information indicating the posture of the image processing device 1.
  • the motion data includes acceleration data indicating acceleration on each of the three axes and angular acceleration data indicating angular acceleration on each of the three axes.
  • the imaging device 70 outputs imaging data obtained by imaging the outside world.
  • the image pickup device 70 includes, for example, a lens, an image pickup device, an amplifier, and an AD converter.
  • the light collected through the lens is converted into an image pickup signal, which is an analog signal, by the image pickup device.
  • the amplifier amplifies the imaging signal and outputs it to the AD converter.
  • the AD converter converts the amplified imaging signal, which is an analog signal, into imaging data, which is a digital signal.
  • the processing device 10 detects various gestures of the user based on the image pickup data, and controls the image processing device 1 according to the detected gestures. That is, the image pickup device 70 functions as an input device for inputting a user's instruction, such as a pointing device and a touch panel.
  • the line-of-sight detection device 80 detects the user's line of sight and outputs line-of-sight data indicating the direction of the user's line of sight based on the detection result.
  • the GPS device 90 receives radio waves from a plurality of satellites and generates position data from the received radio waves.
  • the position data indicates the position of the image processing device 1.
  • the format of the position data may be any format as long as the position of the image processing device 1 can be specified.
  • the position data indicates, for example, the latitude and longitude of the image processing device 1. In this example, it is illustrated that the position data is obtained from the GPS device 90, but the image processing device 1 may acquire the position data by any method.
  • the processing device 10 functions as an acquisition unit 11, a first generation unit 12, a second generation unit 13, and a display control unit 14 by reading and executing the control program P1 from the storage device 20, for example.
  • the acquisition unit 11 acquires motion data generated by the motion detection device 60, imaging data obtained by imaging the surroundings of the user with the imaging device 70, and position data generated by the GPS device 90.
  • the first generation unit 12 generates environment data and operation data related to user operations.
  • Environmental data includes location data about where the user is and status data about the situation around the user.
  • the first generation unit 12 generates motion data based on motion data.
  • the motion data indicates, for example, sitting, standing, and walking as the user's motion.
  • the first generation unit 12 generates motion data using the acceleration data included in the motion data.
  • the first generation unit 12 generates location data based on the location data.
  • the position data is coordinates indicating the position where the user exists, for example, in latitude and longitude.
  • the location data indicates the location where the user exists, such as home, office, train station, road, and specific store.
  • the first generation unit 12 refers to the first table TBL1 stored in the storage device 20 in order to convert the position data into the location data.
  • the coordinates and the facility name are stored in association with each other. Further, when the user's location cannot be specified by referring to the first table TBL1, the first generation unit 12 converts the location data into the location data by using the map service on the Web.
  • the first generation unit 12 may generate location data based on the identifier included in the beacon signal received by the communication device 30. For example, cafes, hotels, train stations, etc. may provide short-range wireless services.
  • the beacon signal transmitted from the access point includes an identifier.
  • the first generation unit 12 may generate location data by referring to a database in which an identification address on a network assigned to an access point and an actual address (location) are associated with each other.
  • the first generation unit 12 generates status data based on the imaging data, the position data, the motion data, and the identifier.
  • the situation around the user indicated by the situation data includes the user moving by train, the user driving a vehicle, and the user being in a crowd.
  • the first generation unit 12 detects that the speed calculated from the acceleration data is within a predetermined range and the movement route is on the railroad track of the train based on the position data, the user moves by train. It is determined that it is. That is, in order to generate situation data indicating that the user is moving by train, it is a prerequisite to acquire a set of motion data including acceleration data and position data.
  • the first generation unit 12 for example, when the image indicated by the captured data includes the steering wheel of the vehicle and the moving speed of the image processing device 1 is equal to or higher than a predetermined speed, the user drives the vehicle. You may generate status data that indicates that you are doing so. In this case, in order to generate the situation data indicating that the vehicle is being driven, it is a prerequisite to acquire a set of motion data including acceleration data and imaging data. Alternatively, the first generation unit 12 detects an NFC (Near Field Communication) signal transmitted from a transmission device embedded in the headrest of the driver's seat, and the user is seated in the driver's seat based on the detection result. You may detect that.
  • NFC Near Field Communication
  • the first generation unit 12 determines that the user is seated in the driver's seat when the signal indicating that the ignition is ON is acquired from the vehicle, and when the ignition is ON, the user drives the vehicle. You may generate status data to indicate that you are doing so.
  • the processing device 10 acquires a signal from the NFC via the communication device 30. Further, a signal indicating that the ignition is ON is transmitted from the control device of the vehicle. The processing device 10 acquires a signal from the vehicle control device via the communication device 30.
  • the first generation unit 12 determines whether or not the number of identifiers output from the communication device 30 is equal to or greater than a predetermined number, and if the determination result is affirmative, the user's situation is in the crowd.
  • the status data shown may be generated. That is, in order to generate situation data indicating that the user is moving by train, it is a prerequisite to acquire a set of motion data including acceleration data and position data.
  • the second generation unit 13 is based on the operation data, the location data, and the situation data, and based on the combination of at least one of the location where the user is and the situation around the user and the operation of the user, the danger regarding the user. Generate risk data indicating the degree of.
  • the display control unit 14 controls the display of the image on the display device 40 located in front of the user's eyes based on the risk level data.
  • the control of image display has the following aspects.
  • the display control unit 14 controls whether or not to display an image on the display device 40 based on the risk level data.
  • the degree to which the user recognizes the surrounding situation is lower than that when the image is not displayed on the display device 40. Since the display or non-display of the image is controlled based on the risk level data, various information is provided to the user when the user is placed in a safe environment, while the user is placed in a dangerous environment. Does not prevent the user from recognizing the surrounding situation.
  • FIG. 4 is a flowchart showing an example of the operation of the image processing device 1.
  • the processing device 10 generates motion data based on the motion data (step S1). The details of generating operation data will be described later.
  • step S2 determines whether or not the location where the user exists is known. In order to generate location data as described above, acquisition of location data is a prerequisite. In step S2, the processing device 10 determines that the location where the user exists is known when the position data can be acquired.
  • step S3 the processing device 10 determines whether the situation around the user is known. In order to generate the situation data as described above, it is a prerequisite to acquire a set of motion data and position data, or to acquire a set of motion data and imaging data. Knowing the situation around the user means that various data necessary for generating situation data can be acquired. In step S3, the processing device 10 determines whether the situation around the user is known depending on whether the set of these data can be acquired. If the determination result in step S3 is negative, the processing device 10 generates location data (step S4). On the other hand, if the determination result in step S3 is affirmative, the processing device 10 generates location data and status data (step S5).
  • step S6 determines whether or not the situation around the user is known. Similar to step S3, the processing device 10 determines in step S6 whether various data necessary for generating status data can be acquired. If the determination result in step S6 is negative, the location where the user exists and the situation around the user are unknown. If both are unknown, the processing apparatus 10 returns the processing to step S1. On the other hand, if the determination result in step S6 is affirmative, the processing device 10 generates status data (step S7). Details of the process of generating location data and status data will be described later.
  • step S8 determines whether or not the user's location is a predetermined location. Specifically, for example, when a predetermined place is set at home, the processing device 10 determines whether the place indicated by the place data is home. If the determination result in step S8 is affirmative, the processing device 10 generates risk data based on the location data and the operation data (step S9). Specifically, the processing device 10 generates risk data with reference to the second table TBL2 stored in the storage device 20. In step S8, it is determined whether or not the user's location is a predetermined location, when the user's location is a predetermined location, the location where the user exists is determined rather than the situation around the user. This is to give priority to generate risk data.
  • FIG. 5 shows the stored contents of the second table TBL2.
  • the degree of danger is stored in association with the pair of the location where the user exists and the operation of the user.
  • the risk of sitting is set to "1"
  • the risk of standing up is set to "2”
  • the risk of walking is set to "3"
  • the place is a station
  • the risk of sitting is set to "1”
  • the risk of standing up is set to "4"
  • the risk of walking is set to "20”.
  • the user is familiar with the arrangement of furniture at home.
  • the user does not know all the positions of benches or pillars at the station. In addition, many people are walking at the station. Therefore, the risk level of the station is set higher than the risk level of the home.
  • step S8 If the determination result in step S8 is negative, or if the status data generation process in step S7 is completed, the processing device 10 generates risk data based on the status data and the operation data (step S10). .. Specifically, the processing device 10 generates risk data with reference to the third table TBL3 stored in the storage device 20.
  • FIG. 6 shows the stored contents of the third table TBL3.
  • the degree of danger is stored in association with the pair of the situation around the user and the action of the user.
  • the risk of sitting is set to "2”
  • the risk of standing up is set to "3”
  • the risk of walking is set to "10”. Will be done.
  • the risk of sitting is set to "7”
  • the risk of standing up is set to "15”
  • the risk of walking is set to "20”.
  • the risk level when the user moves by train is set lower than the risk level of crowds.
  • step S11 the processing device 10 compares the risk level indicated by the risk level data with the reference value, and when the risk level is equal to or higher than the reference value, hides the image on the display device 40 to hide the display device 40. To control. On the other hand, when the degree of risk is less than the reference value, the processing device 10 controls the display device 40 to the display state by displaying an image on the display device 40.
  • the display state and the non-display state of the display device 40 is shown in FIG. 7 according to the combination of the user's operation and the location where the user exists. However, in FIG. 7, the display state is described as "ON" and the non-display state is described as "OFF".
  • the display device 40 is controlled to the display state regardless of the operation of the user. This is because the safety of the user is ensured at home.
  • the display device 40 is controlled to the display state if the user's operation is seated.
  • the display device 40 is controlled to the non-display state if the user's movement is standing or walking. That is, as compared with home, the company is restricted from displaying an image on the display device 40.
  • the display state and the non-display state of the display device 40 are determined as shown in FIG. 8 according to the combination of the situation around the user and the place where the user exists.
  • the display device 40 when the situation around the user is moving by train, the display device 40 is controlled to the non-display state except when the user's operation is seated.
  • the display device 40 is controlled to be in a non-display state regardless of the user's operation.
  • steps S1 to S7 are realized by the processing device 10 functioning as the first generation unit 12. Further, the processes from steps S8 to S10 are realized by the processing device 10 functioning as the second generation unit 13. Further, the process of step S11 is realized by the processing device 10 functioning as the display control unit 14.
  • FIG. 9 is a flowchart showing an example of the operation data generation process.
  • the processing device 10 specifies the direction of gravitational acceleration based on the acceleration data (step S20). In this process, the processing device 10 specifies the direction in which a constant acceleration is applied to the image processing device 1 over a predetermined time as the direction of gravitational acceleration.
  • the processing device 10 integrates the acceleration data twice to calculate the relative position of the image processing device 1 (step S21).
  • the relative position indicates the relative position at the start time with respect to the position of the image processing apparatus 1.
  • the processing device 10 determines whether the relative position of the image processing device 1 changes in the direction of gravitational acceleration and the amount of change in the relative position is within a predetermined range (step S22).
  • the predetermined range is, for example, 40 cm or more and 80 cm or less.
  • step S24 determines whether the relative position of the image processing device 1 changes in the direction opposite to the gravitational acceleration and the amount of change is within a predetermined range (step). S24).
  • the predetermined range is, for example, 40 cm or more and 80 cm or less.
  • step S26 determines whether the speed of the image processing device 1 in the direction orthogonal to the direction of gravitational acceleration is within a predetermined range.
  • the processing device 10 integrates the acceleration data once to calculate the speed.
  • the predetermined range of speeds in the orthogonal direction is, for example, 1 km / hour or more and 4 km / hour or less. If the determination result in step S26 is affirmative, the processing device 10 generates motion data indicating walking (step S27). On the other hand, if the determination result in step S26 is negative, the processing device 10 returns the processing to step S20.
  • FIG. 10 is a flowchart showing an example of the location data generation process.
  • the processing device 10 determines whether or not the communication device 30 has received the GPS signal (step S30). If the determination result in step S30 is affirmative, the processing device 10 determines whether the location can be specified based on the position data (step S31). If the determination result in step S31 is affirmative, the processing device 10 generates location data based on the position data (step S34).
  • step S30 or S31 determines whether or not the communication device 30 has received the beacon signal in the short-range radio. If the determination result in step S32 is affirmative, the processing device 10 determines whether or not the location can be specified based on the identifier included in the beacon signal (step S33). If the determination result in step S33 is affirmative, the processing device 10 generates location data based on the identifier (step S34).
  • step S32 or S33 If the determination result in step S32 or S33 is negative, the location where the user exists is unknown, so the processing device 10 ends the process without generating location data.
  • FIG. 11 is a flowchart showing an example of the situation data generation process.
  • the processing device 10 determines whether or not the image indicated by the captured data includes a handle (step S40). If the determination result in step S40 is affirmative, the processing device 10 determines whether the speed of the image processing device 1 is equal to or higher than a predetermined value (step S41). The processing device 10 calculates the velocity by integrating the acceleration data included in the motion data.
  • step S41 If the determination result in step S41 is affirmative, the processing device 10 generates status data indicating that the user is driving (step S42).
  • step S40 determines whether the GPS signal has been received by the communication device 30 (step S43). If the determination result in step S43 is affirmative, the processing device 10 determines whether the position of the image processing device 1 indicated by the position data has changed along the line (step S44). If the determination result in step S44 is affirmative, the processing device 10 generates status data indicating that the user is moving by train (step S45).
  • step S46 determines whether or not the communication device 30 has received one or more beacon signals. If the determination result in step S46 is affirmative, the processing device 10 determines whether the number of identifiers included in one or more beacon signals is a predetermined number or more (step S47).
  • Terminal devices such as smartphones often transmit beacon signals at predetermined intervals in order to communicate with other devices using short-range wireless communication. Therefore, when there is another user who has a smartphone around the user, the image processing device 1 receives the beacon signal transmitted from the smartphone of the other user.
  • the number of identifiers can be roughly thought of as the number of other users who have smartphones around them.
  • step S48 the processing device 10 generates status data indicating that the user is in a crowd.
  • steps S41, S46 or S47 are negative, the processing device 10 ends the process without generating status data.
  • the image processing device 1 (processing device 10) generates environment data including location data relating to the location where the user exists, status data relating to the situation around the user, and operation data relating to the user's operation. 1 Based on the generation unit 12, the second generation unit 13 that generates risk data indicating the degree of danger related to the user based on at least one of the location data and the situation data, and the operation data, and the risk data.
  • a display control unit 14 that controls the display of an image on the display device 40 located in front of the user's eyes is provided.
  • a risk level is generated in consideration of both the user's operation and the environment in which the user is placed, and the display of the image on the display device 40 is controlled according to the risk level. Therefore, compared to the case where the risk level of the user is calculated by considering only one of the user's behavior and the environment in which the user is placed, the display is performed in consideration of both the user's behavior and the environment in which the user is placed. Since the display of the image on the device is controlled, it is possible to achieve both the safety of the user and the provision of information to the user. A good balance between user safety and providing information to users.
  • the image processing device 1 the image is hidden even though the user is placed in a safe place or situation, or the user is placed in a dangerous place or situation.
  • the problem of displaying an image is solved, and as a result, both the safety of the user and the provision of information to the user can be achieved.
  • the display control unit 14 controls whether or not to display an image on the display device 40 based on the risk level data.
  • the degree to which the user recognizes the surrounding situation is lower than that when the image is not displayed on the display device 40. Since the display control unit 14 does not display an image on the display device 40 according to the risk level data, when the area of the display area to be displayed on the display device 40 is reduced or the position of the display area is shifted from the center when the risk level is high. Compared with, the safety of the user can be increased.
  • the first generation unit 12 generates location data, status data, and operation data.
  • the second generation unit 13 generates risk data based on the situation data and the operation data when the place where the user exists is unknown and the situation around the user is known. Then, when the situation around the user is unknown and the location of the user is known, the second generation unit 13 generates the risk level data based on the location data and the operation data.
  • the image processing device 1 When either the location where the user exists or the situation around the user is unknown, the image processing device 1 generates risk data based on the data of the other and the operation data. Therefore, it is more flexible than when the location of the user and the user's actions are required to generate the risk data, or when the situation around the user and the user's actions are required. Risk level data can be generated.
  • the first generation unit 12 generates location data, status data, and operation data
  • the second generation unit 13 generates operation data when the location of the user indicated by the location data is a predetermined location.
  • the location data are used to generate the risk data, and if the location of the user indicated by the location data is not a predetermined location, the risk data is generated based on the operation data and the situation data.
  • the image processing device 1 When the location indicated by the location data is a predetermined location, the image processing device 1 gives priority to the location data over the status data and generates the risk level data.
  • the location of the user may include a home that is likely to be safe, or a station that is likely to be dangerous.
  • the image processing device 1 gives priority to the user's place over the situation around the user and generates the risk data. Therefore, the image processing device 1 can appropriately generate the risk level of the user as compared with the case where the location data is not prioritized over the situation data.
  • the image processing device 1 includes an acquisition unit 11 that acquires image acquisition data obtained by imaging the surroundings of the user, and the first generation unit 12 determines whether or not the image indicated by the image data includes the steering wheel of the vehicle. Is determined, and if the determination result is affirmative, status data indicating that the user is driving the vehicle is generated.
  • the image of the captured data includes the steering wheel. Therefore, it is possible to detect that the user is seated in the driver's seat by determining that the image indicated by the captured data includes the steering wheel. Since the image processing device 1 makes it a necessary condition that the user is driving that the determination result is affirmative, it can be appropriately determined that the user is driving.
  • the image processing device 1 includes an acquisition unit 11 that is received by a communication device 30 that communicates by short-range wireless communication and acquires an identifier that identifies the source device.
  • the first generation unit 12 determines whether or not the number of identifiers is equal to or greater than a predetermined number, and if the determination result is affirmative, generates situation data indicating that the user's situation is in the crowd. To do. Since the identifier identifies the source device, the number of identifiers means the number of source devices that exist around the user. Since terminal devices such as smartphones often have a short-range wireless function, the number of identifiers is roughly the number of other people around the user. Therefore, the image processing device 1 can determine that the user's situation is in the crowd based on the number of identifiers.
  • the first generation unit 12 specifies the location where the user exists based on the identifier.
  • facilities such as train stations may provide short-range wireless services.
  • beacon signals are transmitted from access points. Therefore, the image processing device 1 can specify the location of the user based on the identifier included in the beacon signal.
  • the image processing device 1 of the second embodiment compares the degree of risk with one reference value, and determines whether or not to display an image on the display device 40 according to the comparison result. On the other hand, in the image processing device 1 of the second embodiment, whether or not to display an image on the display device 40 according to the time when the risk level becomes equal to or higher than the reference value and the time when the risk level becomes lower than the reference value. Is different from the image processing apparatus 1 of the first embodiment in that the image processing apparatus 1 is controlled. That is, the image processing device 1 of the second embodiment is the same as the image processing device 1 of the first embodiment except for the process of step S11 shown in FIG. The differences will be described below.
  • FIG. 12 is a flowchart showing a process related to control of the display device 40 in the image processing device 1 of the second embodiment.
  • the processing device 10 determines whether or not the display device 40 is in the display state (step S50). If the determination result in step S50 is affirmative, the processing device 10 determines whether the risk level indicated by the risk level data has changed from less than the reference value to more than the reference value (step S51). The processing device 10 repeats the determination in step S51 until the determination result in step S51 becomes affirmative. When the determination result in step S51 becomes affirmative, the processing device 10 starts measuring the time (step S52).
  • step S53 determines whether or not the time when the risk level becomes equal to or higher than the reference value continues for the first hour. That is, if the risk level transitions from the reference value or more to less than the reference value before the measurement time reaches the first time, the determination result in step S53 becomes negative. If the determination result is negative, the processing device 10 returns the processing to step S51.
  • step S53 the processing device 10 shifts the display device 40 from the display state to the non-display state (step S54).
  • the determination result in step S50 is negative.
  • the processing device 10 determines whether the risk level indicated by the risk level data has changed from the reference value or more to less than the reference value (step S55). The processing device 10 repeats the determination in step S55 until the determination result in step S55 becomes affirmative.
  • the processing device 10 starts measuring the time (step S56).
  • the processing device 10 determines whether or not the time when the risk level is less than the reference value continues for the second time (step S57). However, the second hour is shorter than the first hour. That is, if the risk level transitions from less than the reference value to more than the reference value before reaching the second time, which is shorter than the first time, the determination result in step S57 is negative. If the determination result is negative, the processing device 10 returns the processing to step S55. On the other hand, if the determination result in step S57 is affirmative, the processing device 10 shifts the display device 40 from the non-display state to the display state (step S58).
  • the processing device 10 functions as the display control unit 14.
  • the display control unit 14 of the second embodiment causes the display device 40 to hide the image from the display state for displaying the image. To transition to the hidden state.
  • the processing device 10 shifts the display device 40 from the non-display state to the display state when the state in which the risk level indicated by the risk level data is less than the reference value continues for the second time or more, which is longer than the first time.
  • the second time which is the reference for transitioning from the non-display state to the display state
  • the first time which is the reference for transitioning the display device 40 from the display state to the non-display state. That is, the transition from the display state to the non-display state of the display device 40 is easier than the transition from the non-display state to the display state.
  • FIG. 13 is a flowchart showing a process related to control of the display device 40 in the image processing device 1 of the third embodiment.
  • the processing device 10 determines whether or not the display device 40 is in the display state (step S60). If the determination result in step S60 is affirmative, the processing device 10 determines whether the risk level indicated by the risk level data has changed from less than the first reference value to more than or equal to the first reference value (step S61). The processing device 10 repeats the determination in step S61 until the determination result in step S61 becomes affirmative. When the determination result in step S61 becomes affirmative, the processing device 10 shifts the display device 40 from the display state to the non-display state (step S62).
  • step S60 When the display device 40 is in the non-display state, the determination result in step S60 is negative. When the determination result in step S60 is negative or the process in step S62 is completed, the processing device 10 determines whether the risk level indicated by the risk level data has changed from the second reference value or more to less than the second reference value ( Step S63). The processing device 10 repeats the determination in step S63 until the determination result in step S63 becomes affirmative. When the determination result in step S63 becomes affirmative, the processing device 10 shifts the display device 40 from the non-display state to the display state (step S58), and returns the process to step S61.
  • the second reference value is smaller than the first reference value.
  • the processing device 10 functions as the display control unit 14.
  • the display control unit 14 of the third embodiment removes the display device 40 from the display state for displaying the image.
  • the display device 40 is changed from the non-display state to the display state. Transition to.
  • the second reference value which is the reference for transitioning the display device 40 from the non-display state to the display state
  • the first reference value which is the reference for transitioning the display device 40 from the display state to the non-display state. That is, the transition from the display state to the non-display state in the display device 40 is easier than the transition from the non-display state to the display state.
  • the environmental data includes location data and status data, but the environmental data may include only one of the location data and status data.
  • the risk data is generated based on the location data and the operation data, or is generated based on the situation data and the operation data.
  • the risk level data is generated based on the location data and the operation data (step S9), and the risk level data is generated based on the situation data and the operation data (step S10).
  • the second generation unit 13 may generate the risk level data based on the location data, the situation data, and the operation data.
  • the fourth table TBL4 shown in FIG. 14 may be stored in the storage device, and the second generation unit 13 may generate the risk level data with reference to the fourth table TBL4.
  • the risk is 5 when the location data indicates the station, the status data indicates the crowd, and the motion data indicates the seating, the location data indicates the station, the status data indicates the crowd, and the motion.
  • the risk level is 13 when the data indicates standing, the risk level is 20 when the location data indicates a station, the situation data indicates crowding, and the motion data indicates walking.
  • the risk level is 7 when the location data indicates the road, the situation data indicates the crowd, and the operation data indicates the crowd.
  • the risk level when standing up is 15, and the risk level when the location data indicates a road, the situation data indicates crowding, and the motion data indicates walking is 20.
  • the risk data generated based on the location data, the situation data, and the operation data is compared with the location data or the risk data generated based on the situation data and the operation data, and the environment surrounding the user. Is more reflected. Therefore, the accuracy of the risk data is improved.
  • the display control unit 14 may control the size of the image display area, the image transmittance, and the position of the image display area as the image display control. For example, the larger the risk, the smaller the size of the display area may be, or the display area may be arranged at a position away from the center of the screen. Further, the display control unit 14 may control the size and position of the display area according to the degree of danger.
  • the image processing device 1 includes a display device 40, but the present disclosure is not limited thereto.
  • the image processing device 1 may be a terminal device such as a smartphone including a processing device 10, a storage device 20, a communication device 30, a motion detection device 60, and a GPS device 90 among the components shown in FIG. Then, the terminal device may control the display of the image on the display device 40 by communication between the smart glasses including the display device 40, the sound output device 50, the image pickup device 70, and the line-of-sight detection device 80 and the terminal device.
  • the image processing device 1 may hide the display device 40 by controlling the display of the image on the display device 40 based on the risk level data.
  • the display device 40 may be in the display state regardless of the risk level data. For example, an image generated by an application that displays various information generated by the national instantaneous warning system on the display device 40 corresponds to this.
  • the first generation unit 12 determines whether or not the image indicated by the captured data includes a steering wheel for controlling the traveling direction of the vehicle (step S40), and at least the determination result is affirmative. In some cases, situational data indicating that the user is driving a vehicle has been generated, but the disclosure is not limited to this.
  • the determination of whether the image indicated by the captured data includes the steering wheel is executed to detect that the user is seated in the driver's seat. Therefore, in step S40, does the first generation unit 12 include at least one member arranged in front of the driver's seat among the plurality of members arranged inside the vehicle in the image indicated by the captured data? It may be determined whether or not.
  • the first generation unit 12 determines whether or not at least one member arranged in front of the driver's seat is included in the image indicated by the imaging data, and if the determination result is affirmative, the user determines. Situational data indicating that the vehicle is being driven may be generated. Modification 7
  • the image processing apparatus 1 has described smart glasses using optical see-through as an example, but the present disclosure is not limited thereto.
  • the image processing device 1 may be a head-mounted display mounted on the user's head.
  • the head-mounted display has the configuration of the image processing device 1 described in each embodiment. That is, the display device 40 is provided on the head-mounted display mounted on the user's head.
  • Such head-mounted displays include video see-through type goggles and video see-through type smart glasses, in addition to optical see-through type smart glasses.
  • the storage device 20 is a recording medium that can be read by the processing device 10, and examples thereof include a ROM and a RAM, but a flexible disk, a magneto-optical disk (for example, a compact disk, a digital disk, etc.) Versatile disks, Blu-ray® disks, smart cards, flash memory devices (eg cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppies (eg Registered Trademarks) Disks, magnetic strips, databases, servers and other suitable storage media.
  • the program may also be transmitted from the network via a telecommunication line.
  • the program may also be transmitted from the communication network via a telecommunication line.
  • Each of the above-described embodiments includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA ( Registered Trademarks), GSM (Registered Trademarks), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), LTE 802.20, UWB (Ultra-WideBand), Bluetooth It may be applied to (Registered Trademarks), other systems that utilize suitable systems and / or next-generation systems that are extended based on them.
  • the described information, signals, etc. may be represented using any of a variety of different techniques.
  • data, instructions, commands, information, signals, bits, symbols, chips, etc. may be voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.
  • the input / output information and the like may be stored in a specific place (for example, a memory), or may be managed using a management table.
  • Input / output information and the like can be overwritten, updated, or added.
  • the output information and the like may be deleted.
  • the input information or the like may be transmitted to another device.
  • the determination may be made by a value represented by 1 bit (0 or 1) or by a boolean value (Boolean: true or false). , May be done by numerical comparison (eg, comparison with a given value).
  • each function illustrated in FIG. 3 is realized by any combination of at least one of hardware and software. Further, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically connected, or directly or indirectly (for example, by two or more devices that are physically or logically separated). , Wired, wireless, etc.) and may be realized using these plurality of devices. The functional block may be realized by combining the software with the one device or the plurality of devices.
  • the software is an instruction, an instruction set, a code, regardless of whether the software is called software, firmware, middleware, microcode, hardware description language, or another name. It should be broadly interpreted to mean code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. ..
  • software, instructions, information, etc. may be transmitted and received via a transmission medium.
  • a transmission medium For example, a website that uses at least one of wired technology (coaxial cable, fiber optic cable, twist pair, digital subscriber line (DSL: Digital Subscriber Line), etc.) and wireless technology (infrared, microwave, etc.) When transmitted from a server, or other remote source, at least one of these wired and wireless technologies is included within the definition of transmission medium.
  • connection In the embodiments described above, the terms “connected”, “coupled”, or any variation thereof, are direct or indirect between two or more elements. It means any connection or connection and can include the presence of one or more intermediate elements between two elements that are “connected” or “connected” to each other.
  • the connection or connection between the elements may be physical, logical, or a combination thereof.
  • connection may be read as "access”.
  • the two elements use at least one of one or more wires, cables and printed electrical connections, and, as some non-limiting and non-comprehensive examples, the radio frequency domain. Can be considered to be “connected” or “coupled” to each other using electromagnetic energies having wavelengths in the microwave and light (both visible and invisible) regions.
  • determining and “determining” used in the present disclosure may include a wide variety of actions.
  • “Judgment” and “decision” are, for example, judgment (judging), calculation (calculating), calculation (computing), processing (processing), derivation (deriving), investigation (investigating), search (looking up, search, inquiry). It may include (eg, searching in a table, database or another data structure), ascertaining as “judgment” or “decision”.
  • judgment and “decision” are receiving (for example, receiving information), transmitting (for example, transmitting information), input (input), output (output), and access. (Accessing) (for example, accessing data in memory) may be regarded as “judgment” or “decision”.
  • judgment and “decision” mean that “resolving”, “selecting”, “choosing”, “establishing”, “comparing”, etc. are regarded as “judgment” and “decision”. Can include. That is, “judgment” and “decision” may include that some action is regarded as “judgment” and “decision”. Further, “judgment (decision)” may be read as “assuming”, “expecting”, “considering” and the like.
  • Each aspect / embodiment described in the present disclosure may be used alone, in combination, or may be switched according to the execution.
  • the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.
  • Image processing device 10 ... Processing device, 11 ... Acquisition unit, 12 ... First generation unit, 13 ... Second generation unit, 14 ... Display control unit, 20 ... Storage device, 30 ... Communication device, 40 ... Display device ..

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Optics & Photonics (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

L'invention concerne un dispositif de traitement (10) comprenant : une première unité de génération (12) destinée à générer des données de mouvement relatives au mouvement d'un utilisateur et des données environnementales comprenant des données d'emplacement relatives à l'emplacement d'un utilisateur et/ou des données de situation relatives à la situation concernant l'environnement de l'utilisateur ; une deuxième unité de génération (13) destinée à générer des données de risque indiquant un degré de risque relatif à l'utilisateur, en fonction des données de mouvement et des données d'emplacement et/ou des données de situation ; et une unité de commande d'affichage (14) destinée, en fonction des données de risque, à effectuer une commande sur une image affichée sur un dispositif d'affichage (40) contenu dans un visiocasque destiné à être porté sur la tête de l'utilisateur.
PCT/JP2020/019460 2019-05-15 2020-05-15 Dispositif de traitement Ceased WO2020230892A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021519504A JP7316354B2 (ja) 2019-05-15 2020-05-15 処理装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019091898 2019-05-15
JP2019-091898 2019-05-15

Publications (1)

Publication Number Publication Date
WO2020230892A1 true WO2020230892A1 (fr) 2020-11-19

Family

ID=73288890

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/019460 Ceased WO2020230892A1 (fr) 2019-05-15 2020-05-15 Dispositif de traitement

Country Status (2)

Country Link
JP (1) JP7316354B2 (fr)
WO (1) WO2020230892A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022157034A (ja) * 2021-03-31 2022-10-14 本田技研工業株式会社 情報提供方法、プログラム及び情報提供装置
JP2025523078A (ja) * 2022-07-15 2025-07-17 グーグル エルエルシー 頭部搭載型ウェアラブルデバイスにおけるコンテンツ出力管理

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005252735A (ja) * 2004-03-04 2005-09-15 Olympus Corp 危険防止機能付き頭部装着型表示装置
JP2009206679A (ja) * 2008-02-27 2009-09-10 Mitsubishi Electric Corp サービス実行装置及びサービス実行プログラム及び記録媒体
JP2012155655A (ja) * 2011-01-28 2012-08-16 Sony Corp 情報処理装置、報知方法及びプログラム
US20140043213A1 (en) * 2012-08-07 2014-02-13 Industry-University Cooperation Foundation Hanyang University Wearable display device having a detection function
WO2017104227A1 (fr) * 2015-12-18 2017-06-22 ソニー株式会社 Appareil et procédé de traitement d'informations, et programme associé

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014038323A1 (fr) * 2012-09-06 2014-03-13 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et programme
WO2018109925A1 (fr) * 2016-12-16 2018-06-21 三菱電機株式会社 Dispositif de prévention de collision de portière de véhicule et procédé de prévention de collision de portière de véhicule
JP2018143413A (ja) * 2017-03-03 2018-09-20 セイコーエプソン株式会社 モニタリング装置、モニタリングシステム、モニタリング方法、モニタリングプログラム、記憶媒体、モニタリングサーバー及び電子機器

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005252735A (ja) * 2004-03-04 2005-09-15 Olympus Corp 危険防止機能付き頭部装着型表示装置
JP2009206679A (ja) * 2008-02-27 2009-09-10 Mitsubishi Electric Corp サービス実行装置及びサービス実行プログラム及び記録媒体
JP2012155655A (ja) * 2011-01-28 2012-08-16 Sony Corp 情報処理装置、報知方法及びプログラム
US20140043213A1 (en) * 2012-08-07 2014-02-13 Industry-University Cooperation Foundation Hanyang University Wearable display device having a detection function
WO2017104227A1 (fr) * 2015-12-18 2017-06-22 ソニー株式会社 Appareil et procédé de traitement d'informations, et programme associé

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022157034A (ja) * 2021-03-31 2022-10-14 本田技研工業株式会社 情報提供方法、プログラム及び情報提供装置
JP2025523078A (ja) * 2022-07-15 2025-07-17 グーグル エルエルシー 頭部搭載型ウェアラブルデバイスにおけるコンテンツ出力管理

Also Published As

Publication number Publication date
JP7316354B2 (ja) 2023-07-27
JPWO2020230892A1 (fr) 2020-11-19

Similar Documents

Publication Publication Date Title
CN104919398B (zh) 可佩戴的基于行为的视觉系统
US12198399B2 (en) Display system and display method
US8498816B2 (en) Systems including mobile devices and head-mountable displays that selectively display content, such mobile devices, and computer-readable storage media for controlling such mobile devices
US9952665B2 (en) Eye vergence detection on a display
EP2933707B1 (fr) Réglage de présentation d'affichage pour visiocasque
US20140098008A1 (en) Method and apparatus for vehicle enabled visual augmentation
KR102222250B1 (ko) 참조 지점을 이용한 경로 안내 제공 방법 및 그 장치
KR20160145976A (ko) 영상 공유 방법 및 이를 수행하는 전자 장치
KR101646356B1 (ko) 웨어러블 기기를 이용한 차량 제어장치 및 방법
US20160221502A1 (en) Cognitive displays
KR101612824B1 (ko) Hmd를 이용한 운전자 상태 모니터링 방법 및 장치
CN106338828A (zh) 车载增强现实系统、方法和设备
US11137600B2 (en) Display device, display control method, and display system
US20160070101A1 (en) Head mounted display device, control method for head mounted display device, information system, and computer program
WO2016199248A1 (fr) Dispositif de présentation d'informations et procédé de présentation d'informations
JP7316354B2 (ja) 処理装置
US20220198794A1 (en) Related information output device
WO2023112838A1 (fr) Dispositif de traitement d'informations
JP7730919B2 (ja) 情報処理装置
WO2023149256A1 (fr) Dispositif de commande d'affichage
JP7835854B2 (ja) 表示制御装置
JP7698142B2 (ja) 特定システム
US12535878B2 (en) Wearable apparatus
JP2016110296A (ja) ウェアラブルデバイス、情報処理方法及びプログラム
JP7267105B2 (ja) 情報処理装置及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20806711

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021519504

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20806711

Country of ref document: EP

Kind code of ref document: A1