WO2016181731A1 - Dispositif de détection de chute, procédé de détection de chute, et dispositif de surveillance d'une personne à surveiller - Google Patents

Dispositif de détection de chute, procédé de détection de chute, et dispositif de surveillance d'une personne à surveiller Download PDF

Info

Publication number
WO2016181731A1
WO2016181731A1 PCT/JP2016/061355 JP2016061355W WO2016181731A1 WO 2016181731 A1 WO2016181731 A1 WO 2016181731A1 JP 2016061355 W JP2016061355 W JP 2016061355W WO 2016181731 A1 WO2016181731 A1 WO 2016181731A1
Authority
WO
WIPO (PCT)
Prior art keywords
head
monitored person
unit
fall
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2016/061355
Other languages
English (en)
Japanese (ja)
Inventor
山下　雅宣
安紀 辻
篤広 野田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Konica Minolta Inc
Original Assignee
Konica Minolta Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Konica Minolta Inc filed Critical Konica Minolta Inc
Priority to JP2016566827A priority Critical patent/JP6115689B1/ja
Publication of WO2016181731A1 publication Critical patent/WO2016181731A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to a fall detection device and a fall detection method for detecting a fall of a monitored person to be monitored. And this invention relates to the to-be-monitored person monitoring apparatus using this fall detection apparatus.
  • Japan is an aging society, more specifically the ratio of population over 65 years old to the total population due to the improvement of living standards accompanying the post-war high economic growth, improvement of sanitary environment and improvement of medical standards, etc. It is a super-aging society with an aging rate exceeding 21%.
  • the total population was about 126.5 million, while the elderly population over the age of 65 was about 25.56 million.
  • the total population was about 124.11 million.
  • the elderly population will be about 34.56 million.
  • nurses who need nursing or nursing care due to illness, injury, elderly age, etc., or those who need nursing care are those who need nursing in a normal society that is not an aging society.
  • the fall detection device disclosed in Patent Document 1 includes a camera, an image processing unit that captures a video signal output from the camera, performs difference processing on the captured image, and a difference area obtained by the difference processing. And a judging means for detecting a falling motion of the human body based on the area. More specifically, the determination means determines whether or not a fall has occurred based on an area change per unit time in the human body.
  • the fall detection device disclosed in Patent Document 1 detects the fall by the area change per unit time in the human body region. Since the size of the region of the head that appears in the image changes depending on whether or not there is a fall, a method of detecting a fall based on the size of the head region can be considered.
  • the present invention has been made in view of the above circumstances, and its object is to provide a fall detection device and a fall detection method that can detect the fall of the monitored person more accurately based on an image obtained by imaging the monitored person from above.
  • the present invention also provides a monitored person monitoring device using the fall detection device.
  • the fall detection device, the fall detection method, and the monitored person monitoring apparatus In the fall detection device, the fall detection method, and the monitored person monitoring apparatus according to the present invention, an image obtained by capturing the monitored person as the monitoring target from above the monitored person is acquired, and the head candidate is obtained from the image. The head candidate area is extracted, and each time change in each of a plurality of preset parameters related to the head is obtained for the head candidate area, and based on each time change in each of the plurality of parameters A fall in the monitored person is detected. Therefore, the fall detection device, the fall detection method, and the monitored person monitoring apparatus according to the present invention can detect the fall of the monitored person with higher accuracy based on an image obtained by imaging the monitored person from above.
  • the monitored person monitoring apparatus includes a behavior detection unit that detects a predetermined behavior set in advance in a monitored person to be monitored, and a notification unit that notifies the predetermined behavior detected by the behavior detection unit to the outside.
  • a monitored person monitoring apparatus may be realized by being integrally configured as one device, and may be realized by a plurality of devices as a system.
  • the behavior detection unit may be mounted on any of the plurality of devices.
  • the first and second embodiments of the monitored person monitoring apparatus will be described in the case where the monitored person monitoring apparatus is realized by a plurality of devices as a system.
  • the monitored person monitoring apparatus includes an image acquisition unit that acquires an image of a monitored person that is a monitoring target from above the monitored person, and an image acquired by the image acquisition unit.
  • a head candidate extraction unit that extracts a head candidate region that is a candidate for each part, and each time in each of a plurality of preset parameters related to the head with respect to the head candidate region extracted by the head candidate extraction unit
  • a parameter calculation unit for obtaining a change, and a fall detection unit for detecting a fall in the monitored person based on each time change in each of the plurality of parameters obtained by the parameter calculation unit, and at least one of the parameters Is extracted from the image acquired by the image acquisition unit.
  • the monitored person monitoring apparatus further includes a distance measuring unit that measures the distance to the head of the monitored person from above the monitored person, and the parameter calculation unit includes the image acquisition unit.
  • a size calculation unit that obtains, as the parameter, the size of the head of the monitored person as viewed from above, the fall detection unit is configured to include the head calculated by the size calculation unit. The fall of the monitored person is detected based on the time change rate of the difference between the size of the part and the distance to the head measured by the distance measuring unit. Note that, even when the monitored person monitoring apparatus is configured integrally as a single device, the monitored person monitoring apparatus can be configured similarly to the following description.
  • the behavior detection unit is mounted on sensor devices SUa and SUb described later together with the notification unit
  • other devices in this system for example, a management server device described later
  • the monitored person monitoring device can be configured in the same manner as described below.
  • the head In the above-described method of detecting a fall by the size of the head region, the head is captured in a substantially circular shape in the image captured from above, and thus a circular region is detected as the head region from the image.
  • a substantially circular object that is not the head such as a pillow, a washbasin, a ball, a trash can, and a futon handle is also reflected, so the area of these objects is erroneously detected as the head area, A fall may be erroneously detected.
  • the erroneous detection can be avoided by removing the area of these objects at the fixed position from the area for detecting the head in advance.
  • these objects do not always exist at a fixed position.
  • the first embodiment addresses such a case.
  • FIG. 1 is a diagram illustrating a configuration of a monitored person monitoring system according to the embodiment.
  • FIG. 2 is a diagram illustrating a configuration of a sensor device in the monitored person monitoring system according to the first embodiment.
  • the monitored person monitoring system MSa in the first embodiment of the example in which the monitored person monitoring apparatus is realized as a system is a monitored person (a watching target person) that is a monitoring target (a watching target) to be monitored (a watching target).
  • a predetermined action set in Ob (Ob-1 to Ob-4) is detected to monitor the monitored person Ob. For example, as shown in FIG.
  • a network such as a LAN (Local Area Network), a telephone network, and a data communication network.
  • NW network, communication line
  • the network NW may be provided with relays such as repeaters, bridges, routers, and cross-connects that relay communication signals.
  • the plurality of sensor devices SUa-1 to SUa-4, the management server device SV, the fixed terminal device SP, and the plurality of portable terminal devices TA-1 and TA-2 are wireless including an access point AP.
  • a LAN for example, a LAN according to the IEEE 802.11 standard
  • NW is connected to be communicable with each other.
  • the monitored person monitoring system MSa is arranged at an appropriate place according to the monitored person Ob.
  • the monitored person (person to be watched) Ob is, for example, a person who needs nursing due to illness or injury, a person who needs care due to a decrease in physical ability, a single person living alone, or the like.
  • the monitored person Ob may be a person who needs the detection when a predetermined inconvenient event such as an abnormal state occurs in the person. preferable.
  • the monitored person monitoring system MSa is suitably arranged in a building such as a hospital, an elderly welfare facility, or a dwelling unit according to the type of the monitored person Ob.
  • the monitored person monitoring system MSa is disposed in a building of a care facility that includes a plurality of rooms RM in which a plurality of monitored persons Ob and a plurality of rooms such as a nurse station.
  • the sensor device SUa is a device that has a communication function for communicating with other devices SV, SP, and TA via the network NW, detects the monitored person Ob, and transmits the detection result to the management server device SV.
  • the sensor device SUa includes a sensor unit 1, a sound input / output unit 2, a control processing unit 3a, a communication interface unit (communication IF unit) 4, and a storage unit 5a. Is provided.
  • the sensor unit 1 is a device that is connected to the control processing unit 3a and detects predetermined various amounts set in advance in the monitored person Ob in accordance with the control of the control processing unit 3a.
  • the predetermined amounts are appropriately determined according to a predetermined action to be detected in the monitored person Ob.
  • the predetermined amounts include the distance to the head of the monitored person Ob.
  • the predetermined actions are awakening, getting out of bed, falling, and abnormal movement of the subject Ob, and in order to detect these, the sensor unit 1 includes, for example, a Doppler sensor 11, And a camera 12.
  • the Doppler sensor 11 is a body motion sensor that transmits a transmission wave, receives a reflection wave of the transmission wave reflected by an object, and outputs a Doppler frequency component Doppler signal based on the transmission wave and the reflection wave. .
  • the frequency of the reflected wave shifts in proportion to the moving speed of the object due to the so-called Doppler effect, so there is a difference (Doppler frequency component) between the frequency of the transmitted wave and the frequency of the reflected wave.
  • Doppler sensor 11 generates a signal of the Doppler frequency component as a Doppler signal and outputs it to the control processing unit 3a.
  • the transmission wave may be an ultrasonic wave, a microwave, or the like, but is a microwave in the present embodiment. Since the microwave can be transmitted through the clothing and reflected from the body surface of the monitored person Ob, the movement of the body surface can be detected even when the monitored person Ob is wearing clothes.
  • the camera 12 is an apparatus that is connected to the control processing unit 3a and generates an image (image data) under the control of the control processing unit 3a.
  • the camera 12 is arranged so as to be able to monitor a space (location space, in the example shown in FIG. 1, where the monitored person Ob that is a monitoring target to be monitored) is located. Is captured from above, an image (image data) overlooking the imaging target is generated, and the image of the imaging target is output to the control processing unit 3a.
  • the camera 12 is expected to be located at the head of the monitored person Ob in the bedding on which the monitored person Ob is lying (for example, a bed).
  • the sensor device SUa uses the camera 12 of the sensor unit 1 to acquire an image of the monitored person Ob taken from above the monitored person Ob, preferably an image taken from directly above the planned head position.
  • the camera 12 is an example of an image acquisition unit.
  • Such a camera 12 may be a device that generates an image of visible light, but in this embodiment, it is a device that generates an infrared image so that the monitored person Ob can be monitored even in a relatively dark place.
  • a camera 12 is, for example, in this embodiment, an imaging optical system that forms an infrared optical image of an imaging target on a predetermined imaging surface, and a light receiving surface that is aligned with the imaging surface.
  • An image sensor that converts an infrared optical image in the imaging target into an electrical signal, and image data that is data representing an infrared image in the imaging target by performing image processing on the output of the image sensor.
  • the imaging optical system of the camera 12 is preferably a wide-angle optical system (so-called wide-angle lens (including a fisheye lens)) having an angle of view that can capture the entire room RM in which the camera 12 is disposed.
  • the sound input / output unit 2 is connected to the control processing unit 3a and is a circuit for acquiring an external sound and inputting it to the sensor device SUa.
  • the sound input / output unit 2 corresponds to an electrical signal representing the sound according to the control of the control processing unit 3a. It is a circuit for generating and outputting sound.
  • the sound input / output unit 2 includes, for example, a microphone that converts sound acoustic vibrations into electrical signals, and a speaker that converts sound electrical signals into sound acoustic vibrations.
  • the sound input / output unit 2 outputs an electric signal representing an external sound to the control processing unit 3a, and converts the electric signal input from the control processing unit 3a into an acoustic vibration of sound and outputs the sound.
  • the communication IF unit 4 is a communication circuit that is connected to the control processing unit 3a and performs communication according to the control of the control processing unit 3a.
  • the communication IF unit 4 generates a communication signal containing the data to be transferred input from the control processing unit 3a according to the communication protocol used in the network NW of the monitored person monitoring system MSa, and generates the generated communication signal. It transmits to other devices SV, SP, TA via the network NW.
  • the communication IF unit 4 receives a communication signal from another device SV, SP, TA via the network NW, extracts data from the received communication signal, and can be processed by the control processing unit 3a. And output to the control processing unit 3a.
  • the communication IF unit 4 further uses, for example, standards such as the Bluetooth (registered trademark) standard, the IrDA (Infrared Data Association) standard, and the USB (Universal Serial Bus) standard to input / output data to / from an external device.
  • standards such as the Bluetooth (registered trademark) standard, the IrDA (Infrared Data Association) standard, and the USB (Universal Serial Bus) standard to input / output data to / from an external device.
  • An interface circuit may be provided.
  • the storage unit 5a is a circuit that is connected to the control processing unit 3a and stores various predetermined programs and various predetermined data under the control of the control processing unit 3a.
  • the various predetermined programs include, for example, control processing programs such as a monitoring processing program for executing information processing related to monitoring of the monitored person Ob.
  • the monitoring processing program includes a behavior detection program for detecting a predetermined behavior in the monitored person Ob based on an output of the sensor unit 1, a notification processing program for notifying the predetermined behavior detected by the behavior detection program to the outside, By using the streaming processing program for streaming the moving image captured by the camera 12 to the fixed terminal device SP or the portable terminal device TA that requested the moving image, and the sound input / output unit 2 or the like, the fixed terminal device SP or the portable terminal A nurse call processing program for making a voice call with the device TA is included.
  • the behavior detection program includes a head candidate extraction program for extracting a head candidate region that is a head candidate from an image acquired by the camera 12, and a head candidate region extracted by the head candidate extraction program.
  • the parameter calculation program for obtaining each time change in each of a plurality of preset parameters related to the head, and the person to be monitored Ob based on each time change in each of the plurality of parameters obtained by the parameter calculation program A fall detection program for detecting fall is included.
  • the various types of predetermined data include data necessary for executing these programs, data necessary for monitoring the monitored person Ob, and the like.
  • the storage unit 5a includes, for example, a ROM (Read Only Memory) that is a nonvolatile storage element, an EEPROM (Electrically Erasable Programmable Read Only Memory) that is a rewritable nonvolatile storage element, and the like.
  • the storage unit 5a includes a RAM (Random Access Memory) that serves as a working memory of the so-called control processing unit 3a that stores data generated during execution of the predetermined program.
  • the control processing unit 3a is a circuit for controlling each unit of the sensor device SUa according to the function of each unit and detecting predetermined amounts set in advance in the monitored person Ob.
  • the control processing unit 3a includes, for example, a CPU (Central Processing Unit) and its peripheral circuits.
  • the control processing unit 3a functionally includes a control unit 31, a behavior detection processing unit 32, a notification processing unit 33, a streaming processing unit 34, and a nurse call processing unit 35 by executing the control processing program.
  • the processing unit 32 functionally includes a moving object extraction unit 321, a head candidate extraction unit 322a, a parameter calculation unit 323a, and a fall detection unit 324a.
  • the control unit 31 controls each part of the sensor device SUa according to the function of each part, and controls the entire sensor device SUa.
  • the behavior detection processing unit 32 detects a predetermined behavior set in advance in the monitored person Ob based on the output of the sensor unit 1.
  • the predetermined behavior is wakeup, getting out of bed, falling, and abnormal movement of the monitored person Ob
  • the behavior detection processing unit 32 is based on the output of the sensor unit 1.
  • Wake-up, getting-off, falling, and minute body movement abnormality in the monitoring person Ob are detected as detection results, and the detection results are notified to the notification processing unit 33.
  • the behavior detection processing unit 32 detects wake-up, getting-off and abnormal body movement of the monitored person Ob based on the output of the sensor unit 1 by a known technique.
  • the behavior detection processing unit 32 extracts a moving body region from the image acquired by the camera 12 of the sensor unit 1 as the human body region of the monitored person Ob, and the posture of the monitored person Ob from the aspect ratio of the extracted moving body area ( For example, standing position, sitting position, lying down, etc.) are detected, the position of the detected moving body region is detected, and the distinction between the rising and leaving is determined based on these determinations and the detected posture and position of the monitored person Ob.
  • the behavior detection processing unit 32 detects body movement of the chest (vertical movement of the chest) accompanying the breathing motion of the monitored person Ob by the Doppler sensor 11 of the sensor unit 1, and the period disturbance in the body movement of the chest When the amplitude in the body motion of the chest that is equal to or less than a preset threshold is detected, it is determined that the minute motion abnormality is present.
  • the behavior detection processing unit 32 functionally includes a moving object extraction unit 321, a head candidate extraction unit 322a, a parameter calculation unit 323a, and a fall detection unit 324a.
  • the fall of the monitored person Ob is detected.
  • the moving object extraction unit 321 extracts a moving object region from the image acquired by the camera 12 of the sensor unit 1 as a human body region of the monitored person Ob by using a method such as a background difference method or an inter-frame difference method.
  • the moving object extraction unit 321 is also used when extracting a moving object in order to detect the above-described wake-up and getting-off, and is shared by these processes.
  • a background image is obtained in advance and stored in advance in the storage unit 5a as one of the various predetermined data, and the presence or absence of moving objects is determined from the difference image between the image generated by the camera 12 and the background image. If there is a moving object as a result of this determination, the area of the moving object is set as a moving object area.
  • the presence / absence of a moving object is determined from a difference image between an image of a current frame and an image of a past frame (for example, the previous frame) generated by the camera 12. If there is a moving object region, this moving object region is set as the moving object region.
  • the moving object extraction unit 321 notifies the head candidate extraction unit 322a of the extracted moving object region.
  • the head candidate extraction unit 322a extracts a head candidate region that is a head candidate from the moving object region extracted by the moving object extraction unit 321. In the image picked up from above, the head appears in a substantially circular shape, so a circular region is extracted as the head candidate region from the body movement region. For example, the head candidate extraction unit 322a extracts a circular area from the body movement area as the head candidate area by using a Hough transform that extracts a circular shape. Further, for example, a head shape pattern is prepared in advance and stored in advance in the storage unit 5a as one of the various predetermined data, and the head candidate extraction unit 322a is stored in the storage unit 5a in advance. The shape pattern is searched from the body movement region by using pattern matching, and a region matching the shape pattern of the head is extracted as the head candidate region. The head candidate extraction unit 322a notifies the parameter calculation unit 323a of the extracted head candidate region.
  • the parameter calculation unit 323a obtains each time change in each of a plurality of preset parameters related to the head with respect to the head candidate region extracted by the head candidate extraction unit 322a. At least one of the plurality of parameters is extracted from the image acquisition unit that acquires an image captured from above the monitored person, which is an image acquired by the camera 12 in this embodiment.
  • the plurality of parameters are, for example, an area of the head candidate region and preset subparameters related to the shape of the head candidate region.
  • the sub-parameter is one or more of a circularity of the head candidate region, an aspect ratio of the head candidate region, a skin area ratio of the head candidate region, and a hair area ratio of the head candidate region. is there.
  • the circularity is a degree representing how close to a perfect circle.
  • the aspect ratio is the ratio of the first length along the first direction and the second length along the second direction orthogonal to the first direction.
  • the skin area ratio is a ratio between the total area of the head candidate area and the area of the skin area (skin area) in the head candidate area.
  • the hair area ratio is a ratio between the total area of the head candidate area and the area of the hair area (hair area) in the head candidate area.
  • the parameter calculation unit 323a performs the time change of the area of the head candidate region (the time differential value of the area, the difference value between the frames of the area) dD and the time change of the circularity (the time of the circularity). A differential value and a difference value between frames of circularity) dS are obtained.
  • the parameter calculation unit 323a notifies the fall detection unit 324a of each time change in each of the obtained plurality of parameters.
  • the fall detection unit 324a detects a fall in the monitored person Ob based on each time change in each of the plurality of parameters obtained by the parameter calculation unit 323a. More specifically, the fall detection unit 324a obtains a time change dVa of the evaluation value Va set in advance based on the plurality of parameters, and the monitored person Ob based on the time change dVa of the obtained evaluation value Va. Detects falls in For example, the fall detection unit 324a obtains a time change dVa of the evaluation value Va set in advance based on the plurality of parameters, and determines whether or not there is a change occurring in the time change dVa of the obtained evaluation value Va. Detects if there is a fall.
  • the fall detection unit 324a determines that the monitored person Ob has not fallen, and when there is a change in the time change dVa of the evaluation value Va.
  • the time change dVa of Va is obtained, and the presence or absence of a fall in the monitored person Ob is detected based on the presence or absence of a change occurring in the time change dVa of the evaluation value Va.
  • the fall detection unit 324a calculates a time change dVa of the evaluation value Va set in advance based on the plurality of parameters, and the time change dVa of the calculated evaluation value Va is within a preset threshold range. In the case (th1 ⁇ dVa ⁇ th2), the fall is determined. When the time change dVa of the evaluation value Va is not within the threshold range (th1> dVa, dVa> th2), the fall detection unit 324a determines that the monitored person Ob has not fallen, while the evaluation value When the time change dVa of Va is within the threshold range (th1 ⁇ dVa ⁇ th2), the fall detection unit 324a determines that the monitored person Ob has fallen. When the fall detection unit 324a detects the fall of the monitored person Ob, the fall detection unit 324a notifies the notification processing unit 33 accordingly.
  • the notification processing unit 33 notifies the predetermined behavior detected in the behavior detection processing unit 32 (in the present embodiment, getting up, getting out of bed, falling, abnormal movement of fine bodies) to the outside. More specifically, the notification processing unit 33 is information representing the detected predetermined action (state, situation) (detected action information (in this embodiment, one of the wake-up, get-off, fall, and abnormal micromotion) A plurality of information)), identifier information for identifying and identifying the monitored person Ob in which the predetermined action is detected (identifier for identifying and identifying the sensor device SUa detecting the monitored person Ob) Information) and a communication signal (monitoring information communication signal) containing the image used for detection of the predetermined action is generated and transmitted to the management server device SV by the communication IF unit 4.
  • the notification processing unit 33 is information representing the detected predetermined action (state, situation) (detected action information (in this embodiment, one of the wake-up, get-off, fall, and abnormal micromotion) A plurality of information)), identifier information for identifying
  • the streaming processing unit 34 when there is a moving image distribution request from the fixed terminal device SP or the portable terminal device TA via the network NW and the communication IF unit 4, makes the request to the fixed terminal device SP or the portable terminal device.
  • the moving image (for example, live moving image) generated by the camera 12 of the sensor unit 1 is distributed to the TA via the communication IF unit 4 and the network NW by streaming reproduction.
  • the nurse call processing unit 35 when receiving an input operation by a not-illustrated nurse call push button switch that receives a nurse call from the monitored person Ob, via the communication IF unit 4 and the network NW, A voice call can be made with the mobile terminal device TA.
  • FIG. 1 shows four first to fourth sensor devices SUa-1 to SUa-4 as an example, and the first sensor device SUa-1 is one of the monitored persons Ob.
  • the second sensor device SUa-2 is disposed in the room RM-1 (not shown) of Ob-1, and the second sensor device SUa-2 is arranged in the room RM-2 (not shown) of Mr. B Ob-2 who is one of the monitored persons Ob.
  • the third sensor device SUa-3 is disposed in a room RM-3 (not shown) of Mr. C Ob-3, one of the monitored people Ob, and the fourth sensor device is a monitored person. It is arranged in the room RM-4 (not shown) of Mr. D Ob-4, one of Ob.
  • the management server device SV has a communication function for communicating with other devices SUa, SP, and TA via the network NW, and receives a detection result regarding the monitored person Ob and an image of the monitored person Ob from the sensor device SUa.
  • This is a device that manages information (monitoring information) related to monitoring of the monitored person Ob.
  • the management server apparatus SV stores (records) the monitoring information related to monitoring the monitored person Ob, and A communication signal (monitoring information communication signal) containing the monitoring information related to the monitoring of the observer Ob is transmitted to the fixed terminal device SP and the portable terminal device TA.
  • the management server device SV provides the client with data corresponding to the request of the client (in this embodiment, the fixed terminal device SP and the portable terminal device TA).
  • a management server device SV can be configured by, for example, a computer with a communication function.
  • the fixed terminal device SP includes a communication function for communicating with other devices SUa, SV, TA via the network NW, a display function for displaying predetermined information, an input function for inputting predetermined instructions and data, and the like.
  • a user interface of the monitored person monitoring system MSa by inputting a predetermined instruction or data to be given to the management server SV or the portable terminal device TA, or displaying a detection result or an image obtained by the sensor device SUa. It is a device that functions as (UI).
  • Such a fixed terminal device SP can be configured by, for example, a computer with a communication function.
  • the mobile terminal device TA communicates with other devices SV, SP, SUa via the network NW, a display function for displaying predetermined information, an input function for inputting predetermined instructions and data, and a voice call. It has a calling function to perform, and inputs a predetermined instruction or data to be given to the management server device SV or the sensor device SUa, or displays the detection result or image obtained by the sensor device SUa by a notification from the management server device SV It is a device that receives and displays the monitoring information related to the monitoring of the monitored person Ob.
  • a portable terminal device TA can be configured by a portable communication terminal device such as a so-called tablet computer, a smartphone, or a mobile phone.
  • the control processing unit 3a includes a control unit 31, a behavior detection processing unit 32, a notification processing unit 33, a streaming processing unit 34, and a nurse call processing unit 35.
  • the behavior detection processing unit 32 a moving object extraction unit 321, a head candidate extraction unit 322a, a parameter calculation unit 323a, and a fall detection unit 324a are functionally configured.
  • the monitored person monitoring system MSa having the above configuration generally monitors each monitored person Ob by the following operation.
  • the sensor device SUa samples the output of the Doppler sensor 11 of the sensor unit 1 at a predetermined sampling period, acquires an image with the camera 12 in synchronization with the output, and outputs the sampled and acquired output of the Doppler sensor 11 and the camera 12.
  • a predetermined action (state, situation) in the monitored person Ob is determined, and as a result of this determination, the predetermined action (for example, in this embodiment, wake-up, getting-off, Communication that contains monitoring result information such as determination result information indicating a determination result determined as the state of the monitored person Ob and image data of a still image of the monitored person Ob
  • monitoring result information such as determination result information indicating a determination result determined as the state of the monitored person Ob and image data of a still image of the monitored person Ob
  • a signal (monitoring information communication signal) is transmitted to the management server device SV via the network NW.
  • the operation of the fall detection for detecting the fall will be described in detail later.
  • the management server device SV When the management server device SV receives the monitoring information communication signal from the sensor device SUa via the network NW, the management server device SV stores monitoring information such as determination result information and still image data stored in the monitoring information communication signal in its storage unit. (Record). Then, the management server device SV transmits a monitoring information communication signal containing monitoring information such as the determination result information and still image data to the terminal device (in this embodiment, the fixed terminal device SP and the portable terminal device TA). . As a result, the state (situation) of the monitored person Ob is notified to a monitor such as a nurse or a caregiver via the terminal devices SP and TA.
  • the fixed terminal device SP and the portable terminal device TA display the monitoring information accommodated in the monitoring information communication signal.
  • the monitored person monitoring system MSa roughly detects each monitored person Ob by each sensor device SUa, the management server device SV, the fixed terminal device SP, and the portable terminal device TA, and each monitored person. Ob is being monitored.
  • FIG. 3 is a flowchart illustrating the operation of the sensor device in the monitored person monitoring system according to the first embodiment.
  • FIG. 4 is a diagram for explaining how to obtain the perimeter of the head candidate region.
  • FIG. 5 is a diagram for explaining a temporal change in the circularity of the head at the time of falling. The left side of FIG. 5 shows a standing state, the right side of FIG. 5 shows a lying state, and the center of FIG. 5 shows a state between these standing and lying sides.
  • FIG. 5 is a flowchart illustrating the operation of the sensor device in the monitored person monitoring system according to the first embodiment.
  • FIG. 4 is a diagram for explaining how to obtain the perimeter of the head candidate region.
  • FIG. 5 is a diagram for explaining a temporal change in the circularity of the head at the time of falling.
  • the left side of FIG. 5 shows a standing state
  • the right side of FIG. 5 shows a lying state
  • the center of FIG. 5 shows a state between these standing and lying sides.
  • FIG. 6 is a diagram illustrating an example of the head area, the temporal differentiation of the head area, the circularity of the head, the temporal differentiation of the circularity of the head, and each time change of the evaluation value during the fall. .
  • the horizontal axis in FIG. 6 is the frame number (time), and the vertical axis is each value.
  • ⁇ represents the area D
  • ⁇ represents the time differential (area change) dD of the area
  • ⁇ represents the circularity S
  • x represents the time differential of the circularity S (circularity).
  • FIG. 7 is a diagram for explaining the change over time of the circularity of the ball when the ball is dropped.
  • FIG. 7 The left side of FIG. 7 shows the ball in the upper position, the right side of FIG. 7 shows the ball in the lower position (floor position), and the center of FIG. 7 shows the ball in the upper and lower positions ( (Floor position).
  • FIG. 8 is a diagram showing an example of each time change of the ball area, the time differentiation of the ball area, the circularity of the ball, the temporal differentiation of the circularity of the ball, and the evaluation value when the ball is dropped. It is.
  • ⁇ represents area D
  • ⁇ time derivative of area (time change in area) dD
  • ⁇ represents circularity S
  • x time derivative of circularity S (circularity).
  • FIG. 9 is a diagram for explaining a temporal change in the circularity of the trash box when the trash box falls.
  • the left side of FIG. 9 shows the state of standing
  • the right side of FIG. 9 shows the state of recumbent
  • the center of FIG. 9 shows the state between these standing and recumbent.
  • Each upper stage in FIG. 9 shows a side view thereof
  • each lower stage shows a top view thereof.
  • the sensor device SUa samples the output of the Doppler sensor 11 of the sensor unit 1 and acquires the image by the camera 12 to detect the fall of the monitored person Ob by operating as follows. is doing.
  • control processing unit 3a acquires an image from the camera 12 of the sensor unit 1 (S1). That is, an image for one frame is acquired from the camera 12.
  • control processing unit 3a uses, for example, a background subtraction method or an interframe subtraction method from the image acquired by the camera 12 by the moving body extraction unit 321 of the behavior detection processing unit 32 to monitor the subject Ob.
  • a moving body region is extracted as the human body region (S2).
  • control processing unit 3a uses the head candidate extraction unit 322a of the behavior detection processing unit 32 from the moving body region extracted by the moving body extraction unit 321 by using a technique such as Hough transform or pattern matching, for example.
  • a candidate head region is extracted (S3).
  • the control processing unit 3a uses the parameter calculation unit 323a of the behavior detection processing unit 32 in each of a plurality of preset parameters related to the head with respect to the head candidate region extracted by the head candidate extraction unit 322a.
  • Each time change is obtained (S4).
  • the parameter calculation unit 323a obtains the area D and the circularity S of the head candidate region as the plurality of parameters, and obtains the respective time changes dD and dS. More specifically, the parameter calculation unit 323a obtains the total number of pixels (total number of pixels) included in the head candidate area EHA extracted by the head candidate extraction unit 322a, for example, as illustrated in FIG.
  • the area D of the head candidate region is obtained, and the total number of pixels constituting the outer periphery of the head candidate region EHA extracted by the head candidate extraction unit 322a ( By calculating the total number of pixels located at the boundary) (total number of pixels) and multiplying the total number of pixels thus determined by the length of one pixel (for example, vertical length, horizontal length, diagonal length, or an average value thereof)
  • D and circularity S are set, for example, at the sampling timing (sample) Ing the number of timing) and in association with the frame number and the like stored in the storage unit 5a.
  • the circularity S indicates that the closer the value is to 1, the closer it is to a perfect circle.
  • the parameter calculation unit 323a calculates the area D (N-1) and the circularity S (N-1) of the head candidate region EHA obtained at the previous (N-1) sampling timing, and the current (N) By obtaining the difference between the area D (N) and the circularity S (N) of the head candidate area EHA obtained at the sampling timing, the time change (time differentiation) dD of the area D and the time change of the circularity S are obtained.
  • Each dS is obtained, and the obtained time change dD of the area D of the head candidate region EHA and the time change dS of the circularity S are associated with, for example, the time of the sampling timing (number of sampling timings), the frame number, In addition, it is stored in the storage unit 5a.
  • the control processing unit 3a determines the presence or absence of a fall in the monitored person Ob based on each time change in each of the plurality of parameters obtained by the parameter calculation unit 323a by the fall detection unit 324a of the behavior detection processing unit 32. Then, the fall of the monitored person Ob is detected (S5).
  • the fall detection unit 324a obtains a time change dVa of the preset evaluation value Va based on a plurality of parameters obtained by the parameter calculation unit 323a, and based on the time change dVa of the obtained evaluation value Va. The fall of the monitored person Ob is detected.
  • the middle posture in the middle of FIG. 5 is changed to the recumbent posture (right side in FIG. 5).
  • the area of the head is away from the position where the camera 12 is arranged.
  • the shape of the head gradually decreases as shown in the center and the right side, and the shape of the head changes from a substantially circular shape shown in the left side of FIG. 5 to a shape shown in the center of FIG. Change to shape.
  • each of the head area D ( ⁇ ) and the circularity S ( ⁇ ) has the profile shown in FIG. 6, and both the frame numbers 1 to 5 before the fall are not so much changed, and the falls Both start to become smaller at frame number 6 where the start of the movement starts, and after frame number 8 when the fall is over due to falling to the floor, both are less changed again.
  • the ball BL goes from the upper position (left side in FIG. 7) to the floor position (right side in FIG. 7) through the middle state in FIG. .
  • the area of the ball BL moves away from the arrangement position of the camera 12, so the left side, the center, and the right side of FIG.
  • the shape of the ball BL remains circular as shown in the left side, center and right side of FIG. 7, but does not change. Therefore, as shown in the graph, the area D ( ⁇ ) of the ball BL becomes the profile shown in FIG.
  • the circularity S ( ⁇ ) of the ball BL does not change so much after the frame number 8 falling on the floor, and the circularity S ( ⁇ ) of the ball BL becomes the profile shown in FIG. As described above, the time variation of the circularity S is different between the fall of the monitored person Ob and the fall of the ball BL.
  • the inverted frustoconical trash can DP moves from the normal arrangement posture before the fall (left side in FIG. 9) to the middle of FIG. After the above state, the posture falls to the side lying on the floor (right side in FIG. 9).
  • the area and shape of the trash bin DP is changed from the circular shape on the lower left side of FIG. The shape changes to a substantially trapezoidal shape on the lower right side of FIG.
  • the area D of the inverted frustoconical trash box DP gradually increases when the overturn starts, while the circularity S suddenly increases from 1 to 0 when the overturn starts. Get closer to.
  • the time change of the area D and the time change of the circularity S between the fall of the monitored person Ob and the fall of the trash can DP.
  • the fall detection unit 324a compares these differences with each other based on the time change dD of the area D and the time change dS of the circularity S in the head candidate region, so that the fall of the monitored subject Ob
  • the fall of the monitored person Ob can be detected by discriminating whether the event has occurred in other than the monitoring person Ob.
  • the fall of the monitored person Ob may be determined based on the time change of the area D ( ⁇ ) and the time change of the circularity S ( ⁇ ), but the evaluation value Va ( ⁇ ) obtained by multiplication of these time derivatives. 6), as can be seen by comparing FIG. 6 and FIG. 8, in FIG. 6, it rises from frame number 5 where the fall started, peaks at frame numbers 6 and 7, and falls at frame number 8, but in FIG. There is no such change. Therefore, the fall of the monitored person Ob is determined by the evaluation value Va rather than the fall of the monitored person Ob by the time change dD of the area D ( ⁇ ) and the time change dS of the circularity S ( ⁇ ). It is easier to determine.
  • the fall detection unit 324a detects the presence or absence of a fall in the monitored person Ob based on the presence or absence of a change that occurs in the time change dVa of the evaluation value Va. In this case, when there is no change in the time change dVa of the evaluation value Va, the fall detection unit 324a determines that the monitored person Ob has not fallen, and there is a change in the time change dVa of the evaluation value Va. In this case, the fall detection unit 324a determines that the monitored person Ob has fallen.
  • the fall detection unit 324a obtains a time change dVa of the preset evaluation value Va based on the plurality of parameters, and the time change dVa of the obtained evaluation value Va is within a preset threshold range. (Th1 ⁇ dVa ⁇ th2), the fall is determined. When the time change dVa of the evaluation value Va is not within the threshold range (th1> dVa, dVa> th2), the fall detection unit 324a determines that the monitored person Ob has not fallen, while the evaluation value When the time change dVa of Va is within the threshold range (th1 ⁇ dVa ⁇ th2), the fall detection unit 324a determines that the monitored person Ob has fallen.
  • the upper and lower limit values th1 and th2 of the threshold range are set so that, for example, a fall can be determined based on the head from a plurality of samples.
  • step S5 If no fall is detected as a result of the determination in step S5 (No), the process at this timing is terminated. On the other hand, if a fall is detected (Yes), the fall detection unit 324a The notification processor 33 is notified of the fall of Ob.
  • the notification processing unit 33 notifies the management server device SV to that effect by the monitoring information communication signal described above, and performs processing at this timing. finish.
  • the monitored person monitoring system MSa and the sensor apparatus SUa of the example using the action detecting apparatus and the action detecting method in the first embodiment which is an example of the monitored person monitoring apparatus are included in the head candidate region.
  • the head candidate area is used to detect the monitored person Ob more accurately. A fall can be detected.
  • the monitored person monitoring system MSa and the sensor device SUa are not only based on the time change dD in the area of the head candidate region, but also based on the time change of a preset subparameter related to the shape of the head candidate region. Thus, since the fall of the monitored person Ob is detected, the head and the other parts except the head can be more discriminated, and the fall of the monitored person Ob can be detected with higher accuracy.
  • the monitored person monitoring system MSa and the sensor device SUa can easily determine the presence or absence of a fall by comparing the time change dVa of the evaluation value Va with the upper and lower limit values th1 and th2 that define the threshold range.
  • the circularity S is used as a sub-parameter related to the shape, but is not limited to this.
  • the aspect ratio of the head candidate area may be used as a sub-parameter related to the shape
  • the skin area ratio of the head candidate area may be used
  • the hair area ratio of the head candidate area may be used. May be.
  • a plurality of the circularity of the head candidate region, the aspect ratio of the head candidate region, the skin area ratio of the head candidate region, and the hair area ratio of the head candidate region may be used.
  • the aspect ratio is the ratio of the first length along the first direction and the second length along the second direction orthogonal to the first direction.
  • the skin area ratio is a ratio between the total area of the head candidate area and the area of the skin area (skin area) in the head candidate area.
  • the hair area ratio is a ratio between the total area of the head candidate area and the area of the hair area (hair area) in the head candidate area.
  • FIG. 10 is a diagram for explaining the temporal change in the aspect ratio of the head during a fall.
  • FIG. 10A is a top view of the head as seen from above
  • FIG. 10B is a front view of the head as seen from the front.
  • FIG. 11 is a diagram for explaining the total head height, head length, and head width.
  • FIG. 11A is a side view
  • FIG. 11B is a front view.
  • FIG. 12 is a diagram for explaining temporal changes in the skin area ratio of the head and the hair area ratio during a fall.
  • the left side of FIG. 12 shows a standing state
  • the right side of FIG. 12 shows a lying state
  • the center of FIG. 12 shows a state between these standing and lying sides.
  • FIG. 13 is a diagram for explaining how to obtain the skin area ratio of the head and the hair area ratio.
  • the aspect ratio is used as the subparameter, in the image acquired by the camera 12 that images the monitored person Ob from above the monitored person Ob, the head is shown in FIG. 10A in the standing posture before the fall. As shown in FIG. 11, the aspect ratio is the head width relative to the head length.
  • Statistical data on the size of each part of a person is created and released by, for example, a public institution. For example, statistical data on the Japanese head is disclosed by the Digital Human Research Group, National Institute of Advanced Industrial Science and Technology (Human Information Research Division) ([online], search on April 23, 2015, Internet ⁇ http: // www .Dh.aist.go.jp / jp / pubbase / data.php>).
  • the head length is 188 mm
  • the time change of the aspect ratio of the trash can can take a value between 1 and 0.
  • the fall detection unit 324a can determine that the fall has occurred when the temporal change in the aspect ratio is within a preset threshold range. In this case, if the time change of the aspect ratio is not within the threshold range, the fall detection unit 324a determines that the monitored person Ob has not fallen, while the time change of the aspect ratio is within the threshold range. If it is, the fall detection unit 324a determines that the monitored person Ob has fallen.
  • the lower limit value is 0.687
  • the upper limit value is 0.840
  • the fall detection unit 324a has an aspect ratio time change of 0.687 or more and 0.840 or less. In some cases, the fall can be determined.
  • the temporal change in the hair area ratio can be used as a sub-parameter, and the skin area and the hair area can be detected by known conventional means, for example, as shown in FIG.
  • the block is divided into small blocks, and for each block, it is determined whether the block is an area where the hair is copied or an area where the skin is copied, which is determined by whether the block has a texture as shown in FIG. 13C, it can be determined by whether or not it has a texture as a feature.
  • Approach can be determined by various known techniques determination method or the like using the determination method and Fourier transform based on the co-occurrence matrix.
  • the size of the head region shown in the image does not change much. For this reason, if it is attempted to determine whether or not there is a fall based only on the size of the head region shown in the image, it is difficult to determine whether it is a fall or a movement in the horizontal direction.
  • the second embodiment deals with such a case.
  • the monitored person monitoring apparatus includes an action detection unit that detects a predetermined action set by a monitored person who is a monitoring target, and the predetermined action detected by the action detection unit. And a notification unit for notifying the device. And in 2nd Embodiment, the said action detection part is the distance from the image acquisition part which acquires the image imaged from the said to-be-monitored person, and the said to-be-monitored person's head above the to-be-monitored person.
  • a distance measuring unit that measures from the above, a size calculating unit that calculates the size of the head of the monitored person viewed from above from the image acquired by the image acquiring unit, and the head that is determined by the size calculating unit And a fall detection unit that detects a fall in the monitored person based on a temporal change rate of a difference between the size of the monitor and the distance to the head measured by the ranging unit.
  • the behavior detection unit is mounted on a sensor device SUb described later together with the notification unit will be described.
  • the monitored person monitoring device can be configured in the same manner as described below.
  • FIG. 14 is a diagram illustrating a configuration of a sensor device in the monitored person monitoring system according to the second embodiment.
  • FIG. 15 is a diagram illustrating an arrangement of the sensor device in the monitored person monitoring system according to the second embodiment.
  • the monitored person monitoring system MSb in the second embodiment of the example in which the monitored person monitoring device is realized as a system is the same as the monitored person monitoring system MSa of the first embodiment.
  • a predetermined action set in Ob (Ob-1 to Ob-4) is detected to monitor the monitored person Ob.
  • the management server device SV, fixed terminal device SP, and one or more portable terminal devices TA (TA-1, TA-2) in the monitored person monitoring system MSb of the second embodiment are respectively the first embodiment. Since this is the same as the management server device SV, fixed terminal device SP, and one or more portable terminal devices TA (TA-1, TA-2) in the monitored person monitoring system MSa, the description thereof is omitted.
  • the sensor device SUb is a device that has a communication function for communicating with other devices SV, SP, and TA via the network NW, detects the monitored person Ob, and transmits the detection result to the management server device SV.
  • the sensor device SUb includes a sensor unit 1, a sound input / output unit 2, a control processing unit 3b, a communication interface unit (communication IF unit) 4, and a storage unit 5b. Is provided.
  • the sensor unit 1, the sound input / output unit 2 and the communication IF unit 4 in the sensor device SUb of the second embodiment are respectively the sensor unit 1, the sound input / output unit 2 and the communication IF unit in the sensor device SUa of the first embodiment. Since it is the same as 4, its description is omitted.
  • the predetermined quantities include the distance to the head of the monitored person Ob and are appropriately determined according to the predetermined action to be detected by the monitored person Ob.
  • the storage unit 5b is a circuit that is connected to the control processing unit 3b and stores various predetermined programs and various predetermined data under the control of the control processing unit 3b.
  • the various predetermined programs include, for example, control processing programs such as a monitoring processing program for executing information processing related to monitoring of the monitored person Ob.
  • the monitoring processing program includes a behavior detection program for detecting a predetermined behavior in the monitored person Ob based on an output of the sensor unit 1, a notification processing program for notifying the predetermined behavior detected by the behavior detection program to the outside, By using the streaming processing program for streaming the moving image captured by the camera 12 to the fixed terminal device SP or the portable terminal device TA that requested the moving image, and the sound input / output unit 2 or the like, the fixed terminal device SP or the portable terminal A nurse call processing program for making a voice call with the device TA is included.
  • the behavior detection program includes a moving body extraction program that extracts a moving body region as a human body region of the monitored person Ob from an image acquired by the camera 12 of the sensor unit 1, and a moving body region extracted from the moving body extraction program.
  • the various types of predetermined data include data necessary for executing these programs, data necessary for monitoring the monitored person Ob, and the like.
  • the storage unit 5b includes, for example, a ROM that is a nonvolatile storage element, an EEPROM that is a rewritable nonvolatile storage element, and the like.
  • the storage unit 5b includes a RAM serving as a working memory of a so-called control processing unit 3b that stores data generated during execution of the predetermined program.
  • the control processing unit 3b is a circuit for controlling each unit of the sensor device SUb according to the function of each unit and detecting predetermined amounts set in advance in the monitored person Ob.
  • the control processing unit 3b includes, for example, a CPU and its peripheral circuits.
  • the control processing unit 3b functionally includes a control unit 31, a behavior detection processing unit 32, a notification processing unit 33, a streaming processing unit 34, and a nurse call processing unit 35 by executing the control processing program.
  • the processing unit 32 functionally includes a moving object extraction unit 321, a head extraction unit 322b, a size calculation unit 323b, a distance calculation unit 324b, and a fall detection unit 325b.
  • the control unit 31 controls each part of the sensor device SUb according to the function of each part, and controls the entire sensor device SUb.
  • the behavior detection processing unit 32 detects a predetermined behavior set in advance in the monitored person Ob based on the output of the sensor unit 1.
  • the predetermined behavior is wakeup, getting out of bed, falling, and abnormal movement of the monitored person Ob
  • the behavior detection processing unit 32 is based on the output of the sensor unit 1.
  • Wake-up, getting-off, falling, and minute body movement abnormality in the monitoring person Ob are detected as detection results, and the detection results are notified to the notification processing unit 33.
  • the behavior detection processing unit 32 detects wake-up, getting-off and abnormal body movement of the monitored person Ob based on the output of the sensor unit 1 by a known technique.
  • the behavior detection processing unit 32 extracts a moving body region from the image acquired by the camera 12 of the sensor unit 1 as the human body region of the monitored person Ob, and the posture of the monitored person Ob from the aspect ratio of the extracted moving body area ( For example, standing position, sitting position, lying down, etc.) are detected, the position of the detected moving body region is detected, and the distinction between the rising and leaving is determined based on these determinations and the detected posture and position of the monitored person Ob.
  • the behavior detection processing unit 32 detects body movement of the chest (vertical movement of the chest) accompanying the breathing motion of the monitored person Ob by the Doppler sensor 11 of the sensor unit 1, and the period disturbance in the body movement of the chest When the amplitude in the body motion of the chest that is equal to or less than a preset threshold is detected, it is determined that the minute motion abnormality is present.
  • the behavior detection processing unit 32 includes a moving object extraction unit 321, a head extraction unit 322b, a size calculation unit 323b, a distance calculation unit 324b, and a fall detection unit 325b. Functionally provided, and these detect the fall of the monitored person Ob.
  • the moving object extraction unit 321 extracts a moving object region from the image acquired by the camera 12 of the sensor unit 1 as a human body region of the monitored person Ob by using a method such as a background difference method or an inter-frame difference method.
  • the moving object extraction unit 321 is also used when extracting a moving object in order to detect the above-described wake-up and getting-off, and is shared by these processes.
  • a background image is obtained in advance and stored in advance in the storage unit 5b as one of the various predetermined data, and the presence / absence of moving objects is determined from the difference image between the image generated by the camera 12 and the background image. If there is a moving object as a result of this determination, the area of the moving object is set as a moving object area.
  • the presence / absence of a moving object is determined from a difference image between an image of a current frame and an image of a past frame (for example, the previous frame) generated by the camera 12. If there is a moving object region, this moving object region is set as the moving object region.
  • the moving object extraction unit 321 notifies the extracted moving object region to the head extraction unit 322b.
  • the head extraction unit 322b extracts a head region (a head region or a head candidate region) from the moving object region extracted by the moving object extraction unit 321.
  • a head shape pattern is prepared in advance and stored in advance in the storage unit 5b as one of the various predetermined data, and the head extraction unit 322b is stored in the storage unit 5b in advance.
  • a pattern is searched from the body movement area by using pattern matching, and an area matching the shape pattern of the head is extracted as the head area.
  • the head is reflected in an image captured from above in a substantially circular shape corresponding to the top of the head. Therefore, a substantially circular pattern is one of the head shape patterns. included.
  • the head extraction unit 322b notifies the size calculation unit 323b of the extracted head region.
  • the size calculation unit 323b obtains the size of the head of the monitored person Ob seen from above by obtaining the area of the head region extracted by the head extraction unit 322b. For example, the size calculation unit 323b calculates the total number of pixels (total number of pixels) included in the head region extracted by the head extraction unit 322b, and multiplies the calculated total number of pixels by the area of one pixel. The area of the head region is obtained, and the size of the head is obtained. The size calculation unit 323b notifies the fall detection unit 325b of the obtained head size.
  • the size calculation unit 323b corresponds to another example of the parameter calculation unit.
  • the distance calculation unit 324b obtains the distance to the head of the monitored person Ob by obtaining the distance from the Doppler signal output from the Doppler sensor 11 of the sensor unit 1.
  • the distance calculation is performed.
  • the distance obtained by the part 324b increases substantially monotonously with time and becomes a relatively large value.
  • the distance obtained by the distance calculation unit 324b is a value resulting from the body movement of the chest accompanying the breathing motion of the monitored person Ob.
  • the monitored person Ob when the monitored person Ob is normal, it changes periodically with the passage of time and becomes a relatively small value. For this reason, it can be distinguished whether the distance calculated
  • the fall detection unit 325b detects a fall in the monitored person Ob based on the time change rate of the difference between the size of the head obtained by the size calculation unit 323b and the distance to the head obtained by the distance calculation unit 324b. To do. When the fall detection unit 325b detects the fall of the monitored person Ob, the fall detection unit 325b notifies the notification processing unit 33 accordingly.
  • the Doppler sensor 11 and the distance calculation unit 324b correspond to an example of a distance measurement unit
  • the camera 12 corresponds to an example of an image acquisition unit.
  • the control processing unit 3b includes a control unit 31, a behavior detection processing unit 32, a notification processing unit 33, a streaming processing unit 34, and a nurse call processing unit 35.
  • the behavior detection processing unit 32 a moving object extraction unit 321, a head extraction unit 322b, a size calculation unit 323b, a distance calculation unit 324b, and a fall detection unit 325b are functionally configured.
  • the general operation in the monitored person monitoring system MSb configured as described above for monitoring each monitored person Ob is the same as that in the first embodiment, except for the operation of overturn detection in the sensor device SUb. Description is omitted.
  • FIG. 16 is a flowchart illustrating the operation of the sensor device in the monitored person monitoring system according to the second embodiment.
  • FIG. 17 is a diagram for explaining the temporal change of the head region during a fall. The left side of FIG. 17 shows the standing position before the fall, the right side of FIG. 17 shows the lying side after the fall, and the center of FIG. 17 shows the state between the standing position and the lying side (state during the fall) ).
  • FIG. 18 is a diagram illustrating an example of changes over time in the distance to the head and the size of the head region during a fall.
  • FIG. 17 is a diagram for explaining the temporal change of the head region during a fall. The left side of FIG. 17 shows the standing position before the fall, the right side of FIG. 17 shows the lying side after the fall, and the center of FIG. 17 shows the state between the standing position and the lying side (state during the fall) ).
  • FIG. 18 is a diagram illustrating an example of changes over time in the distance to the head and the size of the head region
  • FIG. 19 is a diagram illustrating an example of changes over time in the distance to the head and the size of the head region during normal horizontal (lateral) movement.
  • FIG. 20 is a diagram illustrating an example of changes over time of the distance to the head and the size of the head region during normal vertical (vertical) movement.
  • the horizontal axis is time
  • the right vertical axis is the size of the head
  • the left vertical axis is the distance.
  • Graphs ⁇ 1, ⁇ 2, and ⁇ 3 show the relationship between time and the distance to the head
  • graphs ⁇ 1, ⁇ 2, and ⁇ 3 show the relationship between time and the size of the head.
  • the sensor device SUb samples the output of the Doppler sensor 11 of the sensor unit 1 and acquires the image with the camera 12 to detect the fall of the monitored person Ob by operating as follows. is doing.
  • the control processing unit 3b acquires a Doppler signal from the Doppler sensor 11 of the sensor unit 1, and acquires an image (image for one frame) from the camera 12 of the sensor unit 1 (S11).
  • control processing unit 3b uses a technique such as a background subtraction method or an interframe subtraction method from the image acquired by the camera 12 by the moving body extraction unit 321 of the behavior detection processing unit 32, for example.
  • the moving body region is extracted as the human body region (S12).
  • control processing unit 3b uses a method such as pattern matching from the moving body region extracted by the moving body extraction unit 321 by the head extraction unit 322b of the behavior detection processing unit 32, for example. Partial area) is extracted (S13).
  • This head region may be a head candidate region, as in the first embodiment.
  • the control processing unit 3b obtains the size (area) of the head region extracted by the head extraction unit 322b by the size calculation unit 323b of the behavior detection processing unit 32 (S14).
  • the size calculation unit 323b calculates the total number of pixels (total number of pixels) included in the head region extracted by the head extraction unit 322b, and multiplies the calculated total number of pixels by the area of one pixel.
  • the area ⁇ of the head region is obtained as its size, and the obtained area ⁇ of the head region is stored in the storage unit 5b in association with, for example, the sampling timing (number of sampling timings), the frame number, and the like.
  • the control processing unit 3b uses the distance calculation unit 324b of the behavior detection processing unit 32 to obtain the distance as the distance ⁇ from the Doppler signal acquired in step S11, and the obtained distance ⁇ to the head. For example, it is stored in the storage unit 5b in association with the time of sampling timing (number of times of sampling timing), the frame number, etc. (S15). That is, the area ⁇ of the head region obtained in step S14 and the distance ⁇ to the head obtained in step S15 are associated with each other via, for example, the time of sampling timing (the number of sampling timings), the frame number, and the like. It is stored in the storage unit 5b.
  • control processing unit 3b sets the head region size ⁇ calculated by the size calculation unit 323b and the distance ⁇ to the head calculated by the distance calculation unit 324b by the fall detection unit 325b of the behavior detection processing unit 32. Based on this, the presence or absence of a fall in the monitored person Ob is determined, and the fall in the monitored person Ob is detected (S16).
  • the distance ⁇ 2 to the head and the size ⁇ 2 of the head region in each case, and the distance to the head when the monitored object Ob moves in the vertical direction (vertical direction) due to, for example, squatting Changes with time in each of ⁇ 3 and head region size ⁇ 3 will be described with reference to FIGS.
  • the sensor unit 1 is disposed on the ceiling of the living room RM, for example, and the monitored person is viewed from above the monitored person Ob. Since Ob is sensed, the distance ⁇ 2 to the head changes so as to monotonously increase with time, and becomes, for example, a solid line graph ⁇ 2 shown in FIG. 19, and the size of the head (head region) on the image. The length ⁇ 2 changes so as to become slightly smaller as time elapses, for example, a broken line graph ⁇ 2 shown in FIG.
  • the sensor unit 1 senses the monitored person Ob from above the monitored person Ob as described above. For example, a solid line graph ⁇ 3 shown in FIG. 20 is obtained, and the size ⁇ 3 of the head (head region) on the image decreases monotonously with time. For example, a broken line graph ⁇ 3 shown in FIG. 20 is obtained.
  • the size ⁇ 1 of the head (head region) on the image is reduced as the head moves away from the camera 12 because the approximate top of the head appears in the image in the middle of the fall. It changes a lot. However, since the profile of the head appears in the image at the end from the middle of the fall, the size ⁇ 1 of the head (head region) on the image decreases as the head moves away from the camera 12. The change does not change so much by offsetting the change that increases with the change from the top to the profile. For this reason, the size ⁇ 1 of the head (head region) on the image changes relatively large so as to decrease monotonically with the passage of time from the start of the fall, and at the end of the fall from the end.
  • the size ⁇ 1 of the head (head region) changes nonlinearly so as to gradually approach a predetermined value as time passes in the event of a fall.
  • the difference between the distance ⁇ to the head and the size ⁇ of the head region ( ⁇ 1- ⁇ 1, ⁇ 2- ⁇ 2, ⁇ 3- ⁇ 3) only changes monotonically with the passage of time.
  • Each case is difficult to distinguish based on the difference ( ⁇ 1- ⁇ 1, ⁇ 2- ⁇ 2, ⁇ 3- ⁇ 3) between the distance ⁇ to the head portion and the size ⁇ of the head region, but the time change rate (( ⁇ 1- ⁇ 1 ) / ⁇ T1, ( ⁇ 2- ⁇ 2) / ⁇ T2, ( ⁇ 3- ⁇ 3) / ⁇ T3) change only in the case of a fall, and the time change rate (( ⁇ 1- ⁇ 1) / ⁇ T1, ( ⁇ 2- Each case can be distinguished based on ⁇ 2) / ⁇ T2 and ( ⁇ 3- ⁇ 3) / ⁇ T3).
  • the distance ⁇ 1 to the head and the size ⁇ 1 of the head (head region) at the sampling timings T11, T12, T13, and T14 are respectively ⁇ 1 (T11) and ⁇ 1 (T12).
  • the fall detection unit 325b calculates the difference between the head region size ⁇ obtained by the size calculation unit 323b and the distance ⁇ to the head obtained by the distance calculation unit 324b. Based on the time change rate, the presence or absence of a fall in the monitored person Ob is determined, and the fall in the monitored person Ob is detected. More specifically, the fall detection unit 325b calculates the difference between the head region size ⁇ obtained by the size calculation unit 323b and the distance ⁇ to the head obtained by the distance calculation unit 324b at the previous sampling timing.
  • the time change rate is compared with the time change rate of the difference between the head region size ⁇ obtained by the size calculator 323b and the distance ⁇ obtained by the distance calculator 324b at the current sampling timing. If there is a change in the time change rate of the difference, the fall detection unit 325b determines that the monitored person Ob has fallen, and if there is no change in the time change rate of the difference, the fall detection unit 325b. Determines that the monitored person Ob has not fallen.
  • step S16 if no fall is detected (No), the process at this timing is terminated. On the other hand, if a fall is detected (Yes), the fall detection unit 325b detects the person being monitored. The notification processor 33 is notified of the fall of Ob (S17).
  • the notification processing unit 33 notifies the management server device SV to that effect by the above-described monitoring information communication signal, and performs processing at this timing. finish.
  • the monitored person monitoring system MSb which is an example of the monitored person monitoring apparatus
  • the sensor apparatus SUb of the example using the action detecting apparatus and the action detecting method have the size of the head (head region). Since the fall of the monitored person Ob is detected based on the time change rate of the difference between ⁇ 1 and the distance ⁇ 1 to the head, the fall of the monitored person Ob can be detected with higher accuracy.
  • the monitored person monitoring system MSb and the sensor device SUb use an infrared camera as the camera 12 of the sensor unit 1, the fall of the monitored person Ob can be detected even in the dark.
  • the Doppler sensor mounted as a sensor for detecting a minute movement abnormality can also be used as a distance measurement sensor. There is no need to provide it.
  • the sensor devices SUa and SUb include the behavior detection unit and the notification unit.
  • the behavior detection unit and the notification unit are included in the management server device SV.
  • it may be provided in the fixed terminal device SP, or may be provided in the mobile terminal device TA.
  • a communication interface that receives and acquires a communication signal containing an image captured from above the monitored person from the sensor devices SUa and SUb via the network NW corresponds to an example of an image acquisition unit. become.
  • the monitored person monitoring apparatus includes an image acquisition unit that acquires an image of the monitored person that is a monitoring target from above the monitored person, and an image acquired by the image acquisition unit.
  • a head candidate extraction unit that extracts a head candidate region that is a candidate for the head, and each time change in each of a plurality of preset parameters related to the head with respect to the head candidate region extracted by the head candidate extraction unit
  • a fall detection unit that detects a fall in the monitored person based on each time change in each of the plurality of parameters obtained by the parameter computation unit, and at least one of the parameters is , Extracted from the image acquired by the image acquisition unit.
  • the image is expected to be located at the head of the monitored person in the bedding on which the monitored person is lying down.
  • the image is taken from directly above the preset head planned position.
  • the head is reflected in a substantially circular shape in the image captured from above, and thus a circular region is detected as the head region.
  • a substantially circular object that is not the head such as a pillow, a washbasin, a ball, a trash can, and a futon handle is also reflected, so the area of these objects is erroneously detected as the head area, A fall may be erroneously detected.
  • the erroneous detection can be avoided by removing the area of these objects at the fixed position from the area for detecting the head in advance.
  • the fall detection device obtains each time change in each of a plurality of parameters related to the head with respect to the head candidate area, and detects a fall in the monitored person based on each time change. It is possible to detect the fall of the monitored person with higher accuracy.
  • the plurality of parameters are an area of the head candidate region and a preset subparameter related to the shape of the head candidate region.
  • the sub-parameters include the circularity of the head candidate region, the aspect ratio of the head candidate region, the skin area ratio of the head candidate region, and the hair of the head candidate region.
  • the circularity is a degree representing how close to a perfect circle.
  • the aspect ratio is the ratio of the first length along the first direction and the second length along the second direction orthogonal to the first direction.
  • the skin area ratio is a ratio between the total area of the head candidate area and the area of the skin area (skin area) in the head candidate area.
  • the hair area ratio is a ratio between the total area of the head candidate area and the area of the hair area (hair area) in the head candidate area.
  • Such a fall detection device is not only based on a temporal change in the area of the head candidate area, but also based on a time change of a preset subparameter related to the shape of the head candidate area. Therefore, it is possible to discriminate the head from other parts except the head, and to detect the fall of the monitored person with higher accuracy.
  • the fall detection unit obtains a time change of an evaluation value set in advance based on the plurality of parameters, and the time change of the obtained evaluation value is set in advance. If it falls within the set threshold range, the fall is determined.
  • Such a fall detection device can easily determine whether or not a fall has occurred by comparing the time variation of the evaluation value with the upper and lower limit values that define the threshold range.
  • the fall detection device further includes a distance measuring unit that measures a distance to the head of the monitored person from above the monitored person
  • the parameter calculation unit includes the image acquisition
  • a size calculation unit that obtains, as the parameter, the size of the head of the person to be monitored as viewed from above from the image acquired by the unit
  • the fall detection unit includes the size of the head calculated by the size calculation unit The fall of the monitored person is detected based on the time change rate of the difference between the size and the distance to the head measured by the distance measuring unit.
  • the size of the head region shown in the image greatly changes so as to become smaller as the head moves away from the camera.
  • the profile of the head appears in the image from the middle of the fall to the end thereof, the size of the head region reflected in the image changes as the head moves away from the camera and the profile from the top It does not change much by offsetting the change that grows with the change to.
  • the size of the head region shown in the image does not change much.
  • the fall detection device detects a fall in the monitored person based on a time change rate of a difference between the size of the head and the distance to the head, the fall of the monitored person is more accurately detected. It can be detected well.
  • the image acquisition unit is an infrared camera.
  • Such a fall detection device uses an infrared camera, it is possible to detect the fall of the monitored person even in the dark.
  • the distance measuring unit includes a Doppler sensor and a distance calculation unit that obtains a distance from a Doppler signal of the Doppler sensor.
  • the Doppler sensor can be used in the monitored person monitoring device as a sensor for detecting abnormal movement of the monitored person. Since the fall detection device uses this Doppler sensor for distance measurement, it is not necessary to provide a distance measurement sensor separately when it is incorporated in a monitored person monitoring device equipped with a Doppler sensor as a sensor for detecting minute movement abnormality. .
  • the fall detection method includes an image acquisition step of acquiring an image of a monitored person to be monitored from above the monitored person, and a head from the image acquired in the image acquisition step.
  • a head candidate extraction step for extracting a head candidate region that is a candidate for the head, and each time change in each of a plurality of preset parameters related to the head with respect to the head candidate region extracted in the head candidate extraction step
  • a fall detection step of detecting a fall in the monitored person based on each time change in each of the plurality of parameters obtained in the parameter calculation step.
  • Such a fall detection method obtains each time change in each of a plurality of parameters related to the head with respect to the head candidate region, and detects a fall in the monitored person based on each time change.
  • the fall of the monitored person can be detected with higher accuracy using the area.
  • a monitored person monitoring apparatus includes a behavior detection unit that detects a predetermined behavior of a monitored person that is a monitoring target, and a notification unit that notifies the predetermined behavior detected by the behavior detection unit to the outside
  • the predetermined behavior includes a fall
  • the behavior detection unit includes any of the above-described fall detection devices.
  • a fall detection device a fall detection method, and a monitored person monitoring device using the fall detection device can be provided.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Emergency Management (AREA)
  • Business, Economics & Management (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Alarm Systems (AREA)
  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Emergency Alarm Devices (AREA)
  • Image Analysis (AREA)

Abstract

Selon l'invention, dans un dispositif et un procédé de détection de chute, et un dispositif de surveillance d'une personne à surveiller : une image d'une cible de surveillance, c'est-à-dire d'une personne surveillée, est acquise, ladite image étant capturée à partir du-dessus de la personne surveillée ; une région candidate de partie tête, qui est un candidat d'une partie tête, est extraite de l'image ; des changements dans le temps d'une pluralité de paramètres prédéterminés se rapportant à la partie tête sont obtenus par rapport à la région candidate de la partie tête ; et des chutes de la personne surveillée sont détectées sur la base des changements dans le temps dans chacun de la pluralité de paramètres.
PCT/JP2016/061355 2015-05-11 2016-04-07 Dispositif de détection de chute, procédé de détection de chute, et dispositif de surveillance d'une personne à surveiller Ceased WO2016181731A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2016566827A JP6115689B1 (ja) 2015-05-11 2016-04-07 転倒検知装置および転倒検知方法ならびに被監視者監視装置

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2015-096461 2015-05-11
JP2015096461 2015-05-11
JP2015121198 2015-06-16
JP2015-121198 2015-06-16

Publications (1)

Publication Number Publication Date
WO2016181731A1 true WO2016181731A1 (fr) 2016-11-17

Family

ID=57247895

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/061355 Ceased WO2016181731A1 (fr) 2015-05-11 2016-04-07 Dispositif de détection de chute, procédé de détection de chute, et dispositif de surveillance d'une personne à surveiller

Country Status (2)

Country Link
JP (1) JP6115689B1 (fr)
WO (1) WO2016181731A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3553739A4 (fr) * 2016-12-06 2019-12-04 Konica Minolta, Inc. Système et procédé de reconnaissance d'image
CN113925391A (zh) * 2021-09-17 2022-01-14 美智纵横科技有限责任公司 一种基于清洁机器人的摔倒检测方法及装置、清洁机器人

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000253382A (ja) * 1999-02-25 2000-09-14 Matsushita Electric Works Ltd 転倒検知装置
JP2008052631A (ja) * 2006-08-28 2008-03-06 Matsushita Electric Works Ltd 異常検知システム
JP2012113367A (ja) * 2010-11-19 2012-06-14 Nikon Corp 誘導装置
JP2013073445A (ja) * 2011-09-28 2013-04-22 Jvc Kenwood Corp 危険検知装置及び危険検知方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000253382A (ja) * 1999-02-25 2000-09-14 Matsushita Electric Works Ltd 転倒検知装置
JP2008052631A (ja) * 2006-08-28 2008-03-06 Matsushita Electric Works Ltd 異常検知システム
JP2012113367A (ja) * 2010-11-19 2012-06-14 Nikon Corp 誘導装置
JP2013073445A (ja) * 2011-09-28 2013-04-22 Jvc Kenwood Corp 危険検知装置及び危険検知方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3553739A4 (fr) * 2016-12-06 2019-12-04 Konica Minolta, Inc. Système et procédé de reconnaissance d'image
US10846538B2 (en) 2016-12-06 2020-11-24 Konica Minolta, Inc. Image recognition system and image recognition method to estimate occurrence of an event
CN113925391A (zh) * 2021-09-17 2022-01-14 美智纵横科技有限责任公司 一种基于清洁机器人的摔倒检测方法及装置、清洁机器人

Also Published As

Publication number Publication date
JPWO2016181731A1 (ja) 2017-05-25
JP6115689B1 (ja) 2017-04-19

Similar Documents

Publication Publication Date Title
JP6720909B2 (ja) 行動検知装置、該方法および該プログラム、ならびに、被監視者監視装置
JP6150207B2 (ja) 監視システム
JP6142975B1 (ja) 被監視者監視装置および該方法ならびに被監視者監視システム
JP6852733B2 (ja) 生体監視装置及び生体監視方法
JP6048630B1 (ja) 行動検知装置および行動検知方法ならびに被監視者監視装置
JPWO2015037269A1 (ja) 被監視者監視装置および該方法ならびに被監視者監視システム
JP6784213B2 (ja) 被監視者監視システム
WO2019013257A1 (fr) Système d'aide à la surveillance et son procédé de commande, et programme
JP7592987B2 (ja) 制御プログラム、演算装置、制御システム、および制御方法
JPWO2017082037A1 (ja) 被監視者監視システムの中央処理装置および該方法ならびに前記被監視者監視システム
JPWO2017183602A1 (ja) 被監視者監視システムおよび被監視者監視方法
JP6115689B1 (ja) 転倒検知装置および転倒検知方法ならびに被監視者監視装置
JP2020052808A (ja) 見守り装置、見守りシステム、見守りプログラム、および見守り方法
JP7259540B2 (ja) 判定装置、判定装置の制御プログラム、および判定方法
JP2019197263A (ja) システム、およびシステムの制御方法
WO2020003953A1 (fr) Programme exécuté par ordinateur, dispositif de traitement d'informations et procédé exécuté par ordinateur
JP6737355B2 (ja) 頭部検出装置および頭部検出方法ならびに被監視者監視装置
JPWO2020003704A1 (ja) 制御プログラム、レポート出力方法、およびレポート出力装置
JP7435609B2 (ja) 画像処理システム、画像処理プログラム、および画像処理方法
JP6481537B2 (ja) 被監視者監視装置および被監視者監視方法
JP7352197B2 (ja) 移動行動異常判定装置および該方法ならびに被監視者監視支援システム
JP2021065617A (ja) 画像処理装置および画像処理プログラム
WO2016020247A1 (fr) Système d'identification d'un changement dans la vitesse de marche d'une personne
JP7338268B2 (ja) 見守りシステムおよび見守り方法
Srp et al. Possible techniques and issues in fall detection using asynchronous temporal-contrast sensors

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2016566827

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16792456

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16792456

Country of ref document: EP

Kind code of ref document: A1