EP3688663A1 - Procédé et dispositif de détection d'une zone exempte d'obstacle - Google Patents

Procédé et dispositif de détection d'une zone exempte d'obstacle

Info

Publication number
EP3688663A1
EP3688663A1 EP18792355.2A EP18792355A EP3688663A1 EP 3688663 A1 EP3688663 A1 EP 3688663A1 EP 18792355 A EP18792355 A EP 18792355A EP 3688663 A1 EP3688663 A1 EP 3688663A1
Authority
EP
European Patent Office
Prior art keywords
image
obstacle
pixel
pixels
boundary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP18792355.2A
Other languages
German (de)
English (en)
Inventor
Markus RAFFER
Kevin PAJESTKA
Friedrich Fraundorfer
David SCHINAGL
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tec-Innovation GmbH
Original Assignee
Tec-Innovation GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tec-Innovation GmbH filed Critical Tec-Innovation GmbH
Publication of EP3688663A1 publication Critical patent/EP3688663A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H3/06—Walking aids for blind persons
    • A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/3652—Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001—Teaching or communicating with blind persons
    • G09B21/003—Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
    • A61F9/08—Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks

Definitions

  • the invention relates to a method and a device for detecting the image of an obstacle-free region surrounding an image recording unit according to the preamble of patent claim 1.
  • AT 513881 A2 discloses a device for detecting an environment which can be used in rural areas or outdoors or for persons with impaired vision and reduced motor skills.
  • the device can be installed, for example, in a shoe or a cane or long stick and determined by means of a distance sensor, a distance to a potential obstacle. Depending on the distance determined, a warning is sent to the user via a feedback unit.
  • a disadvantage of the known device is that only a single distance value is specified, which determines how far an object is located away from the receiving unit, wherein the distance measurement is measured only along a single direction, starting from the receiving unit.
  • the object of the invention is therefore to improve the known from the prior art device for distance detection and to provide a method that ensures a safe movement of persons with limited vision and reduced motor skills in any environment.
  • the invention solves this problem in a method of the type mentioned, wherein a) with the image pickup unit at least one shot of a portion of the environment of the image pickup unit is created and based on this at least a recording is a digital image is created, with the features specified in claim 1. It is provided according to the invention
  • a boundary curve of the obstacle-free area in particular in the form of a train path, is determined as a curve on which there are pixels having a high limiting indicator value compared to pixels in their surroundings, e) that the image of the obstacle-free area by the Boundary curve and possibly further predetermined previously curves or lines, in particular the boundary edges of the digital image is set.
  • a path is selected as the limiting curve, which runs through a number of pixels of the digital image, wherein for a number of predetermined first parameters, in particular for each image column of the digital image or for a number of image columns each having a predetermined x-coordinate, one pixel each, in particular a pixel, is selected, through which the path runs.
  • an optimization method is carried out to determine the limiting curve in step d) for individual eligible boundary curves in each case for individual pixels on the respective boundary curve, in particular for a plurality of pixels respectively predetermined first parameters or x-coordinates, in each case a dimension value to be optimized is determined, which is composed of the following partial dimension values,
  • the value of the second parameter in particular the y-coordinate, or of a value derived therefrom, and / or
  • step d) an optimization method is carried out for determining the boundary curve in step d), in which for individual eligible boundary curves in each case those pixels on the respective boundary curve, in particular for a plurality of pixels each having predetermined first parameters or x-coordinates are selected
  • the boundary indicator value is calculated pixel-wise by means of a neural network, in particular for all pixels in the same way and independently of the remaining pixels, the neural network being the input value individual brightness or color values within a, in particular for each pixel the same predetermined environment around the respective pixel, in particular with a predetermined relative position to each pixel receives.
  • the neural network is created using a plurality of predetermined training images, with a training bounding curve being given for each of the training images and the neural network thereon is trained, for the respective training picture, in the In the area of the training boundary curve, the pixels to be supplied have a limit indicator value which differs from the limit indicator values determined in the rest of the image area.
  • Another advantage of the invention lies in the fact that, in contrast to known from the prior art method in the context of training only manually determines where the boundaries of the walk-in area are or not.
  • the Annotationsaufwand falls significantly lower than in known methods, which with the pixels each other information such as, for example, that it is a pixel, which z. B. represents a part of a window or a wall is, deposit.
  • the actual size and shape of the imaged obstacle-free area is determined within the surroundings of the image recording unit,
  • parameters of the image acquisition unit in particular resolution and focal length,
  • the at least one receptacle is mounted with a walking stick fixed to a person or to an animal, in particular a shoe, glove, harness or walker arranged, image pickup unit is added.
  • the digital images may be recorded with an image recording unit mounted in a shoe, the height of the image recording unit being determined above the ground when the shoe is resting on the floor and
  • the actual size and shape of the imaged obstacle-free area is determined according to the invention on the assumption that the image-recording unit is at the determined height.
  • a particularly safe locomotion can be made possible for persons with limited vision and reduced motor skills if, in addition to the limit indicator value, an obstacle indicator is also determined in step b) indicating the type of obstacle the obstacle-free area is limited to, and
  • the obstacle indicator is determined for individual pixels on the limiting curve and preferably the person is displayed.
  • a first distance of the limiting curve or at least one point on the boundary curve of the obstacle-free area is determined by the image recording unit
  • a detection area partially overlapping with the recording area of the image recording unit, - That the second distance and the first distance of the obstacle-free area are compared with each other and another distance measurement is created using the first distance and the second distance, preferably in each case deviations in the respective smaller distance is considered as another distance measurement.
  • the invention further relates to a method which ensures that a person with limited vision and reduced motor skills can move independently in an environment, since the person is warned of obstacles.
  • a distance value corresponding to the size of the obstacle-free area is determined or predetermined by one person, and then depending on this distance value at least one body location, in particular in at least one shoe, the person at least one of several sequentially in the direction of the more sensitive Actuator elements, in particular vibration elements, is activated, wherein the selection of the actuator elements is made depending on the distance value and wherein an actuator is selected in a sensitive body region, in particular the closer to the toe is selected, the smaller the Distance value is.
  • the invention further relates to an apparatus for detecting the image of an obstacle-free area surrounding an image recording unit, comprising an image recording unit and a processing unit connected downstream of the image recording unit.
  • the image recording unit is designed to produce at least one image of a subregion of the surroundings of the image recording unit and to transmit the image to the processing unit
  • the processing unit is designed to
  • a limiting indicator value as a function of the pixel-by-pixel predetermined image data of the digital image within an environment of the respective pixel, which indicates with which probability the image of the boundary of an obstacle-free region is located within a given pixel environment
  • the processing unit is designed to
  • predetermined first parameters for a number of predetermined first parameters, in particular for each image column of the digital image or for a number of image columns each having a predetermined x-coordinate, in each case one pixel, in particular a pixel, through which the path extends.
  • the processing unit is adapted to perform an optimization method for determining the boundary curve, wherein the processing unit is adapted for individual eligible boundary curves in each case for individual pixels on the respective Limit curve, in particular for a plurality of pixels each having predetermined first parameters or x-coordinates, each to determine a to be optimized measure value, which is composed of the following Sectionurgilose sentence:
  • the value of the second parameter in particular the y-coordinate, or of a value derived therefrom, and / or
  • the processing unit is adapted to perform the determination of the boundary curve, an optimization method, wherein the processing unit is adapted for each eligible boundary curves in each case those pixels on the respective boundary curve, in particular for a plurality of pixels each having predetermined first parameters or x coordinates, select
  • the processing unit is designed to
  • the processing unit is configured to
  • the processing unit is designed to determine the actual size and shape of the imaged obstacle-free area within the environment of the image recording unit,
  • parameters of the image acquisition unit in particular resolution and focal length,
  • the image pickup unit is fixedly mounted on a person or on an animal, especially in a shoe, glove, harness, helmet walking stick or Walker is arranged.
  • the processing unit is designed to monitor the position and orientation of the image acquisition unit and to carry out further processing steps only
  • the image acquisition unit is designed to create images only then
  • the image acquisition unit is mounted in a shoe, wherein the processing unit is adapted to determine the height of the image acquisition unit above the ground when the shoe is on the ground stands and
  • the processing unit is designed to
  • a device comprises an ultrasound sensor arranged in the region of the image recording unit, wherein the ultrasound sensor
  • an inventive Device comprises a feedback unit, wherein the feedback unit for, in particular wireless, data communication with the processing unit and for displaying the determined distance measured value and / or the determined obstacle indicator value and / or the actual size and shape of the imaged obstacle-free area for the person is formed ,
  • the invention further relates to a device which allows a person with impaired vision and motor skills to move independently in an unknown environment, as the device warns the person of obstacles.
  • the feedback unit is designed as a shoe with actuator elements
  • the processing unit is designed to
  • actuator elements in particular vibration elements, to make the selection of the actuator depending on the distance value, and to select an actuator in a more sensitive body area, in particular all the more closer to the toe, the smaller the distance value.
  • Fig. 1 shows an environment around an image pickup unit.
  • Fig. 2 shows a distribution of the boundary indicator values in an image of the environment.
  • Fig. 2a shows a section of pixels from Fig. 2.
  • Fig. 3 shows schematically the determined boundary curve in an image of the environment.
  • Fig. 4 shows a shoe with actuator elements.
  • Fig. 5 shows the column-by-column calculation of potentials by means of the Viterbi algorithm for the example in Fig. 2a.
  • Fig. 6 shows the calculated potentials for the example in Fig. 2a.
  • Fig. 7 shows the determined course of the limiting curve.
  • Fig. 1 shows an environment 10 in the form of a room in which a person with limited vision wants to move.
  • a window is arranged, wherein the window has a lower edge 12 of the window. Since the person's space is not known, she is dependent on aids that warn her of obstacles, such as two walls in the direction of the person's movement, and recognize, for example, the accessible obstacle-free area.
  • the person therefore wears a tool, such as a shoe 20, in which an image pickup unit 1 is arranged.
  • the image recording unit 1 mounted in the shoe 20 is a watertight, robust camera, which is frequently used, for example, in sports. Such cameras are small enough that they can also fit in a shoe 20.
  • the image recording unit 1 is arranged on the toe in the region of the toes of the wearer of the shoe 20 and aligned in the direction of movement of the wearer of the shoe 20.
  • a processing unit 4 an energy store 3 and an ultrasonic sensor 2 are arranged in the shoe 20.
  • the image recording unit 1 is designed to make recordings in the form of digital images available and to forward digital images D produced by it to the processing unit 4.
  • the processing unit 4 is designed to process the digital images D transmitted to it and to carry out the inventive method steps described in more detail below.
  • a plurality of ultrasonic sensors 2 can also be arranged in the shoe 20.
  • Other devices required for carrying out a method according to the invention or for transmitting the detected obstacles, such as a feedback unit, may likewise be arranged in the shoe 20, or else carried by the person on the body or otherwise carried along with the aid independently of the aids.
  • the image recording unit 1 In order to capture the image of an obstacle-free region which surrounds the image recording unit 1, at least one image of a subregion of the space is first created with the image recording unit 1. There is also the possibility to create several recordings and combine them into a single digital image D, in particular to select one from several different recordings and to create the digital image D based on this shot. Likewise, the digital image D can be created as a result of different intermediate processing steps such as in particular filtering, sharpening, color corrections etc from one or more recordings. Alternatively, these steps may be performed in the processing unit 4 described below.
  • the thus created digital image D is forwarded from the image acquisition unit 1 to the processing unit 4 and examined by the processing unit 4 for the presence of possible limitations B of the obstacle-free region.
  • This boundary B of the obstacle-free area to be found in the digital image D corresponds in the space shown in FIG. 1 to the wall edge 11.
  • a limiting indicator value Bl is determined by the processing unit 4 for individual pixels P, in particular for all pixels, of the digital image D.
  • an environment around each pixel P is defined for the pixel-wise predetermined image data of the digital image D, within which the boundary indicator value Bl for the respective pixel P is determined.
  • the limit indicator value Bl indicates the probability with which the image of the boundary B of an obstacle-free region is located within a given pixel environment.
  • a neural network is used to determine the boundary indicator value Bl in the embodiment shown.
  • the neural network calculates pixel by pixel, in particular for all pixels P in the same way, the boundary indicator value Bl.
  • the individual brightness or color values within an environment around the respective pixel P are transmitted to the neural network.
  • the environment around the respective pixel P can advantageously have a square shape with an edge length of 200 pixels, in the center of which the respective pixel P is located.
  • other environments may be used as input values for the neural network.
  • a probability distribution map is computed by means of the neural network, which includes a boundary indicator value Bl for each pixel P.
  • the size of the probability distribution map essentially corresponds to the size of the digital image, in particular except for border areas.
  • the Limiting indicator value Bl indicates a probability as to whether the searched boundary B of the obstacle-free area is imaged in the respective pixel P.
  • a multiplicity of predefined training images are used in a training phase, a training boundary curve being predetermined for each of the training images and the neural network being trained for the respective training image in the area adjacent to the training boundary curve Pixels to provide a Begrenzungsindikatorwert Bl, which differs from the rest of the image area determined boundary indicator values Bl.
  • training images are used that include both digital images D of indoor and outdoor environments 10, in which various types of indoor environment Limit B of an obstacle-free area are shown.
  • the illustrated boundaries B may be, for example, steps, sidewalk edges, walls or other boundaries B.
  • digital images D of outdoor environments 10 are also used in different weather conditions.
  • additional image transformations such as rotation, horizontal reflection, contrast change, brightness change and blur are applied.
  • FIG. 2 shows a schematic exemplary representation with a significantly reduced number of pixels for boundary value indicator values Bl determined by the neural network, in each case for the surroundings around pixels P of a section of an image of the space shown in FIG.
  • white areas correspond to pixels P for which a small limit indicator value Bl has been determined, ie that they contain the image of a boundary of the obstacle-free area only with low probability.
  • Long-hatched areas correspond to pixels P for which a higher limit indicator value Bl has been determined, so that these pixels P are more likely to contain the image of a boundary B.
  • the highest boundary indicator value Bl has been determined in FIG. 2 for longitudinally and crosshatched regions, so that these pixels P are most likely to image a boundary B.
  • a coordinate parameterization with two parameters is specified for the digital image, which corresponds in particular to the pixel coordinates, for example in the form of an x and y coordinate.
  • the x-coordinate indicates the column number and increases from the left edge of the picture to the right
  • the y-coordinate indicates the line number and decreases from the upper edge of the picture.
  • this parameterization is not mandatory.
  • another coordinate system can be selected in which, starting from a defined camera point, lines of the same distance to the camera point in the form of layer lines in the digital image D are predefined.
  • a limiting curve BK of the obstacle-free region is determined by the processing unit 4 as a curve.
  • the limiting curve BK is selected such that the image pixels are located on it which have a high limiting indicator value Bl compared to pixels in their surroundings. This means that the limiting curve BK is not necessarily selected to run through individual pixels P, but can also be selected to run through pixels interpolated between a plurality of pixels P.
  • the image of the obstacle-free area which is safely accessible to the person, is thus determined by the limiting curve BK and optionally further predetermined curves or straight lines, in particular the boundary edges of the digital image D.
  • the path train is selected such that it runs through a number of pixels P of the digital image D.
  • a pixel, in particular a pixel P is selected for a number of predefined first parameters, in particular for each image column of the digital image D or for a number of image columns each having a predetermined x coordinate, through which the path extends.
  • a pixel in particular a pixel P
  • predefined first parameters in particular for each image column of the digital image D or for a number of image columns each having a predetermined x coordinate, through which the path extends.
  • an optimization method is applied by the processing unit 4 in the exemplary embodiment shown.
  • a respective dimension value to be optimized is determined in each case for individual pixels on the respective limiting curve BK for individual individual boundary curves BK, in particular for a large number of pixels each having predetermined first parameters or x-coordinates.
  • the dimension value consists of the following partial dimension values:
  • the value of the second parameter in particular the y-coordinate, or of a value derived therefrom, and / or
  • obstacles closer to or closer to the person may be rated higher than those further from the person and in the background.
  • possible Limits B which are in the lower region of the digital image D, or a solid boundary B without jumps, are preferred.
  • Viterbi algorithm For the concrete determination of an optimal boundary curve BK with respect to the aforementioned criteria, it is possible, for example, to use the Viterbi algorithm, which is described, for example, in Viterbi, A., Error bounds for convolutional codes and asymptotically optimum decoding algorithm. In: IEEE Transactions on Information Theory. Vol. 13, No. 2, 1967, pp. 260-269.
  • the potential ⁇ to be maximized by the Viterbi algorithm for determining the profile of the limiting curve BK in the digital image D is composed of:
  • the output of the neural network ie the determined boundary indicator values Bl or the probabilities that a pixel P contains a boundary B of an obstacle-free area, weighted with a weighting factor vl n,
  • Value is weighted with a weighting factor.
  • x, or y respectively denotes the x-coordinate or the y-coordinate of the ith point P, on the limiting curve BK.
  • a bounding curve with the points Pi or coordinates xi, yi is searched for, for which the following sum expression ⁇ is maximal for a limiting curve:
  • the Viterbi algorithm is used to determine the boundary curve BK on the entire image.
  • the limit indicator values Bl determined by the neural network are compiled in a tabular form.
  • Table 1 Determined bound indicator values Bl of the selected pixels.
  • the number of columns and rows may range from 1000 to 5000.
  • the weighting factors or functions listed in Table 2 are specified:
  • Table 2 Weighting factors and functions used in the example.
  • the determined potential is in each case the magnitude of the difference of the y-coordinate values of the respective pixels
  • the last column looks for the pixel with the highest potential. Starting from this pixel, the pixel which was used in the calculation of the pixel of the potential is now searched backwards, that is to say, according to the decreasing column number.
  • the pixels thus determined are used in the framework of the Viterbi algorithm for determining the curve profile of the limiting curve BK.
  • FIG. 3 schematically shows the result of the application of such an optimization method for determining the limiting curve BK on the basis of the boundary indicator values Bl determined by the neural network for the pixels P of the section of the image of the environment 10 or of the space shown in FIG.
  • the path length was selected to pass through a number of pixels P of the digital image D.
  • This procedure is also particularly advantageous, for example, if the user of the invention uses them when climbing stairs, since in this case always the level closest to him is first identified so that he is effectively routed from one level to the next.
  • the actual size and shape of the imaged obstacle-free area within the environment 10 of the image acquisition unit 1 is then determined by the processing unit 4 to determine to the user of the method in which area he is safely traveling can, without encountering obstacles.
  • the created digital image D corresponds to the size of the imaged obstacle-free area the freely accessible space that extends between the image pickup unit 1 and the wall edge 1 1 in Fig. 1.
  • the position of the obstacle-free region in the digital image D In order to determine the shape and size of the obstacle-free region, the position of the obstacle-free region in the digital image D, the relative position, in particular height and orientation, of the image recording unit 1 with respect to the environment 10 and optionally parameters of the image recording unit 1, in particular resolution and focal length, are used. Furthermore, in order to determine the shape and size of the obstacle-free region, it is assumed that the obstacle-free region detected in the digital image D lies on a predetermined three-dimensional surface, in particular on a plane.
  • the position and orientation of the image recording unit 1 is advantageously monitored, so that only images are produced by the image recording unit 1 or method steps according to the invention are carried out when the image recording unit 1 is in a predetermined position.
  • a predetermined position for example, both a position can be defined in which both the orientation of the horizontal image axis of the image pickup unit 1, and the viewing direction of the image pickup unit 1 are aligned approximately horizontally, in particular with an angular deviation of at most 30 ° about a horizontal axis, or Also, a layer in which the image pickup unit 1 containing shoe 20 is completely placed on the ground.
  • the height of the image pickup unit 1 mounted in the shoe 20 above the ground occupied by the image pickup unit 1 when the shoe 20 stands on the ground is determined.
  • the actual size and shape of the imaged obstacle-free area is determined only on condition that the image acquisition unit 1 is at the determined height.
  • the height can be used as a further criterion for an energy-saving, efficient procedure.
  • the person wearing the shoe 20 with built-in image acquisition unit 1 rests on a lying surface, the person's feet and thus their shoes are no longer in a seated position and the viewing direction of the image pickup unit 1 is compared to the predetermined one Position rotates.
  • a creation of digital images D is not required because the person does not want to move and the image acquisition unit 1 does not need to be powered by the energy storage device 3.
  • the shoe 20 with the image pickup unit 1 mounted therein has no contact with the ground and thus is not at the determined height, then no digital images D are advantageously produced by the image pickup unit 1. This is the case, for example, when the person wearing the shoe 20 raises his or her feet while walking, or when the person is resting with his legs up.
  • the image acquisition unit 1 does not permanently consume voltage for the production of digital images D, but only when the image acquisition unit 1 is in a position in which the implementation of the method according to the invention is provided, i. that the person wearing the shoe 20 actually requires assistance in locomotion. In this case, it is also ensured at the same time that the size or shape of the obstacle-free area or the distance to potential obstacles can be reliably determined.
  • a first distance A1 of the boundary curve BK or at least one point on the boundary curve BK of the obstacle-free region can be determined by the image acquisition unit 1. From the known position of the image pickup unit 1 and assuming that the person wearing the shoe 20 with integrated image pickup unit 1 is standing on a plane, the distance from the two-dimensional image representing the digital image D taken by the image pickup unit 1 can be obtained become an obstacle.
  • a second distance A2 is determined by measuring the distance by means of ultrasound.
  • an ultrasonic sensor 2 used for this purpose is arranged in the shoe 20 near the image recording unit 1 and aligned in the same direction as the image recording unit 1, so that the recording area of the image recording unit 1 partially overlaps with the detection area of the ultrasonic sensor 2.
  • the first distance A1 and the second distance A2 of the obstacle-free area are compared with each other, and another distance measurement value A is created using the first distance A1 and the second distance A2.
  • the smaller distance is preferably regarded as a further distance measurement value A.
  • such determined distance lines are located in determined distances of 10 cm, 50 cm, 100 cm and 150 cm to the image pickup unit 1.
  • the processing unit 4 or the neural network NN is designed to determine the obstacle indicator H1 for individual pixels on the boundary curve BK and preferably to represent the person. In this way, the person wearing the shoe 20 can be notified, which obstacles or objects such. As sidewalk edges, walls, people, animals or the like, are in their direction of movement, so that they get the opportunity to respond accordingly.
  • the at least one receptacle is accommodated with an image recording unit 1 mounted permanently on a person or on an animal, in particular in a glove, belt, helmet, walking stick or walking aid. It is thus possible, for example, for frail persons who are dependent on walking aids, such as, for example, crutches or walking sticks, to mount the image recording unit 1 in crutches.
  • shoes 20 or collars for assistance dogs can be equipped with such an image recording unit 1, so that persons with impaired vision and reduced motor skills can move particularly safe in an unknown environment, as they assistance from their assistance dog and additionally inventively determined hints and warnings receive.
  • the results obtained are perceptibly displayed to the user of the invention.
  • the processing unit 4 is designed to forward the determined results, for example, to a feedback unit.
  • the user may, for example, wear the feedback unit on the body, so that the results obtained can be transmitted to him in a manner that can be perceived by him.
  • the feedback unit may be, for example, headphones used to carry the person upside down.
  • the processing unit 4 is designed to transmit the determined results on the shape and size of the accessible obstacle-free area to the feedback unit and to control the feedback unit such that, for example, as the distance to the boundary B of the obstacle-free area decreases, the sound sequences are louder or higher As soon as the sound is heard, it will be delivered at a speeding rhythm, so that the person can see that it is approaching an obstacle.
  • the 4 shows a shoe 20 with an image pickup unit 1, an ultrasound sensor 2, an energy store 3, a processing unit 4, and actuator elements 5 mounted therein.
  • the image pickup unit 1 and the ultrasound sensor 2 are installed in the toe, so that both are aligned in the same direction
  • the energy storage 3 and the processing unit 4 are arranged in the embodiment in paragraph 21 of the shoe 20.
  • Six actuator elements 5 are arranged in the region of the toes or the footwear of the wearer of the shoe 20 in the shoe sole region 22, wherein the actuator elements 5 are arranged such that they form a line extending from the ball of the foot in the direction of the toe.
  • the actuator elements 5 are designed as vibrating elements which vibrate when they are actuated, for example, by the processing unit 4.
  • the actuator elements may, for example, also be designed in the form of a pin and, when activated, press upwards into the foot.
  • a distance corresponding to the size of an obstacle-free area around the image pickup unit 1 arranged in the shoe tip, or a person who carries the shoe 20, is determined by the processing unit 4.
  • the processing unit 4 determines the distance value.
  • at least one digital image D and / or the ultrasonic sensor 2 created by the image recording unit 1 is used.
  • the selection of the actuator 5 is made depending on the distance value. As the distance to an obstacle becomes smaller, actuator elements 5 are selected in an all the more sensitive body region, in particular the closer to the tip of the toe.
  • the actuator element 5 is actuated in the area of the person's footing. If the distance to the obstacle becomes smaller, starting from the actuator element 5 in the area of the football, the actuator element 5 located closer to the toe tip is actuated until finally, at a short distance to the obstacle, the actuator element in FIG Area of the person's toes is chosen. This signals to the person that he / she is in danger of encountering an obstacle should it continue to move in the direction it was headed.
  • the invention can not only be used to facilitate the movement in everyday life for persons with limited vision and / or reduced motor skills.
  • the invention can also be used to detect an obstacle-free area around an image pickup unit 1 arranged, for example, in the area of the bumper of a vehicle, and to determine the shape and size of the freely passable area in the direction of travel of the vehicle. This may be of interest, for example, in the area of autonomous driving in order to determine the trafficable environment, or to detect the distance to a preceding vehicle and, for example, to regulate the speed of the autonomously driving vehicle, depending on the distance to the vehicle in front.

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Automation & Control Theory (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Pain & Pain Management (AREA)
  • Veterinary Medicine (AREA)
  • Rehabilitation Therapy (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé de détection de la représentation d'une zone entourant une unité de prise d'images (1), selon lequel a) au moins une image d'une zone partielle de l'environnement (10) de l'unité de prise d'image (1) est créée avec l'unité de prise d'image (1) et une image numérique (D) est créée sur la base de cette ou de ces images. Le procédé est caractérisé : b) en ce que l'image numérique (D) ainsi créée est examinée quant à la présence de possibles délimitations (B) d'une zone exempte d'obstacle, une valeur d'indicateur de délimitation (BI) étant déterminée pour chacun des pixels individuels (P) de l'image numérique (D) en fonction de données d'image, prédéfinies au niveau des pixels, de l'image numérique (D) à l'intérieur d'un environnement du pixel concerné (P), laquelle valeur indique la probabilité avec laquelle la reproduction de la délimitation (B) d'une zone exempte d'obstacle se trouve à l'intérieur d'un environnement de pixel prédéfini, c) en ce que, pour l'image numérique (D), un paramétrage de coordonnées est prédéfini avec deux paramètres qui correspondent en particulier aux coordonnées de pixel, d) en ce qu'une courbe de délimitation (BK) de la zone exempte d'obstacle, en particulier sous forme d'un tronçon de chaussée, est déterminée en tant que virage sur lequel se trouvent des points d'image qui - en comparaison à des points d'image dans leur environnement - présentent une grande valeur d'indicateur de délimitation (BI), e) en ce que la reproduction de la zone exempte d'obstacle est déterminée par la courbe de délimitation (BK) et le cas échéant par d'autres courbes ou droites prédéfinies auparavant, en particulier par les bords de délimitation de l'image numérique (D).
EP18792355.2A 2017-09-26 2018-09-24 Procédé et dispositif de détection d'une zone exempte d'obstacle Pending EP3688663A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ATA50823/2017A AT520579B1 (de) 2017-09-26 2017-09-26 Verfahren und Vorrichtung zur Detektion eines hindernisfreien Bereichs
PCT/AT2018/060223 WO2019060936A1 (fr) 2017-09-26 2018-09-24 Procédé et dispositif de détection d'une zone exempte d'obstacle

Publications (1)

Publication Number Publication Date
EP3688663A1 true EP3688663A1 (fr) 2020-08-05

Family

ID=63962992

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18792355.2A Pending EP3688663A1 (fr) 2017-09-26 2018-09-24 Procédé et dispositif de détection d'une zone exempte d'obstacle

Country Status (4)

Country Link
US (1) US11205074B2 (fr)
EP (1) EP3688663A1 (fr)
AT (1) AT520579B1 (fr)
WO (1) WO2019060936A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200036678A (ko) * 2018-09-20 2020-04-07 삼성전자주식회사 청소 로봇 및 그의 태스크 수행 방법
JP7221549B2 (ja) * 2018-12-17 2023-02-14 学校法人千葉工業大学 情報処理装置および移動ロボット
US20220417704A1 (en) * 2021-06-23 2022-12-29 Aarna RAGHAVAPUDI Visual assistance system
US11867514B2 (en) * 2021-09-24 2024-01-09 Telenav, Inc. Navigation system with independent positioning mechanism and method of operation thereof

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100127007A (ko) 2009-05-25 2010-12-03 이용근 시각 장애인을 위한 보행 안내 장치 및 방법
KR101435546B1 (ko) * 2012-12-26 2014-09-16 건국대학교 산학협력단 카메라와 촉각형 인터페이스를 이용한 시각 장애인 보조 시스템
AT513882A2 (de) 2013-01-08 2014-08-15 Pajestka Kevin Vorrichtung zur Detektierung einer Umgebung
US9915545B2 (en) * 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
WO2015143203A1 (fr) * 2014-03-19 2015-09-24 Schepens Eye Research Institute Systèmes actifs d'imagerie confocale et procédés pour prothèses visuelles
US9811756B2 (en) * 2015-02-23 2017-11-07 Mitsubishi Electric Research Laboratories, Inc. Method for labeling images of street scenes

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Canny edge detector", 23 May 2017 (2017-05-23), XP055830287, Retrieved from the Internet <URL:https://en.wikipedia.org/w/index.php?title=Canny_edge_detector&oldid=781894186> [retrieved on 20210805] *
PARAMONOV VLADIMIR ET AL: "Depth Camera Based on Color-Coded Aperture", 2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 1 July 2016 (2016-07-01), pages 910 - 918, XP055971326, ISBN: 978-1-5090-1437-8, Retrieved from the Internet <URL:https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber=7789608&ref=aHR0cHM6Ly9pZWVleHBsb3JlLmllZWUub3JnL2RvY3VtZW50Lzc3ODk2MDg=> [retrieved on 20221012], DOI: 10.1109/CVPRW.2016.118 *
See also references of WO2019060936A1 *
YOSUKE BANDO ET AL: "Extracting depth and matte using a color-filtered aperture", ACM SIGGRAPH ASIA 2008 PAPERS ON, SIGGRAPH ASIA '08, ACM PRESS, NEW YORK, NEW YORK, USA, 1 December 2008 (2008-12-01), pages 1 - 9, XP058165771, ISBN: 978-1-4503-1831-0, [retrieved on 20221012], DOI: 10.1145/1457515.1409087 *
YU ZHIDING ET AL: "CASENet: Deep Category-Aware Semantic Edge Detection", 2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE COMPUTER SOCIETY, US, 21 July 2017 (2017-07-21), pages 1761 - 1770, XP033249517, ISSN: 1063-6919, [retrieved on 20171106], DOI: 10.1109/CVPR.2017.191 *

Also Published As

Publication number Publication date
US11205074B2 (en) 2021-12-21
US20200293776A1 (en) 2020-09-17
WO2019060936A1 (fr) 2019-04-04
AT520579B1 (de) 2022-07-15
AT520579A1 (de) 2019-05-15

Similar Documents

Publication Publication Date Title
DE112015002934B4 (de) Vorrichtung zum Erfassen eines Fahrunvermögenzustands eines Fahrers
DE69824225T2 (de) Das Bild einer Hand verwendende Hinweisvorrichtung
DE112015002948B4 (de) Vorrichtung zum erfassen eines fahrunvermögenzustands eines fahrers
EP3765927B1 (fr) Procédé servant à générer un jeu de données d&#39;entraînement servant à entraîner un module à intelligence artificielle pour un dispositif de commande d&#39;un véhicule
EP3299996B1 (fr) Machines agricoles dotées d&#39;un système de traitement d&#39;images
EP2382609B1 (fr) Dispositif permettant de déterminer une probabilité de collision d&#39;un véhicule avec un être vivant.
DE10301468B4 (de) Vorrichtung zur Beobachtung der Umgebung eines Fahrzeugs
EP3688663A1 (fr) Procédé et dispositif de détection d&#39;une zone exempte d&#39;obstacle
DE102007019491B4 (de) Fahrzeugumgebungs-Überwachungsvorrichtung, Fahrzeug, Fahrzeugumgebungs-Überwachungsverfahren und Fahrzeugumgebungs-Überwachungsprogramm
WO2009019214A2 (fr) Procédé de détermination d&#39;une zone probable de maintien et de déplacement d&#39;un organisme vivant
EP3803526A1 (fr) Procédé d&#39;apprentissage pour un proceed d&#39;assistance à la conduite, procédé d&#39;assistance à la conduite, appareil de commande et véhicule comprenant cet appareil de commande
EP3299995A1 (fr) Système d&#39;analyse d&#39;images pour machines agricoles
DE112011105445B4 (de) Rote-Augen-Bestimmungsvorrichtung
WO2014029738A1 (fr) Procédé pour compléter une information associée à un objet et procédé de sélection d&#39;objets dans un environnement d&#39;un véhicule
EP2766877A1 (fr) Procédé pour représenter l&#39;environnement d&#39;un véhicule
WO2013029722A2 (fr) Procédé de représentation de l&#39;environnement
EP4064819B1 (fr) Procédé de traitement de plantes dans un champ, de génération de données d&#39;apprentissage et d&#39;entraînement d&#39;un réseau de neurones
DE102017123226A1 (de) Verfahren zum Bestimmen einer kritischen Höhe eines vorausliegenden Streckenabschnitts für ein Fahrzeug, das ein Zugfahrzeug und einen Anhänger umfasst
DE112018007056T5 (de) Fahrunterstützungsvorrichtung, Fahrunterstützungsverfahren und Fahrunterstützungsprogramm
EP3663881B1 (fr) Procédé de commande d&#39;un véhicule autonome en fonction des vecteurs de mouvement estimés
DE102020110417A1 (de) Verfahren und System zur Assistenz eines Fahrers
DE102015211871A1 (de) Objekterkennungsvorrichtung
DE102017217872A1 (de) Verbesserte Ansicht eines Kraftfahrzeugumfelds
DE112017006982T5 (de) Bedienungseignungsbeurteilungsvorrichtung,bedienungseignungsbeurteilungsverfahren undbedienungseignungsbeurteilungsprogramm
DE112018007060T5 (de) Fahrtunterstützungsvorrichtung, Fahrtunterstützungsverfahren und Fahrtunterstützungsprogramm

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200416

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210810

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G06K0009000000

Ipc: G06V0020100000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: A61H 3/06 20060101ALI20230208BHEP

Ipc: G09B 21/00 20060101ALI20230208BHEP

Ipc: G06V 20/10 20220101AFI20230208BHEP

INTG Intention to grant announced

Effective date: 20230310

19U Interruption of proceedings before grant

Effective date: 20230616