WO2020022021A1 - Dispositif de calcul de distance - Google Patents

Dispositif de calcul de distance Download PDF

Info

Publication number
WO2020022021A1
WO2020022021A1 PCT/JP2019/026647 JP2019026647W WO2020022021A1 WO 2020022021 A1 WO2020022021 A1 WO 2020022021A1 JP 2019026647 W JP2019026647 W JP 2019026647W WO 2020022021 A1 WO2020022021 A1 WO 2020022021A1
Authority
WO
WIPO (PCT)
Prior art keywords
distance
information
distance information
monocular
calculation device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/026647
Other languages
English (en)
Japanese (ja)
Inventor
稲田　圭介
裕介 内田
野中　進一
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Astemo Ltd
Original Assignee
Hitachi Automotive Systems Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Automotive Systems Ltd filed Critical Hitachi Automotive Systems Ltd
Priority to CN201980049441.1A priority Critical patent/CN112513571B/zh
Publication of WO2020022021A1 publication Critical patent/WO2020022021A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/87—Combinations of systems using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes

Definitions

  • the present invention relates to a distance calculation device.
  • ⁇ There is known a technique of calculating the distance between a camera and an observation target using a stereo camera and performing recognition processing of the observation target. There is also known a method of realizing high-precision and wide-field monitoring around a vehicle by shifting two cameras constituting a stereo camera and generating a compound eye region in front and a monocular region in left and right sides.
  • Patent Literature 1 is an object detection device that detects an object present on a reference plane, and includes a parallax calculation unit that calculates parallax between images captured by a plurality of imaging devices, based on the parallax, A region calculation unit that calculates a common region of the plurality of images and a non-common region other than the common region; and a rectangle in the common region, and the uniformity of the parallax of each point included in the rectangle.
  • a first generation unit that generates a rectangle larger than a predetermined threshold as a candidate rectangle that is a candidate for a rectangle for detecting the object, and a second generation unit that generates a rectangle in the non-common area as the candidate rectangle.
  • An object detection device is disclosed, comprising: a generation unit; and a determination unit that determines whether the generated candidate rectangle includes an object to be detected.
  • a distance calculation device uses a sensor output, a high-precision distance information generation unit that generates high-precision distance information that is high-precision distance information, and a sensor output.
  • a low-accuracy distance information generation unit that generates low-accuracy distance information with lower accuracy than the high-accuracy distance information, and a distance correction unit that corrects the low-accuracy distance information using the high-accuracy distance information.
  • FIG. 3A shows the first captured image 100
  • FIG. 3B shows the second captured image 101
  • FIG. 3C shows the group.
  • Configuration diagram of the correction unit 20 Configuration diagram of group information generation unit 30
  • Flow chart showing the operation of the distance calculation device 6 Configuration diagram of the correction unit 20 in Modification Example 3
  • eta weighting coefficient
  • FIG. 10 is a diagram illustrating an example of a process of correcting the monocular distance information 102 using the correction coefficient ⁇ illustrated in FIG. 9.
  • FIG. 10A shows a state before correction
  • FIG. 10B shows a state after correction.
  • FIG. 12A shows a state before correction
  • FIG. 12B shows a state after correction.
  • FIG. 15A shows a state before correction
  • FIG. 15B shows a state after correction.
  • Configuration diagram of group information generation section 30 in modification 21 Diagram showing correspondence between subject types and correction methods
  • Schematic diagram of distance calculation system S according to the second embodiment Configuration diagram of distance calculation device 6 according to the second embodiment Schematic diagram of distance calculation system S according to the third embodiment Configuration diagram of distance calculation device 6 according to the third embodiment
  • FIG. 1 is a schematic diagram of a distance calculation system S including a distance calculation device 6.
  • the distance calculation system S is mounted on the vehicle 1 and includes a first imaging unit 2 and a second imaging unit 3.
  • the first imaging unit 2 and the second imaging unit 3 are arranged side by side in the forward direction of the vehicle 1.
  • the viewing angles of the first imaging unit 2 and the second imaging unit 3 are known, and some fields of view overlap as shown in FIG. In FIG. 1, the fields indicated by hatching have overlapping visual fields.
  • FIG. 2 is a configuration diagram of the vehicle control system S according to the first embodiment.
  • the vehicle control system S includes a first imaging unit 2, a second imaging unit 3, a distance calculation device 6, a recognition processing unit 7, and a vehicle control unit 8.
  • the distance calculation device 6 includes a monocular distance information generation unit 10, a compound eye distance information generation unit 11, and a monocular distance information correction unit 12.
  • the monocular distance information correction unit 12 includes a correction unit 20 and a group information generation unit 30.
  • the first imaging unit 2 outputs a first captured image 100, which is an image obtained by capturing, to the distance calculation device 6.
  • the second imaging unit 3 outputs a second captured image 101, which is an image obtained by capturing, to the distance calculation device 6.
  • the distance calculation device 6 may be a single ECU (Electronic Control Unit), or may be a calculation device further provided with other functions.
  • the distance calculation device 6 may be configured integrally with the first imaging unit 2 and the second imaging unit 3.
  • the distance calculation device 6, the recognition processing unit 7, and the vehicle control unit 8 may be realized by an integrated device.
  • the recognition processing unit 7 and the vehicle control unit 8 may be realized by an integrated electronic control device, or may be realized by different electronic control devices.
  • the monocular distance information generation unit 10 generates the monocular distance information 102 using each of the first captured image 100 and the second captured image 101.
  • the monocular distance information generation unit 10 calculates the distance from the monocular image by a known method. That is, the monocular distance information generation unit 10 does not use the first captured image 100 and the second captured image 101 in combination in generating the monocular distance information 102.
  • the monocular distance information generation unit 10 combines the distance information calculated using the first captured image 100 and the distance information calculated using the second captured image 101 into monocular distance information 102.
  • the monocular distance information generation unit 10 assumes, for example, that the imaging is performed in a horizontal direction on a plane, and calculates the distance as the upper part of the captured image is farther. However, the monocular distance information generation unit 10 calculates a distance for each distance information block composed of one or a plurality of pixels.
  • the distance information block may be, for example, 16 pixels composed of 4 vertical pixels ⁇ 4 horizontal pixels, 2 pixels composed of 2 vertical pixels ⁇ 1 horizontal pixel, or 1 vertical pixel ⁇ 1 horizontal pixel, ie, 1 pixel It may be a pixel.
  • the monocular distance information generation unit 10 outputs the generated monocular distance information 102 to the correction unit 20 of the monocular distance information correction unit 12.
  • the compound-eye distance information generation unit 11 combines the first captured image 100 and the second captured image 101, and specifically, uses the region where the fields of view of the first captured image 100 and the second captured image 101 overlap to determine the compound eye distance.
  • the information 103 is generated. Since the positional relationship including the baseline length of the first imaging unit 2 and the second imaging unit 3 is known, the compound eye distance information generation unit 11 calculates the distance from the parallax of the pixels common to the first captured image 100 and the second captured image 101. I do. Note that the compound-eye distance information 103 calculated by the compound-eye distance information generating unit 11 is also calculated for each distance information block.
  • the compound eye distance information 103 has higher accuracy. Therefore, the monocular distance information 102 can be referred to as “low-accuracy distance information”, and the compound-eye distance information 103 can be referred to as “high-accuracy distance information”. Similarly, the monocular distance information generating unit 10 can be called a “low-accuracy distance information generating unit”, and the compound-eye distance information generating unit 11 can be called a “high-accuracy distance information generating unit”.
  • the correction unit 20 outputs the generated compound eye distance information 103 to the correction unit 20 of the single eye distance information correction unit 12 and the group information generation unit 30.
  • the group information generation unit 30 detects a subject and performs a grouping process on a plurality of subjects using the first captured image 100, the second captured image 101, and the compound-eye distance information 103, and corrects the result as group information 106. 20. Details of the operation of the group information generation unit 30 will be described later.
  • the group information 106 generated by the group information generation unit 30 includes the type of the group and the position of the subject included in the group on the image.
  • the group information 106 may include information on the type of the subject.
  • the type of the subject is, for example, a person, an animal, a vehicle, a road surface, a curb, a side road, a white line, a guardrail, a sign, a structure, a signal, a falling object on the road, a hole on the road, a puddle, and the like.
  • the correction unit 20 corrects the monocular distance information 102 using the compound eye distance information 103 and the group information 106, and outputs corrected distance information 104. Details of the operation of the correction unit 20 will be described later.
  • the recognition processing unit 7 performs a recognition process using the corrected distance information 104 generated by the distance calculation device 6.
  • the ⁇ correction unit 20 corrects the monocular distance information 102 using the compound eye distance information 103 and the group information 106, and outputs the corrected monocular distance information 102 and compound eye distance information 103 as corrected distance information 104.
  • the detailed configuration and operation of the correction unit 20 will be described later.
  • the vehicle control unit 8 controls the vehicle 1 using the recognition information 140 generated by the recognition processing unit 7.
  • the vehicle control unit 8 outputs the input information 105 to the distance calculation device 6.
  • the input information 105 includes at least one of travel information of the vehicle 1, recognition information 140, traffic infrastructure information, map information, and vehicle control information.
  • the vehicle control unit 8 can acquire travel information of the vehicle 1 from a sensor (not shown) mounted on the vehicle 1.
  • the vehicle control unit 8 can acquire traffic infrastructure information from outside the vehicle 1 using a communication device (not shown).
  • the vehicle control unit 8 can obtain map information by referring to a map database using a communication device (not shown) or by referring to a map database built in the vehicle 1.
  • Examples of the traveling information include traveling speed information, acceleration information, position information, traveling direction, traveling mode, and the like of the vehicle 1.
  • Examples of the traveling mode include an automatic driving mode, a traveling mode based on a traveling road, a traveling mode based on a traveling situation, a traveling mode based on a surrounding natural environment, and an energy saving mode in which the vehicle travels with low power consumption or low fuel consumption.
  • Examples of the automatic driving mode include a manual driving mode in which vehicle control is performed by human judgment, a driving support mode in which a part of vehicle control is assisted by system judgment, and an automatic driving mode in which vehicle control is executed by system judgment.
  • Examples of the travel mode based on the travel path include a city area travel mode, a highway travel mode, and legal speed information.
  • Examples of the traveling mode based on the traveling situation include a traffic congestion traveling mode, a parking lot mode, and a traveling mode according to the position and movement of surrounding vehicles.
  • Examples of the driving mode based on the surrounding natural environment include a night driving mode and a backlight driving mode.
  • Examples of the map information include position information of the vehicle 1 on the map, road shape information, road surface feature information, road width information, lane information, and road gradient information.
  • Examples of the road shape information include a T-junction and an intersection.
  • Examples of the road surface feature information include a signal, a roadway section, a sidewalk section, a level crossing, a bicycle parking lot, a car parking lot, and a crosswalk.
  • Examples of the map information include position information of the vehicle 1 on the map, road shape information, road surface feature information, road width information, lane information, and road gradient information.
  • Examples of the road shape information include a T-junction and an intersection.
  • Examples of the road surface feature information include a signal, a roadway section, a sidewalk section, a level crossing, a bicycle parking lot, a car parking lot, and a crosswalk.
  • Examples of the recognition information 140 include position information, type information, operation information, and danger information of the subject.
  • An example of the position information includes a direction and a distance from the vehicle 1.
  • Examples of the type information include pedestrians, adults, children, the elderly, animals, falling rocks, bicycles, surrounding vehicles, surrounding structures, and curbs.
  • Examples of the motion information include a pedestrian or bicycle swaying, jumping out, traversing, moving direction, moving speed, and moving trajectory.
  • Examples of danger information include abnormal movements of surrounding vehicles such as pedestrians jumping out, falling rocks, suddenly stopping, suddenly decelerating, and suddenly turning.
  • Examples of the traffic infrastructure information include traffic congestion information, traffic regulation information such as speed limit and traffic prohibition, travel route guidance information for guiding to another travel route, accident information, alert information, peripheral vehicle information, map information, and the like.
  • Examples of vehicle control information include brake control, steering wheel control, accelerator control, in-vehicle lamp control, alarm sound generation, in-vehicle camera control, and observation objects around imaging devices to peripheral vehicles and remote center devices connected via a network. There is information output on things. Further, the vehicle control unit 8 may perform subject detection processing based on information processed by the distance calculation device 6 or the recognition processing unit 7, or may perform a first operation on a display device connected to the vehicle control unit 8.
  • An image obtained through the imaging unit 2 or the second imaging unit 3 or a display for allowing the viewer to recognize the image may be displayed, or an information device that processes traffic information such as map information or congestion information may be displayed.
  • traffic information such as map information or congestion information
  • FIG. 3 is a diagram showing an area of a captured image.
  • FIG. 3A is a diagram illustrating a first captured image 100 acquired by the first imaging unit 2
  • FIG. 3B is a diagram illustrating a second captured image 101 acquired by the second imaging unit 3
  • FIG. 3C shows a group.
  • the first imaging unit 2 and the second imaging unit 3 are arranged in the horizontal direction, and the horizontal direction in FIGS. 3A and 3B is arranged in accordance with the shooting position. That is, the same region is photographed as the region Q of the first photographed image 100 and the region R of the second photographed image 101.
  • the compound eye distance information generation unit 11 can calculate the distance with high accuracy.
  • the information included in the region Q and the region R is collectively referred to as “information of the region T”, including the meaning of using the two information of the region Q and the region R together.
  • the region T is called a “binocular region”.
  • information is obtained only from one of the first imaging unit 2 and the second imaging unit 3, and the monocular distance information
  • the generation unit 10 can calculate the distance only with low accuracy.
  • each of the regions P to S is referred to as a “monocular region”.
  • the subjects shown in FIG. 3 are as follows.
  • a subject 410, a subject 420, and a subject 430 are photographed in the first captured image 100.
  • the subject 430 extends over the region P and the region Q.
  • the subject 430 in the region P is called a partial subject 401
  • the subject 430 in the region Q is called a partial subject 400.
  • the subject 410 exists at the right end of the region Q
  • the subject 420 exists near the center of the region P.
  • the partial subject 401 is captured at the left end of the region R
  • the subject 410 is captured near the right end of the region R.
  • the subject 411, the subject 412, and the subject 413 are photographed from the left end.
  • the shapes of the subject 410, the subject 411, the subject 412, and the subject 413 are substantially the same.
  • a set of the partial subjects 400 and 401 in other words, the subject 430 itself is a group G1
  • a set of the subjects 420 and 430 is a group G2
  • the subjects 410, 411, 412, and 413 Is a group G3.
  • Each of the groups G1, G2, and G3 is also referred to as a first group, a second group, and a third group.
  • the type of the group of the portion in each region of the subject existing over the binocular region and the monocular region that is, the group G1 is referred to as a “cross-border group”.
  • a portion in each region of the subject 430 existing over the binocular region Q and the monocular region P that is, the group G1 of the partial subject 400 and the partial subject 401 is classified as a cross-border group.
  • the group G1 is referred to as a “cross-border group G1”.
  • a subject existing in the monocular region and present in the vicinity of a subject belonging to the cross-border group, and the type of the group of the subject forming the cross-border group, that is, the group G2 is referred to as an “enlarged cross-border group”.
  • the subject 420 existing in the monocular region P near the subject 430 belonging to the cross-border group G1 and the group G2 of the subjects 430 constituting the cross-border group G1 are the enlarged cross-border.
  • the group G2 is referred to as an “extended transboundary group G2”.
  • a subject such as the subject 420 that is included in the enlarged cross-border group G2 but not included in the cross-border group G1 is referred to as an “additional subject”.
  • the type of group composed of a plurality of subjects existing adjacent to the compound eye region and the monocular region that is, the group G3 is referred to as a “simple group”.
  • each subject belonging to the simplex group is provided on condition that at least one of the type of the subject, the shape of the subject, the color of the subject, and the position of the subject in the vertical direction is the same or similar.
  • the group G3 of the subject 411, the subject 412, and the subject 413 existing adjacent to the monocular region S is classified into a single group.
  • the group G3 is referred to as a “single group G3”.
  • FIG. 4 is a configuration diagram of the correction unit 20.
  • the correction unit 20 includes a distance difference information generation unit 22, a correction determination unit 23, and a correction execution unit 21.
  • the distance difference information generation unit 22 generates distance difference information 120 based on the monocular distance information 102, the compound eye distance information 103, and the group information 106. The distance difference information 120 will be described later.
  • the correction determination unit 23 determines a correction method for the monocular distance information 102 based on the distance difference information 120, the compound eye distance information 103, and the input information 105, and outputs correction determination information 121.
  • the correction executing unit 21 performs a correction process on the single-lens distance information 102 based on the distance difference information 120, the group information 106, and the correction determination information 121, and outputs the compound-eye distance information 103 and the corrected distance information 104 including the corrected single-eye distance information 104. Generate.
  • the correction determination information 121 will be described later.
  • the distance difference information generation unit 22 calculates an average value LAm for the partial subject 400 included in the compound eye region T based on the compound eye distance information 103. Next, the distance difference information generation unit 22 calculates the average value LAs based on the monocular distance information 102 of the partial subject 400 calculated using only the monocular image. Then, the distance difference information generation unit 22 calculates the distance difference information 120 using the average value LAm and the average value LAs.
  • the calculation of the average value LAm and the average value LAs may be, for example, a simple average of the entire region of the partial subject 400 or a weighted average in which the weight is increased as the distance from the boundary increases.
  • a calculation example in the case of performing weighted averaging is as follows.
  • the compound eye distance information 103 of the n distance information blocks k constituting the partial subject 400 to be calculated is Lm (k)
  • the monocular distance information 102 is Ls (k)
  • the weighting coefficient for the compound eye distance information 103 is ⁇ (k).
  • the weighting coefficient for the monocular distance information 102 is ⁇ (k).
  • k is an integer of 0 to n-1.
  • the average values LAm and LAs are expressed as Expressions 1 and 2.
  • the distance difference information 120 may be calculated as follows. A difference between the compound eye distance information 103 of the partial subject 400 in the compound eye region T and the monocular distance information 102 of the partial subject 400 in the single eye region Q is calculated for each distance information block unit, and the generated partial subject 400 in the compound eye region T is generated. May be generated based on the difference for each distance information block unit.
  • the correction determination unit 23 determines whether or not the correction of the distance information of the monocular region is necessary using the input information 105 acquired from the vehicle control unit 8 and generates the correction determination information 121.
  • the correction determination information 121 may include parameters used for correction.
  • the correction determination unit 23 determines not to correct the distance information of the monocular region in the following cases, for example. That is, the vehicle speed exceeds a predetermined speed, the driving mode is a predetermined mode, the current position of the vehicle 1 is on a highway, the steering angle is equal to or more than a predetermined angle, and the like. However, these conditions may be used in combination. This determination is based on the importance of monitoring a short distance when traveling at low speed. At a short distance, the same object is photographed in a large size and the parallax is increased, so that the accuracy of the distance is improved.
  • the risk of contact between a person and a car increases in an environment where people tend to be distracted by the car, such as when driving in a city or turning right or left at an intersection in a city. Conversely, on highways and on national roads where vehicles are traveling at high speed, the risk of human contact with vehicles is greatly reduced.
  • the ⁇ correction determination unit 23 determines parameters used for weighting, for example, as follows. Note that the larger the value of this parameter, the greater the distance information of the compound eye region is reflected in the correction.
  • the correction determination unit 23 sets the parameter to be larger as the vehicle speed is faster, and sets the parameter to the maximum when the vehicle speed is higher than the first predetermined value, and sets the parameter to the minimum when the vehicle speed is lower than the second predetermined value.
  • the correction determination unit 23 determines the location where the vehicle 1 is traveling. For example, if the vehicle 1 is traveling in a location where traffic is intense, the parameter is set to a large value. On roads, the parameters are set to a minimum.
  • the correction determination unit 23 sets a larger parameter as the steering angle of the vehicle 1 is larger, and sets a smaller parameter as the steering angle is smaller. Note that these conditions may be combined.
  • the correction determination unit 23 By providing the correction determination unit 23, it is possible to accurately measure the distance to an object around the vehicle, that is, a person, a car, a parked vehicle, or the like during low-speed driving or when turning right or left at an intersection where the risk of contact between the person and the vehicle increases. Thus, the risk of an accident can be greatly reduced.
  • more accurate automatic driving judgment can be realized by improving the accuracy of distance information referred to by the automatic driving system. It is possible to prevent an automatic driving error due to an erroneous detection generated from the erroneous distance information. By limiting at low speed traveling, the processing load at high speed traveling can be reduced.
  • the correction executing unit 21 corrects the monocular distance information 102 based on the group information 106 by a different method for each type of group.
  • the distance difference information 120 output by the distance difference information generation unit 22 is mainly used when processing the cross-border group G1.
  • the correction determination information 121 is used for determining whether or not correction is required for the cross-border group G1, and for determining correction parameters. Hereinafter, the correction process will be described for each type of group.
  • the correction executing unit 21 corrects the monocular distance information 102 of the partial subject 401 existing in the monocular region P. This correction is, for example, to replace the monocular distance information 102 of all or a part of the distance information blocks of the partial subject 401 with the average value LAm of the partial subjects in the compound eye region T. In the case of replacement, a predetermined pixel may be weighted. As an example of the weighting, there is a method of reducing the weighting as the distance from the boundary increases.
  • the correction executing unit 21 corrects the distance information as follows when the processing is performed on the extended cross-border group. That is, the correction executing unit 21 corrects the distance information of the partial subject in the monocular area of the enlarged cross-border group in the same manner as the distance information of the monocular area in the cross-border group.
  • the correction executing unit 21 corrects the distance information of the subject in the monocular region of the enlarged cross-border group using the distance information of the partial subject in the compound eye region. Specifically, in the example shown in FIG. 3, the distance information of the partial subject 401 in the monocular region P of the enlarged cross-border group G2 is the same as the correction of the partial subject 401 in the cross-border group G1.
  • the distance information of the subject 420 in the monocular region P is corrected using the distance information of the partial subject 400 in the compound eye region T.
  • the correction of the distance information of the subject in the monocular region using the distance information of the partial subject in the compound eye region is, for example, as follows. That is, the distance information of each of the distance information blocks of the subject in the single-eye region is set as the average value of the distances of the entire regions of the partial subjects of the compound-eye region and the average value of the distance information of the respective distance information blocks of the subject in the single-eye region.
  • the correction executing unit 21 corrects the single-eye distance information 102 of the subject photographed only in the single-eye region using the compound-eye distance information 103 of the subject belonging to the single group.
  • the subject 411 is calculated using the compound-eye distance information 103 of the subject 410 calculated using the compound-eye image T and the monocular distance information 102 of the subject 410 calculated using the single-eye image R.
  • the monocular distance information 102 of the subject 412 and the subject 413 is corrected.
  • the subject 410 is also photographed in the monocular image Q, since the image is photographed by a camera different from the monocular image S to be corrected, the monocular distance information 102 obtained from the monocular image Q is used for correction in this case. Is not used.
  • the correction executing unit 21 calculates a difference between the compound-eye distance information 103 and the single-lens distance information 102 of the subject 410, and adds the difference to the single-eye distance information 102 of the subjects 411, 412, and 413.
  • FIG. 5 is a configuration diagram of the group information generation unit 30.
  • the group information generation unit 30 includes a subject detection unit 32 and a group determination unit 31.
  • the subject detection unit 32 detects the subject using the first captured image 100 output from the first imaging unit 2 and the second captured image 101 output from the second imaging unit 3 when the group information generation unit 30 detects the subject.
  • the detection information 130 is output.
  • the first captured image 100 or the second captured image 101 may be used, or both the first captured image 100 and the second captured image 101 may be used. Further, the first photographed image 100 or the second photographed image 101 having different photographing times may be used.
  • the first The approximate area where the captured image 100 and the second captured image 101 overlap can be calculated in advance.
  • the subject detection unit 32 calculates a region where the first captured image 100 and the second captured image 101 overlap with each other by using the preliminary calculation, and specifies a monocular region and a compound eye region. Thereby, the boundary between the compound eye region and the monocular region becomes clear.
  • the condition of the subject detected by the subject detection unit 32 may be input as the input information 105.
  • the condition of the subject is the type, size, position information and the like of the subject.
  • the subject detection information 130 includes the type of the subject, the size of the subject, the subject straddling the boundary between the compound eye region and the monocular region, the subject in the monocular region, the position information of the subject, and the information of the distance information block forming the subject. .
  • the group determination unit 31 calculates the group information 106 based on the subject detection information 130, the compound eye distance information 103, and the input information 105.
  • the group determination unit 31 classifies subjects straddling the boundary between the compound eye region and the monocular region into one group and classifies them into a cross-border group.
  • the group determination unit 31 classifies the subject into another extended group including the other subjects, and classifies the group into an enlarged cross-border group.
  • the group determination unit 31 classifies a plurality of subjects existing adjacent to the monocular region into one group, and classifies them into a single group.
  • FIG. 6 is a flowchart showing the operation of the distance calculation device 6.
  • the distance calculation device 6 acquires the first captured image 100 and the second captured image 101 from the first imaging unit 2 and the second imaging unit 3 (S01).
  • the distance calculation device 6 executes S02, S03, and S04 in parallel.
  • S02, S03, and S04 may be executed in any order.
  • S02 is executed by the monocular distance information generation unit 10
  • S03 is executed by the compound eye distance information generation unit 11
  • S04 is executed by the group information generation unit 30.
  • the distance calculation device 6 generates monocular distance information corresponding to the first captured image 100 using the first captured image 100, and monocular distance information corresponding to the second captured image 101 using the second captured image 101. Generate That is, in S02, the distance information of the compound eye region is calculated twice using each image. In S03, the distance calculation device 6 generates compound eye distance information of the overlapping area using the first captured image 100 and the second captured image 101. In S04, the distance calculation device 6 detects a subject, generates a group (S05), and determines whether or not the first group exists (S06).
  • the distance calculation device 6 When the execution of S02 and S03 is completed and the determination of S06 is affirmative, the distance calculation device 6 generates the distance difference information 120 of the first group (S07). If the first group does not exist (S06: NO), the distance calculation device 6 ends the processing.
  • the distance calculation device 6 determines whether the generated distance difference information 120 is larger than a predetermined value (S08). If the distance difference information 120 is larger, the correction execution unit 21 corrects the monocular distance information 102 as described above. The processing is performed, and the corrected distance information 104 is output (S09). If the distance is smaller than the predetermined value, the distance calculation device 6 ends the processing. The distance calculation device 6 repeats these processes for each frame, for example.
  • the distance calculation device 6 uses the output of the sensor to generate the compound eye distance information 103 that is highly accurate distance information, and the compound eye distance information 103 by using the output of the sensor.
  • a monocular distance information generation unit 10 that generates monocular distance information 102 with a lower precision than that, and a monocular distance information correction unit 12 that corrects the monocular distance information 102 using the compound eye distance information 103 are provided. Therefore, it is possible to correct the distance information of an area with low accuracy.
  • the compound-eye distance information generation unit 11 generates compound-eye distance information 103 by using outputs of a plurality of cameras and outputs of compound-eye regions in which fields of view overlap.
  • the monocular distance information generation unit 10 generates monocular distance information 102 of a monocular region using an output of a single camera.
  • the distance calculation device 6 includes a group information generation unit 30 that associates one or more subjects included in each of the compound eye region and the monocular region as a first group based on a predetermined condition.
  • the monocular distance information correction unit 12 corrects the monocular distance information 102 using the compound eye distance information 103 of the subject belonging to the first group generated by the compound eye distance information generating unit 11. Therefore, the monocular distance information 102 can be corrected using the compound eye distance information 103.
  • the subjects belonging to the first group G1 are the same subjects existing over the single-eye region and the compound-eye region.
  • the group information generating unit 30 associates a subject included in the monocular region, which satisfies a predetermined condition, as an additional subject with the first group as a second group.
  • the subject 420 which is an additional subject
  • the first group G1 are associated with each other as a second group G2.
  • the monocular distance information correction unit 12 corrects the monocular distance information of the additional subject by using the distance information of the first group.
  • the monocular distance information correction unit 12 calculates the difference between the compound eye distance information 103 and the monocular distance information 102 for the same area as distance difference information, and determines whether the correction of the monocular distance information 102 is necessary based on the distance difference information. to decide. Therefore, when the effect of the correction is expected to be small, the correction can be omitted to reduce the processing load.
  • the monocular distance information correcting unit 12 uses the compound eye distance information of the first group generated by the compound eye distance information generating unit 11 or the corrected monocular distance information generated by the monocular distance information correcting unit 12 to generate a monocular of the additional subject Correct the distance information.
  • the monocular distance information correction unit 12 may replace the monocular distance information 102 of the partial subject 401 with the compound eye distance information 103 of the partial subject 400 in the correction of the monocular distance of the cross-border group G1.
  • the monocular distance information correction unit 12 uses the monocular distance information 102 of the subjects 411, 412, and 413 existing in the monocular area as the monocular distance information 103 of the subject 410 existing in the compound eye area in the correction of the monocular distance of the separate group G3. It may be replaced.
  • FIG. 7 is a configuration diagram of the correction unit 20 according to the third modification.
  • the correction determining unit 23 in the present modified example does not receive the input information 105 from the vehicle control unit 8.
  • the correction determination unit 23 according to the present modification generates correction determination information 121 that determines a correction method for the monocular distance information 102 based on the distance difference information 120. Specifically, the correction determination unit 23 determines that the monocular distance information 102 is corrected when the distance difference information 120 is equal to or greater than a predetermined threshold, and corrects the monocular distance information 102 when the distance difference information 120 is less than the predetermined threshold.
  • a predetermined threshold a predetermined threshold
  • FIG. 8 is a diagram illustrating a calculation method of the distance difference information 120 and a process of correcting the monocular distance information 102 for the cross-border group G1 according to the fourth modification.
  • Grids shown in the partial subjects 400 and 401 in FIG. 8 are distance information blocks.
  • a row of distance information blocks arranged in the horizontal direction in the drawing in each of the partial subject 400 and the partial subject 401 will be referred to as a “distance block line”.
  • distance difference information 120 is generated for each distance block line.
  • the monocular distance information 102 in the monocular area is replaced with distance difference information 120 generated based on the compound-eye distance information 103 in the compound-eye area T located at the boundary between the compound-eye area T and the monocular area.
  • the single-eye distance information 102 in the single-eye area may be replaced with a difference value between the single-eye distance information 102 and the compound-eye distance information 103 generated for each distance information block based on the compound-eye distance information 700.
  • the distance information block line 700 shown in FIG. 8 is a distance information block line in the partial subject 400 in the compound eye region T.
  • a distance information block line 701 in the figure is a distance information block line in the partial subject 401 in the monocular region.
  • the distance difference information generation unit 22 generates an average value LAm for each distance information block line based on the compound eye distance information 103 in the partial subject in the compound eye region T. Then, an average value LAm for each distance information block line is generated based on the monocular distance information 102 in the partial subject in the monocular region P, and the distance difference information 120 for each distance information block line based on the average value LAm and the average value LAs.
  • the correction executing unit 21 may perform correction for adding the distance difference information 120 to the monocular distance information 102 before correction.
  • a predetermined pixel may be weighted.
  • the corrected monocular distance information represented by the symbol LCs is represented by the following Expression 6 or Expression when the symbol of the monocular distance information 102 is Ls and the symbol of the distance difference information 120 is LD. 7 is calculated.
  • LCs ⁇ ⁇ Ls + ⁇ ⁇ LD (Equation 6)
  • LCs (1 ⁇ ) ⁇ Ls + ⁇ ⁇ LD (Equation 7)
  • FIG. 9 is a diagram showing a method of determining the weighting coefficient ⁇ shown in Expression 6 or Expression 7.
  • the weighting coefficient is referred to as a “correction coefficient”.
  • the correction executing unit 21 determines the correction coefficient ⁇ according to the correction function 1100 using the compound eye distance information 103 as a variable.
  • the distance indicated by the compound eye distance information 103 is referred to as a short distance 1101 when the distance is less than L0, a medium distance 1012 when the distance L0 is less than L1, and a long distance 1013 when the distance L1 or longer.
  • the correction coefficient ⁇ is set to 0.0, and when the compound eye distance information 103 is the long distance 1103, the correction coefficient ⁇ is set to 1.0.
  • the correction coefficient ⁇ is changed according to the compound eye distance information 103.
  • L0 and L1 are predetermined thresholds.
  • FIG. 10 is a diagram showing an example of a process of correcting the monocular distance information 102 using the correction coefficient ⁇ shown in FIG. FIG. 10A shows a state before correction, and FIG. 10B shows a state after correction.
  • the regions 90A and 90B show a partial subject in the compound eye region T of the object which is a region at the intermediate distance 1102 and straddles the compound eye region T and the single eye region P.
  • the partial subjects 90A and 90B are the same.
  • the regions 91A and 91B show partial subjects within the monocular region P of the same subject as the regions 90A and 90B. Note that the partial subjects 91A and 91B are the same.
  • the 92 regions 92A and 92B indicate partial subjects in the compound eye region T of a subject that extends over a long distance 1103 and the compound eye region T and the single eye region P. Note that the partial subjects 92A and 902 are the same.
  • the regions 93A and 93B show partial subjects in the monocular region P of the same subject as the regions 92A and 92B. Symbols A, B, m, M, and n described in each area schematically indicate distance information in each area, and indicate that the distance information in the area of the same symbol is the same.
  • a correction process of the single eye distance information 102 on the partial subject within the single eye region P constituting the same subject is performed. Further, irrespective of the compound eye distance information 103 (92A) of the partial subject within the long distance 1103 and within the compound eye region T, the correction processing of the single eye distance information 102 for the same subject in the single eye region P is not performed.
  • the monocular distance information correction unit 12 determines whether or not to correct the monocular distance information 102 based on the compound eye distance information 103 and determines a parameter used for the correction.
  • the distance difference information 120 may be calculated as follows. For each distance information block unit in the distance information block line, a difference value between the compound eye distance information 103 in the partial subject in the compound eye region T and the monocular distance information 102 in the partial subject in the non-compound eye region is generated. The difference value may be used to calculate an average difference value of the partial subjects in the compound eye region T, and the average difference value may be used as the distance difference information 120.
  • the compound eye distance information 103 in the upper and lower distance information block lines may be used.
  • N integer
  • N tap FIR Finite Impulse Response
  • the distance difference information 120 may be the average value LAm in the distance information block line.
  • the correction executing unit 21 converts the monocular distance information 102 in all or some of the distance information blocks in the distance information block line in the partial It may be replaced with the average value LAm in the distance information block line. In the case of replacement, a predetermined pixel may be weighted. The weighting is reduced, for example, according to the distance from the boundary between the compound eye region T and the monocular region.
  • predetermined pixels may be weighted. For example, the weighting is made smaller as the distance from the boundary between the compound eye region T and the monocular region becomes longer.
  • FIG. 11 is a diagram illustrating a correction process of the distance difference information 120 and the monocular distance information 102 for the cross-border group G1 according to the modification 11.
  • the subject 430 straddles the compound eye region T and the monocular region P.
  • the subject 430 includes a partial subject 401 in the monocular region P and a partial subject 400 in the compound eye region T.
  • the average value of the compound eye distance information 103 generated based on the compound eye distance information 700 in the compound eye region T located at the boundary between the compound eye region T and the single eye region P is calculated, and the average value of the compound eye region P
  • the monocular distance information 102 in the partial subject 401 is replaced.
  • the distance information block sequence 700 is located at a position closest to the monocular region P of the partial subject 400 existing in the compound eye region T, that is, at a boundary with the monocular region P.
  • the distance difference information generation unit 22 generates an average value LAm of the left end distance information block of the partial subject 400 in the compound eye region T based on the compound eye distance information 103 in the partial subject within the compound eye region T. Further, the distance difference information generation unit 22 generates an average value LAs of the distance information block at the left end of the partial subject 400 in the compound eye region T based on the monocular distance information 102 in the partial subject 401 of the monocular region P, and calculates the average value LAm and the average value LAm. The distance difference information 120 for each distance information block line is generated based on the average value LAs.
  • the average value LAm and the average value LAs can be, for example, the average value of all or a part of the distance information in the distance information block at the left end 700 of the partial subject 400 in the compound eye region T. In calculating the average, predetermined pixels may be weighted.
  • the distance difference information 120 can be, for example, a difference value LD between the average value LAm and the average value LAs. When calculating the difference value, the average value LAm or the average value LAs may be weighted.
  • the distance difference information 120 may be an average value LA described below.
  • the average value LA is a difference value between the compound eye distance information 103 in the partial subject 400 in the compound eye region T and the monocular distance information 102 in the partial subject 401 in the single eye region. It is generated based on a difference value for each distance information block in the subject 400.
  • the distance difference information 120 may be the average value LAm in the distance information block at the left end 700 of the partial subject 400 in the compound eye region T.
  • Modification 14 In the correction processing by the correction execution unit 21 for the cross-border group G1, the monocular distance information 102 in all or some of the distance information blocks in the partial subject 401 is replaced with the average value LAm of the partial subject 400 in the compound eye region T. You may. In the case of replacement, a predetermined pixel may be weighted. For example, the weighting can be reduced according to the distance from the boundary between the compound eye region T and the monocular region P.
  • the distance difference information 120 of the same distance information block may be added to the monocular distance information 102.
  • predetermined pixels may be weighted. For example, the weighting can be reduced according to the distance from the boundary between the compound eye region T and the monocular region P.
  • FIG. 12 is a diagram illustrating a correction process performed by the correction execution unit 21 on the enlarged cross-border group G2 according to the twelfth modification.
  • FIGS. 12A and 12B show two subjects arranged in the compound eye region T and the monocular region P and their distance information, respectively.
  • FIG. ) Indicates after correction.
  • the 83 regions 83A and 83B indicate partial subjects in the monocular region P of the subject that straddles the compound eye region T and the monocular region P. Note that the partial subjects 83A and 83B are the same.
  • the regions 84A and 84B show partial subjects in the monocular region P of the subject that straddles the compound eye region T and the monocular region P. Note that the partial subjects 84A and 84B are the same.
  • the regions 85A and 85B show a subject existing only in the monocular region P.
  • the symbols A, n, m, M, and N1 described in the respective regions schematically indicate distance information in the respective regions, and indicate that the distance information in the regions of the same symbol is the same.
  • the correction determination unit 23 first corrects the monocular distance information 102 on the partial subject in the monocular region P constituting the same subject from m to M based on the compound eye distance information 103 (A) on the partial subject in the compound eye region T. . Next, based on the corrected monocular distance information M generated for the partial subject in the monocular region P, the correction determination unit 23 determines the monocular distance of another subject in the monocular region P included in the enlarged cross-border group G2.
  • FIG. 13 is a diagram illustrating a method of determining the weighting coefficient represented by Expression 6 or 7, that is, the correction coefficient ⁇ .
  • the correction coefficient ⁇ is determined using the traveling speed of the own vehicle included in the input information 105.
  • the traveling speed is equal to or less than the predetermined value F0
  • the correction coefficient ⁇ is set to 1.0
  • the correction coefficient ⁇ is set to 0.0.
  • the threshold L1 shown in FIG. 9 may be variable.
  • FIG. 14 is a diagram illustrating the threshold value L1.
  • the threshold value L1 is determined according to the traveling speed of the vehicle 1 included in the input information 105. When the traveling speed is equal to or less than the predetermined value F0, the threshold L1 is the minimum D0. When the traveling speed exceeds the predetermined value F1, the threshold L1 is the maximum D1. When the traveling speed is between F0 and F1, the value of the threshold L1 increases as the traveling speed increases.
  • FIG. 15 is a diagram for explaining the correction for the expanded cross-border group in the modification 19.
  • FIG. 15A shows the state before correction
  • FIG. 15B shows the state after correction.
  • the correction determination unit 23 determines whether or not to correct the single-lens distance information of the additional subject in the enlarged transboundary group as follows. That is, the correction determination information 121 determines that correction is to be performed when the distance between the additional subject and the compound eye region is shorter than a predetermined threshold, and determines that correction is not to be performed when the distance is longer than the predetermined threshold.
  • both the subject 88A and the subject 89A belong to the enlarged cross-border group.
  • the distance to the compound eye region T of the subject 88A is a distance indicated by reference numeral 1500, and is shorter than a predetermined value DB0. Therefore, the monocular distance information of the subject 88A is corrected from n to N.
  • the distance to the compound eye region T of the subject 89A is a distance indicated by reference numeral 1501 and is longer than a predetermined value DB0. Therefore, the monocular distance information of the subject 89B is not corrected and does not change from k.
  • the monocular distance information correction unit 12 determines the presence or absence of the distance information correction and the correction parameters based on the distance between another subject and the boundary between the monocular image and the compound eye image.
  • FIG. 16 is a diagram illustrating the configuration of the correction unit 20 according to the sixteenth modification.
  • the correction unit 20 in the present modification further includes a monocular area stay period holding unit 24 in addition to the configuration in the first embodiment.
  • the monocular region stay period holding unit 24 Based on the group information 106, the monocular region stay period holding unit 24 generates monocular region stay period information 122 indicating the length of time during which the subject is present in the monocular region within the monocular region P, for example, the number of frames, and performs correction. Output to the determination unit 23.
  • the monocular area stay period information 122 there is frame number information and time information when the subject is continuously present in the monocular area in the monocular area P.
  • the single eye region stay period information 122 may be initialized to zero.
  • the monocular area staying period information 122 may be initialized to a maximum value or a predetermined value.
  • FIG. 17 is a diagram illustrating a method of determining the correction coefficient ⁇ by the correction determination unit 23.
  • the correction determination information 121 determines the correction coefficient ⁇ used in Expressions 6 and 7 based on the monocular area stay period information 122.
  • the correction determination information 121 sets the correction coefficient ⁇ to 1.0 when the monocular area stay period information 122 is equal to or smaller than a predetermined value Ta.
  • the correction determination information 121 sets the correction coefficient ⁇ to 0.0.
  • the correction determination information 121 decreases the correction coefficient ⁇ as the monocular area stay period information 122 increases.
  • the distance calculation device 6 includes a monocular region stay period holding unit 24 that generates monocular region stay period information 122 in which the group exists outside the compound eye region.
  • the monocular distance information correction unit 12 determines a distance information correction method based on the monocular area stay period information 122.
  • FIG. 18 is a configuration diagram of the group information generation unit 30 in the modification 21.
  • the group information generation unit 30 further includes a monocular area stay period holding unit 33 in addition to the configuration illustrated in FIG.
  • the monocular region stay period holding unit 33 generates monocular region stay period information 131 indicating the length of time during which the subject is present in the monocular region within the monocular region P, for example, the number of frames, based on the subject detection information 130. , To the group determination unit 31.
  • the single-eye region stay period information 131 there is frame number information and time information when the subject is continuously present in the single-eye region in the single-eye region P.
  • the single eye region stay period information 131 may be initialized to zero.
  • the monocular area staying period information 131 may be initialized to a maximum value or a predetermined value.
  • the correction executing unit 21 may determine a method of correcting the single-lens distance information based on the type of the subject and the speed of the vehicle 1. For example, the correction method is the necessity of correction and the parameter of correction.
  • FIG. 19 is a diagram showing the correspondence between the type of subject and the correction method. However, FIG. 19 also shows the relationship between the speed of the vehicle 1 and the correction method. FIG. 19 is divided into two stages for the sake of drawing. In FIG. 19, “correction ON” indicates that the single-lens distance information is corrected, and “correction OFF” indicates that the single-lens distance information is not corrected.
  • the percentage in FIG. 19 indicates that the correction is to be performed and the value of the parameter used for the correction. That is, the correction method in which the percentage is described is a correction method using parameters as shown in Expressions 6 and 7. However, in the case of “0%”, the correction need not be performed.
  • the correction execution unit 21 uses the correction method 1
  • the monocular distance information is corrected when the type of the subject is a person, a vehicle, and a road surface, but is corrected when the type of the subject is a structure. do not do.
  • the correction executing unit 21 uses the correction method 3
  • the cases are classified according to the speed of the vehicle 1, and when the vehicle is running at low speed, the correction is performed regardless of the type of the subject.
  • the vehicle is running at high speed, the person and the vehicle are not corrected.
  • the time of middle speed running the person and the vehicle are corrected according to the speed.
  • the road surface or the structure is a stationary object having continuity in a large area, it can be easily corrected.
  • the area of a person or a vehicle is relatively small and has no continuity, the correction is performed only at the time of low-speed running where high necessity is required.
  • the distance accuracy of the person is increased during low-speed running, and the distance accuracy of the vehicle is increased during high-speed running.
  • Cost reduction is realized by reducing the overall correction processing load by utilizing the fact that the environment around the vehicle changes depending on the speed, and in particular, changes what is desired to be seen. Specifically, by reducing the calculation capability required for the distance calculation device 6, the distance calculation device 6 can be manufactured using a calculation device having a low processing capability.
  • the correction may be performed only when the distance indicated by the monocular distance information before correction is shorter than a predetermined distance.
  • the correction target may be limited to only those who are within a predetermined distance range from the road surface. Since the road surface has continuity in a large area, the correction accuracy is high. On the other hand, it is necessary to pay more attention to people near the traveling road.
  • the monocular distance information correction unit 12 determines a method of correcting the monocular distance information based on at least one of the traveling state of the vehicle 1 and the type of the subject. Therefore, the distance calculation device 6 can perform the correction based on the running state of the vehicle and the type of the subject.
  • FIG. 20 is a schematic diagram of a distance calculation system S according to the second embodiment.
  • the vehicle 1 includes a first imaging unit 2 and a distance sensor 2000 as sensors for collecting surrounding information.
  • the distance sensor 2000 is a sensor that can acquire distance information, and is, for example, a LiDAR (Light Detection and Ranging), a millimeter wave radar, a TOF (Time of Flight) type distance sensor, a stereo camera, or the like.
  • the distance sensor 2000 acquires distance information of a range indicated by a solid line, that is, an area indicated by hatching in the center of the drawing.
  • the first imaging unit 2 sets the range indicated by the broken line, that is, the wide range including the range in which the distance sensor 2000 acquires the distance information, as the imaging range.
  • a region indicated by hatching corresponds to the compound eye region T in the first embodiment. Regions other than the region indicated by hatching in the imaging range of the first imaging unit 2 correspond to the monocular region P and the monocular region S in the first embodiment.
  • FIG. 21 is a configuration diagram of the distance calculation device 6 according to the second embodiment.
  • the compound-eye distance information generation unit 11 is omitted from the configuration of the first embodiment.
  • Distance sensor 2000 outputs distance information 2010 to monocular distance information correction unit 12.
  • the monocular distance information correction unit 12 treats the distance information 2010 as the compound eye distance information 103 in the first embodiment. Therefore, although there is no actual situation, it can be considered that the compound-eye distance information generation unit 11 virtually exists, and the output of the input sensor is directly output as the high-precision distance information 103.
  • Other configurations are the same as those of the first embodiment.
  • the compound-eye distance information generation unit 11 generates compound-eye distance information 103 using at least one of a TOF distance sensor, a millimeter-wave radar, and an LiDAR output. Therefore, the distance information of the monocular region can be corrected using the information of the distance sensor with higher accuracy.
  • FIGS. A third embodiment of the distance calculation device will be described with reference to FIGS.
  • the same components as those in the first embodiment are denoted by the same reference numerals, and the differences will be mainly described.
  • the points that are not particularly described are the same as in the first embodiment.
  • a sensor mainly mounted on the vehicle 1 is different from that of the first embodiment.
  • FIG. 22 is a schematic diagram of a distance calculation system S according to the third embodiment.
  • the vehicle 1 includes a first imaging unit 2, a second imaging unit 3, a first distance sensor 2200, and a second distance sensor 2201 as sensors for collecting surrounding information.
  • the first distance sensor 2200 and the second distance sensor 2201 are sensors capable of acquiring distance information, and are, for example, a LiDAR, a millimeter wave radar, a TOF distance sensor, a stereo camera, and the like.
  • the fields of view of the first imaging unit 2 and the second imaging unit 3 are the same, and an image of the hatched area in the center of the figure is taken.
  • the field of view of the first distance sensor 2200 is about the front left of the vehicle 1
  • the field of view of the second distance sensor 2201 is about the front right of the vehicle 1.
  • the viewing angles, installation positions, and postures of all sensors are known.
  • the area indicated by hatching corresponds to the compound eye area T in the first embodiment.
  • the areas above and below the hatching correspond to the monocular area P and the monocular area S in the first embodiment.
  • FIG. 23 is a configuration diagram of the distance calculation device 6 according to the third embodiment.
  • the monocular distance information generation unit 10 is omitted.
  • the first distance sensor 2200 and the second distance sensor 2201 output the distance information 2210 and the distance information 2211 to the monocular distance information correction unit 12.
  • the monocular distance information correction unit 12 extracts necessary area information from the input distance information 2210 and the distance information 2211 and treats the extracted information as monocular distance information 102 in the first embodiment. Therefore, although the actual state does not exist, it can be considered that the monocular distance information generation unit 10 virtually exists and the output of the input sensor is output as the monocular distance information 102 as it is.
  • Other configurations are the same as those of the first embodiment.
  • the monocular distance information generation unit 10 generates the monocular distance information 102 by using the output of at least one of a TOF distance sensor, a millimeter wave radar, and a LiDAR.
  • FIGS. A fourth embodiment of the distance calculation device will be described with reference to FIGS.
  • the same components as those in the first embodiment are denoted by the same reference numerals, and the differences will be mainly described.
  • the points that are not particularly described are the same as in the first embodiment.
  • the present embodiment mainly differs from the first embodiment in the viewing angle of the camera mounted on the vehicle 1 and the attitude of the camera.
  • FIG. 24 is a schematic diagram of a distance calculation system S according to the fourth embodiment.
  • the vehicle 1 includes a first imaging unit 2 and a second imaging unit 3 as sensors for collecting surrounding information, as in the first embodiment.
  • the viewing angles of the first imaging unit 2 and the second imaging unit 3 are widened, and the fields of view of both are the same.
  • the first imaging unit 2 and the second imaging unit 3 have, for example, a fisheye lens and are capable of imaging a wide range, but the distortion amount around the lens is large, and the resolution is reduced.
  • a hatched area which corresponds to the compound eye area T in the first embodiment.
  • Regions other than the center of the field of view of the first imaging unit 2 and the second imaging unit 3 have large distortion and low spatial resolution, and therefore correspond to the monocular region P and the monocular region S in the first embodiment. I reckon. In other words, an area with relatively poor shooting conditions is regarded as a monocular area, and an area with relatively good imaging conditions is considered as a compound eye area.
  • FIG. 25 is a configuration diagram of the distance calculation device 6 according to the fourth embodiment.
  • the monocular distance information generation unit 10 is omitted from the configuration of the first embodiment.
  • the monocular distance information correction unit 12 treats only the vicinity of the center of the compound eye distance information 103 output by the compound eye distance information generating unit 11 as the compound eye distance information 103 in the first embodiment, and the peripheral part is the first embodiment. Is handled as the monocular distance information 102 in. Other configurations are the same as those of the first embodiment.
  • the compound-eye distance information generation unit 11 generates compound-eye distance information 103 by using information of an area where the two cameras output and the fields of view overlap and the shooting conditions are relatively good, and the compound-eye distance information 103
  • the generation unit 10 generates the monocular distance information 102 using information of an area output by the two cameras and having an overlapping field of view and in which imaging conditions are inferior to the area used by the compound-eye distance information generation unit 11. . Therefore, the present invention can be applied to a system having no monocular region.
  • the photographing conditions may include not only the distortion of the lens but also the amount of light and dust attached to the lens.
  • distance information of a dark area where the amount of light is small is treated as monocular distance information 102
  • distance information of a brightly and clearly photographed area is compound eye distance information 103. It may be treated as.
  • FIG. 26 is a configuration diagram of a distance calculation system S according to the fifth embodiment.
  • the monocular distance information correction unit 12 is realized by hardware different from the monocular distance information generation unit 10 and the compound eye distance information generation unit 11.
  • the camera processing device 15 includes a monocular distance information generation unit 10, a compound eye distance information generation unit 11, and a feature information generation unit 13.
  • the feature information generation unit 13 creates a feature amount of the input image, for example, an edge image or a histogram, and outputs it as feature information 107 to the image processing unit 60.
  • the ECU 14 includes a monocular distance information correction unit 12, a recognition processing unit 7, and a vehicle control unit 8.
  • the present invention can be implemented by various hardware configurations.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Measurement Of Optical Distance (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Abstract

L'invention concerne un dispositif de calcul de distance comprenant : une unité de génération d'informations de distance de haute précision qui génère, à l'aide d'une sortie provenant d'un capteur, des informations de distance de haute précision constituant des informations sur une distance très précise; une unité de génération d'informations de distance de faible précision qui génère, à l'aide d'une sortie du capteur, des informations de distance de faible précision, dont le degré de précision est inférieur à celui des informations de distance de haute précision; et une unité de correction de distance qui corrige les informations de distance de faible précision à l'aide des informations de distance de haute précision.
PCT/JP2019/026647 2018-07-27 2019-07-04 Dispositif de calcul de distance Ceased WO2020022021A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201980049441.1A CN112513571B (zh) 2018-07-27 2019-07-04 距离计算装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-141891 2018-07-27
JP2018141891A JP7042185B2 (ja) 2018-07-27 2018-07-27 距離算出装置

Publications (1)

Publication Number Publication Date
WO2020022021A1 true WO2020022021A1 (fr) 2020-01-30

Family

ID=69180414

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/026647 Ceased WO2020022021A1 (fr) 2018-07-27 2019-07-04 Dispositif de calcul de distance

Country Status (3)

Country Link
JP (1) JP7042185B2 (fr)
CN (1) CN112513571B (fr)
WO (1) WO2020022021A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025163785A1 (fr) * 2024-01-31 2025-08-07 Ａｓｔｅｍｏ株式会社 Dispositif de reconnaissance externe et procédé de reconnaissance externe
US12467303B2 (en) 2021-04-06 2025-11-11 Inteva Products, Llc Pivoting cursor for frameless glass in a vehicle

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7341079B2 (ja) * 2020-02-10 2023-09-08 住友重機械工業株式会社 距離画像推定装置及び操縦装置
JP7644620B2 (ja) * 2020-05-25 2025-03-12 株式会社Subaru 車外環境認識装置
JP2022041219A (ja) * 2020-08-31 2022-03-11 エスゼット ディージェイアイ テクノロジー カンパニー リミテッド 制御装置、測距センサ、撮像装置、制御方法、及びプログラム
JP7499140B2 (ja) * 2020-10-14 2024-06-13 日立Astemo株式会社 物体認識装置
EP4047388B1 (fr) * 2021-02-22 2024-07-24 Shenzhen Camsense Technologies Co., Ltd Appareil de télémétrie, lidar et robot mobile
JP7759739B2 (ja) * 2021-05-28 2025-10-24 株式会社Subaru 車両の車外撮像装置
JP2023035903A (ja) * 2021-08-30 2023-03-13 株式会社モルフォ 深度推定方法、深度推定装置
JP7797855B2 (ja) * 2021-12-15 2026-01-14 トヨタ自動車株式会社 情報処理装置、情報処理方法及びプログラム
JP7776391B2 (ja) * 2022-07-20 2025-11-26 本田技研工業株式会社 車両用検出装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000121319A (ja) * 1998-10-15 2000-04-28 Sony Corp 画像処理装置および画像処理方法、並びに提供媒体
WO2006121087A1 (fr) * 2005-05-10 2006-11-16 Olympus Corporation Dispositif de traitement d’images, procede de traitement d’images et programme de traitement d’images
WO2006123615A1 (fr) * 2005-05-19 2006-11-23 Olympus Corporation Appareil, procede et programme de mesure de la distance
JP2007263669A (ja) * 2006-03-28 2007-10-11 Denso It Laboratory Inc ３次元座標取得装置
JP2008304202A (ja) * 2007-06-05 2008-12-18 Konica Minolta Holdings Inc 距離画像生成方法、距離画像生成装置及びプログラム
JP2017139631A (ja) * 2016-02-04 2017-08-10 日立オートモティブシステムズ株式会社 撮像装置
JP2019128153A (ja) * 2018-01-19 2019-08-01 本田技研工業株式会社 距離算出装置及び車両制御装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8478066B2 (en) * 2003-10-31 2013-07-02 Mitsubishi Denki Kabushiki Kaisha Image-correction method and image pickup apparatus
EP3193134B1 (fr) * 2014-09-11 2022-10-12 Hitachi Astemo, Ltd. Dispositif de traitement d'images

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000121319A (ja) * 1998-10-15 2000-04-28 Sony Corp 画像処理装置および画像処理方法、並びに提供媒体
WO2006121087A1 (fr) * 2005-05-10 2006-11-16 Olympus Corporation Dispositif de traitement d’images, procede de traitement d’images et programme de traitement d’images
WO2006123615A1 (fr) * 2005-05-19 2006-11-23 Olympus Corporation Appareil, procede et programme de mesure de la distance
JP2007263669A (ja) * 2006-03-28 2007-10-11 Denso It Laboratory Inc ３次元座標取得装置
JP2008304202A (ja) * 2007-06-05 2008-12-18 Konica Minolta Holdings Inc 距離画像生成方法、距離画像生成装置及びプログラム
JP2017139631A (ja) * 2016-02-04 2017-08-10 日立オートモティブシステムズ株式会社 撮像装置
JP2019128153A (ja) * 2018-01-19 2019-08-01 本田技研工業株式会社 距離算出装置及び車両制御装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12467303B2 (en) 2021-04-06 2025-11-11 Inteva Products, Llc Pivoting cursor for frameless glass in a vehicle
WO2025163785A1 (fr) * 2024-01-31 2025-08-07 Ａｓｔｅｍｏ株式会社 Dispositif de reconnaissance externe et procédé de reconnaissance externe

Also Published As

Publication number Publication date
CN112513571B (zh) 2023-08-25
JP2020016628A (ja) 2020-01-30
CN112513571A (zh) 2021-03-16
JP7042185B2 (ja) 2022-03-25

Similar Documents

Publication Publication Date Title
JP7042185B2 (ja) 距離算出装置
JP6648411B2 (ja) 処理装置、処理システム、処理プログラム及び処理方法
KR102772870B1 (ko) 전자기기 및 제어방법
EP3358302B1 (fr) Procédé de commande de déplacement et dispositif de commande de déplacement
US10964217B2 (en) Travel control method and travel control apparatus
US10798319B2 (en) Camera device and method for capturing a surrounding region of a vehicle in a situation-adapted manner
JP6254084B2 (ja) 画像処理装置
US20190156131A1 (en) Image Processing Device, Outside Recognition Device
US20120277990A1 (en) Method and apparatus for determining a plausible lane for guiding a vehicle and an automobile
US20200059613A1 (en) Camera Device and Method for Detecting a Surrounding Area of a Driver's Own Vehicle
JP2016001170A (ja) 処理装置、処理プログラム、及び、処理方法
US10750085B2 (en) Camera device for capturing a surrounding area of a driver's own vehicle and method for providing a driver assistance function
US20200118280A1 (en) Image Processing Device
CN110053625B (zh) 距离计算装置和车辆控制装置
JP6699344B2 (ja) 逆走車検出装置、逆走車検出方法
US12497040B2 (en) Vehicle control system and vehicle driving method using the vehicle control system
US20190026923A1 (en) Image recognition system
US20220327819A1 (en) Image processing apparatus, image processing method, and program
JP2019146012A (ja) 撮像装置
JP3960146B2 (ja) 画像認識装置
JP2025159111A (ja) 情報出力装置
CN109643447B (zh) 图像处理装置、摄像装置
JP6253175B2 (ja) 車両の外部環境認識装置
WO2019039106A1 (fr) Dispositif de reconnaissance d'image
JP7185571B2 (ja) 視認方向推定装置、視認方向推定方法、およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19839831

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19839831

Country of ref document: EP

Kind code of ref document: A1