WO2025197360A1 - Procédé de traitement d'informations, dispositif de traitement d'informations et programme - Google Patents

Procédé de traitement d'informations, dispositif de traitement d'informations et programme

Info

Publication number
WO2025197360A1
WO2025197360A1 PCT/JP2025/004426 JP2025004426W WO2025197360A1 WO 2025197360 A1 WO2025197360 A1 WO 2025197360A1 JP 2025004426 W JP2025004426 W JP 2025004426W WO 2025197360 A1 WO2025197360 A1 WO 2025197360A1
Authority
WO
WIPO (PCT)
Prior art keywords
histogram
distance value
information processing
unit
peaks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2025/004426
Other languages
English (en)
Japanese (ja)
Inventor
裕崇 田中
佑斗 内海
健斗 鈴木
邦昭 野田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Publication of WO2025197360A1 publication Critical patent/WO2025197360A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles

Definitions

  • This disclosure relates to an information processing method, an information processing device, and a program.
  • distance information is acquired. Based on this distance information, obstacle detection, self-position estimation, etc. are performed. In this case, light is shone onto the subject, and the time it takes for the reflected light from the subject to be received is converted into distance information.
  • the accuracy of measuring the distance to the subject may decrease.
  • the present disclosure therefore provides an information processing method, information processing device, and program that can suppress a decrease in the processing accuracy of obstacle detection and self-position estimation, even when light is irradiated through a transparent object.
  • a method for processing information is provided.
  • the distance value based on the peak on the closer side may be used to determine whether an obstacle exists.
  • the distance value based on the peak on the far side may be used to estimate the self-position.
  • the distance value based on the closest peak may be used to determine whether an obstacle exists.
  • the distance value based on the farthest peak may be used to estimate the self-position.
  • the distance value based on that peak may be used to determine an obstacle and estimate the vehicle's own position.
  • the plurality of light emission pulse signals are emitted in a plurality of irradiation directions; in the generating step, the histogram is generated for each of the plurality of irradiation directions; In the signal processing process, if there are two peaks in the histogram for each of the multiple illumination directions, an obstacle may be determined using a first distance value based on the peak at the closest distance, and the vehicle's own position may be estimated using a second distance value based on the peak at the farthest distance.
  • the obstacle determination may be based on the first distance value and the illumination direction from the aircraft corresponding to the first distance value.
  • the estimation of the self-position may be based on the second distance value and an illumination direction from the own aircraft corresponding to the second distance value.
  • the method may further include a determination step for determining whether two or more peaks exist in the histogram, and the signal processing step may be performed based on the determination made in the determination step.
  • the determination step it may be determined that a transparent object is present if two or more peaks exist in the histogram.
  • the obstacle determination may determine that the transparent object exists in an illumination direction in which it is determined that two or more peaks exist.
  • the first distance value and the illumination direction from the aircraft corresponding to the first distance value may be generated as second point cloud data, and if the transparent object is present, the distance value to the transparent object may be included in the second point cloud data.
  • the second distance value and the illumination direction from the aircraft corresponding to the second distance value may be generated as first point cloud data, and if the transparent object is present, the distance value to a subject farther away than the transparent object may be included in the first point cloud data.
  • a time measurement unit that generates a value related to a return time of a reflected light pulse signal corresponding to a plurality of light emission pulse signals that are repeatedly emitted at predetermined time intervals; a histogram generating unit that generates a histogram of the number of measurements of the reflected light pulse signal relative to the value related to the return time; a signal processing unit that, when two peaks exist in the histogram, determines an obstacle relative to the aircraft using a distance value based on one of the peaks, and estimates the aircraft's own position relative to the aircraft using a distance value based on the other peak;
  • An information processing device is provided.
  • the obstacle determination and the self-location estimation may be performed based on the determination of the identification unit.
  • the identification unit may determine that a transparent object is present if two or more peaks exist in the histogram.
  • the obstacle determination may determine that the transparent object exists in an illumination direction in which it is determined that two or more peaks exist.
  • a program is provided that causes an information processing device to execute a signal processing step in which, when two peaks exist in the histogram, a distance value based on one of the peaks is used to determine whether the aircraft is an obstacle, and a distance value based on the other peak is used to estimate the aircraft's own position.
  • FIG. 1 is a block diagram showing a schematic configuration of a distance measuring device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram showing the configurations of a light receiving device, an information processing device, and an operation control device.
  • FIG. 3 is a diagram schematically showing the scanning range of the optical scanning unit.
  • 3A and 3B are diagrams schematically illustrating an example of the internal configuration of a light-emitting unit and a light-receiving range of a light-receiving unit.
  • 4A and 4B are diagrams showing the relationship between the projection pattern of the light-emitting element and the focal point of the light-emitting optical system.
  • FIGS. 10A and 10B are diagrams showing the relationship between the projection pattern of a light-emitting element and the illumination angle of a nearby projection pattern NFP.
  • FIG. 2 is a diagram schematically showing a far-projected pattern FFP received via a light receiving optical system.
  • FIG. 10 is a view showing a projection of each light-emitting element projected in a second direction.
  • FIG. 4 is a diagram showing a projection pattern of light-emitting elements arranged in a first direction.
  • FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system.
  • FIG. 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.
  • FIG. 1 is a block diagram showing a schematic configuration of an autonomously driven mobile body 100 according to the present embodiment.
  • the mobile body 100 may be, for example, an autonomous mobile robot such as an autonomous mobile vacuum cleaner that moves indoors, an autonomous food delivery machine, an autonomously flying drone, an autonomously moving vehicle that moves on land, or an autonomously navigating ship that moves on or underwater.
  • the mobile body 100 includes a light-emitting device 1, a light-receiving device 2, an information processing device 3, an operation control device 4, and a drive unit 5.
  • the technology disclosed herein can be applied to an autonomous mobile robot such as an autonomous mobile vacuum cleaner that moves indoors, an autonomously flying drone, an autonomously moving vehicle that moves on land, or an autonomously navigating ship that moves on or underwater.
  • an autonomous mobile robot such as an autonomous mobile vacuum cleaner that moves indoors, an autonomously flying drone, an autonomously moving vehicle that moves on land, or an autonomously navigating ship that moves on or underwater.
  • the light-emitting device 1 is a device that irradiates an optical pulse signal onto the subject 6. This light-emitting device 1 irradiates an optical pulse signal (TX pulse signal) within a two-dimensional measurement range around the device itself.
  • the light-receiving device 2 detects a reflected optical pulse signal (RX pulse signal) that is the optical pulse signal reflected by the subject 6. This light-receiving device 2 detects the returning light from the two-dimensional measurement range as a reflected optical pulse signal.
  • the subject 6 includes objects, buildings, etc. in the surrounding environment of the device itself (mobile body 100).
  • the information processing device 3 is capable of generating point cloud data containing two-dimensional distance information around the vehicle based on the reflected light pulse signals received by the light receiving device 2. Furthermore, the information processing device 3 is capable of estimating the vehicle's position and detecting obstacles with which it may collide based on this point cloud data.
  • the driving control device 4 controls the autonomous driving of the mobile body 100 so as to avoid collisions with obstacles based on the estimated position of the mobile body generated by the information processing device 3 and the position of the detected obstacle. For example, it controls the driving of the mobile body 100 along the target route.
  • the drive unit 5 is a mechanism for moving the mobile body 100, and includes a running mechanism, a propulsion mechanism, a flight mechanism, etc.
  • the mobile body 100 is configured as an autonomous mobile robot, and the drive unit 5 is composed of the motors, wheels, etc. of the autonomous mobile robot.
  • the drive unit 5 is composed of a screw propeller, etc. as a propulsion mechanism.
  • the drive unit 5 is composed of the motors, propellers, etc. as a flight mechanism.
  • the drive unit 5 is driven according to the control of the operation control device 4, and moves the mobile body 100.
  • Figure 2 is a block diagram showing the configurations of the light-receiving device 2, information processing device 3, and operation control device 4.
  • Figure 3 is a block diagram showing the configurations of the light-emitting device 1, light-receiving device 2, and information processing device 3 according to this embodiment in more detail.
  • the moving object 100 shown in Figures 2 and 3 performs distance measurement processing using, for example, the dToF (direct time of flight) method.
  • the light-emitting device 1 has a laser emitter 10, a measurement pulse generator 11, and an emission optical system (TX optical system) 12.
  • the laser emitter 10 has multiple light-emitting elements 10a.
  • the multiple light-emitting elements 10a are arranged in multiple rows along a predetermined direction (first direction Y).
  • first direction Y is defined as the vertical direction and the second direction X is defined as the horizontal direction.
  • the first direction Y may also be defined as the horizontal direction and the second direction X as the vertical direction.
  • the multiple light-emitting elements 10a may be arranged in the second direction X (horizontal direction).
  • the multiple light-emitting elements 10a may be arranged in both the first direction Y and the second direction X.
  • the multiple light-emitting elements 10a repeatedly emit light-emitting pulse signals (TX pulse signals) at predetermined time intervals.
  • the light-emitting unit 2 can also scan the optical signals emitted by the multiple light-emitting elements 10a in the first direction X. The specific method for scanning the optical signals is not important.
  • the multiple light-emitting elements 10a emit light one by one in sequence, with a time lag.
  • each group of light-emitting elements 10a, each containing two or more light-emitting elements 10a may emit light with a time lag.
  • the multiple light-emitting elements 10a may emit light at the same timing.
  • Each light-emitting element 10a repeatedly emits an optical pulse signal thousands to tens of thousands of times during a single measurement.
  • the measurement pulse generation unit 11 controls the emission timing of the optical pulse signal. That is, the measurement pulse generation unit 11 has a drive circuit 13, a clock generation unit 14, and an emission control unit 15.
  • the drive circuit 13 drives the multiple light-emitting elements 10a based on a control signal from the emission control unit 15. For example, the drive circuit 13 controls the emission timing of the optical pulse signal based on a control signal from the emission control unit 15.
  • the clock generation unit 14 generates a clock signal synchronized with a reference clock signal.
  • the reference clock signal is, for example, a signal input from the operation control device 4.
  • the light-emission control unit 15 is composed of a CPU (Central Processing Unit), memory, etc., and controls the drive circuit 13 and clock generation unit 14 by executing a predetermined program, as described above. In other words, the light-emission control unit 15 generates control signals for controlling the light-emission timing of each light-emitting element 10a in synchronization with the clock signal.
  • the drive circuit 13 described above drives the multiple light-emitting elements 10a based on the control signals output from the light-emission control unit 15.
  • the light emitting optical system (TX optical system) 12 irradiates or scans the light pulse signal emitted from the light emitting unit 2 within a predetermined angular range.
  • the light emitting optical system 12 may have a mechanical scanning mechanism such as a polygon mirror or MEMS mirror.
  • the light emitting optical system 12 can change the angular range of irradiation in synchronization with a control signal from the light emission control unit 15.
  • the central processing unit 40 (see Figure 4) of the operation control device 4 controls the light emitting device 1, the light receiving device 2, and the information processing device 3.
  • the light receiving device 2 has a light detection unit 20 and a light receiving optical system (RX optical system) 21. As shown in Figure 3, the light receiving optical system (RX optical system) 21 focuses reflected light signals from the subject 6 onto the light detection unit 20.
  • the light detection unit (SPAD sensor light detection unit) 20 has a pixel array unit 22, a timing control unit 23, and a drive circuit 24.
  • the timing control unit 23 is composed of a CPU (Central Processing Unit), memory, etc., and executes a predetermined program to control the pixel array unit 22 and drive circuit 24 at timing synchronized with the measurement pulse generation unit 11.
  • the pixel array unit 22 has multiple ranging pixels 22a arranged in two dimensions.
  • the multiple ranging pixels 22a receive reflected light signals from the subject 6.
  • the multiple ranging pixels 22a output electrical signals according to the light intensity of the received reflected light signals.
  • Each of the multiple ranging pixels 22a has a light-receiving element.
  • the light-receiving element is, for example, a SPAD (Single Photon Avalanche Photo Diode).
  • Each ranging pixel 22a may have a quench circuit (not shown). In its initial state, the quench circuit supplies a reverse bias voltage between the anode and cathode of the SPAD, with a potential difference exceeding the breakdown voltage. After the SPAD detects a photon, the drive circuit 24 supplies a reverse bias voltage to the SPAD via the corresponding quench circuit to prepare for detection of the next reflected light pulse signal (RX pulse signal).
  • RX pulse signal reflected light pulse signal
  • each light-emitting element 10a repeatedly emits a light pulse signal thousands to tens of thousands of times, and each corresponding ranging pixel 22a converts this into an electrical signal thousands to tens of thousands of times.
  • the information processing device 3 has a signal processing control unit 30a, a memory unit 30b, a time measurement unit (TDC) 31, a histogram generation unit 32, and a signal processing unit 33.
  • the signal processing unit 33 has an echo identification unit 34, a distance generation unit 35, a self-position estimation unit 36, and an obstacle determination unit 37.
  • the signal processing control unit 30a is composed of a CPU (Central Processing Unit), a memory unit 30b, etc., and controls the time measurement unit (TDC) 31, the histogram generation unit 32, and the signal processing unit 33 by executing a predetermined program.
  • the memory unit 30b stores data generated by the time measurement unit (TDC) 31, the histogram generation unit 32, and the signal processing unit 33.
  • the echo identification unit 34 in this embodiment corresponds to the identification unit.
  • the TDC 31 generates a time digital signal with a predetermined time resolution that corresponds to the light reception time of the reflected light pulse signal received by the SPAD. For example, the TDC 31 synchronizes with the emission timing of the light emission pulse signal several thousand to tens of thousands of times, and generates a time digital signal for each ranging pixel 22a by repeating the process several thousand to tens of thousands of times with a predetermined time resolution.
  • Figure 4 is a diagram showing an example of measuring distance values in the surrounding environment including the subject 6a.
  • This shows an example in which a polygon mirror 53 is used as the optical system 12.
  • the optical system 12 is composed of an exit lens 51, a dichroic mirror 52, and a polygon mirror 53.
  • An optical pulse signal is irradiated onto the scanning range 12r of the polygon mirror 52 via the exit lens 51, dichroic mirror 52, and polygon mirror 53.
  • Polygon mirror 51 is rotated around its axis of rotation by a motor under the control of light emission control unit 15.
  • a reflected light pulse signal with a light intensity according to the reflectance of subject 6a is reflected from subject 6a.
  • This reflected light pulse signal is reflected by polygon mirror 53 and dichroic mirror 52, and is measured in order by ranging pixels 22a of pixel array unit 22 and converted into a distance value. Note that in Figure 4, the distance value corresponding to subject 6a is schematically illustrated as subject d6a.
  • the light emitting position of the laser emitting unit 10, the light emitting time, the rotation angle of the polygon mirror 51, and the ranging pixels 22a of the pixel array unit 22 at this time are associated and stored in the memory unit 30b.
  • the irradiation direction of the light emitting pulse can be calculated from the light emitting position of the laser emitting unit 10 and the rotation angle of the polygon mirror 51.
  • FIG. 5 is a diagram showing an example of first point cloud data in the measurement of FIG. 4.
  • FIG. 5(a) is a diagram showing a schematic diagram of first point cloud data g10.
  • the first point cloud data g10 is used for self-position estimation by the self-position estimation unit 36, which will be described later.
  • the first point cloud data g10 is data associated with each ranging pixel 22a, and includes distance values and irradiation direction information.
  • Figure 5(b) is a diagram showing an example of a histogram used to generate element p20 of the first point cloud data g20.
  • the vertical axis represents the time digital signal generated by the TDC 31, and the vertical axis represents the number of measurements within the bin width. For example, if a light emission pulse signal is emitted several thousand times corresponding to the ranging pixel 22a and all measurements are taken, the sum of the number of measurements within all bins will be several thousand measurements.
  • the histogram generation unit 32 generates a histogram for each ranging pixel 22a with a bin width according to the time resolution of the TDC 31 based on the several thousand to tens of thousands of time digital signals generated by the TDC 31, and stores the generated histogram in the memory unit 30b.
  • the bin width is the width of each frequency unit that makes up the histogram. The higher the time resolution of the TDC 31, the narrower the bin width can be, resulting in a histogram that more accurately reflects the time frequency at which the RX pulse signal is received.
  • the histogram generator 32 adds up the number of time digital signals within a bin width to obtain the number of measurements (photon count) within the bin width. In this way, the histogram generator 32 generates a histogram by arranging the number of measurements within each bin width for each bin width. The histogram generator 32 also generates a histogram corresponding to each ranging pixel 22a.
  • Figure 6 shows an example of measuring the surrounding environment including subject 6b via transparent object A20.
  • transparent object A10 is placed in scanning range 12r of polygon mirror 53.
  • Subject 6b is located on the far side of transparent object A10. Note that in Figure 6, the distance value corresponding to subject 6b is schematically shown as subject d6b.
  • FIG. 7 shows examples of the first point cloud data g10a and second point cloud data g20 in the measurement of FIG. 6.
  • FIG. 7(a) is a diagram showing a schematic diagram of the second point cloud data g20.
  • the second point cloud data g20 is used for obstacle determination by the obstacle determination unit 37, which will be described later.
  • the second point cloud data g20 is data associated with each ranging pixel 22a, and includes distance values and illumination direction information.
  • Figure 7(b) is a diagram schematically showing the first point cloud data g10a.
  • the first point cloud data g10a is used for self-position estimation by the self-position estimation unit 36, which will be described later.
  • Figure 7(c) is a diagram showing an example of the first point cloud data g10a and a histogram used to generate element p20 of the second point cloud data g20.
  • the vertical axis represents the time digital signal generated by the TDC 31, and the vertical axis represents the number of measurements within the bin width.
  • the first reflected light pulse signal whose light intensity corresponds to the reflectance of transparent object A10
  • the second reflected light pulse signal whose light intensity corresponds to the reflectance of subject 6b
  • the histogram when passing through transparent object A10 is generally measured as two or more peaks, h20 and h10a.
  • the second point cloud data g20 is distance value data based on the position of the peak h20 on the close distance side.
  • the distance value of transparent object A10 is used in the second point cloud data g20.
  • the first point cloud data g10a is distance value data based on the position of the second peak h10a on the long distance side.
  • the first point cloud data g10a uses a distance value based on the position of the second peak h10a on the far side of transparent object A10.
  • the distance values of areas other than transparent object A10 are the same in the first point cloud data g10a and the second point cloud data g20.
  • distance value data is generated in the first point cloud data g10a based on the position of the peak h10a on the farthest side.
  • the first point cloud data g10a uses a distance value based on the position of the peak h10a on the farthest side from the transparent object A10 instead of the distance value of the transparent object A10.
  • the distance values of areas other than the transparent object A10 are the same in the first point cloud data g10a and the second point cloud data g20.
  • the echo identification unit 34 of the signal processing unit 33 determines the number of peaks in the histogram for each ranging pixel 22a. If there are two or more peaks, the echo identification unit 34 determines that measurement was performed through a transparent object. In other words, if there are two or more peaks in the histogram, the echo identification unit 34 determines that a transparent object is present.
  • the echo discrimination unit 34 outputs information about the first peak of the histogram, which corresponds to the second peak on the long distance side, to the distance generation unit 35.
  • the echo discrimination unit 34 determines that there are two peaks in the histogram, it outputs information about the second peak of the histogram, which corresponds to the shortest distance side, in addition to information about the first peak, to the distance generation unit 35.
  • the echo discrimination unit 34 can determine that measurement was performed through a transparent object.
  • the echo discrimination unit 34 determines that there are two peaks in the histogram, it outputs information about the second peak of the second histogram, which corresponds to the shortest distance side, in addition to information about the first histogram, to the distance generation unit 35. In other words, when there are two peaks in the histogram, the echo discrimination unit 34 can determine that measurement was performed through a transparent object.
  • the echo discrimination unit 34 sets the histogram corresponding to the farthest side as the first histogram and outputs information about the first peak of the first histogram to the distance generation unit 35.
  • the echo discrimination unit 34 determines that the histogram has three or more peaks, it outputs information about the second peak of the second histogram corresponding to the closest side to the distance generation unit 35 in addition to the information about the first histogram. In other words, when the histogram has two or more peaks, the echo discrimination unit 34 can determine that measurement was performed through a transparent object.
  • the echo discrimination unit 34 determines that the histogram has three or more peaks, it sets the histogram corresponding to the farthest side as the first histogram and outputs information about the second peak of the second histogram corresponding to the closest side to the distance generation unit 35 in addition to the information about the first histogram. In other words, if there are two or more peaks in the histogram, the echo identification unit 34 can determine that the measurement was performed through a transparent object.
  • the distance generation unit 35 calculates first distance information to the subject 6 for each ranging pixel 22a, for example, by calculating the center of gravity of the RX pulse signal in the first histogram based on the information of the first peak, and outputs this information to the obstacle determination unit 37.
  • the distance generation unit 35 generates first point cloud data, which is information on the first distance and the illumination direction from the aircraft at the first distance, based on the information in the storage unit 30b, and supplies this to the self-position estimation unit 36.
  • the illumination direction from the aircraft corresponds to, for example, the illumination direction from the aircraft centered on the illumination point of the optical system 12.
  • the distance generation unit 35 calculates the first distance to the subject 6 by, for example, calculating the center of gravity of the RX pulse signal in the first histogram based on the first peak information; if second peak information is available, the distance generation unit 35 calculates the second distance to the subject 6 for each ranging pixel 22a by, for example, calculating the center of gravity of the RX pulse signal in the second histogram based on the second peak information.
  • the distance generation unit 35 then generates information on the first distance and information on the direction of illumination from the aircraft at the first distance as first point cloud data based on the information in the storage unit 30b, and supplies this to the self-position estimation unit 36. Furthermore, if there is information on the first distance, information on the direction of illumination from the aircraft at the first distance, and information on the second distance, the distance generation unit 35 generates information on the second distance and information on the direction of illumination from the aircraft at the second distance as second point cloud data based on the information in the storage unit 30b, and supplies this to the obstacle determination unit 37.
  • the self-position estimation unit 36 generates map information of the self-position and the surrounding environment of the self-aircraft based on the first distance and the illumination direction from the self-aircraft. In other words, the self-position estimation unit 36 uses the first point cloud data to create the shapes of surrounding objects. It is known that detecting objects that are as far away as possible is effective in improving the accuracy of the self-position.
  • the obstacle determination unit 37 detects obstacles in the direction of the aircraft's path based on the second distance and the direction of illumination from the aircraft. In other words, the obstacle determination unit 37 can create the shape and position of surrounding obstacles using the second point cloud data. Details of the self-position estimation unit 36 and the obstacle determination unit 37 will be described later.
  • the driving control device 4 has a central processing unit 40, a motion sensor 41, an encoder 42, and a main memory device 43.
  • the central processing unit 40 is composed of a CPU (Central Processing Unit) and executes a predetermined program to control the drive unit 5 and cause the moving body 100 to move. In other words, the central processing unit 40 controls the movement of the moving body 100 based on control information generated by the self-position estimation unit 36 and the obstacle determination unit 37.
  • CPU Central Processing Unit
  • the motion sensor 41 measures the angular velocity and angular acceleration of the wheels of the mobile body 100, and supplies a velocity information signal containing angular velocity and angular acceleration information to the self-position estimation unit 36.
  • the encoder 42 is an encoder for the wheels of the mobile body 100, measures the rotation angle of the wheels, and supplies a direction information signal containing rotation angle information to the self-position estimation unit 36.
  • the self-position estimation unit 36 calculates the self-position, velocity vector, and acceleration vector of the vehicle (mobile body 100).
  • the self-position estimation unit 36 calculates the velocity vector and acceleration vector of the vehicle based on the angular velocity, angular acceleration, and wheel rotation angle of the wheels supplied from the motion sensor 41 and encoder 42.
  • the self-position estimation unit 36 calculates the vehicle's relative position from the point where it started moving by accumulating the wheel angular velocity and the trajectory of the wheel rotation angle.
  • the self-position estimation unit 36 also uses the vehicle's relative position at the time the first point cloud data was acquired as the reference coordinate.
  • the self-position estimation unit 36 uses the acquired first point cloud data to create a first map indicating the distance value to a subject (object) 6 in the surrounding environment, while estimating the vehicle's position. If a second map of the driving environment exists, the self-position estimation unit 36 can also match the position information of the second map with the position information of the generated first map to generate the vehicle's absolute position.
  • the obstacle determination unit 37 determines that a transparent object exists in the irradiation direction in which two or more peaks exist, and at the distance corresponding to the peak on the short-distance side.
  • the obstacle determination unit 37 determines the possibility of the moving body 100 colliding with an obstacle based on the velocity vector and acceleration vector at the moving body's own position obtained by the self-position estimation unit 36, and the second point cloud data.
  • This obstacle determination unit 37 divides the spatial region in the moving body's 100's direction of travel into small regions that are continuous in the moving body's direction of travel, based on the velocity vector. For each divided small region, the obstacle determination unit 37 configures the moving obstacle with a distance value based on the second point cloud data.
  • the obstacle determination unit 37 also calculates the risk of collision with an obstacle based on the velocity vector and acceleration vector.
  • the obstacle determination unit 37 sets an obstacle area where an obstacle may exist based on the collision risk for each small area.
  • the obstacle determination unit 37 then outputs obstacle control information to the central processing unit 40 for controlling the movement of the vehicle (mobile body 100) based on the distance from the vehicle to the obstacle area.
  • the central processing unit 40 calculates the angular velocity, angular acceleration, and rotational angle of the wheels based on the first map information and the vehicle's own position, and controls the drive unit 5 to the next destination. At this time, the vehicle's travel path is changed based on the obstacle control information from the obstacle determination unit 37. Alternatively, the vehicle's speed can be reduced. In this way, even when a transparent object is positioned as an obstacle, it is possible to avoid collisions and control driving based on the position information from the first map.
  • Figure 8 is a flowchart showing an example of processing by the information processing device 3.
  • an example of generating point cloud data after the histogram generation unit 32 generates a histogram is described.
  • the pixel array unit 22 has a two-dimensional array of ranging pixels 22a, 0 to 56 vertically (Y) and 0 to 192 horizontally (X).
  • the echo identification unit 34 acquires 57 x 193 histograms corresponding to each ranging pixel 22a from the storage unit 30b (step S10).
  • the distance generation unit 35 determines whether the number of peaks n associated with (X, Y) is 1 (step S18). If the distance generation unit 35 determines that n is 1 (Yes in step S18), it performs depth conversion (calculates distance) based on the peak position (step S20). Then, the distance value, illuminated target direction, and (X, Y) coordinates are associated with each other and supplied to the obstacle determination unit 37 as part of the second point cloud data (step S22). Next, the distance value, illuminated target direction, and (X, Y) coordinates are associated with each other and supplied to the self-position estimation unit 36 as part of the first point cloud data (step S24).
  • the distance generation unit 35 determines that n is not 1 (No in step S18), it performs depth conversion (calculates distance) based on the peak position on the closest side (step S32), associates the distance value, illuminated target direction, and (X, Y) coordinates, and supplies them to the obstacle determination unit 37 as part of the second point cloud data (step S34). Then, the processing from step S26 is repeated.
  • the distance generation unit 35 determines that n is not 1 (No in step S18), it performs depth conversion (calculates distance) based on the peak position on the second closest side (step S36), associates the distance value, illuminated target direction, and (X, Y) coordinates, and supplies them to the self-position estimation unit 36 as part of the first point cloud data (step S38). Then, the processing from step S26 is repeated.
  • the distance generation unit 35 performs depth conversion (calculates distance) based on the peak position on the farthest side (step S36), associates the distance value, target direction, and (X, Y) coordinates, and supplies them to the self-position estimation unit 36 as part of the first point cloud data (step S38).
  • first point cloud data and second point cloud data are generated according to the number of peaks in the histogram.
  • the echo identification unit 34 determines the number of peaks in the histogram generated by the histogram generation unit 32, and when there are multiple peaks, the distance generation unit 35 supplies the obstacle determination unit 37 with a distance value based on the short-distance peak, and supplies the self-position estimation unit 36 with a distance value based on the long-distance peak.
  • This allows the obstacle determination unit 37 to perform obstacle determination based on the short-distance distance value, and the self-position estimation unit 36 to perform self-position estimation based on the long-distance distance value, even when distance values for the surrounding environment are generated through a transparent object. Therefore, even when a transparent object is present, it is possible to suppress a decrease in the processing accuracy of the obstacle determination unit 37 and the self-position estimation unit 36.
  • the moving body 100 according to the modification of the first embodiment differs from the moving body 100 according to the first embodiment in that, when there are two or more peaks in the histogram, the light emission intensity is changed to enable false shadow detection.
  • the differences from the moving body 100 according to the first embodiment will be described below.
  • Figure 9 shows an example histogram when a false shadow occurs.
  • the vertical axis shows the time digital signal generated by the TDC 31, and the vertical axis shows the number of measurements within the bin width.
  • Peak h30a is a peak caused by lens flare/ghosting.
  • Peak h30b is a peak caused by reflection from the subject 6.
  • the central processing unit 40 reduces the emission intensity of the laser emission unit 10 and causes it to emit light again.
  • the echo identification unit 34 compares, for example, a first ratio of the heights of peaks h30a and h30b in the initially generated histogram with a second ratio of the heights of peaks h30a and h30b in the regenerated histogram, and determines that there is no artifact if the ratio is within a predetermined range.
  • the echo identification unit 34 compares the first ratio with the second ratio, and determines that there is an artifact if the ratio is outside the predetermined range. If there is an artifact, it is possible to invalidate the information in the histogram associated with that ranging pixel 22a. Alternatively, the emission intensity can be changed again and measured again. This makes it possible to prevent unnecessary obstacle avoidance operations from being performed.
  • the technology according to the present disclosure can be applied to various products.
  • the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.
  • Figure 10 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology disclosed herein can be applied.
  • the vehicle control system 12000 includes multiple electronic control units connected via a communication network 12001.
  • the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an inside vehicle information detection unit 12040, and an integrated control unit 12050.
  • the functional configuration of the integrated control unit 12050 also includes a microcomputer 12051, an audio/video output unit 12052, and an in-vehicle network I/F (Interface) 12053.
  • the drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain in accordance with various programs.
  • the drivetrain control unit 12010 functions as a control device for a driveforce generating device such as an internal combustion engine or drive motor that generates vehicle driveforce, a driveforce transmission mechanism that transmits driveforce to the wheels, a steering mechanism that adjusts the vehicle's steering angle, and a braking device that generates vehicle braking force.
  • the body system control unit 12020 controls the operation of various devices installed in the vehicle body according to various programs.
  • the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps.
  • radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020.
  • the body system control unit 12020 accepts these radio waves or signal inputs and controls the vehicle's door lock device, power window device, lamps, etc.
  • the outside vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000.
  • the outside vehicle information detection unit 12030 is connected to an imaging unit 12031.
  • the outside vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images.
  • the outside vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.
  • the imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received.
  • the imaging unit 12031 can output the electrical signal as an image, or as distance measurement information.
  • the light received by the imaging unit 12031 may be visible light, or may be invisible light such as infrared light.
  • the in-vehicle information detection unit 12040 detects information inside the vehicle. Connected to the in-vehicle information detection unit 12040 is, for example, a driver state detection unit 12041 that detects the driver's state.
  • the driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.
  • Microcomputer 12051 can calculate control target values for the driving force generating device, steering mechanism, or braking device based on information inside and outside the vehicle acquired by external vehicle information detection unit 12030 or internal vehicle information detection unit 12040, and output control commands to drivetrain control unit 12010.
  • microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following based on inter-vehicle distance, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.
  • ADAS Advanced Driver Assistance System
  • the microcomputer 12051 controls the driving force generating device, steering mechanism, braking device, etc. based on information about the vehicle's surroundings acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040, thereby enabling cooperative control aimed at autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.
  • the microcomputer 12051 can output control commands to the body system control unit 12020 based on information outside the vehicle acquired by the vehicle exterior information detection unit 12030.
  • the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the vehicle exterior information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.
  • the audio/video output unit 12052 transmits at least one audio and/or video output signal to an output device capable of visually or audibly notifying vehicle occupants or the outside of the vehicle of information.
  • an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices.
  • the display unit 12062 may include, for example, at least one of an on-board display and a head-up display.
  • Figure 11 shows an example of the installation location of the imaging unit 12031.
  • the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.
  • the imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100.
  • the imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly capture images of the front of the vehicle 12100.
  • the imaging units 12102 and 12103 provided on the side mirrors mainly capture images of the sides of the vehicle 12100.
  • the imaging unit 12104 provided on the rear bumper or back door mainly captures images of the rear of the vehicle 12100.
  • the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.
  • Imaging range 12111 indicates the imaging range of imaging unit 12101 provided on the front nose
  • imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 provided on the side mirrors, respectively
  • imaging range 12114 indicates the imaging range of imaging unit 12104 provided on the rear bumper or back door.
  • At least one of the image capturing units 12101 to 12104 may have a function for acquiring distance information.
  • at least one of the image capturing units 12101 to 12104 may be a stereo camera consisting of multiple image capturing elements, or an image capturing element having pixels for phase difference detection.
  • the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the closest three-dimensional object on the path of the vehicle 12100 that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km/h or higher). Furthermore, the microcomputer 12051 can set the inter-vehicle distance that should be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.
  • automatic braking control including follow-up stop control
  • automatic acceleration control including follow-up start control
  • the microcomputer 12051 can classify and extract three-dimensional object data regarding three-dimensional objects into categories such as motorcycles, standard vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into those that are visible to the driver of the vehicle 12100 and those that are difficult to see.
  • the microcomputer 12051 determines the collision risk, which indicates the risk of collision with each obstacle, and when the collision risk is equal to or exceeds a set value and a collision is possible, it can provide driving assistance to avoid a collision by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drivetrain control unit 12010.
  • At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays.
  • the microcomputer 12051 can recognize pedestrians by determining whether or not a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points in the images captured by the image capturing units 12101 to 12104 as infrared cameras, and performing pattern matching processing on a series of feature points that indicate the outline of an object to determine whether or not the object is a pedestrian.
  • the audio/video output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis.
  • the audio/video output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian in a desired position.
  • the technology disclosed herein can be applied to the outside vehicle information detection unit 12030 and the image capture unit 12031. Because more accurate distance images can be obtained, it becomes possible to further improve the accuracy of cooperative control aimed at autonomous driving and other autonomous driving.
  • This technology can be configured as follows:
  • the plurality of light emission pulse signals are emitted in a plurality of irradiation directions; in the generating step, the histogram is generated for each of the plurality of irradiation directions;
  • the information processing method described in (1) wherein, in the signal processing step, when two peaks exist in the histogram for each of the plurality of illumination directions, an obstacle is determined using a first distance value based on the peak on the closest side, and the self-position is estimated using a second distance value based on the peak on the farthest side.
  • a time measurement unit that generates a value related to the return time of a reflected light pulse signal corresponding to a plurality of light pulse signals that are repeatedly emitted at predetermined time intervals; a histogram generating unit that generates a histogram of the number of measurements of the reflected light pulse signal relative to the value related to the return time; a signal processing unit that, when two peaks exist in the histogram, determines an obstacle relative to the aircraft using a distance value based on one of the peaks, and estimates the aircraft's own position relative to the aircraft using a distance value based on the other peak;
  • An information processing device comprising:

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Abstract

Le problème décrit par la présente invention est de fournir un procédé de traitement d'informations, un dispositif de traitement d'informations et un programme qui permettent d'empêcher un déclin de la précision de traitement d'une détection d'obstacle et d'une estimation de position de véhicule hôte même si la lumière est éclairée à travers un objet transparent. La solution selon l'invention porte sur un procédé de traitement d'informations comprenant : une étape de mesure consistant à générer des valeurs relatives à des temps de retour de signaux d'impulsion de lumière réfléchie correspondant à une pluralité de signaux d'impulsion d'émission de lumière émis de manière répétée à des intervalles de temps prédéterminés ; une étape de génération consistant à générer un histogramme représentant le nombre de mesures des signaux d'impulsion de lumière réfléchie pour chacune des valeurs associées aux temps de retour ; et une étape de traitement de signal consistant, lorsque l'histogramme comprend deux pics, à déterminer la présence d'un obstacle par rapport à un véhicule hôte à l'aide d'une valeur de distance sur la base de l'un des pics et à estimer la position du véhicule hôte à l'aide d'une valeur de distance sur la base de l'autre pic.
PCT/JP2025/004426 2024-03-19 2025-02-10 Procédé de traitement d'informations, dispositif de traitement d'informations et programme Pending WO2025197360A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2024-043922 2024-03-19
JP2024043922 2024-03-19

Publications (1)

Publication Number Publication Date
WO2025197360A1 true WO2025197360A1 (fr) 2025-09-25

Family

ID=97139238

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2025/004426 Pending WO2025197360A1 (fr) 2024-03-19 2025-02-10 Procédé de traitement d'informations, dispositif de traitement d'informations et programme

Country Status (1)

Country Link
WO (1) WO2025197360A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021024685A1 (fr) * 2019-08-05 2021-02-11 ソニー株式会社 Dispositif, procédé et programme de traitement d'informations
US20210132213A1 (en) * 2019-10-31 2021-05-06 Samsung Electronics Co., Ltd. Apparatus and methods for multi-sensor slam systems
WO2023281802A1 (fr) * 2021-07-06 2023-01-12 ソニーグループ株式会社 Dispositif de traitement de signal et procédé de traitement de signal

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021024685A1 (fr) * 2019-08-05 2021-02-11 ソニー株式会社 Dispositif, procédé et programme de traitement d'informations
US20210132213A1 (en) * 2019-10-31 2021-05-06 Samsung Electronics Co., Ltd. Apparatus and methods for multi-sensor slam systems
WO2023281802A1 (fr) * 2021-07-06 2023-01-12 ソニーグループ株式会社 Dispositif de traitement de signal et procédé de traitement de signal

Similar Documents

Publication Publication Date Title
US12571892B2 (en) Distance measurement device and distance measurement method
JP7494200B2 (ja) 照明装置、照明装置の制御方法、および、測距モジュール
US20220381913A1 (en) Distance measurement sensor, signal processing method, and distance measurement module
WO2021065495A1 (fr) Capteur de télémétrie, procédé de traitement de signal, et module de télémétrie
JP7490653B2 (ja) 測定装置および測定方法、並びにプログラム
WO2021241360A1 (fr) Dispositif de détection, système de détection et procédé de détection
WO2025197360A1 (fr) Procédé de traitement d'informations, dispositif de traitement d'informations et programme
WO2024195531A1 (fr) Dispositif de détection optique et système de télémétrie
US20240272285A1 (en) Light source control device, light source control method, and distance measuring device
JP7645226B2 (ja) センシングシステム、および、測距システム
US20230341556A1 (en) Distance measurement sensor, signal processing method, and distance measurement module
US20240168159A1 (en) Distance measuring device, distance measuring system, and distance measuring method
US20240319341A1 (en) Distance measuring device and distance measuring method
WO2025192071A1 (fr) Dispositif et procédé de mesure de distance
KR20240168357A (ko) 측거 장치 및 측거 방법
WO2023190279A1 (fr) Dispositif de télémétrie
WO2023189917A1 (fr) Dispositif de télémétrie et procédé de télémétrie
WO2022269995A1 (fr) Dispositif, procédé et programme de mesure de distance
JP2024129985A (ja) カメラシステムおよびアプリケーションプロセッサ
WO2025121217A1 (fr) Dispositif de mesure de distance
JP2024132574A (ja) 発光装置、及び測距システム
WO2024190189A1 (fr) Élément de détection de lumière, procédé de télémétrie et système de télémétrie
JP2024125795A (ja) 光検出装置および測距システム
WO2024203104A1 (fr) Dispositif de commande, dispositif de mesure de distance et procédé de commande
WO2024009739A1 (fr) Capteur de télémétrie optique et système de télémétrie optique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25773915

Country of ref document: EP

Kind code of ref document: A1