WO2019196476A1 - 基于激光传感器生成地图 - Google Patents

基于激光传感器生成地图 Download PDF

Info

Publication number
WO2019196476A1
WO2019196476A1 PCT/CN2018/121051 CN2018121051W WO2019196476A1 WO 2019196476 A1 WO2019196476 A1 WO 2019196476A1 CN 2018121051 W CN2018121051 W CN 2018121051W WO 2019196476 A1 WO2019196476 A1 WO 2019196476A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
data
cloud data
glass
laser
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/121051
Other languages
English (en)
French (fr)
Inventor
程保山
申浩
郝立良
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sankuai Online Technology Co Ltd
Original Assignee
Beijing Sankuai Online Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sankuai Online Technology Co Ltd filed Critical Beijing Sankuai Online Technology Co Ltd
Priority to EP18914482.7A priority Critical patent/EP3772041A4/en
Priority to US17/044,317 priority patent/US11315264B2/en
Publication of WO2019196476A1 publication Critical patent/WO2019196476A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
    • G05D1/024—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors in combination with a laser
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/56—Particle system, point based geometry or rendering

Definitions

  • the present application relates to generating a map based on a laser sensor.
  • a laser sensor can be used to measure environmental data around an object, and the measured environmental data is subjected to Simultaneous Localization and Mapping (SLAM) calculation to construct a map.
  • SLAM Simultaneous Localization and Mapping
  • current laser sensors do not recognize glass, making SLAM limited during use.
  • the present application provides a laser sensor-based map generation method and apparatus, a mobile device, and a computer readable storage medium.
  • a laser sensor-based map generation method comprising:
  • a laser sensor-based map generating apparatus comprising:
  • An acquisition module configured to acquire image data, which is obtained by a visual sensor
  • a determining module configured to determine, according to the image data, first point cloud data belonging to a glass-like region in the laser data, the laser data being acquired by the laser sensor and corresponding to the image data, and the vision
  • the time period during which the sensor acquires the image data is the same as the time period during which the laser sensor collects the laser data;
  • An adjusting module configured to adjust a weight of the laser data according to the first point cloud data
  • a processing module configured to perform fusion processing on the first point cloud data and the second point cloud data belonging to the non-glass-like area in the laser data based on the adjusted weight of the laser data to generate a map.
  • a computer readable storage medium when the computer program is called by a processor, the computer program causes the processor to execute the laser sensor-based map generation method described above.
  • a mobile device comprising a vision sensor, a laser sensor, a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor executing
  • the above-described computer program implements the above-described laser sensor-based map generation method.
  • the weight of the laser data is adjusted according to the first point cloud data, and then the first point cloud data and the laser are compared based on the adjusted weight of the laser data.
  • the second point cloud data belonging to the non-class glass area in the data is subjected to fusion processing to generate a map.
  • the first point cloud data with small weight can appear in the merged map data with a very small probability, thereby improving the accuracy of the generated map and high availability.
  • FIG. 1 is a flowchart of a laser sensor-based map generation method according to an exemplary embodiment of the present application
  • FIG. 2A is a flowchart of a method for determining point cloud data belonging to a glass-like region in associated laser data based on image data, according to an exemplary embodiment of the present application;
  • 2B is a schematic view of a pattern corresponding to a glass-like region shown in an exemplary embodiment of the present application
  • 2C is a schematic diagram of a pattern corresponding to a glass-like region shown by another exemplary embodiment of the present application.
  • 2D is a schematic diagram of a pattern corresponding to a glass-like region shown in still another exemplary embodiment of the present application.
  • 2E is a schematic diagram of a pattern corresponding to a glass-like region shown in still another exemplary embodiment of the present application.
  • 2F is a schematic diagram of a glass-like region in a current image identified by a first recognition model, according to an exemplary embodiment of the present application
  • 2G is a schematic diagram of a glass-like region in a current image identified by a second recognition model, shown by another exemplary embodiment of the present application;
  • FIGS. 2F and 2G are schematic diagrams of a glass-like region in the current image based on FIGS. 2F and 2G;
  • FIG. 3A is a flowchart of a method for reducing a first weight corresponding to first point cloud data according to an exemplary embodiment of the present application
  • FIG. 3B is a flowchart of a method for increasing a second weight corresponding to second point cloud data according to an exemplary embodiment of the present application
  • FIG. 4 is a flowchart of a method for performing fusion processing on first point cloud data and second point cloud data based on weights of adjusted laser data according to an exemplary embodiment of the present application;
  • FIG. 5 is a hardware structural diagram of a mobile device according to an exemplary embodiment of the present application.
  • FIG. 6 is a structural block diagram of a laser sensor-based map generating apparatus according to an exemplary embodiment of the present application.
  • FIG. 7 is a structural block diagram of a laser sensor-based map generating apparatus according to another exemplary embodiment of the present application.
  • first, second, third, etc. may be used to describe various information in this application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information without departing from the scope of the present application.
  • second information may also be referred to as the first information.
  • word "if” as used herein may be interpreted as "when” or “when” or “in response to a determination.”
  • the present application provides a method of generating a map based on a laser sensor and a vision sensor.
  • the method can be applied to a mobile device having a vision sensor (eg, a camera) and a laser sensor, which can include, but is not limited to, an unmanned vehicle, a robot, etc., and can also be applied to a server, and the mobile device will collect the collected data.
  • a vision sensor eg, a camera
  • a laser sensor which can include, but is not limited to, an unmanned vehicle, a robot, etc.
  • the server Transferred to the server, the server generates a map based on the data collected by the mobile device and transmits the map to the mobile device.
  • the method includes steps S101-S104:
  • Step S101 acquiring image data, which is acquired by a visual sensor.
  • the image data may be acquired by a visual sensor on the mobile device, or the image data may be acquired by other visual sensors, and the collected image data may be transmitted to the mobile device.
  • Step S102 determining first point cloud data belonging to the glass-like region in the laser data based on the image data.
  • the laser data is acquired by the laser sensor and corresponds to the image data
  • the time period during which the visual sensor acquires the image data is the same as the time period during which the laser sensor collects the laser data .
  • the glass-like area includes a light-transmissive medium area that the laser sensor can not detect, but the visual sensor can capture. Since the vision sensor can capture the glass-like region, the first point cloud data belonging to the glass-like region in the laser data can be determined based on the image data.
  • the laser data is referred to as the associated laser data of the image data.
  • the image data and laser data respectively captured by the vision sensor and the laser conventional device for the region 1 at the time of 12:00 to 12:01 Beijing time may be referred to as associated data.
  • the specific time period and the division of the area 1 may be set according to the device condition, the network condition, the environment complexity of the shooting area, etc., and no specific limitation is made here.
  • the first point cloud data belonging to the glass-like region in the associated laser data is determined based on the image data, and real-time detection of the glass-like region can be realized.
  • determining, based on the image data, the first point cloud data belonging to the glass-like region in the laser data may include steps S1021-S1022:
  • Step S1021 identifying a glass-like region in the image data.
  • identifying the glass-like data in the image data comprises: dividing the image corresponding to the image data into a plurality of sub-images; determining whether each sub-image belongs to the glass-like region by one or more pre-trained recognition models; The sub-images belonging to the glass-like region determine the glass-like regions in the image data.
  • the mobile device or the server can acquire the training picture set, and can use the convolutional neural network (Faster R-CNN) to train one or more recognition models, and then use the trained one or more recognition models to detect the image data. Glass-like area.
  • Faster R-CNN convolutional neural network
  • determining whether each sub-image belongs to the glass-like region by using the pre-trained recognition model may include: inputting each sub-image into the recognition model, and obtaining a probability that the sub-image belongs to the glass-like region based on the recognition model; If it is determined that the probability that the sub-image belongs to the glass-like region is greater than the probability threshold corresponding to the recognition model based on the recognition model, determining that the sub-image belongs to the glass-like region; and determining that the sub-image belongs to the glass-like region based on the recognition model If the probability of the region is less than or equal to the probability threshold corresponding to the identification model, it is determined that the sub-image belongs to a non-glass-like region.
  • the image corresponding to the image data may be segmented into a plurality of sub-images according to a segmentation manner used in training the recognition model, and each sub-image is input into the recognition model, and it is determined that the sub-image belongs to the class based on the recognition model.
  • the probability of the glass area may be determined that the sub-image belongs to the class based on the recognition model.
  • the area 21 and the area 22 represent the upper and lower portions of a scene
  • the area 23 represents the middle of the scene and may be a frosted glass-like area or a film-like glass-like area
  • the area 21 and the area 22 are the area 23 cut off.
  • the sub-image 3, the sub-image 4, the sub-image 5, and the sub-image 6 of a certain image each include the pattern shown in FIG. 2C, it can be determined that the area composed of the sub-image 3, the sub-image 4, the sub-image 5, and the sub-image 6 is A glass-like area in image data.
  • pre-training a plurality of recognition models, and determining, by the plurality of recognition models, whether the sub-images belong to the glass-like region comprises: inputting the sub-images into each of the recognition models, respectively, to obtain a recognition model based on the recognition model Determining a probability that the sub-image belongs to a glass-like region; and determining that the sub-image belongs to when a probability that the sub-image belongs to a glass-like region is determined to be greater than a probability threshold corresponding to the recognition model based on each of the recognition models Glass-like area.
  • the probability that the sub-image belongs to the glass-like region is determined to be greater than the probability threshold corresponding to the recognition model based on each of the recognition models, and the obtained glass-like region is more accurate.
  • Determining the glass-like region in the image data according to the sub-images belonging to the glass-like region includes: combining the sub-images belonging to the glass-like region as the glass-like region in the image data, and the glass-like region thus obtained is more accurate.
  • the identified glass-like region can be used as the glass-like region in the image data by determining that the probability that the sub-image belongs to the glass-like region is greater than the probability threshold corresponding to the recognition model based on a certain recognition model.
  • FIG. 2F is a glass-like region 24 in the current image identified by the first recognition model
  • FIG. 2G is a glass-like region 25 in the current image identified by the second recognition model.
  • the union of the glass-like region 24 and the glass-like region 25, i.e., the glass-like region 26, is used as the glass-like region of the image, as shown in Fig. 2H.
  • the probability that the sub-images belong to the glass-like region is greater than the probability threshold corresponding to each of the recognition models based on the plurality of recognition models, and the identified glass-like regions can be used as the glass-like regions in the image data. For example, when the probability that the sub-image belongs to the glass-like region is greater than the probability threshold corresponding to the first recognition model based on the first recognition model, and the probability that the sub-image belongs to the glass-like region based on the second recognition model is greater than the second recognition model At the probability threshold, it is determined that the sub-image belongs to a glass-like region. In this way, the resulting glass-like region is more accurate.
  • the probability that the sub-image belongs to the glass-like region based on any of the recognition models is greater than the probability threshold corresponding to the recognition model, it is determined that the sub-image belongs to the glass-like region, and thus the obtained glass-like region is not missed.
  • Step S1022 Determine first point cloud data in the associated laser data according to an external parameter between the glass-like region in the image data and the pre-received visual sensor and the laser sensor.
  • the external parameters may be manually calibrated by a Robust Automatic Detection in Laser Of Calibration Chessboards (RADLOCC) toolbox (TOOLBOX) and input into the mobile device.
  • RADLOCC Robust Automatic Detection in Laser Of Calibration Chessboards
  • the above external parameter refers to a parameter representing a spatial rotation and translation relationship between a coordinate system in which the visual sensor is located and a coordinate system in which the laser sensor is located, and the representation form may be a 3*3 rotation matrix and a 3*1 translation vector. It can be a 4*1 quaternion vector and a 3*1 translation vector.
  • determining the first point cloud data belonging to the glass-like region in the associated laser data can effectively reduce the workload of manually correcting the glass wall Improve the efficiency of map generation.
  • Step S103 adjusting weights of the laser data according to the first point cloud data.
  • Adjusting the weight of the associated laser data according to the first point cloud data includes any one or more of the following: reducing a first weight corresponding to the first point cloud data; and increasing a non-glass type region in the laser data
  • the second point of cloud data corresponds to the second weight.
  • the first weight corresponding to the first point cloud data may be reduced, the second weight corresponding to the second point cloud data may be increased, and the first weight corresponding to the first point cloud data may be decreased and the second is increased.
  • the second weight corresponding to the point cloud data may be reduced, the second weight corresponding to the second point cloud data may be increased, and the first weight corresponding to the first point cloud data may be decreased and the second is increased. The second weight corresponding to the point cloud data.
  • reducing the first weight corresponding to the first point cloud data may include steps S1031 - S1032 :
  • Step S1031 reducing the confidence of the first point cloud data.
  • the accuracy of the first point cloud data is relatively poor, in order to improve the accuracy of generating the map, the confidence of the first point cloud data can be reduced.
  • the confidence of the first point cloud data can be reduced to 0-0.1.
  • step S1032 the first weight corresponding to the first point cloud data is reduced according to the reduced confidence.
  • the first weight corresponding to the first point cloud data may be reduced according to the reduced confidence, that is, the reduced first weight is related to the reduced confidence. For example, if the reduced confidence is 0, the first weight can be reduced to 0.01 or 0, and the like. It should be noted that the foregoing 0.01 or 0 is only an example. In an actual application, the first weight size may be adjusted as needed.
  • increasing the second weight corresponding to the second point cloud data includes steps S1033-S1034.
  • step S1033 the confidence of the second point cloud data is increased.
  • the accuracy of the second point cloud data is relatively good, in order to improve the accuracy of generating the map, the confidence of the second point cloud data can be improved.
  • the confidence of the second point cloud data can be increased to 0.9 to 1.
  • step S1034 according to the improved confidence, the second weight corresponding to the second point cloud data is increased.
  • the second weight corresponding to the second point cloud data may be increased according to the increased confidence, that is, the second weight after the improvement is related to the improved confidence. For example, if the increased confidence is 0.9, the second weight can be increased to 0.99 or 1 or the like. It should be noted that the above 0.99 or 1 is only an example. In practical applications, the second weight size may be adjusted as needed.
  • the first weight corresponding to the first point cloud data is decreased according to the reduced confidence, and the second weight corresponding to the second point cloud data is increased according to the improved confidence, and the associated laser data is further fused according to the adjusted weight.
  • Processing provides the conditions that provide the conditions for generating high-precision map data.
  • Step S104 Perform fusion processing on the first point cloud data and the second point cloud data based on the weight of the adjusted laser data to generate a map.
  • the mobile device may perform fusion processing on the first point cloud data and the second point cloud data based on the weight of the adjusted laser data to obtain an initial map, and then optimize the initial map to generate a map, for example,
  • the initial map can be optimized for loop detection to generate a map to further improve the accuracy of generating the map.
  • the fusion processing of the first point cloud data and the second point cloud data based on the weight of the adjusted laser data may include:
  • Step S1041 Register the first point cloud data to generate the registered first point cloud data.
  • Step S1042 registering the second point cloud data to generate the registered second point cloud data.
  • Step S1043 based on the coordinate vector of the registered first point cloud data, the coordinate vector of the registered second point cloud data, the reduced first weight, the second weight after the enhancement, and the registered point cloud data The pose parameter, the cost function is calculated.
  • the cost function is the formula (1):
  • Wi is the weight corresponding to the registration point cloud data, wherein if the registered point cloud data is located in the glass-like region, that is, the registration For a little cloud data, Wi represents the first weight after the decrease, for example 0. If the registered point cloud data is located in the non-glass-like area, that is, the registered second point cloud data, Wi represents the second weight after the increase.
  • R, t represents the attitude parameter between the registered point cloud data, for example, R represents a rotation variation matrix of Xm to Xn, and t represents a displacement variation vector of Xm to Xn.
  • step S1044 an optimized attitude parameter is obtained by performing an iterative operation on the cost function.
  • the estimated values of R and t can be obtained by the code wheel sensor of the mobile device, then R and t are derived by F_cost, and the gradient is used to iterate continuously, when the value of F_cost is after two iterations. The iteration is stopped when the value changes very small, and R' and t' obtained at this time are optimized attitude parameters.
  • Step S1045 Perform fusion processing on the registered first point cloud data and the registered second point cloud data based on the optimized posture parameters.
  • the registered first point cloud data and the registered second point cloud data may be merged based on the optimized pose parameters.
  • the optimized attitude parameter is obtained based on the cost function, and the first point cloud data and the second point cloud data of the registration are merged based on the optimized attitude parameter, and the implementation manner is simple.
  • the first point cloud data and the second point cloud data are merged based on the adjusted weights, so that the first point cloud data with small weight appears in the merged map data with a very small probability, thereby improving the generated map. Accuracy and high availability.
  • the present application also provides an embodiment of a laser sensor-based map generation device.
  • Embodiments of the laser sensor based map generating apparatus of the present application can be applied to a mobile device.
  • the mobile device can be an unmanned vehicle, a robot, or the like. It can also be applied to a server, which generates a map and then transmits the map to the robot.
  • the device embodiment may be implemented by software, or may be implemented by hardware or a combination of hardware and software.
  • FIG. 5 is a hardware structural diagram of a mobile device according to an embodiment of the present application.
  • the mobile device includes a visual sensor 510, a laser sensor 520, a processor 530, a memory 540, and is stored in the memory 540 and can be A computer program running on the processor 530, which executes the above-described laser sensor-based map generation method when the computer program executes the computer program.
  • the mobile device may include other hardware according to the actual function of generating the map, and details are not described herein again.
  • FIG. 6 is a structural block diagram of a laser sensor-based map generating apparatus according to an exemplary embodiment of the present application, which may be applied to a mobile device having a vision sensor and a laser sensor, as shown in FIG.
  • the acquisition module 61 is configured to acquire image data, which is acquired by a visual sensor.
  • the determining module 62 is configured to determine, according to the image data, first point cloud data belonging to the glass-like region in the laser data, the laser data is acquired by the laser sensor and corresponding to the image data, and the vision
  • the time period during which the sensor acquires the image data is the same as the time period during which the laser sensor acquires the laser data.
  • the adjusting module 63 is configured to adjust the weight of the laser data according to the first point cloud data.
  • the processing module 64 is configured to perform fusion processing on the first point cloud data and the second point cloud data belonging to the non-glass-like area of the laser data based on the adjusted weight of the laser data to generate a map.
  • FIG. 7 is a structural block diagram of a laser sensor-based map generating apparatus according to another exemplary embodiment of the present application. As shown in FIG. 7, on the basis of the foregoing embodiment shown in FIG. 6, the determining module 62 may include:
  • the identification sub-module 621 is configured to identify a glass-like region in the image data
  • the determining sub-module 622 is configured to determine, according to the glass-like region in the image data identified by the recognition sub-module 621 and the external parameter between the pre-received visual sensor and the laser sensor, the first point cloud data belonging to the glass-like region in the laser data.
  • the outer parameter represents a spatial rotation and translation relationship between a first coordinate system in which the visual sensor is located and a second coordinate system in which the laser sensor is located.
  • the identifying sub-module 621 is further configured to: divide an image corresponding to the image data into a plurality of sub-images; determine, by using one or more pre-trained recognition models, whether each of the sub-images belongs to a class. a glass region; and determining a glass-like region in the image data based on a sub-image belonging to the glass-like region.
  • the identification sub-module 621 is further configured to: input the sub-image into the recognition model, and obtain a probability that the sub-image belongs to a glass-like region based on the recognition model; and When the recognition model determines that the probability that the sub-image belongs to the glass-like region is greater than a probability threshold corresponding to the recognition model, it is determined that the sub-image belongs to a glass-like region.
  • the identification sub-module 621 is further configured to: input the sub-images into each of the recognition models, and obtain a probability that the sub-image belongs to a glass-like region based on the recognition model; and And determining, according to each of the recognition models, that the probability that the sub-image belongs to the glass-like region is greater than a probability threshold corresponding to the recognition model, determining that the sub-image belongs to a glass-like region.
  • the identification sub-module 621 is further configured to use a union of sub-images belonging to the glass-like region as a glass-like region in the image data.
  • the adjusting module 63 is further configured to: reduce a first weight corresponding to the first point cloud data; and/or increase a second weight corresponding to the second point cloud data.
  • the adjusting module 63 is further configured to: reduce a confidence level of the first point cloud data; and decrease the first weight according to the reduced confidence.
  • the adjusting module 63 is further configured to: increase a confidence level of the second point cloud data; and increase the second weight according to the improved confidence.
  • the processing module 64 is further configured to perform fusion processing on the first point cloud data and the second point cloud data based on the weight of the adjusted laser data to obtain an initial map;
  • the initial map is optimized to generate the map.
  • the processing module 64 is further configured to register the first point cloud data to generate the registered first point cloud data, and register the second point cloud data to generate a match. a second point cloud data; a coordinate vector based on the registered first point cloud data, a coordinate vector of the registered second point cloud data, and a corresponding reduction of the first point cloud data of the registration Calculating a cost function by calculating a cost function by the first weight, the second weight of the second point cloud data corresponding to the registration, and the posture parameter between the registered point cloud data; and iterating through the cost function Computing, optimizing the gesture parameter; and performing fusion processing on the registered first point cloud data and the second point cloud data based on the optimized posture parameter.
  • a computer readable storage medium storing a computer program that, when invoked by a processor, causes the processor to perform the laser-based A map generation method of a sensor
  • the computer readable storage medium may include a nonvolatile machine readable storage medium, a read only memory (ROM), a random access memory (RAM), a compact disk read only memory (CD-ROM), Tapes, floppy disks, and optical data storage devices.
  • the device embodiment since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the present application. Those of ordinary skill in the art can understand and implement without any creative effort.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Electromagnetism (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Optics & Photonics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请提供一种基于激光传感器的地图生成方法。在一实施例中，所述方法包括：获取图像数据，该图像数据是通过视觉传感器采集得到的；基于图像数据确定激光数据中属于类玻璃区域的第一点云数据；根据所述第一点云数据调整所述激光数据的权重；基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非类玻璃区域的第二点云数据进行融合处理，生成地图。

Description

基于激光传感器生成地图
相关申请的交叉引用
本专利申请要求于2018年4月9日提交的、申请号为201810312694.0、发明名称为“一种基于激光传感器的地图生成方法及装置和机器人”的中国专利申请的优先权，该申请的全文以引用的方式并入本文中。
技术领域
本申请涉及基于激光传感器生成地图。
背景技术
随着计算机技术和人工智能的发展，智能移动机器人成为机器人领域的一个重要研究方向和研究热点。移动机器人的定位和地图创建是移动机器人领域的热点研究问题。
目前，可以利用激光传感器测量物体周围的环境数据，并将测量的环境数据进行即时定位与地图创建(Simultaneous localization and mapping，简称SLAM)计算来构建地图。然而，当前激光传感器无法识别出玻璃，使得在使用过程中SLAM受到了限制。
发明内容
有鉴于此，本申请提供一种基于激光传感器的地图生成方法及装置、移动设备和计算机可读存储介质。
根据本公开实施例的第一方面，提供一种基于激光传感器的地图生成方法，所述方法包括：
获取图像数据，所述图像数据是通过视觉传感器采集得到的；
基于所述图像数据确定激光数据中属于类玻璃区域的第一点云数据，所述激光数据是通过激光传感器采集得到的且与所述图像数据对应同一区域，并且所述视觉传感器采集所述图像数据的时间段与所述激光传感器采集所述激光数据的时间段相同；
根据所述第一点云数据调整所述激光数据的权重；和
基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非 类玻璃区域的第二点云数据进行融合处理，生成地图。
根据本公开实施例的第二方面，提供一种基于激光传感器的地图生成装置，所述装置包括：
获取模块，用于获取图像数据，所述图像数据是通过视觉传感器采集得到的；
确定模块，用于基于所述图像数据确定激光数据中属于类玻璃区域的第一点云数据，所述激光数据是通过激光传感器采集得到的且与所述图像数据对应同一区域，并且所述视觉传感器采集所述图像数据的时间段与所述激光传感器采集所述激光数据的时间段相同；
调整模块，用于根据所述第一点云数据调整所述激光数据的权重；
处理模块，用于基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非类玻璃区域的第二点云数据进行融合处理，生成地图。
根据本公开实施例的第三方面，提供一种计算机可读存储介质，在所述计算机程序被处理器调用时，所述计算机程序促使所述处理器执行上述基于激光传感器的地图生成方法。
根据本公开实施例的第四方面，提供一种移动设备，包括视觉传感器、激光传感器、处理器、存储器及存储在所述存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述基于激光传感器的地图生成方法。
在基于激光传感器的地图生成方法中，根据所述第一点云数据调整所述激光数据的权重，然后基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非类玻璃区域的第二点云数据进行融合处理，生成地图。在该方法中，权重小的第一点云数据可以极小的概率出现在融合后的地图数据中，从而提高生成的地图的精度，可用性高。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本发明的实施例，并与说明书一起用于解释本发明的原理。
图1是本申请一示例性实施例示出的一种基于激光传感器的地图生成方法的流程图；
图2A是本申请一示例性实施例示出的基于图像数据确定关联的激光数据中属于类玻璃区域的点云数据的方法的流程图；
图2B是本申请一示例性实施例示出的类玻璃区域对应的图案的示意图；
图2C是本申请另一示例性实施例示出的类玻璃区域对应的图案的示意图；
图2D是本申请再一示例性实施例示出的类玻璃区域对应的图案的示意图；
图2E是本申请又一示例性实施例示出的类玻璃区域对应的图案的示意图；
图2F是本申请一示例性实施例示出的第一识别模型识别出的当前图像中的类玻璃区域的示意图；
图2G是本申请另一示例性实施例示出的第二识别模型识别出的当前图像中的类玻璃区域的示意图；
图2H是基于图2F和图2G示出的当前图像中的类玻璃区域的示意图；
图3A是本申请一示例性实施例示出的降低第一点云数据对应的第一权重的方法的流程图；
图3B是本申请一示例性实施例示出的提高第二点云数据对应的第二权重的方法的流程图；
图4是本申请一示例性实施例示出的基于调整后的激光数据的权重，对第一点云数据和第二点云数据进行融合处理的方法的流程图；
图5是本申请一示例性实施例示出的移动设备的硬件结构图；
图6是本申请一示例性实施例示出的一种基于激光传感器的地图生成装置的结构框图；
图7是本申请另一示例性实施例示出的一种基于激光传感器的地图生成装置的结构框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施 例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本申请可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
基于激光传感器生成地图时，激光传感器无法探测出类玻璃区域，即激光传感器对于存在类玻璃区域的环境建图困难。有鉴于此，本申请提供基于激光传感器和视觉传感器生成地图的方法。该方法可应用于具有视觉传感器(例如，摄像机)和激光传感器的移动设备上，该移动设备可以包括但不局限于无人车、机器人等，还可以应用于服务器上，移动设备将采集的数据传输给服务器，服务器基于移动设备采集的数据生成地图并将地图传输给移动设备。如图1所示，该方法包括步骤S101-S104：
步骤S101，获取图像数据，该图像数据是通过视觉传感器采集得到的。
其中，可以通过移动设备上的视觉传感器获取图像数据，也可以通过其他的视觉传感器采集图像数据，再将采集到的图像数据传输给该移动设备。
步骤S102，基于图像数据确定激光数据中属于类玻璃区域的第一点云数据。其中，所述激光数据是通过激光传感器采集得到的且与所述图像数据对应同一区域，并且所述视觉传感器采集所述图像数据的时间段与所述激光传感器采集所述激光数据的时间段相同。
其中，类玻璃区域包括激光传感器检测不到，但视觉传感器可以拍摄到的透光介质区域。由于视觉传感器可以拍摄到类玻璃区域，因此，可以基于图像数据确定激光数据中属于类玻璃区域的第一点云数据。
当图像数据和激光数据对应于同一拍摄区域时，而且视觉传感器获取图像数据的时间段与激光传感器采集激光数据的时间段相同，该激光数据就称为该图像数据的关联的 激光数据。例如，视觉传感器和激光传统器在北京时间12：00～12:01这个时间段对区域1分别进行拍摄的图像数据及激光数据可以称为关联的数据。具体时间段以及区域1的划分可以根据设备情况、网络情况、拍摄区域的环境复杂度等来进行设置，这里不做具体限制。
基于图像数据确定关联的激光数据中属于类玻璃区域的第一点云数据，能够实现类玻璃区域的实时检测。
如图2A所示，基于图像数据确定激光数据中属于类玻璃区域的第一点云数据可以包括步骤S1021-S1022：
步骤S1021，识别图像数据中的类玻璃区域。
在一实施例中，识别图像数据中的类玻璃数据包括：将图像数据对应的图像分割成多个子图像；通过预先训练的一个或多个识别模型判断每个子图像是否属于类玻璃区域；并根据属于类玻璃区域的子图像确定图像数据中的类玻璃区域。
其中，移动设备或服务器可以获取训练图片集，并可以采用卷积神经网络(Faster R-CNN)训练出一个或多个识别模型，然后采用训练出的一个或多个识别模型检测图像数据中的类玻璃区域。
在一实施例中，通过预先训练的识别模型判断每个子图像是否属于类玻璃区域可以包括：将每个子图像输入识别模型，得到基于所述识别模型判定所述子图像属于类玻璃区域的概率；若基于所述识别模型判定该子图像属于类玻璃区域的概率大于所述识别模型对应的概率阈值，则确定该子图像属于类玻璃区域；和若基于所述识别模型判定该子图像属于类玻璃区域的概率小于或等于所述识别模型对应的概率阈值，则确定该子图像属于非类玻璃区域。
在一实施例中，可以按照训练识别模型时采用的分割方式，将图像数据对应的图像分割成多个子图像，并将每个子图像输入识别模型，得到基于该识别模型判定所述子图像属于类玻璃区域的概率。
图2B至图2E分别表示类玻璃区域对应的图案。其中，图2E中，区域21和区域22表示一场景的上部和下部，区域23表示该场景的中部并且可以为磨砂类玻璃区域或者贴有薄膜的类玻璃区域，区域21和区域22被区域23隔断。假设某个图像的子图像3、子图像4、子图像5和子图像6均包括图2C所示的图案，则可以确定子图像3、子图像4、子图像5和子图像6组成的区域为该图像数据中的类玻璃区域。
在一实施例中，预先训练多个识别模型，通过多个识别模型判断所述子图像是否属于类玻璃区域包括：将所述子图像分别输入每个所述识别模型，得到基于所述识别模型判定所述子图像属于类玻璃区域的概率；和当基于每个所述识别模型判定所述子图像属于类玻璃区域的概率均大于所述识别模型对应的概率阈值时，确定所述子图像属于类玻璃区域。在这种情况下，基于每个所述识别模型判定所述子图像属于类玻璃区域的概率均大于所述识别模型对应的概率阈值，获得的类玻璃区域更加准确。
根据属于类玻璃区域的子图像确定所述图像数据中的类玻璃区域，包括：将属于类玻璃区域的子图像的并集作为图像数据中的类玻璃区域，这样获得的类玻璃区域更准确。
假设，基于某个识别模型判定出子图像属于类玻璃区域的概率大于该识别模型对应的概率阈值，则可以将所识别出的类玻璃区域作为图像数据中的类玻璃区域。例如，图2F中示出的是第一识别模型识别出的当前图像中的类玻璃区域24，图2G中示出的是第二识别模型识别出的当前图像中的类玻璃区域25，则可以将类玻璃区域24和类玻璃区域25的并集即类玻璃区域26作为该图像的类玻璃区域，如图2H所示。
假设，基于多个识别模型判定出子图像属于类玻璃区域的概率均大于各个识别模型对应的概率阈值，则可以将识别出的类玻璃区域作为图像数据中的类玻璃区域。例如，当基于第一识别模型判定子图像属于类玻璃区域的概率大于第一识别模型对应的概率阈值，且基于第二识别模型判定该子图像属于类玻璃区域的概率大于第二识别模型对应的概率阈值时，确定该子图像属于类玻璃区域。这样，得到的类玻璃区域更加准确。或者，若基于任一识别模型判定子图像属于类玻璃区域的概率大于该识别模型对应的概率阈值，则确定该子图像属于类玻璃区域，这样，得到的类玻璃区域不会有遗漏。
步骤S1022，根据图像数据中的类玻璃区域和预先接收的视觉传感器与激光传感器之间的外参数，确定关联的激光数据中的第一点云数据。
其中，上述外参数可以由人工通过基于棋盘格标定板的激光数据鲁棒自动检测(Robust Automatic Detection in Laser Of Calibration Chessboards，简称RADLOCC)工具箱(TOOLBOX)标定，并输入到移动设备中。
上述外参数是指视觉传感器所在的坐标系和激光传感器所在的坐标系之间表示空间旋转和平移关系的参数，其表现形式可以是一个3*3的旋转矩阵和3*1的平移向量，也可以是一个4*1的四元数向量和一个3*1的平移向量。
根据图像数据中的类玻璃区域和预先接收的视觉传感器与激光传感器之间的外参数， 确定关联的激光数据中属于类玻璃区域的第一点云数据，可以有效减少人工修正玻璃墙的工作量，提高地图的生成效率。
步骤S103，根据所述第一点云数据调整激光数据的权重。
根据所述第一点云数据调整关联的激光数据的权重包括以下任意一项或多项：降低所述第一点云数据对应的第一权重；和提高所述激光数据中属于非类玻璃区域的第二点云数据对应的第二权重。
在该实施例中，可以降低第一点云数据对应的第一权重，可以提高第二点云数据对应的第二权重，还可以降低属于第一点云数据对应的第一权重和提高第二点云数据对应的第二权重。
继续图1进行描述，如图3A所示，降低第一点云数据对应的第一权重可以包括步骤S1031-S1032：
步骤S1031，降低第一点云数据的置信度。
由于第一点云数据的准确性相对差，因此为了提高生成地图的精度，可以降低第一点云数据的置信度。
例如，可以将第一点云数据的置信度降低至0～0.1。
步骤S1032，根据降低后的置信度，降低第一点云数据对应的第一权重。
其中，可以根据降低后的置信度，降低第一点云数据对应的第一权重，即降低后的第一权重与降低后的置信度相关。例如，若降低后的置信度为0，则可以将第一权重降低至0.01或0等。需要说明的是，上述0.01或0仅为示例，在实际应用中，可以根据需要调整第一权重大小。
继续图1进行描述，如图3B所示，提高所述第二点云数据对应的所述第二权重包括步骤S1033-S1034。
步骤S1033，提高第二点云数据的置信度。
由于第二点云数据的准确性相对好，因此为了提高生成地图的精度，可以提高第二点云数据的置信度。
例如，可以将第二点云数据的置信度提高至0.9～1。
步骤S1034，根据提高后的置信度，提高第二点云数据对应的第二权重。
其中，可以根据提高后的置信度，提高第二点云数据对应的第二权重，即提高后的第二权重与提高后的置信度相关。例如，若提高后的置信度为0.9，则可以将第二权重提高至0.99或1等。需要说明的是，上述0.99或1仅为示例，在实际应用中，可以根据需要调整第二权重大小。
根据降低后的置信度降低第一点云数据对应的第一权重，根据提高后的置信度提高第二点云数据对应的第二权重，为后续基于调整后的权重对关联的激光数据进行融合处理提供了条件，也即为生成高精度的地图数据提供了条件。
步骤S104，基于调整后的激光数据的权重，对第一点云数据和第二点云数据进行融合处理，生成地图。
继续图1进行描述，例如移动设备可以基于调整后的激光数据的权重对第一点云数据和第二点云数据进行融合处理，得到初始地图，然后对初始地图进行优化后生成地图，例如，可以对初始地图进行回环检测优化后生成地图，以进一步提高生成地图的精度。
在该实施例中，如图4所示，基于调整后的激光数据的权重对第一点云数据和第二点云数据进行融合处理可以包括：
步骤S1041，对所述第一点云数据进行配准，以生成配准的第一点云数据。
步骤S1042，对所述第二点云数据进行配准，以生成配准的第二点云数据。
步骤S1043，基于配准的第一点云数据的坐标向量、配准的第二点云数据的坐标向量、降低后的第一权重、提高后的第二权重和配准的点云数据之间的姿态参数，计算代价函数。
假设，点云数据m_i与点云数据n_i配准，i表示点索引，则代价函数为公式(1)：
F_cost＝∑Wi*(Xn_i–(R(Xm_i)+t))   (1)
其中，Xm_i,Xn_i表示配准点云数据的坐标向量(x,y,z)，Wi为配准点云数据对应的权重，其中，若配准的点云数据位于类玻璃区域，即配准的第一点云数据，则Wi表示降低后的第一权重，例如0,若配准的点云数据位于非类玻璃区域，即配准的第二点云数据，则Wi表示提高后的第二权重，例如1，R，t表示配准的点云数据之间的姿态参数，例如R表示Xm到Xn的旋转变化矩阵，t表示Xm到Xn的位移变化向量。
步骤S1044，通过对代价函数进行迭代运算，获得优化的姿态参数。
在该实施例中，可以通过移动设备的码盘传感器获得R和t的估计值，然后通过 F_cost对R和t求导，利用梯度下降方法不断迭代，当F_cost的值在两次迭代之后的差值变化很小时停止迭代，此时得到的R′和t′为优化的姿态参数。
步骤S1045，基于优化的姿态参数对配准的第一点云数据和配准的第二点云数据进行融合处理。
在得到优化的姿态参数之后，可以基于优化的姿态参数对配准的第一点云数据和配准的第二点云数据进行融合处理。
在该实施例中，基于代价函数获得优化的姿态参数，并基于优化的姿态参数对配准的第一点云数据和配准的第二点云数据进行融合处理，实现方式简单。
基于调整后的权重对第一点云数据和第二点云数据进行融合处理，使得权重小的第一点云数据以极小的概率出现在融合后的地图数据中，从而提高生成的地图的精度，可用性高。
与前述基于激光传感器的地图生成方法的实施例相对应，本申请还提供了基于激光传感器的地图生成装置的实施例。
本申请基于激光传感器的地图生成装置的实施例可以应用在移动设备上。其中，该移动设备可以为无人车、机器人等。也可以应用在服务器上，由服务器生成地图，再将地图传输给机器人。装置实施例可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。如图5所示，为本申请本申请一实施例提供的移动设备的硬件结构图，该移动设备包括视觉传感器510、激光传感器520、处理器530、存储器540及存储在存储器540上并可在处理器530上运行的计算机程序，该处理器530执行该计算机程序时实现上述基于激光传感器的地图生成方法。除了图5所示的处理器530及存储器540之外，该移动设备可根据生成地图的实际功能，包括其他硬件，对此不再赘述。
图6是本申请一示例性实施例示出的一种基于激光传感器的地图生成装置的结构框图，该装置可以应用于具有视觉传感器和激光传感器的移动设备上，如图6所示，该装置包括：获取模块61、确定模块62、调整模块63和处理模块64。
获取模块61用于获取图像数据，该图像数据是通过视觉传感器采集得到的。
确定模块62用于基于所述图像数据确定激光数据中属于类玻璃区域的第一点云数据，所述激光数据是通过激光传感器采集得到的且与所述图像数据对应同一区域，并且所述视觉传感器采集所述图像数据的时间段与所述激光传感器采集所述激光数据的时间段相同。
调整模块63用于根据所述第一点云数据调整所述激光数据的权重。
处理模块64用于基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非类玻璃区域的第二点云数据进行融合处理，生成地图。
图7是本申请另一示例性实施例示出的基于激光传感器的地图生成装置的结构框图，如图7所示，在上述图6所示实施例的基础上，确定模块62可以包括：
识别子模块621用于识别图像数据中的类玻璃区域；
确定子模块622用于根据识别子模块621识别出的图像数据中的类玻璃区域和预先接收的视觉传感器与激光传感器之间的外参数，确定激光数据中属于类玻璃区域的第一点云数据。所述外参数表示所述视觉传感器所在的第一坐标系和所述激光传感器所在的第二坐标系之间的空间旋转和平移关系。
在一实施例中，所述识别子模块621还用于：将所述图像数据对应的图像分割成多个子图像；通过预先训练的一个或多个识别模型判断每个所述子图像是否属于类玻璃区域；和根据属于类玻璃区域的子图像确定所述图像数据中的类玻璃区域。
在一实施例中，所述识别子模块621还用于：将所述子图像输入所述识别模型，得到基于所述识别模型判定所述子图像属于类玻璃区域的概率；和当基于所述识别模型判定所述子图像属于类玻璃区域的概率大于所述识别模型对应的概率阈值时，确定所述子图像属于类玻璃区域。
在一实施例中，所述识别子模块621还用于：将所述子图像分别输入每个所述识别模型，得到基于所述识别模型判定所述子图像属于类玻璃区域的概率；和当基于每个所述识别模型判定所述子图像属于类玻璃区域的概率均大于所述识别模型对应的概率阈值时，确定所述子图像属于类玻璃区域。
在一实施例中，所述识别子模块621还用于：将属于类玻璃区域的子图像的并集作为所述图像数据中的类玻璃区域。
在一实施例中，调整模块63还用于降低所述第一点云数据对应的第一权重；和/或提高所述第二点云数据对应的第二权重。
在一实施例中，调整模块63还用于：降低所述第一点云数据的置信度；和根据所述降低后的置信度，降低所述第一权重。
在一实施例中，调整模块63还用于：提高所述第二点云数据的置信度；和根据提高 后的置信度，提高所述第二权重。
在一实施例中，处理模块64还用于基于调整后的所述激光数据的权重对所述第一点云数据和所述第二点云数据进行融合处理，得到初始地图；和对所述初始地图进行优化以生成所述地图。
在一实施例中，处理模块64还用于对所述第一点云数据进行配准，以生成配准的第一点云数据；对所述第二点云数据进行配准，以生成配准的第二点云数据；基于所述配准的第一点云数据的坐标向量、所述配准的第二点云数据的坐标向量、所述配准的第一点云数据对应的降低后的第一权重、所述配准的第二点云数据对应的提高后的第二权重、以及配准的点云数据之间的姿态参数，计算代价函数；通过对所述代价函数进行迭代运算，优化所述姿态参数；和基于所述优化后的姿态参数对所述配准的所述第一点云数据和所述第二点云数据进行融合处理。
上述装置中各个单元的功能和作用的实现过程具体详见上述方法中对应步骤的实现过程，在此不再赘述。
在示例性实施例中，还提供了一种计算机可读存储介质，该存储介质存储有计算机程序，在所述计算机程序被处理器调用时，所述计算机程序促使所述处理器执行上述基于激光传感器的地图生成方法，其中，计算机可读存储介质可以包括非易失性机器可读存储介质、只读存储器(ROM)、随机存取存储器(RAM)、光盘只读存储器(CD-ROM)、磁带、软盘和光数据存储设备等。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本申请方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (15)

  1. 一种基于激光传感器的地图生成方法，包括：
    获取图像数据，所述图像数据是通过视觉传感器采集得到的；
    基于所述图像数据确定激光数据中属于类玻璃区域的第一点云数据，所述激光数据是通过激光传感器采集得到的且与所述图像数据对应同一区域，并且所述视觉传感器采集所述图像数据的时间段与所述激光传感器采集所述激光数据的时间段相同；
    根据所述第一点云数据调整所述激光数据的权重；和
    基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非类玻璃区域的第二点云数据进行融合处理，生成地图。
  2. 根据权利要求1所述的方法，其中，根据所述第一点云数据调整所述激光数据的权重，包括以下任意一项或多项：
    降低所述第一点云数据对应的第一权重，和
    提高所述第二点云数据对应的第二权重。
  3. 根据权利要求1所述的方法，其特征在于，基于所述图像数据确定所述激光数据中的所述第一点云数据，包括：
    识别所述图像数据中的类玻璃区域；和
    根据所述图像数据中的类玻璃区域和预先接收的所述视觉传感器与所述激光传感器之间的外参数，确定所述激光数据中的所述第一点云数据。
  4. 根据权利要求3所述的方法，其特征在于，其中，所述外参数表示所述视觉传感器所在的第一坐标系和所述激光传感器所在的第二坐标系之间的空间旋转和平移关系。
  5. 根据权利要求3所述的方法，其特征在于，识别所述图像数据中的类玻璃区域，包括：
    将所述图像数据对应的图像分割成多个子图像；
    通过预先训练的一个或多个识别模型判断每个所述子图像是否属于类玻璃区域；和
    根据属于类玻璃区域的子图像确定所述图像数据中的类玻璃区域。
  6. 根据权利要求5所述的方法，其特征在于，通过所述预先训练的所述识别模型判断所述子图像是否属于类玻璃区域，包括：
    将所述子图像输入所述识别模型，得到基于所述识别模型判定所述子图像属于类玻璃区域的概率；和
    当基于所述识别模型判定所述子图像属于类玻璃区域的概率大于所述识别模型对应 的概率阈值时，确定所述子图像属于类玻璃区域。
  7. 根据权利要求5所述的方法，其特征在于，通过所述预先训练的多个所述识别模型判断所述子图像是否属于类玻璃区域，包括：
    将所述子图像分别输入每个所述识别模型，得到基于所述识别模型判定所述子图像属于类玻璃区域的概率；和
    当基于每个所述识别模型判定所述子图像属于类玻璃区域的概率均大于所述识别模型对应的概率阈值时，确定所述子图像属于类玻璃区域。
  8. 根据权利要求5所述的方法，其特征在于，根据属于类玻璃区域的子图像确定所述图像数据中的类玻璃区域，包括：
    将属于类玻璃区域的子图像的并集作为所述图像数据中的类玻璃区域。
  9. 根据权利要求2所述的方法，其特征在于，降低所述第一点云数据对应的所述第一权重，包括：
    降低所述第一点云数据的置信度；和
    根据所述降低后的置信度，降低所述第一权重。
  10. 根据权利要求2所述的方法，其特征在于，提高所述第二点云数据对应的所述第二权重，包括：
    提高所述第二点云数据的置信度；和
    根据提高后的置信度，提高所述第二权重。
  11. 根据权利要求1-10任一项所述的方法，其特征在于，基于调整后的所述激光数据的权重对所述第一点云数据和所述第二点云数据进行融合处理，生成地图，包括：
    基于调整后的所述激光数据的权重对所述第一点云数据和所述第二点云数据进行融合处理，得到初始地图；
    对所述初始地图进行优化以生成所述地图。
  12. 根据权利要求11所述的方法，其特征在于，基于调整后的所述激光数据的权重对所述第一点云数据和所述第二点云数据进行融合处理，包括：
    对所述第一点云数据进行配准，以生成配准的第一点云数据；
    对所述第二点云数据进行配准，以生成配准的第二点云数据；
    基于所述配准的第一点云数据的坐标向量、所述配准的第二点云数据的坐标向量、所述配准的第一点云数据对应的降低后的第一权重、所述配准的第二点云数据对应的提高后的第二权重、以及配准的点云数据之间的姿态参数，计算代价函数；
    通过对所述代价函数进行迭代运算，优化所述姿态参数；和
    基于所述优化后的姿态参数对所述配准的所述第一点云数据和所述第二点云数据进行融合处理。
  13. 一种基于激光传感器的地图生成装置，包括：
    获取模块，用于获取图像数据，所述图像数据是通过视觉传感器采集得到的；
    确定模块，用于基于所述图像数据确定激光数据中属于类玻璃区域的第一点云数据，所述激光数据是通过激光传感器采集得到的且与所述图像数据对应同一区域，并且所述视觉传感器采集所述图像数据的时间段与所述激光传感器采集所述激光数据的时间段相同；
    调整模块，用于根据所述第一点云数据调整所述激光数据的权重；和
    处理模块，用于基于调整后的所述激光数据的权重，对所述第一点云数据和所述激光数据中属于非类玻璃区域的第二点云数据进行融合处理，生成地图。
  14. 一种计算机可读存储介质，所述存储介质存储有计算机程序，在所述计算机程序被处理器调用时，所述计算机程序促使所述处理器执行上述权利要求1-12任一所述的方法。
  15. 一种移动设备，包括视觉传感器、激光传感器、处理器、存储器及存储在所述存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述权利要求1-12任一所述的方法。
PCT/CN2018/121051 2018-04-09 2018-12-14 基于激光传感器生成地图 Ceased WO2019196476A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18914482.7A EP3772041A4 (en) 2018-04-09 2018-12-14 LASER SENSOR BASED MAP GENERATION
US17/044,317 US11315264B2 (en) 2018-04-09 2018-12-14 Laser sensor-based map generation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810312694.0A CN108665541B (zh) 2018-04-09 2018-04-09 一种基于激光传感器的地图生成方法及装置和机器人
CN201810312694.0 2018-04-09

Publications (1)

Publication Number Publication Date
WO2019196476A1 true WO2019196476A1 (zh) 2019-10-17

Family

ID=63783158

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/121051 Ceased WO2019196476A1 (zh) 2018-04-09 2018-12-14 基于激光传感器生成地图

Country Status (4)

Country Link
US (1) US11315264B2 (zh)
EP (1) EP3772041A4 (zh)
CN (1) CN108665541B (zh)
WO (1) WO2019196476A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116529767A (zh) * 2020-10-01 2023-08-01 高通股份有限公司 使用捕获多个光谱的相机的同时定位和建图

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108665541B (zh) 2018-04-09 2019-06-07 北京三快在线科技有限公司 一种基于激光传感器的地图生成方法及装置和机器人
CN112116698B (zh) * 2019-06-21 2025-06-17 宝马股份公司 用于点云融合的方法和装置
EP4070130A4 (en) * 2019-12-04 2023-12-27 Waymo LLC EFFICIENT ALGORITHM FOR PROJECTING WORLD POINTS ONTO A ROLLER SHUTTER IMAGE
CN112270251B (zh) 2020-10-26 2021-07-06 清华大学 一种基于互信息的自适应多传感器数据融合方法及系统
CN114612929A (zh) * 2020-12-08 2022-06-10 山东新松工业软件研究院股份有限公司 基于信息融合的人体跟踪方法、系统及可读存储介质
US12607481B2 (en) 2020-12-22 2026-04-21 Beijing Robint Technologies Co., Ltd. Mobile robot map construction method and apparatus, and computer device and storage medium
US20220349725A1 (en) * 2021-04-21 2022-11-03 Nvidia Corporation High definition mapping for autonomous systems and applications
CN113467450A (zh) * 2021-07-01 2021-10-01 北京小狗吸尘器集团股份有限公司 无人机器控制方法、装置、计算机设备及存储介质
CN113724382B (zh) * 2021-07-23 2024-07-02 北京搜狗科技发展有限公司 地图生成方法、装置及电子设备
CN114322987B (zh) * 2021-12-27 2024-02-23 北京三快在线科技有限公司 一种构建高精地图的方法及装置
CN114577198B (zh) * 2022-01-18 2024-02-02 辽宁华盾安全技术有限责任公司 一种高反物体定位方法、装置及终端设备
CN116500640A (zh) * 2022-01-19 2023-07-28 中国移动通信有限公司研究院 机器人建图方法、装置及相关设备
CN114078151B (zh) * 2022-01-19 2022-04-22 季华实验室 一种点云融合方法、装置、电子设备及存储介质
CN115127537B (zh) * 2022-05-16 2024-12-20 安克创新科技股份有限公司 地图生成方法、装置、存储介质及电子设备
CN116546424A (zh) * 2023-02-07 2023-08-04 智道网联科技(北京)有限公司 激光建图方法及其装置、激光定位方法及其装置
CN119414410B (zh) * 2024-10-29 2025-07-22 佛山市测绘地理信息研究院有限公司 基于优化算法的复杂地形环境下高精度不动产测绘方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107472135A (zh) * 2016-06-07 2017-12-15 松下知识产权经营株式会社 图像生成装置、图像生成方法以及程序
CN107703935A (zh) * 2017-09-12 2018-02-16 安徽胜佳和电子科技有限公司 多个数据权重融合进行避障的方法、存储装置及移动终端
CN108665541A (zh) * 2018-04-09 2018-10-16 北京三快在线科技有限公司 一种基于激光传感器的地图生成方法及装置和机器人

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103459099B (zh) * 2011-01-28 2015-08-26 英塔茨科技公司 与一个可移动的远程机器人相互交流
CN103884330B (zh) * 2012-12-21 2016-08-10 联想(北京)有限公司 信息处理方法、可移动电子设备、引导设备和服务器
CN104574387B (zh) * 2014-12-29 2017-11-21 张家港江苏科技大学产业技术研究院 水下视觉slam系统中的图像处理方法
CN104916216A (zh) 2015-06-26 2015-09-16 深圳乐行天下科技有限公司 一种地图构建方法及系统
CN105136155B (zh) * 2015-09-24 2018-12-14 联想(北京)有限公司 一种导航方法及电子设备
CN106043169A (zh) * 2016-07-01 2016-10-26 百度在线网络技术（北京）有限公司 环境感知设备和应用于环境感知设备的信息获取方法
CN106092104B (zh) * 2016-08-26 2019-03-15 深圳微服机器人科技有限公司 一种室内机器人的重定位方法及装置
CN106813672B (zh) * 2017-01-22 2020-02-14 深圳悉罗机器人有限公司 移动机器人的导航方法及移动机器人
CN107860390A (zh) * 2017-12-21 2018-03-30 河海大学常州校区 基于视觉ros系统的非完整机器人远程定点自导航方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107472135A (zh) * 2016-06-07 2017-12-15 松下知识产权经营株式会社 图像生成装置、图像生成方法以及程序
CN107703935A (zh) * 2017-09-12 2018-02-16 安徽胜佳和电子科技有限公司 多个数据权重融合进行避障的方法、存储装置及移动终端
CN108665541A (zh) * 2018-04-09 2018-10-16 北京三快在线科技有限公司 一种基于激光传感器的地图生成方法及装置和机器人

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3772041A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116529767A (zh) * 2020-10-01 2023-08-01 高通股份有限公司 使用捕获多个光谱的相机的同时定位和建图

Also Published As

Publication number Publication date
US20210082132A1 (en) 2021-03-18
US11315264B2 (en) 2022-04-26
EP3772041A4 (en) 2021-05-26
CN108665541B (zh) 2019-06-07
CN108665541A (zh) 2018-10-16
EP3772041A1 (en) 2021-02-03

Similar Documents

Publication Publication Date Title
WO2019196476A1 (zh) 基于激光传感器生成地图
CN111780764B (zh) 一种基于视觉地图的视觉定位方法、装置
US10198823B1 (en) Segmentation of object image data from background image data
US9965865B1 (en) Image data segmentation using depth data
CN114140527B (zh) 一种基于语义分割的动态环境双目视觉slam方法
CN113052907B (zh) 一种动态环境移动机器人的定位方法
CN110322500A (zh) 即时定位与地图构建的优化方法及装置、介质和电子设备
CN117593650B (zh) 基于4d毫米波雷达与sam图像分割的动点滤除视觉slam方法
CN108955718A (zh) 一种视觉里程计及其定位方法、机器人以及存储介质
CN113570713B (zh) 一种面向动态环境的语义地图构建方法及装置
KR20220004009A (ko) 키 포인트 검출 방법, 장치, 전자 기기 및 저장 매체
WO2024012333A1 (zh) 位姿估计方法及相关模型的训练方法、装置、电子设备、计算机可读介质和计算机程序产品
CN112967340A (zh) 同时定位和地图构建方法、装置、电子设备及存储介质
KR102615412B1 (ko) 비주얼 로컬라이제이션을 수행하기 위한 방법 및 장치
CN115797400B (zh) 一种多无人系统协同长期目标跟踪方法
KR20230049969A (ko) 글로벌 측위 장치 및 방법
CN113781563A (zh) 一种基于深度学习的移动机器人回环检测方法
US20230386164A1 (en) Method for training an object recognition model in a computing device
CN112069997B (zh) 一种基于DenseHR-Net的无人机自主着陆目标提取方法及装置
CN115088244B (zh) 用于基于来自单目相机的数据进行定位的深度传感器激活
CN118840529A (zh) 一种基于场景识别和深度学习的视觉定位方法及系统
CN117274313A (zh) 动态场景下轻量化视觉slam系统、vslam方法
CN114972953A (zh) 基于深度学习模型的回环检测方法
CN114445885A (zh) 训练多目标检测模型的方法、检测方法及相关装置
CN119444806A (zh) 一种点云配准方法、装置及智能移动设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18914482

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018914482

Country of ref document: EP

Effective date: 20201109