WO2021077941A1 - 机器人定位方法、装置、智能机器人和存储介质 - Google Patents

机器人定位方法、装置、智能机器人和存储介质 Download PDF

Info

Publication number
WO2021077941A1
WO2021077941A1 PCT/CN2020/115046 CN2020115046W WO2021077941A1 WO 2021077941 A1 WO2021077941 A1 WO 2021077941A1 CN 2020115046 W CN2020115046 W CN 2020115046W WO 2021077941 A1 WO2021077941 A1 WO 2021077941A1
Authority
WO
WIPO (PCT)
Prior art keywords
robot
pose
scene
sensor data
collected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/115046
Other languages
English (en)
French (fr)
Inventor
钟立扬
李同
邵长东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecovacs Commercial Robotics Co Ltd
Original Assignee
Ecovacs Commercial Robotics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ecovacs Commercial Robotics Co Ltd filed Critical Ecovacs Commercial Robotics Co Ltd
Priority to EP20879071.7A priority Critical patent/EP4050449B1/en
Priority to US17/771,428 priority patent/US12415277B2/en
Publication of WO2021077941A1 publication Critical patent/WO2021077941A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1628—Program controls characterised by the control loop
    • B25J9/1653—Program controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0248—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means in combination with a laser
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/246—Arrangements for determining position or orientation using environment maps, e.g. simultaneous localisation and mapping [SLAM]
    • G05D1/2462—Arrangements for determining position or orientation using environment maps, e.g. simultaneous localisation and mapping [SLAM] using feature-based mapping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene

Definitions

  • the present invention relates to the field of artificial intelligence technology, in particular to a robot positioning method, device, intelligent robot and storage medium.
  • a robot is a kind of mechanical device that can accept human commands and perform corresponding tasks.
  • various intelligent robots are increasingly entering people's lives, such as service robots, cleaning robots, and self-moving vending robots.
  • the intelligent robot will locate the intelligent robot according to the data collected by various sensors configured by itself, and further plan the motion trajectory according to the positioning result. The intelligent robot only needs to move according to the trajectory. User instructions.
  • the embodiments of the present invention provide a robot positioning method, device, and storage medium to improve the accuracy of intelligent robot positioning.
  • the embodiment of the present invention provides a robot positioning method, which includes:
  • the pose of the robot is determined according to the target sensor data corresponding to the scene among the various sensor data.
  • the embodiment of the present invention provides a robot positioning device, including:
  • An acquisition module for acquiring images collected by a camera on the robot and various sensor data collected by various sensors on the robot;
  • An extraction module for extracting semantic information contained in the image An extraction module for extracting semantic information contained in the image
  • a recognition module configured to recognize the scene where the robot is located according to the semantic information
  • the pose determination module is used to determine the pose of the robot according to the target sensor data corresponding to the scene among the multiple sensor data.
  • An embodiment of the present invention provides an intelligent robot, including: a processor and a memory; wherein the memory is used to store one or more computer instructions, where the one or more computer instructions are implemented when executed by the processor :
  • the pose of the robot is determined according to the target sensor data corresponding to the scene among the various sensor data.
  • the embodiment of the present invention provides a computer-readable storage medium storing computer instructions.
  • the computer instructions are executed by one or more processors, the one or more processors are caused to perform at least the following actions:
  • the pose of the robot is determined according to the target sensor data corresponding to the scene among the various sensor data.
  • a camera and various sensors are configured on the robot, and the robot can obtain images collected by the camera and various sensor data collected by various sensors. Then, the robot first extracts the semantic information contained in the collected images, and recognizes the scene where the robot is currently located based on the semantic information. Finally, the current position of the robot is determined according to the target sensor data corresponding to the scene where the robot is located.
  • the sensor data used in determining the pose of the robot is not all sensor data, but the target sensor data corresponding to the scene. This makes the basis for determining the pose more targeted, thereby further improving Accuracy of pose.
  • Fig. 1 is a flowchart of a robot positioning method provided by an embodiment of the present invention
  • Figure 3a is a flowchart of a method for determining a robot pose provided by an embodiment of the present invention
  • 3b is a flowchart of another method for determining the pose of a robot according to an embodiment of the present invention.
  • Figure 3c is a flowchart of yet another method for determining the pose of a robot according to an embodiment of the present invention.
  • Figure 3d is a flowchart of yet another method for determining the pose of a robot according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a robot positioning device provided by an embodiment of the present invention.
  • Fig. 5 is a schematic structural diagram of an intelligent robot corresponding to the robot positioning device provided by the embodiment shown in Fig. 4.
  • the words “if” and “if” as used herein can be interpreted as “when” or “when” or “in response to determination” or “in response to detection”.
  • the phrase “if determined” or “if detected (statement or event)” can be interpreted as “when determined” or “in response to determination” or “when detected (statement or event) )” or “in response to detection (statement or event)”.
  • Fig. 1 is a flowchart of a robot positioning method provided by an embodiment of the present invention.
  • the execution subject of the method may be a robot. As shown in Fig. 1, the method may include the following steps:
  • the camera on the robot is used to collect images corresponding to the scene in which the robot is located, and the various sensors on the robot are respectively used to collect various sensor data, so that the robot can obtain images and various sensor data.
  • the camera that collects the image can also be considered as a visual sensor.
  • sensors can include laser sensors, motion speed meters, motion angle meters, and so on.
  • what the laser sensor collects is the corresponding laser point coordinates when the laser is irradiated on the object.
  • What the movement speed meter collects is the movement speed of the robot, and the movement distance of the robot can be further calculated according to this speed.
  • the angular velocity of the robot is collected by the movement angle meter, and the angular velocity of the robot can be further calculated based on this angular velocity.
  • the aforementioned camera may be a monocular or binocular camera
  • the motion speed measurement meter may specifically be a wheel odometer
  • the motion angle measurement meter may specifically be an inertial measurement unit (IMU).
  • the robot can locate the robot in the following manner based on the acquired images and various sensor data.
  • the sensor data collected by each sensor corresponds to the sensor's own coordinate system.
  • the inconsistency of the coordinate system will obviously affect the subsequent positioning process. Make an impact. Therefore, in order to ensure the accuracy of the robot's positioning, optionally, for multiple types of sensor data that have been acquired, coordinate conversion can be performed on them, that is, the sensor data in different coordinate systems are converted to target coordinates Tie down.
  • the target coordinate system can be a coordinate system corresponding to any sensor.
  • the coordinate system corresponding to the laser sensor is usually determined as the target coordinate system. The conversion relationship between different coordinate systems has been pre-configured, and the robot can call it directly.
  • the robot acquires the sensor data collected by the sensor, it immediately reads the current system time of the robot, and determines the read system time as the sensor data collection time. What is used in the robot positioning process is the acquisition time of sensor data within the preset time difference.
  • the robot will perform semantic recognition on the image collected by the camera to extract the semantic information contained in the image.
  • a pre-trained semantic recognition model may be configured in the robot, and the image captured by the camera will be directly input to the model, so that the model outputs the recognition result. Then, the robot can determine the scene where the robot is based on the recognition result.
  • the aforementioned semantic recognition model may specifically be a Conditional Random Field (CRF) model, a Convolutional Neural Networks (CNN) model, a Recurrent Neural Networks (RNN) model, etc. Wait.
  • CCF Conditional Random Field
  • CNN Convolutional Neural Networks
  • RNN Recurrent Neural Networks
  • the robot can determine that the scene it is in is a corridor or a glass walkway.
  • the robot can determine that the scene it is in is a crowd.
  • the image semantic information does not include preset types of buildings or crowds, the robot can determine that it is in a common positioning scene. Since in practical applications, robots are usually used in public places, at this time, the general positioning scene can be the halls of public places, such as shopping mall halls, bank halls, hospital halls, and so on.
  • the robot can be pre-configured with the corresponding relationship between the scene and the target sensor data. Based on this corresponding relationship and the scene in which the robot is located, the robot can determine which of the various sensor data is the target sensor data, and Determine the pose of the robot based on the target sensor data.
  • the robot when the robot is in a crowd scene, the robot can determine the pose of the robot according to the sensor data collected by the movement speed meter and the sensor data collected by the movement angle meter.
  • the robot can determine the pose of the robot according to the sensor data collected by the motion speed meter, the motion angle meter, the laser sensor, and the camera.
  • a camera and various sensors are configured on the robot, and the robot can obtain images collected by the camera and various sensor data collected by various sensors.
  • the robot first extracts the semantic information contained in the collected images, and recognizes the scene where the robot is currently located based on the semantic information.
  • the current position of the robot is determined according to the target sensor data corresponding to the scene where the robot is located.
  • the sensor data used in determining the pose of the robot is not all sensor data, but the target sensor data corresponding to the scene. This makes the basis for determining the pose more targeted, thereby further improving Accuracy of pose.
  • the robot may also have an inability to move failure during the movement.
  • the robot can continue to locate itself in the manner provided in the foregoing embodiment, and send the positioning result to the maintenance server.
  • Maintenance personnel can find the robot according to the positioning result to maintain it. Since the positioning result determined according to the foregoing embodiment has high accuracy, the maintenance personnel can also quickly and accurately find the malfunctioning robot, thereby improving the maintenance efficiency of the robot.
  • robots with mobile capabilities are often used in public places, such as shopping malls, hospitals, and so on.
  • the robot can provide navigation services for the user while positioning its own pose, that is, the robot can lead the user to the target location where the user wants to go.
  • the target location that the user wants to reach can be, for example, a certain store in a shopping mall or a certain clinic in a hospital, and so on.
  • FIG. 2 is a flowchart of another robot positioning method provided by an embodiment of the present invention. As shown in FIG. 2, after the above step 104, the method may further include The following steps:
  • the user can make the robot obtain the target position by interacting with the robot.
  • the robot may be equipped with an operation screen for the user to input the target position.
  • the target position is in the form of text.
  • the robot can also be equipped with a pickup device such as a microphone. When the user speaks the target location, the robot can collect the target location in the form of voice through the pickup device.
  • the robot can plan a navigation path with the robot pose determined in step 104 as the starting point and the target position input by the user as the end point.
  • the robot only needs to move along this navigation path to lead the user to the target position.
  • the pose of the robot and the target position input by the user will be marked on a pre-established grid map, and the robot can plan an optimal navigation path based on the positional relationship between these two positions in the grid map.
  • each grid in the grid map is marked as having obstacles or no obstacles.
  • the established grid map also corresponds to scene A, and the map construction is completed before the robot starts to provide positioning and navigation services in scene A.
  • the grid map can be constructed based on historical sensor data collected by a laser sensor configured by the robot. Specifically, before the robot provides positioning and navigation services, it can first traverse the environmental movement in this scene A, and at this time, the plane area where the scene A is located has been pre-divided into several sub-areas, and each sub-area is called a grid. As the robot moves, the laser sensor configured by the robot will collect sensing data, which is the aforementioned historical sensing data. Then, the robot then determines which locations in the scene A have obstacles based on historical sensor data, and further calculates the two-dimensional coordinates of the obstacles in the grid map. Finally, each grid in the grid map is labeled according to the two-dimensional coordinates to construct a grid map.
  • the robot when the robot is placed in a position in scene A for the first time, it does not know its current pose, which can be called the initial pose.
  • the robot may determine the initial pose of the robot in the following manner.
  • the method may further include the following steps:
  • the robot is placed in scene A for the first time.
  • the robot can rotate in place so that the camera can collect images corresponding to various directions.
  • the image collected at this time may be referred to as an initial image.
  • the robot can perform semantic recognition on each initial image.
  • the semantic information included in the initial image is compared with the semantic information included in the pre-built semantic grid map to determine the initial pose of the robot.
  • each grid in the semantic grid map is not only marked whether there is an obstacle, but also what kind of object the obstacle is.
  • the establishment of this map is also a preprocessing process.
  • the semantic grid map is constructed after the grid map and before the robot starts to provide positioning and navigation services in scene A. Assuming that the robot provides positioning and navigation services in scene A, the established semantic grid map also corresponds to scene A.
  • Semantic raster maps can be constructed from historical images collected by cameras and raster maps that have been constructed. Specifically, the robot first generates a grid map according to the method disclosed in step 201, and while the robot can move in this scene A, the camera configured by the robot can collect several images, which can be called historical images. The semantic recognition model is used to identify what kind of objects are included in the historical image, and the identified objects are marked in the grid map to construct the semantic grid map.
  • the target position input by the user may also be used to provide navigation services for the user. Since the pose determined by the robot has a high degree of accuracy, the navigation service provided also has a high degree of accuracy, which improves the service level of the robot.
  • the robot can determine the scene where the robot is located according to the semantic information of the image collected by the camera. It can be seen that the accuracy of image semantic information recognition can directly affect the accuracy of scene determination.
  • the brightness of the image can also be taken into consideration, that is, the objects contained in the image and the environmental brightness value corresponding to the image are also considered in the process of semantic recognition.
  • the environmental brightness value is used to indicate the light intensity of the image acquisition environment. If the light intensity is too small or too large, it will affect the semantic recognition of the image, and further lead to the insufficient accuracy of the recognized scene.
  • the robot can simultaneously recognize the scene where the robot is located according to the brightness value of the environment corresponding to the image and the objects contained in the image.
  • the robot first converts the collected image into a grayscale image, and then determines the environmental brightness value corresponding to the image according to the grayscale value of each pixel. For example, it is possible to calculate the average value of the gray values of all pixels, and determine this average value as the environmental brightness value corresponding to the image.
  • step 104 in the embodiment shown in FIG. 1 is an optional implementation manner, as shown in FIG. 3a, the method may include the following steps:
  • the movement speed meter can be a wheel odometer
  • the movement angle meter can be an IMU.
  • the robot can calculate the movement distance of the robot based on the rotational speed of the left and right wheels of the robot collected by the wheel odometer, and on the other hand can calculate the movement angle of the robot based on the angular velocity collected by the IMU. Then, according to the calculated movement distance, movement angle and the pose of the robot at the previous moment, the first pose is determined, and the first pose is the pose coordinates in the world coordinate system.
  • the robot will map the first pose to the pre-built raster map, that is, convert the first pose in the world coordinate system to the raster map coordinate system to obtain the second pose.
  • the second pose is the current pose of the robot, which is the positioning result of the robot.
  • the grid map used in the above process can be constructed with reference to the related description in the embodiment shown in FIG. 2.
  • the robot when the scene where the robot is located is a scene with too high or too low light intensity, the robot is based on the sensor data collected by the movement speed measuring meter and the movement angle measuring meter and the data collected by the laser sensor.
  • the raster map constructed in advance with sensor data locates the robot, and the basis for determining the pose is more targeted, which improves the accuracy of positioning.
  • the robot calculates that the brightness value of the environment corresponding to the image is within the preset value range, it indicates that the current scene of the robot is a scene with suitable light intensity, such as general positioning scenes such as shopping mall halls and hospital halls.
  • suitable light intensity such as general positioning scenes such as shopping mall halls and hospital halls.
  • Each map point in the visual feature map corresponds to a pose coordinate in the world coordinate system. Since the first pose is also a coordinate in the world coordinate system, the coordinates of the first pose in the world coordinate system can be set Compare with the coordinates of each map point in the visual feature map in the world coordinate system, and determine the map point closest to the position coordinates of the first pose in the visual feature map as the target map point. This target map point corresponds to The pose coordinates are also the third pose. The above mapping process is actually the process of determining the target map point.
  • the established visual feature map also corresponds to scene A, and the map construction is completed before the robot starts to provide positioning and navigation services in scene A.
  • the visual feature map can be constructed based on the images collected by the camera configured by the robot. Specifically, before the robot provides positioning and navigation services, it can move at will in this scene A first, so that the camera configured by the robot can collect images. In order to distinguish from the images used in step 101, the images collected at this time can be Called historical images, sparse feature points are extracted from the collected historical images, and a visual feature map is constructed according to the extracted sparse feature points.
  • the robot will merge the second and third poses obtained after the mapping, and the result of the fusion is the positioning result of the robot.
  • An optional fusion method can set different weight values for the second pose and the third pose respectively, and perform a weighted summation of the two, and the result of the sum is the fusion result.
  • an extended Kalman filter method may also be used to fuse the second pose and the third pose to obtain the fusion result. This fusion method is a relatively mature technology, and the fusion process will not be described in detail in this application.
  • the robot when the scene where the robot is located is a scene with suitable light intensity, the robot can be based on the sensor data collected by the motion speed meter and the motion angle meter and the sensor data collected by the laser sensor.
  • the pre-built grid map and the pre-built visual feature map using the image collected by the camera can locate the robot.
  • the basis for determining the pose is more targeted, thereby improving the accuracy of the pose.
  • step 104 in the foregoing embodiment is another optional implementation manner, as shown in FIG. 3c, the method may include the following steps:
  • the fourth pose obtained in this embodiment is actually the third pose in the embodiment shown in FIG. 3b, and the different names are only for distinguishing between different embodiments.
  • the robot is in a building with a preset structure, and the light intensity inside the building is also appropriate, that is, in this scenario, the environment brightness value corresponding to the image collected by the camera on the robot It is within the preset value range.
  • the robot when the robot is in a building with a preset structure with suitable light intensity, the robot can be based on the sensing data collected by the movement speed measuring meter and the movement angle measuring meter and the sensing data collected by the laser sensor.
  • the data pre-built grid map and the pre-built visual feature map using the image collected by the camera can locate the robot.
  • the basis for determining the pose is more targeted to improve the accuracy of the pose.
  • the pre-built grid map, semantic grid map, and visual feature map are all established when the scene A is in the state I.
  • the scene A may be a shopping mall scene, and the state I indicates a way of placing shelves in the shopping mall.
  • the way of placing shelves often changes, so that the mall is in state II.
  • This state II represents another way of placing shelves in the mall.
  • step 104 in the foregoing embodiment is another optional implementation manner, as shown in FIG. 3d, the method may include the following steps:
  • step 601 The execution process of the foregoing step 601 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 3b, which is not repeated here.
  • the camera on the robot will collect several images, and the robot will match the two images that are adjacent to each other in the collection time.
  • two images with adjacent acquisition times may be referred to as the first image and the second image, respectively.
  • it can be determined which pixel point in the first image is the same pixel point in the second image.
  • the fifth pose is determined according to the pixel coordinates of the pixels having the matching relationship in the first image and the second image, respectively.
  • step 603 The execution process of the foregoing step 603 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 3b, which is not repeated here.
  • the robot is in state II scene A, and the light intensity in this scene is also appropriate, that is, the environment brightness value corresponding to the image collected by the camera on the robot is at the preset value Within range.
  • the robot when the robot is in scene A in state II, and the light intensity of scene A in this state is suitable, the robot can use the sensor data collected by the movement speed meter, the movement angle meter, and the camera collection Image to locate the robot.
  • the basis for determining the pose is more targeted, thereby improving the accuracy of the pose.
  • the scene A in state II may also be a scene with too high or too low light intensity.
  • the robot can directly base on the sensor data collected by the motion speed meter and the motion angle meter.
  • the sensor data determines the pose of the robot.
  • the specific determination process of the pose please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.
  • the robot When the robot recognizes that a crowd is included according to the image collected by the camera, it can be considered that the scene where the robot is located is a scene with dense crowds.
  • the robot can directly determine the pose of the robot based on the sensor data collected by the movement speed measuring instrument and the sensor data collected by the movement angle measuring instrument.
  • the specific determination process of the pose please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.
  • the robot positioning device according to one or more embodiments of the present invention will be described in detail below. Those skilled in the art can understand that all of these human-computer interaction devices can be configured by using commercially available hardware components through the steps taught in this solution.
  • Fig. 4 is a schematic structural diagram of a robot positioning device provided by an embodiment of the present invention. As shown in Fig. 4, the device includes:
  • the acquisition module 11 is used to acquire images collected by a camera on the robot and various sensor data collected by various sensors on the robot.
  • the extraction module 12 is used to extract semantic information contained in the image.
  • the recognition module 13 is configured to recognize the scene where the robot is located according to the semantic information.
  • the pose determination module 14 is configured to determine the pose of the robot according to target sensor data corresponding to the scene among the various sensor data.
  • the robot positioning device may further include:
  • the navigation module 21 is configured to determine a navigation path according to the pose of the robot and the target position input by the user, so that the robot can move to the target position according to the navigation path.
  • the recognition module 13 in the robot positioning device is specifically configured to: recognize the object according to the environmental brightness value corresponding to the image and/or the object contained in the image The scene where the robot is located.
  • the scene corresponds to that the ambient brightness value is not within a preset value range.
  • the pose determination module 14 in the robot positioning device is used for:
  • the first pose is determined according to the sensing data collected by the motion speed measuring meter and the sensing data collected by the motion angle measuring meter; the first pose is mapped to a pre-built raster map to obtain the first pose Two poses, the grid map is constructed based on historical sensor data collected by the laser sensor and determines that the second pose is the pose of the robot.
  • the scene corresponds to that the ambient brightness value is within a preset numerical range.
  • the pose determination module 14 in the robot positioning device is specifically further configured to map the first pose to a pre-built visual feature map to obtain a third pose, and the visual feature map is based on the data collected by the camera. Historical image construction; and for fusing the second pose and the third pose to determine that the fusion result is the pose of the robot.
  • the scene corresponds to a building containing a preset structure in the image.
  • the pose determination module 14 in the robot positioning device is specifically further configured to: determine the first pose according to the sensing data collected by the movement speed measuring meter and the sensing data collected by the movement angle measuring meter; A pose is mapped to a pre-built visual feature map to obtain a fourth pose, the visual feature map is constructed based on historical images collected by the camera; and the fourth pose is determined to be the pose of the robot .
  • the scene corresponds to a crowd included in the image.
  • the pose determination module 14 in the robot positioning device is specifically further configured to determine the pose of the robot according to the sensing data collected by the movement speed measuring meter and the sensing data collected by the movement angle measuring meter.
  • the scene where the robot is located is a scene where the grid map cannot be used.
  • the pose determination module 14 in the robot positioning device is specifically further used to determine the first pose according to the sensing data collected by the movement speed measuring meter and the sensing data collected by the movement angle measuring meter; The collected images determine the fifth pose; and merge the first pose and the fifth pose to determine that the fusion result is the pose of the robot.
  • the robot positioning device shown in FIG. 4 can execute the robot positioning method provided in the embodiments shown in FIGS. 1 to 3d.
  • parts that are not described in detail in this embodiment please refer to the related descriptions of the embodiments shown in FIGS. 1 to 3d. , I won’t repeat it here.
  • the technical effects that can be achieved by this embodiment can also be referred to the description in the embodiment shown in FIG. 1 to FIG. 3d.
  • the intelligent robot may include: a processor 31 and a memory 32 .
  • the memory 32 is used to store a program that supports the intelligent robot to execute the robot positioning method provided in the embodiment shown in FIG. 1 to FIG. 3d
  • the processor 31 is configured to execute the program stored in the memory 32 program of.
  • the program includes one or more computer instructions, wherein, when the one or more computer instructions are executed by the processor 31, the following steps can be implemented:
  • the pose of the robot is determined according to the target sensor data corresponding to the scene among the various sensor data.
  • the processor 31 is further configured to execute all or part of the steps in the embodiment shown in FIG. 1 to FIG. 3d.
  • the structure of the intelligent robot may also include a communication interface 33 for communicating with other devices or communication networks.
  • an embodiment of the present invention provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform at least the following actions:
  • the pose of the robot is determined according to the target sensor data corresponding to the scene among the various sensor data.
  • the device embodiments described above are merely illustrative, and the modules described as separate components may or may not be physically separated. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • smart terminal equipment such as service robots may be installed in the lobby of the shopping mall.
  • the robot can move around at any position in the shopping mall, so that users at different locations in the shopping mall can issue instructions to the robot.
  • the user can send a consultation instruction or a navigation request instruction to the robot.
  • the camera on the robot will collect images in real time, and the movement speed measuring meter and movement angle measuring meter on the robot will also collect sensor data in real time.
  • the robot will perform semantic recognition on it to determine the objects contained in the image, and at the same time determine the corresponding environmental brightness value of the image, and further determine the object contained in the image and the environmental brightness value of the image The scene where the robot is located.
  • this scene is a scene with suitable light intensity, that is, the environment brightness value corresponding to the image in this scene is within the preset value range.
  • the robot will determine the first pose according to the sensor data collected by the motion speed measuring meter and the motion angle measuring meter, and then map the first pose to the pre-built grid map and visual feature map respectively.
  • the second pose and the third pose are obtained respectively, and the second pose and the third pose are finally merged, and the fusion result is determined as the current pose of the robot, which is the positioning information of the robot.
  • both the grid map and the visual feature map correspond to shopping malls.
  • the robot When it is recognized that the robot is in a crowded area in the shopping mall, the robot will determine the current pose or positioning information of the robot according to the sensor data collected by the motion speed meter and the motion angle meter.
  • the sensor data used to determine the current pose of the robot is pertinent, so as to ensure the accuracy of the robot's positioning.
  • the maintenance personnel can quickly find the robot according to the accurate positioning information to repair the robot, which ensures the maintenance efficiency of the robot.
  • the user can send a navigation request instruction to the robot in the form of text or voice, such as "I want to go to shop I".
  • the robot will mark the location of shop I on the grid map corresponding to the shopping mall.
  • the robot will determine its current scene based on the images collected by the camera and various sensor data.
  • the robot can determine the current pose of the robot according to the sensor data collected by the motion speed meter and the motion angle meter, as well as the pre-built grid map and visual feature map. It is marked on the grid map so that the robot plans the first navigation path for the robot according to the pose and the target location of the shop I. After that, the robot will start to move along this navigation path. During the movement along the first navigation path, the camera and various sensors on the robot will still collect images and sensor data in real time, and continuously determine the scene where the robot is based on the collected images.
  • the robot will determine the pose of the robot according to the sensor data collected by the motion speed meter and the angle speed meter and the pre-built visual feature map. . At this time, the robot can re-plan the second navigation path according to the pose of the robot at the first moment and the target location where the shop I is located, and the robot will continue to move along this second navigation path. If it is determined at the second moment that the robot has moved from the corridor to the crowded area in the shopping mall, the sensor data collected by the robot motion speed meter and the angle speed meter will determine the robot's pose, and further based on this When the pose continues to plan the third navigation path. According to the above process, the robot will continue to plan the navigation path according to its current pose until it leads the user to the door of the shop I.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Automation & Control Theory (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Mechanical Engineering (AREA)
  • Robotics (AREA)
  • Electromagnetism (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Optics & Photonics (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)

Abstract

一种机器人定位方法、装置、智能机器人和存储介质，该方法包括：机器人上配置有摄像头以及多种传感器，机器人能够获取摄像头采集的图像以及多种传感器采集到的多种传感数据（步骤101）。接着，机器人先提取采集到的图像包含的语义信息（步骤102），并根据语义信息识别出机器人当前所处的场景（步骤103）。最终，根据与机器人所处场景对应的目标传感数据来确定机器人当前的位姿（步骤104）。可见，在该方法中，确定机器人位姿时使用到的传感数据不是全部的传感数据，而是与场景对应的目标传感数据，这样使得位姿确定的依据更有针对性，从而进一步提高位姿的准确性。

Description

机器人定位方法、装置、智能机器人和存储介质
交叉引用
本申请引用于2019年10月24日递交的名称为“机器人定位方法、装置、智能机器人和存储介质”、申请号为2019110178268的中国专利申请，其通过引用被全部并入本申请。
技术领域
本发明涉及人工智能技术领域，尤其涉及一种机器人定位方法、装置、智能机器人和存储介质。
背景技术
机器人是能够接受人类指挥并执行相应工作的一种机器装置。随着人工智能技术的发展，各种智能机器人越来越多地进入人们的生活，比如服务机器人、清洁机器人、自移动售货机器人等。
以能够自由移动的智能机器人为例，智能机器人会根据自身配置的各类传感器采集到的数据对智能机器人进行定位，并进一步根据定位结果来规划运动轨迹，智能机器人只需依轨迹运动即可完成用户指令。
发明内容
本发明实施例提供一种机器人定位方法、装置和存储介质，用以提高智能机器人定位的准确性。
本发明实施例提供一种机器人定位方法，该方法包括：
获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
提取所述图像中包含的语义信息；
根据所述语义信息识别所述机器人所处的场景；
根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
本发明实施例提供一种机器人定位装置，包括：
获取模块，用于获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
提取模块，用于提取所述图像中包含的语义信息；
识别模块，用于根据所述语义信息识别所述机器人所处的场景；
位姿确定模块，用于根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
本发明实施例提供一种智能机器人，包括：处理器和存储器；其中，所述存储器用于存储一条或多条计算机指令，其中，所述一条或多条计算机指令被所述处理器执行时实现：
获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
提取所述图像中包含的语义信息；
根据所述语义信息识别所述机器人所处的场景；
根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
本发明实施例提供了一种存储计算机指令的计算机可读存储介质，当所述计算机指令被一个或多个处理器执行时，致使所述一个或多个处理器至少执行以下的动作：
获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
提取所述图像中包含的语义信息；
根据所述语义信息识别所述机器人所处的场景；
根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器 人的位姿。
在本发明实施例中，机器人上配置有摄像头以及多种传感器，机器人能够获取摄像头采集的图像以及多种传感器采集到的多种传感数据。接着，机器人先提取采集到的图像包含的语义信息，并根据语义信息识别出机器人当前所处的场景。最终，根据与机器人所处场景对应的目标传感数据确定机器人当前的位置。根据上述描述可知，在确定机器人位姿时使用到的传感数据不是全部的传感数据，而是与场景对应的目标传感数据，这样使得位姿确定的依据更有针对性，从而进一步提高位姿的准确性。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种机器人定位方法的流程图；
图2为本发明实施例提供的另一种机器人定位方法的流程图；
图3a为本发明实施例提供的一种机器人位姿确定方法的流程图；
图3b为本发明实施例提供的另一种机器人位姿确定方法的流程图；
图3c为本发明实施例提供的又一种机器人位姿确定方法的流程图；
图3d为本发明实施例提供的又一种机器人位姿确定方法的流程图；
图4为本发明实施例提供的一种机器人定位装置的结构示意图；
图5为与图4所示实施例提供的机器人定位装置对应的智能机器人的结构示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述， 显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在本发明实施例中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本发明。在本发明实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式。除非上下文清楚地表示其他含义，“多个”一般包含至少两个。
取决于语境，如在此所使用的词语“如果”、“若”可以被解释成为“在……时”或“当……时”或“响应于确定”或“响应于检测”。类似地，取决于语境，短语“如果确定”或“如果检测(陈述的条件或事件)”可以被解释成为“当确定时”或“响应于确定”或“当检测(陈述的条件或事件)时”或“响应于检测(陈述的条件或事件)”。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的商品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种商品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的商品或者系统中还存在另外的相同要素。
下面结合以下的实施例对本文提供的人机交互方法进行详细介绍。同时，下述各方法实施例中的步骤时序仅为一种举例，而非严格限定。
在实际应用中，该机器人定位方法可以由诸如清洁机器人、迎宾机器人、自移动售货机器人等智能机器人来执行。此种机器人能够在一定空间内自由移动，从而完成用户下达的指令。图1为本发明实施例提供的一种机器人定位方法的流程图，该方法的执行主体可以为机器人，如图1所示，该方法可以包括如下步骤：
101、获取机器人上的摄像头采集的图像以及机器人上的多种传感器采集的多种传感数据。
机器人身上的摄像头用于采集机器人所处场景对应的图像，机器人身上的多种传感器分别用于采集多种传感数据，以使机器人获取到的图像和多种传感数据。其中，采集图像的摄像头也可以认为是视觉传感器。并且多种传感器可以包括激光传感器、运动速度测量计、运动角度测量计等等。与此对应的，激光传感器采集到的是激光照射在物体上时对应的激光点坐标。运动速度测量计采集到的是机器人的运动速度，根据此速度可以进一步计算出机器人的运动距离。运动角度测量计采集到的是机器人的运动角速度，根据此角速度可以进一步计算出机器人的运动角度。在实际应用中，上述的摄像头可以是单目或者双目摄像头，运动速度测量计具体可以为轮式里程计，运动角度测量计具体可以为惯性测量单元(Inertial Measurement Unit，简称IMU)。
此时，机器人即可根据获取到的图像和多种传感数据按照下述方式对机器人进行定位。但值得说明的是，由于多种传感器配置于机器人身上的不同位置，每个传感器采集到的传感数据都分别对应于此传感器自身的坐标系，坐标系的不统一显然会对后续的定位过程产生影响。因此，为了保证机器人定位的准确性，可选地，对于已经获取到的多种传感数据，可以对其进行坐标转换，也即是将处于不同坐标系下的传感数据均转换至目标坐标系下。其中，目标坐标系可以是任一种传感器对应的坐标系。当然，在实际应用中，通常会将激光传感器对应的坐标系确定为目标坐标系。不同坐标系之间的转换关系已经预先配置完成，机器人可以直接调用。
除了坐标系的统一之外，在实际应用中，由于不同传感器采集数据的频率不同，因此，传感数据往往存在时间戳不统一的问题。为了进一步保证机器人定位的准确性，则还需要对采集到的多种传感数据进行时间对齐。可选地，当机器人获取到传感器采集到传感数据后，立即读取当前机器人的系统时间，并将读取到的此系统时间确定为传感数据的采集时间。在机器人定位过程中使用到的是采集时间处于预设时间差内的传感数据。
102、提取图像中包含的语义信息。
103、根据语义信息识别机器人所处的场景。
然后，机器人会对摄像头采集到的图像进行语义识别，以提取出图像包含的语义信息。一种可选地方式，机器人中可以配置有预先训练好的语义识别模型，摄像头拍得的图像会直接输入至此模型中，以由模型输出识别结果。接着，机器人便可以根据识别结果来确定机器人所处的场景。其中，上述的语义识别模型具体可以为条件随机场(Conditional Random Field，简称CRF)模型、卷积神经网络(Convolutional Neural Networks，简称CNN)模型、循环神经网络(Recurrent Neural Networks，简称RNN)模型等等。
举例来说，当图像的语义信息包括大面积墙面或者玻璃等预设类型的建筑物，则机器人可以确定自身所处的场景为长廊或者玻璃走道。当图像的语义信息包括多人时，则机器人可以确定自身所处的场景为人群。当图像语义信息中既不包括预设类型的建筑物也不包括人群，则机器人可以确定自身处于普通定位场景。由于在实际应用中，机器人通常使用在公共场所中，则此时，普通定位场景就可以是公共场所的大厅，比如商场大厅、银行大厅、医院大厅等等。
104、根据多种传感数据中与场景对应的目标传感数据确定机器人的位姿。
最后，机器人中可以预先配置有场景与目标传感数据之间的对应关系，基于此对应关系以及机器人所处的场景，机器人便可确定出多种传感数据中哪些为目标传感数据，并根据此目标传感数据来确定机器人的位姿。
举例来说，当机器人处于人群场景中，则机器人可以根据运动速度测量计采集到的传感数据以及运动角度测量计采集到的传感数据确定机器人的位姿。当机器人处于普通定位场景中，则机器人可以根据运动速度测量计、运动角度测量计、激光传感器、摄像头各自采集到的传感数据来确定机器人的位姿。而具体如何实现在不同场景中根据此场景对应的目标传感数据对机器人进行定位的，可以参见下述各实施例中的详细描述。
本发明提供的实施例中，机器人上配置有摄像头以及多种传感器，机器人能够获取摄像头采集的图像以及多种传感器采集到的多种传感数据。接着，机器人先提取采集到的图像包含的语义信息，并根据语义信息识别出机器人 当前所处的场景。最终，根据与机器人所处场景对应的目标传感数据确定机器人当前的位置。根据上述描述可知，在确定机器人位姿时使用到的传感数据不是全部的传感数据，而是与场景对应的目标传感数据，这样使得位姿确定的依据更有针对性，从而进一步提高位姿的准确性。
在实际应用中，机器人在运动过程中还有可能出现无法移动的故障，此时，机器人可以继续按照上述实施例提供的方式对自身进行定位，并将定位结果发送至维护服务器。维护人员则可以根据定位结果找到机器人，以对其进行维护。由于按照上述实施例确定出的定位结果具有较高的准确性，因此，维护人员也就能够快速、准确地找到出现故障的机器人，从而提高机器人的维护效率。
在上面的描述中也已经提到具有移动能力的机器人往往会用于公共场合中，比如商场、医院等等。此时，机器人在定位自己位姿的同时还可以为用户提供导航服务，也即是机器人可以带领用户到达用户要去的目标位置。用户想要到达的目标位置比如可以是商场内某一店铺或者医院内某一诊室等等。
基于上述描述，在图1所示实施例基础上，图2为本发明实施例提供的另一种机器人定位方法的流程图，如图2所示，在上述步骤104之后，该方法还可以包括如下步骤：
201、根据机器人的位姿与用户输入的目标位置确定导航路径，以供机器人根据导航路径移动到目标位置。
用户可以通过与机器人交互的方式来使机器人获取到目标位置。可选地，机器人可以配置有供用户输入目标位置的操作屏幕，此时，目标位置表现为文字形式。另外，机器人上还可以安装有诸如麦克风等拾音器件，当用户对说出目标位置时，机器人便可以通过此拾音器件采集到语音形式的目标位置。
然后，机器人可以以步骤104中确定出的机器人位姿为起点，以用户输入的目标位置为终点规划出一条导航路径。机器人只需沿此导航路径运动，即可带领用户达到目标位置。可选地，机器人的位姿以及用户输入的目标位 置会被标注于预先建立的栅格地图中，机器人会可以根据这两个位置在栅格地图中位置关系，规划出一条最佳导航路径。其中，栅格地图中的每个栅格都被标注为有障碍物或没有障碍物。
对于上述的栅格地图，其构建是一个预处理过程。假设机器人是在场景A内提供定位和导航服务，则建立的栅格地图也对应于场景A，并且地图构建是在机器人开始在场景A中提供定位和导航服务之前完成的。
可选地，栅格地图可以根据机器人配置的激光传感器采集到的历史传感数据构建。具体地，机器人在提供定位和导航服务之前，可以在先此场景A中遍历环境运动，并且此时场景A所在的平面区域已经被预先划分为若干子区域，每个子区域称为一个栅格。随着机器人的运动，机器人配置的激光传感器便会采集到传感数据，此传感数据即为前述的历史传感数据。然后，机器人再根据历史传感数据确定此场景A中哪些位置存在障碍物，进一步计算出障碍物在栅格地图中的二维坐标。最后，根据此二维坐标对栅格地图中的每个栅格进行标注，从而构建起栅格地图。
另外，当机器人初次被放置在场景A中一位置时，它并不知晓自己当前的位姿，此位姿可以称为初始位姿。可选地，机器人可以采用以下方式确定机器人的初始位姿，则在上述步骤101之前，该方法还可以包括如下步骤：
202、识别摄像头采集的初始图像包含的语义信息。
203、根据语义信息以及预先构建的栅格语义地图中包含的语义信息确定机器人的初始位姿。
具体来说，机器人初次被放置在场景A中，可选地，机器人可以原地旋转，以使摄像头采集到对应于各个方向的图像。为了与上述实施例步骤101中采集到的图像区分，此时采集到的图像可以称为初始图像。然后，机器人可以对每张初始图像进行语义识别，具体的语义识别过程可以参见图1所示实施例步骤102中的相关步骤，在此不再赘述。然后，再将初始图像中包括的语义信息与预先构建的语义栅格地图中包含的语义信息进行比对，从而确定机器人的初始位姿。其中，语义栅格地图中每个栅格既标注有是否存在障 碍物，还标注有障碍物具体是哪种物体。
对于语义栅格地图，可选地，此地图的建立也是一个预处理过程。语义栅格地图是在栅格地图之后构建的，也是在机器人开始在场景A中提供定位和导航服务之前完成的。假设机器人是在场景A内提供定位和导航服务，则建立的语义栅格地图也对应于场景A。
语义栅格地图可以根摄像头采集到的历史图像以及已经构建的栅格地图构建。具体地，机器人先按照步骤201公开方式生成栅格地图，并且机器人可以在此场景A中运动的过程中，机器人配置的摄像头可以采集有若干张图像，此若干张图像可以称为历史图像。利用语义识别模型识别出历史图像中包括哪种物体，并将识别出的物体标注在栅格地图中，以构建起语义栅格地图。
本实施例中，在机器人确定出机器人在场景中的位姿后，还可以用户输入的目标位置为用户提供导航服务。由于机器人确定出的位姿具有较高的准确度，则提供的导航服务也具有较高的准确度，提高机器人的服务水平。
根据图1所示实施例可知，机器人可以根据摄像头采集到图像的语义信息来确定机器人所处的场景。可见，图像的语义信息识别的准确性能够直接影响到场景确定的准确性。为了保证语义识别的准确性，还可以将图像的亮度也考虑进来，也即是在进行语义识别过程中同时考虑图像中包含的物体以及图像对应的环境亮度值。环境亮度值用于表明图像采集环境的光照强度，光照强度过小或过大，都会对图像的语义识别造成影响，并进一步导致识别出的场景不够准确。
此时，机器人可以同时根据图像对应的环境亮度值以及图像中包含的物体来识别机器人所处场景。对于环境亮度值的确定过程，一种可选地方式，机器人先将采集到的图像转换为灰度图像，再根据每个像素点的灰度值确定图像对应的环境亮度值。比如，可以计算所有像素点灰度值的平均值，并将此平均值确定为图像对应的环境亮度值。
若计算出图像对应的环境亮度值不处于预设数值范围内时，则表明机器人当前所处场景是一种光照强度不适宜，比如光照强度过大或者过小的场景，举例来说可以是商场内灯光较强的表演区域等等。这种情况下，图1所示实施例中步骤104一种可选地实现方式，如图3a所示，该方法可以包括如下步骤：
301、根据运动速度测量计采集的传感数据和运动角度测量计采集的传感数据确定第一位姿。
302、将第一位姿映射到预先构建的栅格地图中，以得到第二位姿。
303、确定第二位置为机器人的位姿。
具体来说，正如上述实施例提到的运动速度测量计可以为轮式里程计，运动角度测量计可以为IMU。则基于此，机器人一方面可以根据轮式里程计采集到的机器人左右轮子的转速计算出机器人的运动距离，另一方面可以根据IMU采集到的角速度计算出机器人的运动角度。然后，根据计算出的运动距离、运动角度以及上一时刻机器人的位姿确定出第一位姿，此第一位姿是世界坐标系下的位姿坐标。然后，机器人会将第一位姿映射到预先构建的栅格地图中，也即是将处于世界坐标系下的第一位姿转换到栅格地图坐标系中，以得到第二位姿，此第二位姿也即是机器人当前时刻的位姿，也即是机器人的定位结果。其中，在上述过程中使用到的栅格地图可以参照图2所示实施例中的相关描述构建。
本实施例中，当机器人所处的场景是一种光照强度过大或者过小的场景时，机器人是根据运动速度测量计、运动角度测量计采集到的传感数据以及利用激光传感器采集到的传感数据预先构建的栅格地图对机器人进行定位的，位姿确定的依据更有针对性，提高了定位的准确性。
若机器人计算出图像对应的环境亮度值处于预设数值范围内，则表明机器人当前所处场景是一种光照强度适宜的场景，比如商场大厅、医院大厅等普通定位场景。这种情况下，上述实施例中步骤104另一种可选地实现方式，如图3b所示，该方法可以包括如下步骤：
401、根据运动速度测量计采集的传感数据和运动角度测量计采集的传感数据确定第一位姿。
402、将第一位姿映射到预先构建的栅格地图中，以得到第二位姿。
上述步骤401～402的执行过程与前述实施例的相应步骤相似，可以参见如图3a所示实施例中的相关描述，在此不再赘述。
403、将第一位姿映射到预先构建的视觉特征地图中，以得到第三位姿。
视觉特征地图中每个地图点都对应于世界坐标系下的一个位姿坐标，由于第一位姿也是在世界坐标系下的坐标，因此，可以将第一位姿在世界坐标系下的坐标与视觉特征地图中每个地图点在世界坐标系下的坐标进行比对，在视觉特征地图中确定与第一位姿的位置坐标最接近的地图点为目标地图点，此目标地图点对应的位姿坐标也即是第三位姿。上述映射过程实际上就是确定目标地图点的过程。
对于上述的视觉特征地图，其构建同样是一个预处理过程。假设机器人是在场景A内提供定位和导航服务，则建立的视觉特征地图也对应于场景A，并且地图构建是在机器人开始在场景A中提供定位和导航服务之前完成的。
可选地，视觉特征地图可以根据机器人配置的摄像头采集到的图像构建。具体地，机器人在提供定位和导航服务之前，可以在先此场景A中随意运动，以使机器人配置的摄像头采集到图像，为了与步骤101中使用到的图像区分，此时采集到的图像可以称为历史图像，从采集到的历史图像提取稀疏特征点，并根据提取的稀疏特征点构成视觉特征地图。
404、融合第二位姿和第三位姿，以确定融合结果为机器人的位姿。
最终，机器人会对映射后得到的第二位姿和第三位姿进行融合，融合结果也即是机器人的定位结果。一种可选地融合方式，可以分别为第二位姿和第三位姿设置不同的权重值，并对二者进行加权求和，求和结果即为融合结果。另一种可选地融合方式，还可以采用扩展卡尔曼滤波方式对第二位姿和第三位姿进行融合，以得到融合结果。这种融合方式是一种较为成熟的技术，本申请不再详细描述融合过程。
本实施例中，当机器人所处的场景是一种光照强度适宜的场景，则可以机器人是根据运动速度测量计、运动角度测量计采集到的传感数据以及利用激光传感器采集到的传感数据预先构建的栅格地图、利用摄像头采集的图像预先构建的视觉特征地图来对机器人进行定位。位姿确定的依据更有针对性，从而提高位姿的准确性。
除了上述场景外，当机器人从摄像头采集的图像中识别出图像包含预设结构的建筑物，比如长廊、玻璃走道等等。此时，可以认为机器人处于一种预设结构的建筑物内。这种情况下，上述实施例中步骤104又一种可选地实现方式，如图3c所示，该方法可以包括如下步骤：
501、根据运动速度测量计采集的传感数据和运动角度测量计采集的传感数据确定第一位姿。
502、将第一位姿映射到预先构建的视觉特征地图中，以得到第四位姿。
503、确定第四位姿为机器人的位姿。
上述步骤501～503的执行过程与前述实施例的相应步骤相似，可以参见如图3b所示实施例中的相关描述，在此不再赘述。
在此需要说明的有，一方面，本实施例中得到的第四位姿实际上就是图3b所示实施例中的第三位姿，名称不同只是为了在不同实施例中加以区分。另一方面，本实施例中机器人处于预设结构的建筑物内，并且此建筑物内部的光照强度也是适宜的，也即是在这种场景中机器人上摄像头采集到的图像对应的环境亮度值处于预设数值范围内。
本实施例中，当机器人处于一个光照强度适宜的预设结构的建筑物内，则可以机器人是根据运动速度测量计、运动角度测量计采集到的传感数据以及利用激光传感器采集到的传感数据预先构建的栅格地图、利用摄像头采集的图像预先构建的视觉特征地图来对机器人进行定位。位姿确定的依据更有针对性，以提高位姿的准确性。
另外，根据上述各实施例中的描述可知，预先构建的栅格地图、语义栅格地图以及视觉特征地图都是在场景A处于状态I时建立的。举例来说，场景A可以是商场场景，则状态I表示的是商场内一种货架摆放方式。而对于商场场景，货架的摆放方式往往会发生变动，从而使商场处于状态II，此状态II表示的是商场的另一种货架摆放方式。基于上述描述，当机器人根据摄像头采集到的图像确定出机器人所处的是状态II的场景A，而不是状态I的场景A时，则预先构建的对应于处于状态I的各种地图便是无法使用的。在这种情况下，上述实施例中步骤104又一种可选地实现方式，如图3d所示，该方法可以包括如下步骤：
601、根据运动速度测量计采集的传感数据和运动角度测量计采集的传感数据确定第一位姿。
上述步骤601的执行过程与前述实施例的相应步骤相似，可以参见如图3b所示实施例中的相关描述，在此不再赘述。
602、根据摄像头采集到的图像确定第五位姿。
机器人上的摄像头会采集到若干张图像，机器人会对这些图像中采集时间相邻的两张图像进行匹配。为了便于描述，可以将采集时间相邻的两张图像分别称为第一图像和第二图像。经过匹配后，可以确定出第一图像中的各像素点分别与第二图像中的哪个图像点是相同像素点。然后，根据具有匹配关系的像素点分别在第一图像和第二图像中的像素坐标确定第五位姿。
603、融合第一位姿和第五位姿，以确定融合结果为机器人的位姿。
上述步骤603的执行过程与前述实施例的相应步骤相似，可以参见如图3b所示实施例中的相关描述，在此不再赘述。
需要说明的有，本实施例中机器人处于的是状态II的场景A中，并且此场景内的光照强度也是适宜的，也即是机器人上摄像头采集到的图像对应的环境亮度值处于预设数值范围内。
本实施例中，当机器人在处于状态II的场景A中，其此状态的场景A一个光照强度适宜时，则机器人可以根据运动速度测量计、运动角度测量计采集 到的传感数据以及摄像头采集的图像来对机器人进行定位。位姿确定的依据更有针对性，从而提高了位姿的准确性。
当然，在实际应用中，处于状态II的场景A也可能是一个光照强度过大或过小的场景，此时，机器人可以直接根据运动速度测量计采集的传感数据和运动角度测量计采集的传感数据确定机器人的位姿。位姿的具体确定过程可以参见上述各实施例中的相关描述，在此不再赘述。
当机器人根据摄像头采集到的图像识别出其中包括人群时，则可以认为机器人所处的场景是一种人流密集的场景。这种情况下，机器人可以直接根据运动速度测量计采集的传感数据和运动角度测量计采集的传感数据确定机器人的位姿。位姿的具体确定过程可以参见上述各实施例中的相关描述，在此不再赘述。
以下将详细描述本发明的一个或多个实施例的机器人定位装置。本领域技术人员可以理解，这些人机交互装置均可使用市售的硬件组件通过本方案所教导的步骤进行配置来构成。
图4为本发明实施例提供的一种机器人定位装置的结构示意图，如图4所示，该装置包括：
获取模块11，用于获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据。
提取模块12，用于提取所述图像中包含的语义信息。
识别模块13，用于根据所述语义信息识别所述机器人所处的场景。
位姿确定模块14，用于根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
可选地，该机器人定位装置还可以包括：
导航模块21，用于根据所述机器人的位姿与用户输入的目标位置确定导航路径，以供所述机器人根据所述导航路径移动到所述目标位置。
为识别所述机器人所处的场景，可选地，该机器人定位装置中的识别模 块13具体用于：根据所述图像对应的环境亮度值和/或所述图像中包含的物体，识别所述机器人所处的场景。
可选地，所述场景对应于所述环境亮度值不处于预设数值范围内。
该机器人定位装置中的位姿确定模块14用于：
根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定第一位姿；将所述第一位姿映射到预先构建的栅格地图中，以得到第二位姿，所述栅格地图根据所述激光传感器器采集到的历史传感数据构建以及确定所述第二位姿为所述机器人的位姿。
可选地，所述场景对应于所述环境亮度值处于预设数值范围内。
该机器人定位装置中的位姿确定模块14具体还用于将所述第一位姿映射到预先构建的视觉特征地图中，以得到第三位姿，所述视觉特征地图根据所述摄像头采集的历史图像构建；以及用于融合所述第二位姿和所述第三位姿，以确定融合结果为所述机器人的位姿。
可选地，所述场景对应于所述图像中包含预设结构的建筑物。
该机器人定位装置中的位姿确定模块14具体还用于：根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定第一位姿；将所述第一位姿映射到预先构建的视觉特征地图中，以得到第四位姿，所述视觉特征地图根据所述摄像头采集的历史图像构建；以及确定所述第四位姿为所述机器人的位姿。
可选地，所述场景对应于所述图像中包含人群。
该机器人定位装置中的位姿确定模块14具体还用于：根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定所述机器人的位姿。
可选地，所述机器人所处的场景为无法使用所述栅格地图的场景。
该机器人定位装置中的位姿确定模块14具体还用于：根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定第一位姿；根据所述摄像头采集到的图像确定所述第五位姿；以及融合所述第一位姿和 所述第五位姿，以确定融合结果为所述机器人的位姿。
图4所示的机器人定位装置可以执行前述图1至图3d所示实施例提供的机器人定位方法，本实施例未详细描述的部分，可参考对图1至图3d所示实施例的相关说明，在此不再赘述。本实施例所能达到的技术效果也可以参见图1至图3d所示实施例中的描述。
以上描述了机器人定位装置的内部功能和结构，在一个可能的设计中，机器人定位装置的结构可实现为机器人中的一部分，如图5所示，该智能机器人可以包括：处理器31和存储器32。其中，所述存储器32用于存储支持该智能机器人执行前述图1至图3d所示实施例中提供的机器人定位方法的程序，所述处理器31被配置为用于执行所述存储器32中存储的程序。
所述程序包括一条或多条计算机指令，其中，所述一条或多条计算机指令被所述处理器31执行时能够实现如下步骤：
获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
提取所述图像中包含的语义信息；
根据所述语义信息识别所述机器人所处的场景；
根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
可选地，所述处理器31还用于执行前述图1至图3d所示实施例中的全部或部分步骤。
其中，所述智能机器人的结构中还可以包括通信接口33，用于与其他设备或通信网络通信。
另外，本发明实施例提供了一种存储计算机指令的计算机可读存储介质，当所述计算机指令被一个或多个处理器执行时，致使所述一个或多个处理器至少执行以下的动作：
获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集 的多种传感数据；
提取所述图像中包含的语义信息；
根据所述语义信息识别所述机器人所处的场景；
根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助加必需的通用硬件平台的方式来实现，当然也可以通过硬件和软件结合的方式来实现。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以计算机产品的形式体现出来。
为便于理解，结合如下的应用场景对以上提供的机器人定位方法的具体实现进行示例性说明。
以商场这种公共场合为例，商场大厅中可能会设置诸如服务机器人等的智能终端设备。机器人可以在商场的任意位置随意走动，这样可以以使处于商场不同位置的用户都能向机器人发出指令。其中，用户可以向机器人发出咨询指令或者导航请求指令。
机器人在商场内随意走动的过程中，机器人上的摄像头会实时采集图像，并且机器人上的运动速度测量计、运动角度测量计也会实时采集传感器数据。对于采集到的每张图像，机器人都会对其进行语义识别，以确定图像中包含的物体，同时还会确定图像对应的环境亮度值，并进一步根据图像中包含的物体以及图像的环境亮度值确定机器人所处的场景。
当识别出机器人处于商场大厅时，此场景是一个光照强度适宜的场景，即此场景中图像对应的环境亮度值处于预设数值范围内。这种场景下，机器人会根据运动速度测量计、运动角度测量计采集的传感数据确定第一位姿， 再将第一位姿分别映射到预先构建的栅格地图和视觉特征地图中，以分别得到第二位姿和第三位姿，最终将第二位姿和第三位姿进行融合，将融合结果确定为机器人当前的位姿，此也即是机器人的定位信息。其中，栅格地图和视觉特征地图都是对应于商场的。
当识别出机器人处于商场内人流密集区时，机器人会根据运动速度测量计、运动角度测量计采集的传感数据确定机器人当前的位姿即定位信息。
根据上述描述可知，当机器人处于不同的场景时，用于确定机器人当前位姿所使用到的传感数据是具有针对性的，从而能够保证机器人定位的准确性。在机器人在运动过程中出现运动故障时，则维护人员便可以根据准确的定位信息快速找到机器人，以对机器人进行维修，保证了机器人的维修效率。
另外，用户可以通过文字或者语音的形式向机器人发出导航请求指令，比如“我想要去店铺I”。机器人响应于此导航请求指令，会将店铺I所在的位置标注在商场对应的栅格地图中。同时，机器人还会根据摄像头采集的图像以及多种传感数据确定自己当前所处的场景。
若机器人当前处于商场大厅时，则机器人可以根据运动速度测量计、运动角度测量计采集的传感数据以及预先构建的栅格地图、视觉特征地图来确定出机器人当前的位姿，此位姿会被标注于栅格地图上，以使机器人根据此位姿与店铺I的目标位置为机器人规划出第一导航路径。之后，机器人便会沿此导航路径开始运动。在沿第一导航路径运动的过程中，机器人上的摄像头以及多种传感器依旧会实时采集图像和传感数据，并不断根据采集到的图像确定机器人所处的场景。
若在第一时刻确定出机器人已经从商场大厅运动到长廊时，则机器人会根据运动速度测量计、角度速度测量计各自采集到的传感数据以及预先构建的视觉特征地图确定机器人的位姿。此时，机器人可以根据第一时刻机器人的位姿以及店铺I所在的目标位置重新规划第二导航路径，机器人会沿此第二导航路径继续运动。若在第二时刻确定出机器人已经从长廊运动到商场内的人流密集区域时，则机器人运动速度测量计、角度速度测量计各自采集到的 传感数据确定机器人的位姿，并进一步根据此时的位姿继续规划第三导航路径。按照上述过程，机器人会不断根据自己当前的位姿规划导航路径直至将用户带领至店铺I门口。
根据上述描述可知，机器人从商场大厅运动到店铺I门口的过程中，在不同场景运动时，会使用不同的数据来对机器人进行实时定位，直至完成导航。使用针对性的数据对机器人进行定位，能够保证定位的准确性，由于定位信息是导航路径规划的基础，所以，也能够进一步地保证导航的准确性，保证机器人的服务质量。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (12)

  1. 一种机器人定位方法，其特征在于，所述方法包括：
    获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
    提取所述图像中包含的语义信息；
    根据所述语义信息识别所述机器人所处的场景；
    根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    根据所述机器人的位姿与用户输入的目标位置确定导航路径，以供所述机器人根据所述导航路径移动到所述目标位置。
  3. 根据权利要求1所述的方法，其特征在于，所述多种传感器包括：运动速度测量计、运动角度测量计以及激光传感器。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述语义信息识别所述机器人所处的场景，包括：
    根据所述图像对应的环境亮度值和/或所述图像中包含的物体，识别所述机器人所处的场景。
  5. 根据权利要求4所述的方法，其特征在于，所述场景对应于所述环境亮度值不处于预设数值范围内；
    所述根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿，包括：
    根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定第一位姿；
    将所述第一位姿映射到预先构建的栅格地图中，以得到第二位姿，所述栅格地图根据所述激光传感器器采集到的历史传感数据构建；
    确定所述第二位姿为所述机器人的位姿。
  6. 根据权利要求4所述的方法，其特征在于，所述场景对应于所述环境亮度值处于所述预设数值范围内；
    所述根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿，包括：
    将所述第一位姿映射到预先构建的视觉特征地图中，以得到第三位姿，所述视觉特征地图根据所述摄像头采集的历史图像构建，所述第一位姿根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定；
    融合所述第二位姿和所述第三位姿，以确定融合结果为所述机器人的位姿，所述第二位姿根据所述第一位姿和所述栅格地图确定。
  7. 根据权利要求4所述的方法，其特征在于，所述场景对应于所述图像中包含预设结构的建筑物；
    所述根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿，包括：
    根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定第一位姿；
    将所述第一位姿映射到预先构建的视觉特征地图中，以得到第四位姿，所述视觉特征地图根据所述摄像头采集的历史图像构建；
    确定所述第四位姿为所述机器人的位姿。
  8. 根据权利要求4所述的方法，其特征在于，所述场景对应于所述图像中包含人群；
    所述根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿，包括：
    根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定所述机器人的位姿。
  9. 根据权利要求5所述的方法，其特征在于，所述机器人所处的场景为无法使用所述栅格地图的场景；
    所述根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿，包括：
    根据所述运动速度测量计采集的传感数据和所述运动角度测量计采集的传感数据确定第一位姿；
    根据所述摄像头采集到的图像确定所述第五位姿；
    融合所述第一位姿和所述第五位姿，以确定融合结果为所述机器人的位姿。
  10. 一种机器人定位装置，其特征在于，包括：
    获取模块，用于获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
    提取模块，用于提取所述图像中包含的语义信息；
    识别模块，用于根据所述语义信息识别所述机器人所处的场景；
    位姿确定模块，用于根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
  11. 一种智能机器人，其特征在于，包括：处理器和存储器；其中，所述存储器用于存储一条或多条计算机指令，其中，所述一条或多条计算机指令被所述处理器执行时实现：
    获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集的多种传感数据；
    提取所述图像中包含的语义信息；
    根据所述语义信息识别所述机器人所处的场景；
    根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
  12. 一种存储计算机指令的计算机可读存储介质，其特征在于，当所述计算机指令被一个或多个处理器执行时，致使所述一个或多个处理器至少执行以下的动作：
    获取机器人上的摄像头采集的图像以及所述机器人上的多种传感器采集 的多种传感数据；
    提取所述图像中包含的语义信息；
    根据所述语义信息识别所述机器人所处的场景；
    根据所述多种传感数据中与所述场景对应的目标传感数据确定所述机器人的位姿。
PCT/CN2020/115046 2019-10-24 2020-09-14 机器人定位方法、装置、智能机器人和存储介质 Ceased WO2021077941A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20879071.7A EP4050449B1 (en) 2019-10-24 2020-09-14 Method and device for robot positioning, smart robot, and storage medium
US17/771,428 US12415277B2 (en) 2019-10-24 2020-09-14 Robot positioning method and apparatus, intelligent robot, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911017826.8A CN112711249B (zh) 2019-10-24 2019-10-24 机器人定位方法、装置、智能机器人和存储介质
CN201911017826.8 2019-10-24

Publications (1)

Publication Number Publication Date
WO2021077941A1 true WO2021077941A1 (zh) 2021-04-29

Family

ID=75540805

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/115046 Ceased WO2021077941A1 (zh) 2019-10-24 2020-09-14 机器人定位方法、装置、智能机器人和存储介质

Country Status (4)

Country Link
US (1) US12415277B2 (zh)
EP (1) EP4050449B1 (zh)
CN (1) CN112711249B (zh)
WO (1) WO2021077941A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114494848A (zh) * 2021-12-21 2022-05-13 重庆特斯联智慧科技股份有限公司 一种机器人视距路径确定方法和装置
CN119610113A (zh) * 2024-12-23 2025-03-14 山东核电有限公司 一种应用图像识别算法的核电站流道联合清淤机器人

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112161618B (zh) * 2020-09-14 2023-03-28 灵动科技（北京）有限公司 仓储机器人定位与地图构建方法、机器人及存储介质
CN113361420B (zh) * 2021-06-10 2024-11-26 煤炭科学技术研究院有限公司 基于机器人的矿井火灾监测方法、装置、设备及存储介质
CN113341991B (zh) * 2021-06-18 2022-08-09 重庆大学 一种基于动态窗口和冗余节点过滤的路径优化方法
CN113455965B (zh) * 2021-06-30 2022-05-27 广州科语机器人有限公司 清洁机器人控制方法、装置、介质和清洁机器人
CN114390064B (zh) * 2021-12-30 2024-10-01 科沃斯商用机器人有限公司 设备定位方法、装置、机器人和存储介质
CN116974270A (zh) * 2022-04-21 2023-10-31 广州高新兴机器人有限公司 一种视觉语义辅助激光定位的方法、装置及机器人
WO2024001339A1 (zh) * 2022-07-01 2024-01-04 华为云计算技术有限公司 确定位姿的方法、装置以及计算设备
CN115730236B (zh) * 2022-11-25 2023-09-22 杭州电子科技大学 一种基于人机交互药物识别获取方法、设备及存储介质
CN117710702B (zh) * 2023-07-31 2025-09-05 荣耀终端股份有限公司 视觉定位方法、装置、存储介质和程序产品

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103558856A (zh) * 2013-11-21 2014-02-05 东南大学 动态环境下服务动机器人导航方法
CN103712617A (zh) * 2013-12-18 2014-04-09 北京工业大学 一种基于视觉内容的多层语义地图的创建方法
CN106272423A (zh) * 2016-08-31 2017-01-04 哈尔滨工业大学深圳研究生院 一种针对大尺度环境的多机器人协同制图与定位的方法
CN106840148A (zh) * 2017-01-24 2017-06-13 东南大学 室外作业环境下基于双目摄像机的可穿戴式定位与路径引导方法
CN107066507A (zh) * 2017-01-10 2017-08-18 中国人民解放军国防科学技术大学 一种基于云机器人混合云架构的语义地图构建方法
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
CN110220517A (zh) * 2019-07-08 2019-09-10 紫光云技术有限公司 一种结合环境语意的室内机器人鲁棒slam方法
CN110275540A (zh) * 2019-07-01 2019-09-24 湖南海森格诺信息技术有限公司 用于扫地机器人的语义导航方法及其系统
WO2019185170A1 (en) * 2018-03-30 2019-10-03 Toyota Motor Europe Electronic device, robotic system and method for localizing a robotic system

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9020263B2 (en) * 2008-02-15 2015-04-28 Tivo Inc. Systems and methods for semantically classifying and extracting shots in video
CN103900583B (zh) * 2012-12-25 2018-02-27 联想(北京)有限公司 用于即时定位与地图构建的设备和方法
CN103278170B (zh) * 2013-05-16 2016-01-06 东南大学 基于显著场景点检测的移动机器人级联地图创建方法
CN103278130B (zh) 2013-05-27 2016-05-25 杭州电子科技大学 一种离合器摩擦片平整度测试设备及其使用方法
CN112923936B (zh) * 2015-02-10 2023-04-18 御眼视觉技术有限公司 用于车辆的导航系统、方法及计算机可读介质
JP6117901B1 (ja) * 2015-11-30 2017-04-19 ファナック株式会社 複数の物品の位置姿勢計測装置及び該位置姿勢計測装置を含むロボットシステム
WO2018006082A2 (en) * 2016-07-01 2018-01-04 Uber Technologies, Inc. Autonomous vehicle control using submaps
US20180005052A1 (en) * 2016-07-01 2018-01-04 Uber Technologies, Inc. Static object detection for operating autonomous vehicle
KR102017148B1 (ko) * 2017-03-03 2019-09-02 엘지전자 주식회사 장애물을 학습하는 인공지능 이동 로봇 및 그 제어방법
US10552979B2 (en) * 2017-09-13 2020-02-04 TuSimple Output of a neural network method for deep odometry assisted by static scene optical flow
CN107782304B (zh) * 2017-10-26 2021-03-09 广州视源电子科技股份有限公司 移动机器人的定位方法及装置、移动机器人及存储介质
WO2019086465A1 (en) * 2017-11-02 2019-05-09 Starship Technologies Oü Visual localization and mapping in low light conditions
US10878294B2 (en) 2018-01-05 2020-12-29 Irobot Corporation Mobile cleaning robot artificial intelligence for situational awareness
US10345822B1 (en) * 2018-01-26 2019-07-09 Ford Global Technologies, Llc Cognitive mapping for vehicles
US11188091B2 (en) * 2018-03-06 2021-11-30 Zoox, Inc. Mesh decimation based on semantic information
CN109682368B (zh) * 2018-11-30 2021-07-06 上海肇观电子科技有限公司 机器人及地图构建方法、定位方法、电子设备、存储介质
CN109724603A (zh) * 2019-01-08 2019-05-07 北京航空航天大学 一种基于环境特征检测的室内机器人导航方法
CN110319832B (zh) * 2019-07-05 2024-05-17 京东科技信息技术有限公司 机器人定位方法、装置、电子设备及介质
CN110340877B (zh) * 2019-07-11 2021-02-05 深圳市杉川机器人有限公司 移动机器人及其定位方法和计算机可读存储介质

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103558856A (zh) * 2013-11-21 2014-02-05 东南大学 动态环境下服务动机器人导航方法
CN103712617A (zh) * 2013-12-18 2014-04-09 北京工业大学 一种基于视觉内容的多层语义地图的创建方法
CN106272423A (zh) * 2016-08-31 2017-01-04 哈尔滨工业大学深圳研究生院 一种针对大尺度环境的多机器人协同制图与定位的方法
CN107066507A (zh) * 2017-01-10 2017-08-18 中国人民解放军国防科学技术大学 一种基于云机器人混合云架构的语义地图构建方法
CN106840148A (zh) * 2017-01-24 2017-06-13 东南大学 室外作业环境下基于双目摄像机的可穿戴式定位与路径引导方法
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
WO2019185170A1 (en) * 2018-03-30 2019-10-03 Toyota Motor Europe Electronic device, robotic system and method for localizing a robotic system
CN110275540A (zh) * 2019-07-01 2019-09-24 湖南海森格诺信息技术有限公司 用于扫地机器人的语义导航方法及其系统
CN110220517A (zh) * 2019-07-08 2019-09-10 紫光云技术有限公司 一种结合环境语意的室内机器人鲁棒slam方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4050449A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114494848A (zh) * 2021-12-21 2022-05-13 重庆特斯联智慧科技股份有限公司 一种机器人视距路径确定方法和装置
CN114494848B (zh) * 2021-12-21 2024-04-16 重庆特斯联智慧科技股份有限公司 一种机器人视距路径确定方法和装置
CN119610113A (zh) * 2024-12-23 2025-03-14 山东核电有限公司 一种应用图像识别算法的核电站流道联合清淤机器人

Also Published As

Publication number Publication date
EP4050449B1 (en) 2025-12-24
US12415277B2 (en) 2025-09-16
EP4050449A4 (en) 2022-11-16
EP4050449A1 (en) 2022-08-31
US20220362939A1 (en) 2022-11-17
CN112711249A (zh) 2021-04-27
CN112711249B (zh) 2023-01-03

Similar Documents

Publication Publication Date Title
CN112711249B (zh) 机器人定位方法、装置、智能机器人和存储介质
US10818188B2 (en) Method for dispatching a vehicle to a user's location
CN111958592B (zh) 一种变电站巡检机器人图像语义分析系统及方法
WO2022052660A1 (zh) 仓储机器人定位与地图构建方法、机器人及存储介质
KR20210029586A (ko) 이미지 내의 특징적 객체에 기반하여 슬램을 수행하는 방법 및 이를 구현하는 로봇과 클라우드 서버
TW201715476A (zh) 運用擴增實境技術之導航系統
CN107390721A (zh) 机器人随行控制方法、装置及机器人
Elgendy et al. A novel marker detection system for people with visual impairment using the improved tiny-yolov3 model
CN107966989A (zh) 一种机器人自主导航系统
Ye et al. 6-DOF pose estimation of a robotic navigation aid by tracking visual and geometric features
He et al. Wearable ego-motion tracking for blind navigation in indoor environments
CN110664593A (zh) 基于HoloLens的盲人导航系统及方法
CN115808170B (zh) 一种融合蓝牙与视频分析的室内实时定位方法
WO2021212609A1 (zh) 地图构建方法、装置、计算机设备和存储介质
US12188772B2 (en) Information processing apparatus and information processing method for self-position estimation
WO2020155075A1 (zh) 一种导航装置、方法及相关设备
WO2020114214A1 (zh) 导盲方法和装置，存储介质和电子设备
CN207965645U (zh) 一种机器人自主导航系统
CN117570960A (zh) 一种用于导盲机器人的室内定位导航系统及方法
Hile et al. Information overlay for camera phones in indoor environments
Jadhav et al. Ai guide dog: Egocentric path prediction on smartphone
CN115588036A (zh) 图像采集方法及装置、机器人
CN115908549A (zh) 一种移动作业机器人的实时动态安全防护方法
CN110991383B (zh) 一种多相机联合的周界区域人员定位方法
WO2022246812A1 (zh) 定位方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20879071

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020879071

Country of ref document: EP

Effective date: 20220524

WWG Wipo information: grant in national office

Ref document number: 17771428

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2020879071

Country of ref document: EP