WO2024001596A1 - 机器人运动控制方法以及装置 - Google Patents

机器人运动控制方法以及装置 Download PDF

Info

Publication number
WO2024001596A1
WO2024001596A1 PCT/CN2023/095050 CN2023095050W WO2024001596A1 WO 2024001596 A1 WO2024001596 A1 WO 2024001596A1 CN 2023095050 W CN2023095050 W CN 2023095050W WO 2024001596 A1 WO2024001596 A1 WO 2024001596A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
motion
robot
reference object
edge line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/095050
Other languages
English (en)
French (fr)
Other versions
WO2024001596A9 (zh
Inventor
郝越凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Geekplus Technology Co Ltd
Original Assignee
Beijing Geekplus Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Geekplus Technology Co Ltd filed Critical Beijing Geekplus Technology Co Ltd
Priority to EP23829782.4A priority Critical patent/EP4549103A4/en
Priority to US18/864,725 priority patent/US20250312922A1/en
Publication of WO2024001596A1 publication Critical patent/WO2024001596A1/zh
Publication of WO2024001596A9 publication Critical patent/WO2024001596A9/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/646—Following a predefined trajectory, e.g. a line marked on the floor or a flight path
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B25J19/023—Optical sensing devices including video camera means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/242—Means based on the reflection of waves generated by the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/243—Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2105/00—Specific applications of the controlled vehicles
    • G05D2105/20—Specific applications of the controlled vehicles for transportation
    • G05D2105/28—Specific applications of the controlled vehicles for transportation of freight
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2107/00—Specific environments of the controlled vehicles
    • G05D2107/70—Industrial sites, e.g. warehouses or factories
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2109/00—Types of controlled vehicles
    • G05D2109/10—Land vehicles
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2111/00—Details of signals used for control of position, course, altitude or attitude of land, water, air or space vehicles
    • G05D2111/10—Optical signals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2111/00—Details of signals used for control of position, course, altitude or attitude of land, water, air or space vehicles
    • G05D2111/10—Optical signals
    • G05D2111/17—Coherent light, e.g. laser signals

Definitions

  • the embodiments of this specification relate to the field of robot visual navigation technology, and in particular to a robot motion control method.
  • Robot technology has achieved rapid development and is playing an increasingly important role in industrial production. It is widely used to complete tasks with high repeatability, high risk and high precision requirements; human beings hope Robots can better serve humans and even replace humans in completing a variety of tasks. This requires robots not only to have the ability to complete tasks, but also to have the ability to go to the task location as required, which is the robot's navigation technology.
  • the robot determines its running path by identifying the identification codes laid on the ground, and then restricts the running direction of the robot; however, the identification codes laid on the ground will have manufacturing costs during the manufacturing process. And the laying of identification codes has strict laying standards. To meet the laying standards, sufficient manpower and material resources need to be invested. Finally, after the identification codes are laid, they need to be maintained regularly to avoid the identification codes from being damaged due to wear, aging, etc. cannot be recognized by the robot. Therefore, a method is urgently needed to solve the above problems encountered in the process of robot visual navigation.
  • embodiments of this specification provide a robot motion control method.
  • One or more embodiments of this specification simultaneously relate to a robot motion control device, a computing device, a computer-readable storage medium, and a computer program to solve technical deficiencies existing in the prior art.
  • a robot motion control method including:
  • the moving scene image collected by the visual sensor detect the target reference object based on the moving scene image, and obtain the target reference object detection result, wherein the target reference object is the reference object on both sides of the movement track of the target robot;
  • the motion adjustment parameters of the target robot are determined based on the positioning information, and the motion state of the target robot is adjusted according to the motion adjustment parameters.
  • detecting the target reference object based on the moving scene image and obtaining the target reference object detection result includes:
  • the target reference object detection result is obtained.
  • the target reference object is a shelf; scanning the sports scene image to obtain edge segments within a preset angle range includes:
  • the target reference object detection result is obtained.
  • the target reference object detection result includes a plurality of edge line segments; and determining the edge line of the motion track according to the target reference object detection result includes:
  • each edge line segment in the target scene image is fitted to obtain the edge line of the motion track.
  • determining the positioning information of the target vanishing point according to the edge line includes:
  • determining the motion adjustment parameters of the target robot based on the positioning information includes:
  • Motion adjustment parameters of the target robot are determined based on the orbit direction.
  • determining the track direction of the motion track based on the positioning information includes:
  • the orbit direction of the motion orbit is calculated.
  • determining the motion adjustment parameters of the target robot based on the orbit direction includes:
  • the step of detecting the target reference object based on the moving scene image and obtaining the target reference object detection result further includes:
  • a robot motion control device including:
  • the acquisition module is configured to acquire the motion scene image collected by the visual sensor, detect the target reference object based on the motion scene image, and obtain the target reference object detection result, wherein the target reference object is the motion track of the target robot.
  • a determination module configured to determine the edge line of the motion track based on the target reference object detection result
  • a positioning module configured to determine the positioning information of the target vanishing point according to the edge line
  • the adjustment module is configured to determine the motion adjustment parameters of the target robot based on the positioning information, and adjust the motion state of the target robot according to the motion adjustment parameters.
  • a computing device including:
  • the memory is used to store computer-executable instructions
  • the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, any one of the steps of the robot motion control method is implemented.
  • a computer-readable storage medium which stores computer-executable instructions. When the instructions are executed by a processor, any one of the steps of the robot motion control method is implemented.
  • a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned robot motion control method.
  • One embodiment of this specification realizes the detection of the target reference object in the application scene image, and determines the edge line of the target robot's motion track based on the detection result, and then calculates the target vanishing point based on the edge line, and finally calculates the target vanishing point.
  • the obtained motion adjustment parameters adjust the motion state of the target robot, and the motion state of the target robot can be adjusted in real time, and the implementation process only relies on the existing geometric line structure features in the motion track of the target robot, without the need to add additional
  • the reference object ensures the flexibility, accuracy and cost controllability of the target robot's motion state adjustment.
  • Figure 1 is a schematic structural diagram of a target robot in a robot motion control method provided by an embodiment of this specification
  • Figure 2 is a flow chart of a robot motion control method provided by an embodiment of this specification
  • Figure 3 is a schematic diagram of a robot motion control method implemented in a warehousing system according to an embodiment of this specification
  • Figure 4 is a process flow chart of a robot motion control method provided by an embodiment of this specification.
  • Figure 5 is a schematic structural diagram of a robot motion control device provided by an embodiment of this specification.
  • Figure 6 is a structural block diagram of a computing device provided by an embodiment of this specification.
  • first, second, etc. may be used to describe various information in one or more embodiments of this specification, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other.
  • the first may also be called the second, and similarly, the second may also be called the first.
  • the word "if” as used herein may be interpreted as "when” or “when” or “in response to determining.”
  • Eliminating point Parallel lines in a three-dimensional scene do not intersect, or the parallel lines intersect at an infinity point. When projected onto a two-dimensional screen, the intersecting infinity point is visible. At this time, the infinity point is It is the vanishing point.
  • Internal parameter matrix Different depth cameras have different characteristic parameters. In computer vision, this set of parameters is the internal parameter matrix of the camera.
  • a robot motion control method is provided.
  • This specification also relates to a robot motion control device, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.
  • the movement of the robot is controlled through visual navigation, and the movement of the robot is guided by laying identification codes on the movement track of the robot; specifically, the robot collects environmental images through visual sensors, and then scans the environmental images. Identify the identification code, determine whether the direction of movement conforms to the preset operating track regulations based on the laying position of the identification code, and adjust the direction of movement based on the judgment result.
  • the identification code in order for the robot to identify the identification code contained in the collected environment image, the identification code will be designed and manufactured in a unique style, so that the identification code has visual characteristics that are different from irrelevant things in the robot's operating environment, and can be recognized by the robot from the environment image.
  • the identification code is arranged in the running track of the robot, which requires strict construction standards so that the identification code is posted at the preset layout position to ensure that the running direction of the robot will not be affected due to insufficient layout accuracy; finally, the identification code After the code layout is completed, regular maintenance is required to ensure that the identification code will not be blurred due to aging or wear, causing the robot to be unable to recognize the running direction and not meet the expected direction; from the above, it can be seen that the identification code is in production , layout and maintenance processes all require costs, and since the production, layout and maintenance processes all rely on the work capabilities of the corresponding execution parties, errors in any link will seriously affect the accuracy of the robot's operation, and even cause property losses. .
  • this embodiment provides a robot motion control method, which avoids the process of adding identification codes and saves the production of identification codes by collecting moving scene images and analyzing the geometric line structure of the target reference object in the moving scene images. , layout and maintenance costs, and because the execution subject does not involve manual labor, it can effectively reduce human errors that may occur when implementing the robot motion control method, and help improve the accuracy of robot operation.
  • Figure 1 shows a schematic structural diagram of a target robot in a robot motion control method according to an embodiment of this specification.
  • the target robot integrates a visual image acquisition component, an information processing component, a control component, a driving component and an energy component.
  • the energy component is a visual image acquisition component, an information processing component, a control component,
  • the driving component provides energy, which can be in the form of electrical energy, chemical energy, etc. The specific energy form is determined by the actual usage scenario and is not limited in this embodiment.
  • the visual image acquisition component collects images of the external operating environment, and then transmits the acquisition results to the information processing component for processing. Based on the images of the external operating environment, it calculates the movement direction of the target robot and the motion trajectory of the target robot. Orbital direction, and motion adjustment parameters indicating that the target robot needs to adjust its own motion direction; then the information processing component instructs the control component to adjust the target robot's motion direction through the motion adjustment parameters.
  • the control component can be understood as the motion control of the target robot.
  • the device includes a control chip, a control switch, etc.
  • the motion control device constituting the control component is determined by the actual usage scenario and is not limited in this embodiment; then the control component controls the traveling component to realize the movement of the target robot, and the traveling component It can be composed of one or more combinations of ratchet devices, gear mechanisms, power transmission shafts, directional transmission shafts, tires, crawler tracks, etc.
  • the specific combination form is determined by the actual use scenario and is not limited in this embodiment.
  • figure (b) in Figure 1 also shows a schematic structural diagram of another target robot.
  • the visual image acquisition component and information processing component are not integrated in the target robot.
  • the visual image acquisition module and the information processing component interact with each other through wired communication or wireless communication.
  • Figure (b) shows that neither the visual image acquisition component nor the information processing component is integrated in the target.
  • the information processing component is not integrated in the target robot; or the visual image acquisition device is not integrated in the target robot, but the information processing component is not integrated in the target robot.
  • the method of information interaction is similar to the information interaction device of each module in Figure (b), and will not be described again here.
  • Figure 2 shows a flow chart of a robot motion control method according to an embodiment of this specification, which specifically includes the following steps.
  • Step S202 Obtain the motion scene image collected by the visual sensor, detect the target reference object based on the motion scene image, and obtain the target reference object detection result, wherein the target reference object is the reference on both sides of the target robot's motion track. things.
  • the robot motion control method provided by this embodiment can be implemented in a variety of scenarios.
  • it can be applied in industrial places, so that the moving machinery therein can move according to preset settings. Movement trajectories; it can be used in warehousing sites to allow the handling robots to carry goods; it can be used in the field of autonomous vehicle driving, combined with the obstacle avoidance function, to enable vehicles to drive along prescribed roads; for the convenience of understanding, This embodiment will only describe the robot motion control method implemented in the warehouse.
  • visual sensors can be understood as laser scanners, digital cameras and other devices in actual usage scenarios.
  • the specific devices used are The setting is determined by the actual usage scenario and usage requirements, and is not limited in this embodiment;
  • the motion scene image can be understood as an image of the external environment where the target robot is located;
  • the target reference object can be understood as the beams of the shelves on both sides of the corridor in the storage system , road shoulders on both sides of the highway, etc.
  • Specific target reference objects can be set by the user based on actual usage needs, and are not limited in this embodiment.
  • the visual sensor collects the moving scene image of the external environment during the movement of the target robot, and then scans and detects the moving scene image to determine the position of the preset target reference object, and the target reference object is the target
  • the target reference object is the target
  • the target robot is running in a warehouse.
  • the motion scene image obtained at this time includes rows of shelves.
  • the target robot’s movement track is the target robot’s current location. corridor, then the target reference object can be understood as the shelf adjacent to the corridor.
  • the filtering process includes filtering methods such as bilateral filtering and median filtering.
  • the specific filtering method used is determined by the actual usage scenario and is not limited in this embodiment.
  • the collected running scene images are filtered, and the noise in the moving scene images is filtered to avoid the impact of noise on the subsequent moving scene image processing.
  • a moving robot moves goods along a designated corridor
  • the target reference object is preset as a warehouse shelf.
  • the visual sensor integrated on the cargo handling robot collects the image of the corridor where the cargo handling robot is at this time, and then filters the collected images through bilateral filtering to obtain the moving scene image, and then detects the moving scene.
  • the shelves on both sides of the corridor where the cargo handling robot is located are obtained, and the detection results of the shelves on both sides of the corridor are obtained.
  • the noise in the collected operating scene images is achieved, ensuring the accuracy of detecting the target reference object in the operating scene images, and further ensuring the accuracy of the subsequent adjustment of the target robot's motion state.
  • the process of adjusting the movement direction of the target robot based on the moving scene image relies on the geometric line structure features in the target robot's movement track, and the many visual features contained in the moving scene image are not useful in adjusting the movement direction of the target robot. Help, and will affect the subsequent calculation process as an interference item.
  • the specific implementation is as follows:
  • the preset angle can be understood as a preset angle range. If the line segment in the sports scene image is not within this angle range, it can be regarded as an irrelevant line segment and can be deleted or ignored; the edge line segment can be understood as, in Among the line segments corresponding to the edge of the target reference object in the moving scene image, the line segments meet the preset angle requirements.
  • the moving scene image is scanned, and the line segments corresponding to all edges of the target reference object in the moving scene image are scanned, and then it is judged whether the angles of these line segments are within the preset angle range, and line segments that are not within this range are ignored.
  • the line segment required by the angle range is used to obtain the target reference object detection result.
  • the moving scene image is filtered, it is scanned to determine the line segments corresponding to the edges of the shelves on both sides of the corridor, and then based on the preset angle range [0°, 90°) ⁇ (90°, 180° ), in this way, the line segments corresponding to the vertical beams in the shelf are deleted, and the edge line segments containing only the horizontal beams are obtained.
  • the detection results of the shelf are obtained based on the obtained edge line segments.
  • Figure 3 a schematic diagram of a robot motion control method implemented in a warehousing system, the moving robot is located at point O. After collecting the image, it removes the corresponding line segments of the vertical support beams of shelves A and B, leaving only the horizontal beams. line segment.
  • redundant visual features in sports scene images are removed through the above methods.
  • the overall visual image of the shelf is stripped away, and only the edge line segments within the preset angle range are retained, which greatly reduces the amount of data and interference. item, which will help improve the speed of subsequent processing and reduce the consumption of processing resources.
  • the process of determining the edge line segments within the preset angle range can be implemented as follows:
  • the shelves within the preset angle range in the motion scene image are scanned to obtain the crossbeam boundary line segments of the shelves; and the target reference object detection results are obtained based on the crossbeam boundary line segments.
  • the shelves corresponding to the target reference object are scanned.
  • the motion scene image contains the corner part of the shelf in the warehouse, and at this time, it is necessary to determine the movement of the target robot before moving to the corner. direction, so the preset angle range is used to screen the shelves, and only the beam boundary lines of the shelves before the corners are detected to obtain the target reference object calibration results.
  • the visual sensor integrated on the handling robot collects images of the corridor including the corners. At this time, since the handling robot has not yet moved to the corner of the corridor, it only It is enough to detect the shelves in front of the corridor corner. According to the preset angle, remove the shelves behind the corridor corner, and then scan and detect the shelves in front of the corridor corner. The process has been described in the above steps of this embodiment, so No further details will be given here.
  • the target reference objects in the moving scene images are screened, so that the movement state of the target robot can be adjusted more accurately.
  • Step S204 Determine the edge line of the motion track according to the target reference object detection result.
  • the edge of the motion track can be determined based on the detection results, which facilitates subsequent determination of the adjustment of the motion state of the target robot.
  • the edge line of the motion track can be understood as a line segment indicating the edge of the motion track of the target robot.
  • the straight line on the edge of the target robot's movement track is determined. Subsequently, the direction of the movement track can be determined through this straight line, and the movement direction of the target robot can be further adjusted.
  • edge line segment there may be more than one edge line segment in the target reference object. In this case, if subsequent processing is performed directly based on these edge line segments, there will be multiple processing results, causing confusion. In order to avoid getting The processing results are confused.
  • the specific implementation is as follows:
  • the preset length threshold can be understood as the minimum length that the length of the specified edge line segment must reach. Anything that does not meet this length can be considered as interference items and be deleted or ignored during the processing. It should be noted that when the three-dimensional image is projected to In the scene of two-dimensional images, there is a rule of "large near, small far". Therefore, the length of objects far away from the visual sensor will become shorter in the moving scene image. Therefore, the length threshold of the gradient is used, that is, according to different areas of the moving scene image. Set different length thresholds. For example, if the visual sensor is close to the ground, set the length threshold of the bottom area of the sports scene image to 1 cm, and set the length threshold of the top area of the sports scene image to 1 mm. Specifically, the length threshold The setting is determined by the actual usage scenario and is not limited in this embodiment.
  • the length of each edge line segment is determined and compared with the corresponding preset length threshold, and the edge line segments whose length is lower than the preset length threshold are deleted.
  • the target scene image is obtained; then the target scene image is set
  • the first reference direction and the second reference direction mark the end points of each edge line segment in the target scene image along the first reference direction.
  • the one at the front end is the starting point, and the other end is the end point.
  • the fitting process can use straight line fitting technology to convert the target scene image into All edge line segments in are processed as above to obtain the edge lines of the motion track.
  • the basis for the approximation to be closer to the endpoint can be based on the preset direction threshold and distance threshold. Within the range specified by these two thresholds, it can be determined that the direction is approximate and the distance is close. Finally, the motion trajectory is obtained based on the results of straight line fitting. edge line.
  • the obtained edge line is shown in Figure 3, a schematic diagram of a robot motion control method implemented in a warehousing system, as shown in the crossbeam, , and the line segments corresponding to the crossbeam and the parallel warehouse crossbeam on warehouses A and B.
  • the interference line segments in the collected images can be further removed, and some edge line segments can be merged into one line segment, which reduces the number of line segments that need to be processed and further achieves the effect of lightening the computing pressure.
  • Step S206 Determine the positioning information of the target vanishing point according to the edge line.
  • the edge of the motion track is also determined, and then the target fade point and the positioning information of the target fade point can be calculated based on this edge.
  • the target erasure point can be understood as the intersection point in the two-dimensional image of the straight line where the edge line of the motion track is located; the positioning information can be understood as the information indicating the location of the target erasure point.
  • the target vanishing point is calculated, as well as the positioning information containing the position of the target vanishing point.
  • the shelf beams corresponding to the same layer on the shelves on both sides of the movement track should be integrated to calculate the vanishing point.
  • the first layer The shelf beams are also parallel to the shelf beams on the second floor, and a vanishing point can also be obtained. This vanishing point does not correspond to the direction of the motion track.
  • the motion state adjustment of the target robot is regulated through this vanishing point, and it is impossible to realize the use of The target robot proceeds along the expected route.
  • the specific implementation is as follows:
  • the initial erasure point can be understood as the erasure point formed by the intersection of the straight lines of any edge line
  • the extension line of the edge line can be understood as the ray that starts from the endpoint of the edge line and extends along the direction of the edge line.
  • intersection point between the extension lines of each edge line.
  • the obtained intersection point is the initial erasure point.
  • determine the number of extended lines passing through each initial erasure point and select the one with the largest number of extension lines passing by the edge line.
  • the initial fade point is used as the target fade point, and then the position of the target fade point is detected to obtain the positioning information of the target fade point.
  • the target erasing point you can also determine the intersection point of the straight line where each edge line is located. The obtained intersection point is the initial erasing point. Then determine the number of times each initial erasing point is passed by the straight line of each edge line, and select the number of passes. The most initial vanishing point is used as the target vanishing point.
  • the specific calculation process is: select any two from all the shelf edge lines to calculate the cross product, obtain their intersection point in the two-dimensional image, and then calculate the process.
  • the number of shelf edge lines at the intersection point is cycled N times.
  • the intersection point with the largest number of votes is the target vanishing point, and the position information of the target vanishing point is calculated to determine the positioning information.
  • the target vanishing point obtained is shown in the schematic diagram of a robot motion control method implemented in a warehousing system in Figure 3, which is the intersection point of the straight line where the beam is located.
  • the target vanishing points corresponding to the edges on both sides of the moving track can be determined, and subsequently the track direction of the moving track can be determined through this target disappearing point.
  • Step S208 Determine motion adjustment parameters of the target robot based on the positioning information, and adjust the motion state of the target robot according to the motion adjustment parameters.
  • the track direction of the movement track can be further determined, and the movement direction of the target robot is adjusted based on this track direction, so that the target robot moves along the movement track.
  • the motion adjustment parameters can be understood as parameters used to adjust the movement direction of the target robot;
  • the motion state can be understood as the movement method adopted by the target robot, such as first rotating a certain angle and then performing linear motion; or making The orientation of the target robot does not change, but moves diagonally at a certain angle with the orientation direction through the universal wheel at the bottom; or it moves in a certain arc. degree of curved motion.
  • the motion state indicates that the movement mode of the target robot is determined by the actual usage scenario, and is not limited in this embodiment.
  • the target robot needs to move in the motion track.
  • ignoring the direction of the motion track will cause the target robot to escape from the constraints of the motion track.
  • the specific implementation method is as follows:
  • the track direction of the motion track is determined; and the motion adjustment parameters of the target robot are determined based on the track direction.
  • the orbit direction can be understood as a vector indicating the motion orbit calculated through positioning information.
  • this vector is more biased towards the unit vector, that is, its role is reflected in the indicated direction rather than in size.
  • the direction of the movement track is calculated based on the positioning information. After obtaining the direction of the movement track, the movement adjustment parameters for adjusting the movement direction during the subsequent movement of the target robot are further determined. .
  • the track direction of the motion track determined based on the collected images will also be different.
  • the specific implementation is as follows:
  • the target internal parameter matrix can be understood as the characteristic parameters of the visual sensor configuration.
  • the characteristic parameters of the visual sensor configuration that is, the target internal parameter matrix
  • the track direction of the motion track is calculated based on the positioning information of the target vanishing point.
  • the acquisition angle when the visual sensor acquires the moving scene image can be positioned, and then the orbit direction is determined based on this acquisition angle to standardize the determination of the orbit direction.
  • the specific implementation method is as follows:
  • the movement direction of the target robot is determined based on the movement scene image; and the movement adjustment parameters of the target robot are calculated based on the movement direction and the track direction.
  • the visual sensor is integrated on the target robot
  • the second is that the visual sensor is not integrated on the target robot
  • the vision sensor is fixed on the target robot and cannot move, the angle between the shooting angle of the data sensor and the movement direction of the target robot is fixed.
  • the initial direction its corresponding vector is used as the origin vector in the three-dimensional space
  • the visual sensor can scan the target robot to determine the fixed parts on the target robot as a reference point, and then further determine the target robot's position. Pointing direction, as the movement direction of the target robot.
  • This parameter is the movement adjustment parameter.
  • the movement direction of the cargo-handling robot is determined, as shown in the schematic diagram of a robot motion control method implemented in a warehousing system in Figure 3. Then use the following formula 2 to calculate the rotation angle R of the cargo handling robot relative to the track direction of the running track.
  • the cargo handling robot is rotated by R angle and then continues to move.
  • the movement direction of the target robot is modified, so that it can move in the direction of the running track, and the movement trajectory of the target robot is constrained.
  • One embodiment of this specification realizes the detection of the target reference object in the application scene image, and determines the edge line of the target robot's motion track based on the detection result, and then calculates the target vanishing point based on the edge line, and finally calculates the target vanishing point.
  • the obtained motion adjustment parameters adjust the motion state of the target robot, and the motion state of the target robot can be adjusted in real time, and the implementation process only relies on the existing geometric line structure features in the motion track of the target robot, without the need to add additional
  • the reference object ensures the flexibility, accuracy and cost controllability of the target robot's motion state adjustment.
  • FIG. 4 shows a process flow chart of a robot motion control method provided by an embodiment of this specification, which specifically includes the following steps.
  • Step S402 Obtain the motion scene image collected by the visual sensor.
  • road images are collected, and the road images are median filtered to obtain moving scene images.
  • Step S404 Scan the motion scene image to obtain edge segments within a preset angle range.
  • the moving scene image is scanned, the edge line segments of the target reference object road shoulder are collected, and vertical angle line segments are filtered out of the collected road shoulder edges.
  • Step S406 Obtain the target reference object detection result according to the edge line segment.
  • Step S408 Determine the length of each edge line segment, and delete edge line segments whose length is lower than the preset length threshold to obtain the target scene image.
  • the line segments with an edge line segment less than 1 meter are deleted to obtain the target scene image.
  • Step S410 Determine the starting point position, end point position and line segment direction of each edge line segment in the target scene image.
  • the starting position, end position and respective shoulder direction of each road shoulder line segment in the target scene image are determined.
  • Step S412 Based on the starting position, end position and line segment direction of each edge line segment in the target scene image, fit each edge line segment in the target scene image to obtain the edge line of the motion track.
  • Step S414 Determine the intersection point of the extension lines of each edge line as the initial erasure point.
  • Step S416 For each initial erasure point, count the number of extension lines that intersect the initial erasure point.
  • Step S418 Determine a target erasure point from each of the initial erasure points based on the number of extension lines corresponding to each of the initial erasure points.
  • Step S420 Determine the positioning information of the target vanishing point.
  • the position information of the target elimination point is determined.
  • Step S422 Obtain the target internal parameter matrix of the visual sensor.
  • Step S424 Calculate the track direction of the motion track based on the positioning information and the target internal parameter matrix.
  • the direction of the road is calculated based on the obtained internal parameter matrix and the position information of the target vanishing point.
  • Step S426 Determine the movement direction of the target robot based on the movement scene image.
  • the direction of the origin in the three-dimensional space is regarded as the movement direction of the self-driving car.
  • Step S428 Calculate the motion adjustment parameters of the target robot according to the motion direction and the orbit direction.
  • the angle at which the car needs to be memorized to turn is determined.
  • Step S430 Adjust the motion state of the target robot according to the motion adjustment parameter.
  • the running direction of the car is adjusted based on the obtained steering angle.
  • One embodiment of this specification realizes the detection of the target reference object in the application scene image, and determines the edge line of the target robot's motion track based on the detection result, and then calculates the target vanishing point based on the edge line, and finally calculates the target vanishing point.
  • the obtained motion adjustment parameters adjust the motion state of the target robot, and the motion state of the target robot can be adjusted in real time, and the implementation process only relies on the existing geometric line structure features in the motion track of the target robot, without the need to add additional
  • the reference object ensures the flexibility, accuracy and cost controllability of the target robot's motion state adjustment.
  • FIG. 5 shows a schematic structural diagram of a robot motion control device provided by an embodiment of this specification. As shown in Figure 5, the device includes:
  • the acquisition module 502 is configured to acquire the motion scene image collected by the visual sensor, detect the target reference object based on the motion scene image, and obtain the target reference object detection result, wherein the target reference object is the motion track of the target robot. reference objects on both sides;
  • the determination module 504 is configured to determine the edge line of the motion track according to the target reference object detection result
  • the positioning module 506 is configured to determine the positioning information of the target vanishing point according to the edge line;
  • the adjustment module 508 is configured to determine motion adjustment parameters of the target robot based on the positioning information, and adjust the motion state of the target robot according to the motion adjustment parameters.
  • the acquisition module 502 is further configured to:
  • the acquisition module 502 is further configured to:
  • the shelves within the preset angle range in the motion scene image are scanned to obtain the crossbeam boundary line segments of the shelves; and the target reference object detection results are obtained based on the crossbeam boundary line segments.
  • the determination module 504 is further configured to:
  • the positioning module 506 is further configured to:
  • the adjustment module 508 is further configured to:
  • the track direction of the motion track is determined; and the motion adjustment parameters of the target robot are determined based on the track direction.
  • the adjustment module 508 is further configured to:
  • the adjustment module 508 is further configured to:
  • the movement direction of the target robot is determined based on the movement scene image; and the movement adjustment parameters of the target robot are calculated based on the movement direction and the track direction.
  • the robot motion control device further includes:
  • a filtering module is configured to perform filtering processing on the moving scene image to obtain the moving scene image with noise removed.
  • the robot motion control device provided in one embodiment of this specification can execute the robot motion control method provided in one embodiment of this specification, and further adjust the motion state of the target robot in real time, and the implementation process only relies on the motion trajectory of the target robot.
  • the existing geometric line structure features in the robot eliminate the need to add additional reference objects, ensuring the flexibility, accuracy and cost controllability of the target robot's motion state adjustment.
  • Figure 6 shows a structural block diagram of a computing device 600 provided according to an embodiment of this specification.
  • Components of the computing device 600 include, but are not limited to, memory 610 and processor 620 .
  • the processor 620 and the memory 610 are connected through a bus 630, and the database 650 is used to save data.
  • Computing device 600 also includes an access device 640 that enables computing device 600 to communicate via one or more networks 660 .
  • networks include the Public Switched Telephone Network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communications networks such as the Internet.
  • Access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)), wired or wireless, such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, Worldwide Interconnection for Microwave Access ( Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, etc.
  • NIC network interface card
  • the above-mentioned components of the computing device 600 and other components not shown in FIG. 6 may also be connected to each other, such as through a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 6 is for illustrative purposes only and does not limit the scope of this description. Those skilled in the art can add or replace other components as needed.
  • Computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet computer, personal digital assistant, laptop computer, notebook computer, netbook, etc.), a mobile telephone (e.g., smartphone ), a wearable computing device (e.g., smart watch, smart glasses, etc.) or other type of mobile device, or a stationary computing device such as a desktop computer or PC.
  • a mobile computer or mobile computing device e.g., tablet computer, personal digital assistant, laptop computer, notebook computer, netbook, etc.
  • a mobile telephone e.g., smartphone
  • a wearable computing device e.g., smart watch, smart glasses, etc.
  • stationary computing device such as a desktop computer or PC.
  • Computing device 600 may also be a mobile or stationary server.
  • the processor 620 is configured to execute the following computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned robot motion control method are implemented.
  • the above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned robot motion control method belong to the same concept. For details that are not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned robot motion control method. .
  • An embodiment of the present specification also provides a computer-readable storage medium that stores computer-executable instructions.
  • the computer-executable instructions are executed by a processor, the steps of the above-mentioned robot motion control method are implemented.
  • An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned robot motion control method.
  • the computer instructions include computer program code, which may be in the form of source code, object code, executable file or some intermediate form.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media, etc. need It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium Excludes electrical carrier signals and telecommunications signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)

Abstract

本说明书实施例提供机器人运动控制方法以及装置，其中所述机器人运动控制方法包括：获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果；根据所述目标参考物检测结果，确定所述运动轨道的边缘线；根据所述边缘线，确定目标消影点的定位信息；基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整，可以实时的进行目标机器人的运动状态调整，并且实施过程仅依赖目标机器人的运动轨道中已有的几何线结构特征，无需增设额外参照物，保证目标机器人运动状态调整的灵活性、精准性与成本可控性。

Description

机器人运动控制方法以及装置
本申请要求于2022年06月30日提交中国专利局、申请号为202210762032.X、发明名称为“机器人运动控制方法以及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本说明书实施例涉及机器人视觉导航技术领域，特别涉及一种机器人运动控制方法。
背景技术
随着当前工业自动化的迅速发展，机器人技术取得了飞速的发展，在工业生产中发挥着越来越重要的作用，被广泛用于完成重复性高、危险性大和精度要求高的工作；人类希望机器人可以更好地服务于人类甚至代替人类完成多种多样的工作，这需要机器人不仅有完成任务的能力，还要有根据要求前往任务地点的能力，也就是机器人的导航技术。
机器人常用的导航方式分为雷达导航、惯性导航、卫星导航、视觉导航；在这些导航方式中，基于成本因素，雷达导航的适用范围并不广泛，基于精度因素，惯性导航与卫星导航难以面对具有较高精度要求的场景；由于视觉导航具备实施成本低，导航灵活度高，且拥有足够的导航精度等特点，成为各种场景下被最广泛应用的机器人导航方式。
现有技术中，视觉导航实施过程中，机器人通过识别地面铺设的标识码，确定其运行路径，进而对机器人的运行方向进行约束；然而，地面铺设的标识码在制造过程中会有制造成本，且标识码的铺设具有严格的铺设标准，满足铺设标准需要投入足够的人力、物力资源，最后在标识码铺设完成之后，还需要对其进行定时的维护，避免标识码由于磨损、老化等情况导致无法被机器人识别的情况。因此，亟待一种方法解决机器人视觉导航过程中，遇到的以上问题。
发明内容
有鉴于此，本说明书实施例提供了一种机器人运动控制方法。本说明书一个或者多个实施例同时涉及一种机器人运动控制装置，一种计算设备，一种计算机可读存储介质以及一种计算机程序，以解决现有技术中存在的技术缺陷。
根据本说明书实施例的第一方面，提供了一种机器人运动控制方法，包括：
获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，其中，所述目标参考物为目标机器人的运动轨道的两侧参考物；
根据所述目标参考物检测结果，确定所述运动轨道的边缘线；
根据所述边缘线，确定目标消影点的定位信息；
基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整。
可选地，所述基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果包括：
扫描所述运动场景图像，获得预设角度范围内的边缘线段，其中，所述边缘线段表征目标参考物的边缘；
根据所述边缘线段，得到目标参考物检测结果。
可选地，所述目标参考物为货架；所述扫描所述运动场景图像，获得预设角度范围内的边缘线段包括：
对所述运动场景图像中预设角度范围内的货架进行扫描，获得所述货架的横梁边界线段；
根据所述横梁边界线段，得到目标参考物检测结果。
可选地，所述目标参考物检测结果包括多条边缘线段；所述根据所述目标参考物检测结果，确定所述运动轨道的边缘线包括：
确定各条边缘线段的长度，并删除长度低于预设长度阈值的边缘线段，得到目标场景图像；
确定所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向；
基于所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向，对所述目标场景图像中各条边缘线段进行拟合，得到所述运动轨道的边缘线。
可选地，所述根据所述边缘线，确定目标消影点的定位信息包括：
确定各边缘线的延长线交点为初始消影点；
针对各初始消影点，统计相交于该初始消影点的延长线数量；
基于所述各初始消影点对应的延长线数量，从所述各初始消影点中确定目标消影点；
确定所述目标消影点的定位信息。
可选地，所述基于所述定位信息确定所述目标机器人的运动调整参数包括：
基于所述定位信息，确定所述运动轨道的轨道方向；
基于所述轨道方向确定所述目标机器人的运动调整参数。
可选地，所述基于所述定位信息，确定所述运动轨道的轨道方向包括：
获取所述视觉传感器的目标内参矩阵；
基于所述定位信息与所述目标内参矩阵，计算所述运动轨道的轨道方向。
可选地，所述基于所述轨道方向确定所述目标机器人的运动调整参数包括：
基于所述运动场景图像确定所述目标机器人的运动方向；
根据所述运动方向与所述轨道方向计算所述目标机器人的运动调整参数。
可选地，所述基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果之前还包括：
对所述运动场景图像进行滤波处理，得到去除噪声的所述运动场景图像。
根据本说明书实施例的第二方面，提供了一种机器人运动控制装置，包括：
获取模块，被配置为获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，其中，所述目标参考物为目标机器人的运动轨道的两侧参考物；
确定模块，被配置为根据所述目标参考物检测结果，确定所述运动轨道的边缘线；
定位模块，被配置为根据所述边缘线，确定目标消影点的定位信息；
调整模块，被配置为基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整。
根据本说明书实施例的第三方面，提供了一种计算设备，包括：
存储器和处理器；
所述存储器用于存储计算机可执行指令，所述处理器用于执行所述计算机可执行指令，该指令被处理器执行时实现任意一项所述机器人运动控制方法的步骤。
根据本说明书实施例的第四方面，提供了一种计算机可读存储介质，其存储有计算机可执行指令，该指令被处理器执行时实现任意一项所述机器人运动控制方法的步骤。
根据本说明书实施例的第五方面，提供了一种计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行上述机器人运动控制方法的步骤。
本说明书一个实施例实现了通过对运用场景图像中的目标参考物进行检测，并根据检测结果确定目标机器人运动轨道的边缘线，之后基于边缘线计算目标消影点，最后通过目标消影点计算得出的运动调整参数对所述目标机器人的运动状态进行调整，可以实时的进行目标机器人的运动状态调整，并且实施过程仅依赖目标机器人的运动轨道中已有的几何线结构特征，无需增设额外参照物，保证目标机器人运动状态调整的灵活性、精准性与成本可控性。
附图说明
图1是本说明书一个实施例提供的一种机器人运动控制方法中目标机器人的结构示意图；
图2是本说明书一个实施例提供的一种机器人运动控制方法的流程图；
图3是本说明书一个实施例提供的一种机器人运动控制方法在仓储系统中实施的示意图；
图4是本说明书一个实施例提供的一种机器人运动控制方法的处理过程流程图；
图5是本说明书一个实施例提供的一种机器人运动控制装置的结构示意图；
图6是本说明书一个实施例提供的一种计算设备的结构框图。
具体实施方式
在下面的描述中阐述了很多具体细节以便于充分理解本说明书。但是本说明书能够以很多不同于在此描述的其它方式来实施，本领域技术人员可以在不违背本说明书内涵的情况下做类似推广，因此本说明书不受下面公开的具体实施的限制。
在本说明书一个或多个实施例中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本说明书一个或多个实施例。在本说明书一个或多个实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本说明书一个或多个实施例中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本说明书一个或多个实施例中可能采用术语第一、第二等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本说明书一个或多个实施例范围的情况下，第一也可以被称为第二，类似地，第二也可以被称为第一。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
首先，对本说明书一个或多个实施例涉及的名词术语进行解释。
消影点：三维场景下的平行线不相交，或者说平行线相交于无穷远点，而将其投影至二维画面上时，相交的无穷远点是可见的，此时该无穷远点即为消影点。
内参矩阵：不同的深度摄像头具有不同的特征参数，在计算机视觉里面，这组参数为相机的内参矩阵。
在本说明书中，提供了一种机器人运动控制方法，本说明书同时涉及一种机器人运动控制装置，一种计算设备，以及一种计算机可读存储介质，在下面的实施例中逐一进行详细说明。
实际应用中，通过视觉导航的方式进行机器人运动的控制，会通过在机器人的运动轨道上铺设标识码的方式，引导机器人的运动；具体的，机器人通过视觉传感器采集环境图像，之后扫描环境图像，辨别其中的标识码，根据标识码的铺设位置，判断自身移动方向是否符合预设的运行轨道规定，根据判断的结果去调整运动的方向。
然而，为了使得机器人辨别出采集的环境图像中包含的标识码，标识码会进行专属样式的设计与制造，令标识码具有区别于机器人运行环境中无关事物的视觉特征，能被机器人从环境图像中区分；此外，标识码布置在机器人的运行轨道中，需要严格的施工标准，使标识码被张贴在预设的布置位置，保证不会因为布置精度不足，影响机器人的运行方向；最后，标识码布置完成之后，还需要定期的进行维护，保证标识码不会因为出现老化或是磨损的情况导致其图案模糊，使机器人由于无法识别导致运行方向不符合预期；通过以上可知，标识码在制作、布置与维护过程中，都需要消耗成本，并且由于制作、布置与维护过程都依赖于对应执行方的工作能力，任意环节出现失误，都会严重影响机器人运行的精准性，甚至由此造成财产损失。
有鉴于此，本实施例提供了一种机器人运动控制方法，通过采集运动场景图像，并分析运动场景图像中目标参考物的几何线结构，避免了增设标识码的过程，节省了标识码的制作、布置与维护成本，且由于执行主体不涉及人工，有效减少机器人运动控制方法实施时可能出现的人为失误，有助于提升机器人运行的精准性。
图1示出了根据本说明书一个实施例提供的一种机器人运动控制方法中目标机器人的结构示意图。
其中，图1中的图(a)，在目标机器人中集成了视觉图像采集组件、信息处理组件、控制组件、行驶组件与能源组件，能源组件为视觉图像采集组件、信息处理组件、控制组件、行驶组件提供能源，能源的形式可以为电能、化学能等，具体能源形式由实际使用场景决定，本实施例不进行限定。
机器人运动控制方法执行过程中，视觉图像采集组件采集外部运行环境的图像，之后将采集结果传输至信息处理组件进行处理，根据外部运行环境的图像计算出目标机器人的运动方向，目标机器人运动轨道的轨道方向，以及指示目标机器人需要对自身运动方向进行调整的运动调整参数；之后信息处理组件通过运动调整参数，指示控制组件对目标机器人的运动方向进行调整，控制组件可以理解为目标机器人的运动控制装置，包括控制芯片、控制开关等，需要说明的是，构成控制组件的运动控制装置由实际使用场景决定，本实施例不进行限定；之后控制组件控制行驶组件，实现目标机器人的运动，行驶组件可以由棘轮装置、齿轮机构、动力传动轴、方向传动轴、轮胎、履带等部分中的一个或几个组合而成，其具体组合形式本由实际使用场景决定，本实施例不进行限定。
此外，在图1中的图(b)还展示了另外一种目标机器人的结构示意图，与图(a)相比，视觉图像采集组件与信息处理组件未被集成在目标机器人中，在这种情况下，视觉图像采集模块与信息处理组件，通过有线通信或无线通信的形式进行信息的交互，需要说明的是，图(b)示出了视觉图像采集组件与信息处理组件都没有集成在目标机器人中，除此之外还包括将视觉图像采集装置集成在目标机器人中，将信息处理组件不集成在目标机器人中；或是将视觉图像采集装置不集成在目标机器人中，而将信息处理组件集成在目标机器人中，其中进行信息交互的方式与图(b)中各模块的信息交互装置类似，在此不进行赘述。
图2示出了根据本说明书一个实施例提供的一种机器人运动控制方法的流程图，具体包括以下步骤。
步骤S202：获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，其中，所述目标参考物为目标机器人的运动轨道的两侧参考物。
具体的，实施例提供的一种机器人运动控制方法在实际应用中，本实施例提供的机器人运动控制方法可在多种场景实施，例如可以应用在工业场所，使其中的运动机械按照预设的运动轨迹进行移动；可以应用于仓储场所，使其中的搬运机器人进行货物的搬运；可以应用于车辆自动驾驶领域，结合障碍物躲避功能，实现令车辆沿着规定的马路进行行驶；为方便理解，本实施例后续只对仓储场所实施的机器人运动控制方法进行描述。
其中，视觉传感器在实际使用场景中，可以理解为激光扫描器、数字摄像机等装置，具体采用的装 置由实际使用场景与使用需求决定，本实施例不进行限定；运动场景图像可以理解为，目标机器人所处的外部环境的图像；目标参考物可以理解为，仓储系统中甬道两侧货架的横梁，公路上公路两侧的路肩等，具体的目标参考物可以由用户基于实际使用需求进行设定，本实施例不进行限定。
基于此，视觉传感器采集到目标机器人运动过程中，所处的外部环境的运动场景图像，之后对运动场景图像进行扫描与检测，确定其中预先设定的目标参考物位置，而且目标参考物是目标机器人运行过程中，运行轨道两侧的参考物，如目标机器人运行在某仓库中，此时获取的运动场景图像中包括一排排的货架，这时目标机器人的运动轨道为目标机器人当前所处的甬道，那么目标参考物就可以理解为，与该甬道相邻的货架。
进一步的，在图像采集与处理过程中，存在包括椒盐噪声、高频噪声、低频噪声等图像噪声，直接对采集到的图像进行处理，这些噪声会极大的影响处理质量，为了解决该问题，在本实施例中，具体实现方式如下：
对所述运动场景图像进行滤波处理，得到去除噪声的所述运动场景图像。
其中，滤波处理包括双边滤波、中值滤波等滤波方法，具体采用的滤波方法由实际使用场景决定，本实施例不进行限定。
基于此，对采集到的运行场景图像进行滤波处理，将运动场景图像中的噪声进行过滤，避免噪声对后续运动场景图像处理过程造成影响。
举例说明，在仓储系统中，搬货机器人搬运货物沿着指定的甬道行驶，预先设定目标参考物为仓库货架。在搬货机器人运动的过程中，集成在搬货机器人上的视觉传感器采集搬货机器人此时所在甬道的图像，之后通过双边滤波对采集到的图像进行滤波，得到运动场景图像，之后检测运动场景图像中搬货机器人所在的甬道两侧的货架，得到甬道两侧的货架的检测结果。
综上，通过以上方法，实现了对采集的运行场景图像中的噪声进行去除，保证了在运行场景图像中检测目标参考物的准确性，进一步的保证后续对目标机器人的运动状态调整的准确性。
进一步的，基于运动场景图像，调整目标机器人的运动方向的过程中，依赖的是目标机器人运动轨道中的几何线结构特征，而运动场景图像中包含的众多视觉特征对于调整目标机器人的运动方向没有帮助，且会作为干扰项影响后续的计算过程，为了解决该问题，在本实施例中，具体实施方式如下：
扫描所述运动场景图像，获得预设角度范围内的边缘线段，其中，所述边缘线段表征目标参考物的边缘；根据所述边缘线段，得到目标参考物检测结果。
其中，预设角度可以理解为，预先设定的一个角度范围，运动场景图像中的线段不在此角度范围内，即可认定其是无关线段，可以进行删除或忽略；边缘线段可以理解为，在运动场景图像里目标参考物的边缘所对应的线段中，满足预设角度要求的线段。
基于此，对运动场景图像进行扫描，扫描运动场景图像中目标参考物的全部边缘对应的线段，之后判断这些线段的角度是否在预设角度范围内，将不在此范围内的线段忽略，根据满足角度范围要求的线段，得到目标参考物检测结果。
沿用上例，在运动场景图像经过滤波之后，对其进行扫描，确定甬道两侧的货架的边缘对应的线段，之后根据预设的角度范围[0°，90°)∪(90°，180°)，通过这种方式将货架中垂直的横梁对应的线段删除，得到只包含水平的横梁的边缘线段，根据得到的边缘线段得到货架的检测结果。如图3的一种机器人运动控制方法在仓储系统中实施的示意图所示，搬货机器人位于O点，其采集图像后将货架A、B的竖直的支撑梁对应线段去除，只保留水平横梁的线段。
此外，还可以不对运动场景图像中的货架边缘对应线段进行角度确定，而是直接使用只对水平线段敏感的水平算子，采集运动场景图像中的货架边缘的线段，水平算子由于对垂直线段不敏感，所以会将垂直线段忽略，同样实现了得到货架的检测结果的目的。
综上，通过以上方式去除了运动场景图像中多余的视觉特征，如举例说明中，就将货架的整体视觉形象剥离，只保留了预设角度范围内的边缘线段，大大降低了数据量与干扰项，有利于提升后续处理的速度，减少处理资源消耗。
进一步的，在目标参考物为货架的情况下，确定预设角度范围内的边缘线段的过程，在本实施例中，具体实现方式还可以如下：
对所述运动场景图像中预设角度范围内的货架进行扫描，获得所述货架的横梁边界线段；根据所述横梁边界线段，得到目标参考物检测结果。
其中，在目标参考物对应的货架中，将满足预设角度范围的货架进行扫描，如在运动场景图像中包含仓库中货架的拐角部分，而此时需要确定目标机器人在运动至拐角前的运动方向，因而采用预设角度范围对货架进行筛选，只检测在拐角前的货架的横梁边界线，得到目标参考物校测结果。
沿用上例，在搬货机器人在仓库中进行搬货时，集成在搬货机器人上的视觉传感器采集包含拐角的甬道图像，此时由于搬货机器人还没有运动到甬道拐角位置，所以此时仅对甬道拐角前的货架进行检测即可，根据预设角度，去除甬道拐角后的货架，后续对甬道拐角前的货架进行扫描与检测的过程，在本实施例的上述步骤中已经进行描述，故在此不进行赘述。
综上，通过以上方式，对运动场景图像中的目标参考物进行筛选，使得对目标机器人的运动状态的调整更加精准。
步骤S204：根据所述目标参考物检测结果，确定所述运动轨道的边缘线。
具体的，对目标参考物进行检测之后，因为目标参考物存在于运动轨道的两侧，所以可以实现根据检测的结果确定运动轨道的边缘，便于后续的确定对目标机器人的运动状态的调整。
其中，运动轨道的边缘线可以理解为，指示着目标机器人的运动轨道边缘的线段。
基于此，根据对目标参考物检测得到的检测结果，确定目标机器人运动轨道边缘的直线，后续可以通过这个直线确定运动轨道的方向，并进一步调整目标机器人的运动方向。
进一步的，目标参考物中存在的边缘线段存在不只一条的情况，在这种情况下，如果直接基于这些边缘线段进行后续处理，得到的处理结果也会出现多种，造成混淆，为了避免的得到的处理结果出现混淆，在本实施例中，具体实现方式如下：
确定各条边缘线段的长度，并删除长度低于预设长度阈值的边缘线段，得到目标场景图像；确定所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向；基于所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向，对所述目标场景图像中各条边缘线段进行拟合，得到所述运动轨道的边缘线。
其中，预设长度阈值可以理解为，指定边缘线段长度所要达到的最小长度，不满足此长度的都可以认为是干扰项，处理过程中进行删除或忽视，需要说明的是，在三维图像投影到二维图像的场景下，存在着“近大远小”的规律，故此远离视觉传感器的物体长度反应在运动场景图像中会变短，所以采用梯度的长度阈值，即根据运动场景图像的不同区域设定不同的长度阈值，如视觉传感器贴近地面，此时将运动场景图像底部区域的长度阈值设定为1厘米，而将运动场景图像顶部区域的长度阈值设定为1毫米，具体对长度阈值的设定由实际使用场景决定，本实施例不进行限定。
基于此，确定各个边缘线段的长度，并与对应的预设长度阈值进行比较，将长度低于预设长度阈值的边缘线段删除，经过此处理之后得到目标场景图像；随后为目标场景图像设定第一参考方向与第二参考方向，沿着第一参考方向对目标场景图像中各条边缘线段的端点进行标记，处在前端的为起点位置，另一端为终点位置，在边缘线段与第一参考方向平行的情况下，沿第二参考方向对平行于第一参考方向的边缘线段的端点进行标记，处在前端的为起点位置，另一端为终点位置。
随后，确定边缘线段的方向，选择方向范围小于预设角度的，并且其中一条边缘线段的起始位置与另一条边缘线段的终止位置之间的距离，小于预设距离的两条边缘线段，将这两条边缘线段距离小于预设距离的起始位置与终止位置的端点进行拟合，得到新的线段，需要说明的是，此处的拟合过程可以采用直线拟合技术，将目标场景图像中的全部边缘线段进行如上处理，得到运动轨道的边缘线。
沿用上例，在仓库中的货架，其上若摆放着货箱，在对货架进行检测过程中，很容易将货箱的边缘也作为检测的结果，这种情况下，将短于预设的长度的边缘线段删除，得到目标场景图像，之后又由于货架摆放过程中可能摆放的并不紧密，或者货架摆放的角度有一些误差，摆放并不整齐，这种情况下， 确定目标场景图像中货架横梁边缘对应线段的起始位置、终止位置与线段方向，之后选择线段方向近似，端点距离也较近的横梁边缘对应线段进行直线拟合，需要说明的，此处判断方向近似与端点距离较近的依据，可以根据预设的方向阈值与距离阈值，在这两个阈值规定的范围内就可以认定方向近似，距离较近，最后根据直线拟合的结果得到运动轨道的边缘线。得到的边缘线如图3的一种机器人运动控制方法在仓储系统中实施的示意图中，横梁、，以及货仓A、B上与横梁、平行的仓库横梁对应的线段所示。
综上，通过以上步骤，可以进一步的去除采集到的图像中的干扰线段，并使部分边缘线段融合成为一条线段，减少了需要进行处理的线段数量，进一步实现了轻计算压力的效果。
步骤S206：根据所述边缘线，确定目标消影点的定位信息。
具体的，确定边缘线之后也就确定的运动轨道的边缘，之后可以根据这个边缘计算出目标消影点以及目标消影点的定位信息。
其中，目标消影点可以理解为，运动轨道边缘线所在直线在二维图像中的交点；定位信息可以理解为，指示目标消影点位置的信息。
基于此，根据得到的目标机器人运动轨道的边缘线，计算目标消影点，以及包含目标消影点位置的定位信息。
进一步的，在实际使用场景中，为了计算运动轨道的方向，如仓储系统中的货架，应该综合运动轨道两侧的货架中，同一层对应的货架横梁去计算消影点，但是由于第一层的货架横梁与第二层的货架横梁也是平行的，也可以得到一个消影点，这个消影点并不能对应运动轨道的方向，通过该消影点规范目标机器人的运动状态调整，无法实现使目标机器人沿着预期路线进行，为了解决这个问题，在本实施例中，具体实施方式如下：
确定各边缘线的延长线交点为初始消影点；针对各初始消影点，统计相交于该初始消影点的延长线数量；基于所述各初始消影点对应的延长线数量，从所述各初始消影点中确定目标消影点；确定所述目标消影点的定位信息。
其中，初始消影点可以理解为，任意边缘线所在直线的交点形成的消影点，边缘线的延长线可以理解为，以边缘线的端点为起点，沿着边缘线的方向延长的射线。
基于此，确定各个边缘线的延长线之间的交点，得到的交点就是初始消影点，之后确定各个初始消影点的被延长线经过的数量，选择被边缘线的延长线经过数量最多的初始消影点，作为目标消影点，之后检测目标消影点的位置，得到目标消影点的定位信息。
此外，目标消影点的确定，还可以确定各个边缘线所在直线的交点，得到的交点就是初始消影点，之后确定每个初始消影点被各个边缘线所在直线经过的数量，选择经过数量最多的初始消影点作为目标消影点。
沿用上例，使用RANSAC算法计算货架边缘线的目标消影点，具体的计算过程为：从所有的货架边缘线中任意选两条求叉乘，得到其在二维图像中的交点然后计算经过该交点的货架边缘线数目，循环N次，得到投票数最多的交点即为目标消影点，并计算目标消影点的位置信息，确定定位信息。其中得到的目标消影点如图3的一种机器人运动控制方法在仓储系统中实施的示意图中所示，横梁、所在直线的交点。
综上，通过以上方法，可以确定出对应运动轨道两侧边缘的目标消影点，后续可以通过这个目标消影点去确定运动轨道的轨道方向。
步骤S208：基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整。
具体的，在确定了目标机器人运动轨道对应的目标消影点之后，可以进一步的确定运动轨道的轨道方向，并基于这个轨道方向调整目标机器人的运动方向，使目标机器人沿着运动轨道运动。
其中，运动调整参数可以理解为，用于对目标机器人的运动方向进行调整的参数；运动状态可以理解为，目标机器人采取的运动方式，如先旋转一定角度，之后再进行直线运动；或是令目标机器人的朝向不发生变化，而是通过其底部的万向轮进行与朝向方向呈一定角度的斜线运动；又或者是进行一定弧 度数的曲线运动，需要说明的是，运动状态指示目标机器人的运动方式由实际使用场景决定，本实施例不进行限定。
基于此，确定与目标机器人的运动轨道对应的目标消影点之后，根据其定位信息，计算出指示目标机器人后续进行运动过程中需要的遵循的运动调整参数，最后根据该运动调整参数调整目标机器人的运动状态。
进一步的，目标机器人需要在运动轨道中进行运动，那么对目标机器人运动状态进行调整的过程中，忽视运动轨道的方向会使得目标机器人脱离运动轨道的约束，为了避免这种情况，在本实施例中，具体实现方式如下：
基于所述定位信息，确定所述运动轨道的轨道方向；基于所述轨道方向确定所述目标机器人的运动调整参数。
其中，轨道方向可以理解为，通过定位信息计算出的指示运动轨道的矢量，而这个矢量在实际使用场景中更偏向于单位矢量，即其作用体现在指示方向上，而非体现在大小上。
基于此，基于目标消影点的定位信息，之后根据定位信息计算出运动轨道的方向，在得到了运动轨道的方向之后，进一步的确定目标机器人后续运动过程中对运动方向进行调整的运动调整参数。
综上，通过以上方式，保证了目标机器人会被运动轨道约束，使目标机器人在运动过程中，不会脱离运动轨道的范围。
进一步的，在计算运动轨道方向的过程中，由于视觉传感器采集图像的角度不同，会导致根据采集的图像确定出的运动轨道的轨道方向也是不同的，为了使轨道方向标准化，在本实施例中，具体实现方式如下：
获取所述视觉传感器的目标内参矩阵；基于所述定位信息与所述目标内参矩阵，计算所述运动轨道的轨道方向。
其中，目标内参矩阵可以理解为，视觉传感器配置的特征参数。
基于此，读取视觉传感器配置的特征参数，即目标内参矩阵，之后结合目标消影点的定位信息计算出运动轨道的轨道方向。
综上，通过视觉传感器配置的特征参数，可以定位视觉传感器获取运动场景图像时的采集角度，之后结合此采集角度确定轨道方向，使轨道方向的确定标准化。
进一步的，确定目标机器人需要进行的运动方向的调整，需要结合目标机器人的原运动方向，在本实施例中，具体实现方式如下：
基于所述运动场景图像确定所述目标机器人的运动方向；根据所述运动方向与所述轨道方向计算所述目标机器人的运动调整参数。
其中，在确定目标机器人的运动方向过程中，存在两种情况，第一种是视觉传感器被集成在目标机器人上，第二种是视觉传感器没有被集成在目标机器人上；在第一种情况中，若视觉传感器被固定在目标机器人上不能进行移动，此时数据传感器的拍摄角度与目标机器人的运动方向之间的角度是固定不动的，这种情况下，优选的将目标机器人的运行方向作为初始方向，即将其对应向量在三维空间中作为原点向量；在第二种情况中，视觉传感器可以通过对目标机器人进行扫描，确定目标机器人上的固定部件作为参考点，之后进一步确定目标机器人的指向方向，作为目标机器人的运动方向。
之后，通过确定的目标机器人的运动方向，根据运动方向与轨道方向，计算目标机器人需要进行的运动方向调整的参数，这个参数即为运动调整参数。
沿用上例，读取视觉传感器的目标内参矩阵K，之后根据得到的定位信息，通过如下公式1，计算视觉传感器坐标系中的归一化的运行轨道的方向向量。
公式1
再之后，根据目标传感器与搬货机器人之间的集成关系，确定搬货机器人的运动方向，如图3的一种机器人运动控制方法在仓储系统中实施的示意图中的所示。再通过如下公式2，计算搬货机器人相对于运行轨道的轨道方向进行的旋转角度R。
公式2
最后令搬货机器人旋转R角度后，继续进行运动。
综上，通过上述步骤实现了对目标机器人的运动方向的修改，使之可以在运行轨道中，沿着运行轨道的方向运动，实现了对目标机器人运动轨迹的约束。
本说明书一个实施例实现了通过对运用场景图像中的目标参考物进行检测，并根据检测结果确定目标机器人运动轨道的边缘线，之后基于边缘线计算目标消影点，最后通过目标消影点计算得出的运动调整参数对所述目标机器人的运动状态进行调整，可以实时的进行目标机器人的运动状态调整，并且实施过程仅依赖目标机器人的运动轨道中已有的几何线结构特征，无需增设额外参照物，保证目标机器人运动状态调整的灵活性、精准性与成本可控性。
下述结合附图4，以本说明书提供的机器人运动控制方法在自动驾驶的应用为例，对所述机器人运动控制方法进行进一步说明。其中，图4示出了本说明书一个实施例提供的一种机器人运动控制方法的处理过程流程图，具体包括以下步骤。
步骤S402：获取视觉传感器采集的运动场景图像。
具体的，根据集成在自动驾驶汽车上的视觉传感器，采集马路图像，并将马路图像进行中值滤波，得到运动场景图像。
步骤S404：扫描所述运动场景图像，获得预设角度范围内的边缘线段。
具体的，扫描运动场景图像，对其中的目标参考物路肩的边缘线段进行采集，并且采集的路肩边缘中，过滤垂直角度的线段。
步骤S406：根据所述边缘线段，得到目标参考物检测结果。
步骤S408：确定各条边缘线段的长度，并删除长度低于预设长度阈值的边缘线段，得到目标场景图像。
具体的，根据采集到包含路肩边缘的图像中，将其中边缘线段小于1米的线段删除，得到目标场景图像。
步骤S410：确定所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向。
具体的，确定目标场景图像中的每一个路肩线段的起点位置、终点位置以及各自的路肩方向。
步骤S412：基于所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向，对所述目标场景图像中各条边缘线段进行拟合，得到所述运动轨道的边缘线。
具体的，根据确认的路肩线段的起点位置、终点位置以及路肩方向，对各个路肩线段进行直线拟合，得到边缘线。
步骤S414：确定各边缘线的延长线交点为初始消影点。
步骤S416：针对各初始消影点，统计相交于该初始消影点的延长线数量。
步骤S418：基于所述各初始消影点对应的延长线数量，从所述各初始消影点中确定目标消影点。
具体的，得到了3个消影点，确定消影点上经过的边缘线数量最多的初始消影点为目标消影点。
步骤S420：确定所述目标消影点的定位信息。
具体的，确定目标消影点的位置信息。
步骤S422：获取所述视觉传感器的目标内参矩阵。
具体的，获取自动驾驶汽车上的视觉传感器的内参矩阵。
步骤S424：基于所述定位信息与所述目标内参矩阵，计算所述运动轨道的轨道方向。
具体的，根据得到的内参矩阵和目标消影点的位置信息，计算马路的方向。
步骤S426：基于所述运动场景图像确定所述目标机器人的运动方向。
具体的，由于视觉传感器在自动驾驶汽车上，将三维空间内的原点方向作为自动驾驶汽车的运动方向。
步骤S428：根据所述运动方向与所述轨道方向计算所述目标机器人的运动调整参数。
具体的，根据汽车的运行方向与马路的方向，确定汽车需要记性转向的角度。
步骤S430：根据所述运动调整参数对所述目标机器人的运动状态进行调整。
具体的，根据得到的转向角度，调整汽车的运行方向。
本说明书一个实施例实现了通过对运用场景图像中的目标参考物进行检测，并根据检测结果确定目标机器人运动轨道的边缘线，之后基于边缘线计算目标消影点，最后通过目标消影点计算得出的运动调整参数对所述目标机器人的运动状态进行调整，可以实时的进行目标机器人的运动状态调整，并且实施过程仅依赖目标机器人的运动轨道中已有的几何线结构特征，无需增设额外参照物，保证目标机器人运动状态调整的灵活性、精准性与成本可控性。
与上述方法实施例相对应，本说明书还提供了机器人运动控制装置实施例，图5示出了本说明书一个实施例提供的一种机器人运动控制装置的结构示意图。如图5所示，该装置包括：
获取模块502，被配置为获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，其中，所述目标参考物为目标机器人的运动轨道的两侧参考物；
确定模块504，被配置为根据所述目标参考物检测结果，确定所述运动轨道的边缘线；
定位模块506，被配置为根据所述边缘线，确定目标消影点的定位信息；
调整模块508，被配置为基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整。
一个可选的实施例中，所述获取模块502还被配置为：
扫描所述运动场景图像，获得预设角度范围内的边缘线段，其中，所述边缘线段表征目标参考物的边缘；根据所述边缘线段，得到目标参考物检测结果。
一个可选的实施例中，所述获取模块502还被配置为：
对所述运动场景图像中预设角度范围内的货架进行扫描，获得所述货架的横梁边界线段；根据所述横梁边界线段，得到目标参考物检测结果。
一个可选的实施例中，所述确定模块504还被配置为：
确定各条边缘线段的长度，并删除长度低于预设长度阈值的边缘线段，得到目标场景图像；确定所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向；基于所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向，对所述目标场景图像中各条边缘线段进行拟合，得到所述运动轨道的边缘线。
一个可选的实施例中，所述定位模块506还被配置为：
确定各边缘线的延长线交点为初始消影点；针对各初始消影点，统计相交于该初始消影点的延长线数量；基于所述各初始消影点对应的延长线数量，从所述各初始消影点中确定目标消影点；确定所述目标消影点的定位信息。
一个可选的实施例中，所述调整模块508还被配置为：
基于所述定位信息，确定所述运动轨道的轨道方向；基于所述轨道方向确定所述目标机器人的运动调整参数。
一个可选的实施例中，所述调整模块508还被配置为：
获取所述视觉传感器的目标内参矩阵；基于所述定位信息与所述目标内参矩阵，计算所述运动轨道的轨道方向。
一个可选的实施例中，所述调整模块508还被配置为：
基于所述运动场景图像确定所述目标机器人的运动方向；根据所述运动方向与所述轨道方向计算所述目标机器人的运动调整参数。
一个可选的实施例中，所述机器人运动控制装置还包括：
滤波模块，被配置为对所述运动场景图像进行滤波处理，得到去除噪声的所述运动场景图像。
本说明书一个实施例提供的机器人运动控制装置，可以执行本说明书一个实施例提供的机器人运动控制方法，进一步实时的进行目标机器人的运动状态调整，并且实施过程仅依赖目标机器人的运动轨道 中已有的几何线结构特征，无需增设额外参照物，保证目标机器人运动状态调整的灵活性、精准性与成本可控性。
上述为本实施例的一种机器人运动控制装置的示意性方案。需要说明的是，该机器人运动控制装置的技术方案与上述的机器人运动控制方法的技术方案属于同一构思，机器人运动控制装置的技术方案未详细描述的细节内容，均可以参见上述机器人运动控制方法的技术方案的描述。
图6示出了根据本说明书一个实施例提供的一种计算设备600的结构框图。该计算设备600的部件包括但不限于存储器610和处理器620。处理器620与存储器610通过总线630相连接，数据库650用于保存数据。
计算设备600还包括接入设备640，接入设备640使得计算设备600能够经由一个或多个网络660通信。这些网络的示例包括公用交换电话网(PSTN)、局域网(LAN)、广域网(WAN)、个域网(PAN)或诸如因特网的通信网络的组合。接入设备440可以包括有线或无线的任何类型的网络接口(例如，网络接口卡(NIC))中的一个或多个，诸如IEEE802.11无线局域网(WLAN)无线接口、全球微波互联接入(Wi-MAX)接口、以太网接口、通用串行总线(USB)接口、蜂窝网络接口、蓝牙接口、近场通信(NFC)接口，等等。
在本说明书的一个实施例中，计算设备600的上述部件以及图6中未示出的其他部件也可以彼此相连接，例如通过总线。应当理解，图6所示的计算设备结构框图仅仅是出于示例的目的，而不是对本说明书范围的限制。本领域技术人员可以根据需要，增添或替换其他部件。
计算设备600可以是任何类型的静止或移动计算设备，包括移动计算机或移动计算设备(例如，平板计算机、个人数字助理、膝上型计算机、笔记本计算机、上网本等)、移动电话(例如，智能手机)、可佩戴的计算设备(例如，智能手表、智能眼镜等)或其他类型的移动设备，或者诸如台式计算机或PC的静止计算设备。计算设备600还可以是移动式或静止式的服务器。
其中，处理器620用于执行如下计算机可执行指令，该计算机可执行指令被处理器执行时实现上述机器人运动控制方法的步骤。
上述为本实施例的一种计算设备的示意性方案。需要说明的是，该计算设备的技术方案与上述的机器人运动控制方法的技术方案属于同一构思，计算设备的技术方案未详细描述的细节内容，均可以参见上述机器人运动控制方法的技术方案的描述。
本说明书一实施例还提供一种计算机可读存储介质，其存储有计算机可执行指令，该计算机可执行指令被处理器执行时实现上述机器人运动控制方法的步骤。
上述为本实施例的一种计算机可读存储介质的示意性方案。需要说明的是，该存储介质的技术方案与上述的机器人运动控制方法的技术方案属于同一构思，存储介质的技术方案未详细描述的细节内容，均可以参见上述机器人运动控制方法的技术方案的描述。
本说明书一实施例还提供一种计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行上述机器人运动控制方法的步骤。
上述为本实施例的一种计算机程序的示意性方案。需要说明的是，该计算机程序的技术方案与上述的机器人运动控制方法的技术方案属于同一构思，计算机程序的技术方案未详细描述的细节内容，均可以参见上述机器人运动控制方法的技术方案的描述。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
所述计算机指令包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需 要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
需要说明的是，对于前述的各方法实施例，为了简便描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本说明书实施例并不受所描述的动作顺序的限制，因为依据本说明书实施例，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定都是本说明书实施例所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其它实施例的相关描述。
以上公开的本说明书优选实施例只是用于帮助阐述本说明书。可选实施例并没有详尽叙述所有的细节，也不限制该发明仅为所述的具体实施方式。显然，根据本说明书实施例的内容，可作很多的修改和变化。本说明书选取并具体描述这些实施例，是为了更好地解释本说明书实施例的原理和实际应用，从而使所属技术领域技术人员能很好地理解和利用本说明书。本说明书仅受权利要求书及其全部范围和等效物的限制。

Claims (13)

  1. 一种机器人运动控制方法，包括：
    获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，其中，所述目标参考物为目标机器人的运动轨道的两侧参考物(S202)；
    根据所述目标参考物检测结果，确定所述运动轨道的边缘线(S204)；
    根据所述边缘线，确定目标消影点的定位信息(S206)；
    基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整(S208)。
  2. 根据权利要求1所述的方法，所述基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，包括：
    扫描所述运动场景图像，获得预设角度范围内的边缘线段，其中，所述边缘线段表征目标参考物的边缘(S404)；
    根据所述边缘线段，得到目标参考物检测结果(S406)。
  3. 根据权利要求2所述的方法，所述目标参考物为货架；
    所述扫描所述运动场景图像，获得预设角度范围内的边缘线段，包括：
    对所述运动场景图像中预设角度范围内的货架进行扫描，获得所述货架的横梁边界线段；
    根据所述横梁边界线段，得到目标参考物检测结果。
  4. 根据权利要求1所述的方法，所述目标参考物检测结果包括多条边缘线段；
    所述根据所述目标参考物检测结果，确定所述运动轨道的边缘线，包括：
    确定各条边缘线段的长度，并删除长度低于预设长度阈值的边缘线段，得到目标场景图像(S408)；
    确定所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向(S410)；
    基于所述目标场景图像中各条边缘线段的起点位置、终点位置以及线段方向，对所述目标场景图像中各条边缘线段进行拟合，得到所述运动轨道的边缘线(S412)。
  5. 根据权利要求1所述的方法，所述根据所述边缘线，确定目标消影点的定位信息，包括：
    确定各边缘线的延长线交点为初始消影点(S414)；
    针对各初始消影点，统计相交于该初始消影点的延长线数量(S416)；
    基于所述各初始消影点对应的延长线数量，从所述各初始消影点中确定目标消影点(S418)；
    确定所述目标消影点的定位信息(S420)。
  6. 根据权利要求1所述的方法，所述基于所述定位信息确定所述目标机器人的运动调整参数，包括：
    基于所述定位信息，确定所述运动轨道的轨道方向；
    基于所述轨道方向确定所述目标机器人的运动调整参数。
  7. 根据权利要求6所述的方法，所述基于所述定位信息，确定所述运动轨道的轨道方向，包括：
    获取所述视觉传感器的目标内参矩阵(S422)；
    基于所述定位信息与所述目标内参矩阵，计算所述运动轨道的轨道方向(S424)。
  8. 根据所述权利要求6或7所述的方法，所述基于所述轨道方向确定所述目标机器人的运动调整参数，包括：
    基于所述运动场景图像确定所述目标机器人的运动方向(S426)；
    根据所述运动方向与所述轨道方向计算所述目标机器人的运动调整参数(S428)。
  9. 根据权利要求1-7中的任一项所述的方法，所述基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果之前，还包括：
    对所述运动场景图像进行滤波处理，得到去除噪声的所述运动场景图像。
  10. 一种机器人运动控制装置，包括：
    获取模块(502)，被配置为获取视觉传感器采集的运动场景图像，基于所述运动场景图像，对目标参考物进行检测，得到目标参考物检测结果，其中，所述目标参考物为目标机器人的运动轨道的两侧参考物；
    确定模块(504)，被配置为根据所述目标参考物检测结果，确定所述运动轨道的边缘线；
    定位模块(506)，被配置为根据所述边缘线，确定目标消影点的定位信息；
    调整模块(508)，被配置为基于所述定位信息确定所述目标机器人的运动调整参数，根据所述运动调整参数对所述目标机器人的运动状态进行调整。
  11. 一种计算设备(600)，包括：
    存储器(610)和处理器(620)；
    所述存储器(610)用于存储计算机可执行指令，所述处理器(620)用于执行所述计算机可执行指令，该计算机可执行指令被处理器(620)执行时实现权利要求1-9中的任意一项所述机器人运动控制方法的步骤。
  12. 一种计算机可读存储介质，其存储有计算机可执行指令，该计算机可执行指令被处理器执行时实现权利要求1-9中的任意一项所述机器人运动控制方法的步骤。
  13. 一种计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行权利要求1-9任意一项所述机器人运动控制方法的步骤。
PCT/CN2023/095050 2022-06-30 2023-05-18 机器人运动控制方法以及装置 Ceased WO2024001596A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP23829782.4A EP4549103A4 (en) 2022-06-30 2023-05-18 METHOD AND DEVICE FOR CONTROLLING ROBOT MOVEMENT
US18/864,725 US20250312922A1 (en) 2022-06-30 2023-05-18 Robot motion control method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210762032.X 2022-06-30
CN202210762032.XA CN117359608A (zh) 2022-06-30 2022-06-30 机器人运动控制法以及装置

Publications (2)

Publication Number Publication Date
WO2024001596A1 true WO2024001596A1 (zh) 2024-01-04
WO2024001596A9 WO2024001596A9 (zh) 2024-08-08

Family

ID=89383180

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/095050 Ceased WO2024001596A1 (zh) 2022-06-30 2023-05-18 机器人运动控制方法以及装置

Country Status (4)

Country Link
US (1) US20250312922A1 (zh)
EP (1) EP4549103A4 (zh)
CN (1) CN117359608A (zh)
WO (1) WO2024001596A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120422246A (zh) * 2025-06-26 2025-08-05 上海复数时空科技有限公司 具身智能机器人的控制方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117876363B (zh) * 2024-03-11 2024-06-07 宝鸡杭叉工程机械有限责任公司 一种带式输送机托辊运行状态在线检测方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01133183A (ja) * 1987-11-18 1989-05-25 Toshiba Corp 移動体位置検出装置
CN104077809A (zh) * 2014-06-24 2014-10-01 上海交通大学 基于结构性线条的视觉slam方法
KR20180040314A (ko) * 2016-10-12 2018-04-20 충북대학교 산학협력단 소실점 위치를 이용한 이동 로봇의 복도 주행 방법 및 장치
CN109341692A (zh) * 2018-10-31 2019-02-15 江苏木盟智能科技有限公司 一种沿线导航方法及机器人

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9135705B2 (en) * 2012-10-16 2015-09-15 Qualcomm Incorporated Sensor calibration and position estimation based on vanishing point determination
US9113077B2 (en) * 2013-01-17 2015-08-18 Qualcomm Incorporated Orientation determination based on vanishing point computation
US11327504B2 (en) * 2018-04-05 2022-05-10 Symbol Technologies, Llc Method, system and apparatus for mobile automation apparatus localization
KR102047303B1 (ko) * 2018-06-01 2019-11-21 엘지전자 주식회사 저조도 영상 내 소실점에 기반하여 방향을 추정하는 로봇 및 방법
WO2020184752A1 (ko) * 2019-03-12 2020-09-17 엘지전자 주식회사 설치물과 평행하게 이동하는 카트 및 이동 방법
KR102340543B1 (ko) * 2019-12-04 2021-12-20 충남대학교산학협력단 부착형 농작업 경로 인식 장치

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01133183A (ja) * 1987-11-18 1989-05-25 Toshiba Corp 移動体位置検出装置
CN104077809A (zh) * 2014-06-24 2014-10-01 上海交通大学 基于结构性线条的视觉slam方法
KR20180040314A (ko) * 2016-10-12 2018-04-20 충북대학교 산학협력단 소실점 위치를 이용한 이동 로봇의 복도 주행 방법 및 장치
CN109341692A (zh) * 2018-10-31 2019-02-15 江苏木盟智能科技有限公司 一种沿线导航方法及机器人

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4549103A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120422246A (zh) * 2025-06-26 2025-08-05 上海复数时空科技有限公司 具身智能机器人的控制方法及系统

Also Published As

Publication number Publication date
US20250312922A1 (en) 2025-10-09
EP4549103A4 (en) 2025-06-18
EP4549103A1 (en) 2025-05-07
WO2024001596A9 (zh) 2024-08-08
CN117359608A (zh) 2024-01-09

Similar Documents

Publication Publication Date Title
US20240203139A1 (en) Systems and methods for depth map sampling
US11460855B1 (en) Systems and methods for sensor calibration
RU2656711C2 (ru) Способ и система обнаружения и сопровождения движущихся объектов на основе данных трехмерного датчика
WO2024001596A1 (zh) 机器人运动控制方法以及装置
Wang et al. Autonomous landing of multi-rotors UAV with monocular gimbaled camera on moving vehicle
CN111947647A (zh) 一种视觉与激光雷达融合的机器人精准定位方法
Omrani et al. Dynamic and static object detection and tracking in an autonomous surface vehicle
Vodichev et al. Real-time distance estimation algorithm for objects in warehouse based on monocular camera data for an autonomous unmanned forklift
WO2024036984A1 (zh) 目标定位方法及相关系统、存储介质
Yang et al. Degeneracy‐Resistant LiDAR‐SLAM Algorithm Based on Geometric and Visual Features' Fusion
Yang et al. Zero-shot metric depth estimation via monocular visual-inertial rescaling for autonomous aerial navigation
CN108564626A (zh) 用于确定安装于采集实体的相机之间的相对姿态角的方法和装置
Munguía et al. Method for SLAM Based on Omnidirectional Vision: A Delayed‐EKF Approach
Mohottala et al. Fusion of a camera and a laser range sensor for vehicle recognition
Sun et al. Detection and state estimation of moving objects on a moving base for indoor navigation
Lu et al. Study on LiDAR inertial tightly coupled SLAM method for outdoor environment
Wang et al. PCR-DAT: a new point cloud registration method for lidar inertial odometry via distance and Gauss distributed
Magrin et al. Enhanced Geodetic Ground-Truth Methodology and Experimental Validation for Mobile Robots: Insights from the VRI4WD UFPR-MAP Dataset
Li et al. A Fast Method for 3D Object Segmentation Using Sparse Point Clouds
Sun et al. High-precision localization of intelligent vehicles in indoor parking lots based on path planning
권우진 Robust Pose Estimation using LiDAR for Autonomous Vehicles on Urban Roads
Bazzaza Detection and localization of individual free street parking spaces using artificial intelligence and motion estimation
CN118209106A (zh) 核电站中机器人导航方法、装置、计算机设备与存储介质
Van Crombrugge et al. Fast Ground Detection for Range Cameras on Road Surfaces Using a Three-Step Segmentation
CN121104993A (zh) 机器人对接方法、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23829782

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18864725

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2023829782

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023829782

Country of ref document: EP

Effective date: 20250130

WWP Wipo information: published in national office

Ref document number: 2023829782

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 18864725

Country of ref document: US