WO2024021402A1 - Material taking and goods unloading method based on visual positioning, and apparatus therefor - Google Patents

Abstract

A material taking and goods unloading method based on visual positioning, and an apparatus therefor, which relate to the technical fields of logistics and production. The method comprises the following steps: S1, acquiring image data of a target object within a field-of-view range by means of a coarse-positioning vision device (6); S2, performing calculation on the image data of the target object and performing division to obtain working areas, and then calculating the coordinates of the geometric center of the target object within a single working area; S3, a robotic arm (3) carrying a fine-positioning vision device (7) and moving to a single working area, and acquiring image data of the target object in the working area; S4, calculating the contour and position of the target object in the working area, and outputting the geometric coordinates of the target object; and S5, the robotic arm (3) moving to the precise position of the target object to take down the target object and perform goods unloading. A robotic arm (3) is guided by means of visual positioning to automatically perform precise material taking, and by means of the combined use of a coarse-positioning vision device (6) and a fine-positioning vision device (7), the functions of automatically identifying a target object within a large field of view and performing fine positioning can be achieved.

Description

A method and device for picking and unloading materials based on visual positioning Technical field

The invention relates to the field of logistics production technology, and in particular to a visual positioning-based material picking and unloading method and its device.

Background technique

As the modern logistics industry develops more and more efficiently, automation equipment has begun to be widely used in the logistics loading and unloading process. In the process of realizing automated operations, loading and unloading equipment needs to be equipped with visual positioning technology to position the goods to guide the operation of the machine arm. Although the existing visual positioning can identify the required objects, it can only provide an approximate coordinate position, making the machine The arm can only perform unloading operations in a certain general area. For unloading operations that require high precision, this visual positioning method has certain limitations.

Contents of the invention

The present invention provides a visual positioning-based retrieval and unloading method and a device thereof, aiming to solve the existing problems of low visual positioning accuracy and unfavorable unloading operation for efficient operations.

The invention provides a visual positioning-based picking and unloading method, which includes the following steps:

S1. Acquire image data of target objects within the field of view through coarse positioning vision equipment;

S2. Calculate the image data of the target object and divide the work area, and then calculate the geometric center coordinates of the target object in a single work area;

S3. The robot arm carries the precise positioning vision device and moves to a single work area to obtain image data of the target object in the work area;

S4. Calculate the outline and position of the target object in the work area, and output the geometric coordinates of the target object;

S5. The robot arm moves to the precise location of the target object, picks up the target object, and unloads it.

As a further improvement of the present invention, step S1 specifically includes:

S11. Form 3D imaging through radar scanning of target objects in coarse positioning vision equipment;

S12. Obtain the 2D image of the target object through the 2D camera in the coarse positioning vision device.

As a further improvement of the present invention, step S2 specifically includes:

Fusion of 3D imaging point cloud data and 2D image data, identifies the 3D position distribution of a single target item, divides the work area according to specified rules, calculates and outputs the three-dimensional coordinates of the geometric center of a single work area, and uses this as the robot arm's location The rough positioning position of the material.

As a further improvement of the present invention, the specified rules for dividing the work area include:

Divide a workspace according to a single item; divide it into a workspace according to a group of multiple items; or divide it into a workspace according to the set area size.

As a further improvement of the present invention, step S3 specifically includes:

Set the photographing distance between the 3D structured light camera in the precision positioning vision equipment and the target object. The picking end of the robot arm moves to the geometric center coordinates of the target object in a single work area and maintains the set photographing distance from the target object. The 3D structure A light camera captures an image of the target.

As a further improvement of the present invention, step S4 specifically includes:

The 3D structured light camera identifies the precise contour and relative distance of the target object, and calculates and outputs the precise three-dimensional coordinates of the material taken.

As a further improvement of the present invention, step S5 specifically includes:

The conveyor belt swings to the position of the target object to be picked up. The picking end of the robot arm grabs the target object and drags it to the conveyor belt. The conveyor belt transports the target object and completes unloading.

As a further improvement of the present invention, before executing step S1, it also includes:

Unify the positions of the radar and the robot arm in the coarse positioning vision equipment. The visual calibration content includes: the coordinate system origins of the radar and the robot arm are unified; the directions of the X/Y/Z/θ axes in the coordinate system are unified; and the radar , the actual mapping relationship between the robot arm coordinates and the space is unified.

As a further improvement of the present invention, before executing the method, it also includes:

Visual training for the built-in programs of coarse positioning vision equipment and fine positioning vision equipment: input a large amount of picture data to learn to recognize the outline of the target object. The picture data includes a single category of target object pictures or multiple categories of target object pictures.

As a further improvement of the present invention, a visual positioning-based picking and unloading method is performed, including

Rough positioning vision equipment is used to roughly position and image targets within the field of view and divide the work area;

Precision positioning vision equipment, used for precise positioning and imaging of targets in the work area;

The robot arm is equipped with a picking end, moves to the vicinity of the target object to be picked based on the results of rough positioning, acquires the target object and performs a dragging action based on the results of fine positioning;

The conveyor belt swings to the location of the target object to be retrieved based on the precise positioning results, receives the target object dragged out by the robot arm, and transports the target object for unloading.

The beneficial effects of the present invention are:

(1) The problem of guiding the robot arm to automatically and accurately pick up materials through visual positioning. Through the comprehensive use of radar + 2D camera + 3D structured light camera, the function of automatically identifying target objects in a large field of view and performing precise positioning can be achieved.

(2) Using side drag + assembly line swing to accept, "lifting goods" can be changed to "drag goods", and only need to overcome the friction of the goods, thus greatly increasing the weight of the goods being transported.

Description of drawings

Figure 1 is a flow chart of the visual positioning-based picking and unloading method of the present invention;

Figure 2 is a schematic diagram of the target in the field of view in visual positioning according to the present invention;

Figure 3 is a structural diagram of the material picking and unloading device of the present invention;

Figure 4 is a structural side view of the material picking and unloading device of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

Example 1:

As shown in Figures 1 to 4, a visual positioning-based picking and unloading method of the present invention is implemented as follows:

Perform visual training on the built-in programs of the coarse positioning vision device 6 and the fine positioning vision device 7: input a large amount of picture data to learn to recognize the outline of the target object. The picture data includes a single category of target images or multiple categories of target images. Through a large amount of picture data, the visual positioning program learns to identify the outline of a certain type or categories of target items, so that the coarse positioning vision device 6 and the fine positioning vision device 7 can more quickly identify the corresponding item data when collecting image data. , and quickly obtain the spatial position coordinates and contour of the object to feed back to the robot arm 3 and the conveyor belt 2 to perform the swing operation.

Unified visual calibration of the positions of the radar and the robot arm 3 in the rough positioning vision equipment 6. The visual calibration content includes: the coordinate system origins of the radar and the robot arm 3 are unified; the directions of the X/Y/Z/θ axes in the coordinate system are unified. ; And the actual mapping relationship between the radar, robot arm 3 coordinates and space is unified. The radar and robot arm 3 will be visually calibrated in advance to achieve unified coordinates to avoid deviations between the position data obtained by the radar and the true position of the robot arm 3, which will affect the accurate picking of materials by the robot arm 3.

After completing the above preliminary operations, as shown in Figure 1 and Figure 2, perform the following steps:

S1. When starting work, the whole machine stops at a suitable position, and image data of the target object within the field of view is acquired through the rough positioning vision device 6.

Step S1 specifically includes:

S11. 3D imaging is formed by scanning the target object with the radar in the coarse positioning vision device 6;

S12. Obtain the 2D image of the target object through the 2D camera in the coarse positioning vision device 6 .

S2. Calculate the image data of the target object and divide the work area, then calculate the geometric center coordinates of the target object in a single work area, and perform rough positioning.

Step S2 specifically includes: fusing the point cloud data of the radar 3D imaging and the 2D image data of the 2D camera, identifying the 3D position distribution of a single target item, dividing the work area according to specified rules, and calculating and outputting the 3D geometric center of the single work area. The coordinates are used as the rough positioning position for the robot arm 3 to pick up materials.

The fusion of point cloud data from 3D radar imaging and 2D image data from 2D cameras is as follows:

Similarity matching is performed between Xa and Ya in the radar 3D low-precision point cloud data coordinates (Xa, Ya, Za) and the high-precision coordinates Xb and Yb in the 2D camera. After matching, a corresponding relationship is established, and data replacement and fusion are performed. Finally Obtain high-precision X, Y and low-precision Z combined data: (Xb, Yb, Za). This method can greatly improve the shortcomings of poor accuracy of 3D radar and only two-dimensional data of 2D cameras, and obtain high-precision three-dimensional data.

The specified rules for dividing the work area include: dividing it into a work area according to a single item; dividing it into a work area according to a group of multiple items; or dividing it into a work area according to the set area size. The rules for dividing work areas are not limited to these three methods, and can also be changed according to actual operation needs to meet diverse material retrieval needs.

S3. The robot arm 3 carries the precision positioning vision device 7 and moves to a single work area to obtain image data of the target object in the work area. The picking end 4 of the robot arm 3 moves to a position at a certain distance from the rough positioning coordinate value. This distance value can be set in advance. This position is the photographing position of the 3D structured light camera.

Step S3 specifically includes: setting the photographing distance between the 3D structured light camera in the precision positioning vision device 7 and the target object, moving the material picking end 4 of the robot arm 3 to the geometric center coordinates of the target object in a single work area, and keeping it in contact with the target object. At the set photographing distance, the 3D structured light camera captures the image of the target object.

The 3D structured light camera takes pictures because of its high accuracy in all directions. It can identify the precise outline and distance of the target item and output more accurate three-dimensional coordinates of the material.

S4. Calculate the outline and position of the target object in the work area, output the geometric coordinates of the target object, and perform precise positioning.

Step S4 specifically includes: the 3D structured light camera identifies the precise contour and relative distance of the target object, and calculates and outputs the precise three-dimensional coordinates of the material taken.

The characteristics of radar and 2D cameras: wide field of view, slightly lower accuracy, can be used to collect image data from a large field of view to a designated work area; the characteristics of 3D structured light cameras: small field of view, high accuracy, can be used to determine certain The target object in the work area is operated to obtain the specific outline and precise position coordinates of the object.

S5. The robot arm 3 moves to the precise position of the target object, picks up the target object, and unloads it.

Step S5 specifically includes:

The conveyor belt 2 swings to the position where the target object is to be picked up. The picking end 4 of the robot arm 3 picks up the target object and drags it to the conveyor belt 2. The conveyor belt 2 transports the target object and completes unloading. The position of the material receiving end of the conveyor belt 2 can be flush with the bottom of the goods or slightly lower than the bottom of the goods. The robot arm 3 only needs to overcome the friction between the goods to pull the goods out and drop them onto the conveyor belt 2. Compared with the traditional method, it needs to be lifted and moved. The repositioning method and the dragging method of picking up materials reduce the moving distance of the robot arm 3, save the time for picking up materials, and also significantly increase the weight of the goods to be transported.

Example 2:

As shown in Figures 3 and 4, a visual positioning-based picking and unloading device of the present invention is used to execute the picking and unloading method in the embodiment. The picking and unloading device includes:

The coarse positioning vision device 6 is used to perform coarse positioning and imaging of targets within the field of view and divide the work area; the coarse positioning vision device 6 includes a radar that scans and forms 3D imaging and a 2D camera that acquires 2D images.

The precise positioning vision equipment 7 is used for precise positioning and imaging of targets in the work area; the precision positioning vision equipment 7 includes a 3D structured light camera.

The robot arm 3 is provided with a material picking end 4, moves to the vicinity of the target object to be picked up according to the results of rough positioning, grabs the target object according to the results of fine positioning, and performs a dragging action; the robot arm 3 is preferably a multi-axis robot, which picks up materials End 4 is preferably a suction cup structure with a suction head, and the suction cup is connected to the end of the multi-axis robot. The material picking end 4 is not limited to a suction cup structure, and may also be a clamp structure, or other material picking carriers that play a role in grabbing goods.

The conveyor belt 2 swings to the location of the target object to be retrieved based on the precise positioning results, receives the target object dragged out by the robot arm 3, and transports the target object for unloading.

Equipment base 1, radar and 2D camera can be connected to the equipment base 1 through brackets to collect images of targets in the field of view from a high place. The pan/tilt of the robot arm 3 is connected to the equipment base 1, with the pan/tilt as the center. Swing and grab the designated target through the suction cup structure. The 3D structured light camera is connected to the pickup end 4 of the robot arm 3 to collect image data of the target object from the perspective of the pickup end 4 and obtain the specific outline and specific position coordinates of the target object, thereby completing the precise positioning of the robot arm 3 Reclaimer.

One end of the conveyor belt 2 is hinged on the equipment base 1, and the other end of the conveyor belt 2 completes the lifting and swinging under the action of the telescopic power device. One end of the telescopic power device is fixed on the equipment base 1, and the other end is connected to the conveyor belt 2. The feed end of the conveyor belt 2 is located within the swing range of the machine arm 3, so that the conveyor belt 2 can be docked with the pickup end 4 of the machine arm 3 to complete the unloading operation.

When goods of a specified height are to be taken, the telescopic power device drives the material-receiving end of the conveyor belt 2 to lift or lower to the target goods. The material-receiving end of the conveyor belt 2 will be flush with or slightly lower than the bottom of the goods, so that the machine The arm 3 only needs to drive the suction cup structure to drag the target goods out of the palletized goods onto the conveyor belt 2, which reduces the moving distance of the robot arm 3, and the dragging method only needs to overcome the friction between the goods, not the friction between the goods. The gravity of the cargo itself allows the robot arm 3 to grab and drag cargo with a larger volume or weight, thereby meeting the unloading operations in more situations.

The picking and unloading device also includes a moving mechanism 5, which is connected to the bottom of the equipment frame. As an AGV trolley, the moving mechanism 5 is a moving part of the whole machine. The moving mechanism 5 can be automatically controlled through a program or manually. This makes the movement of the entire machine more controllable and flexible.

The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be concluded that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all of them should be regarded as belonging to the protection scope of the present invention.

Claims (10)

A method of picking and unloading materials based on visual positioning, which is characterized by including the following steps: S1. Acquire image data of target objects within the field of view through coarse positioning vision equipment; S2. Calculate the image data of the target object and divide the work area, and then calculate the geometric center coordinates of the target object in a single work area; S3. The robot arm carries the precise positioning vision device and moves to a single work area to obtain image data of the target object in the work area; S4. Calculate the outline and position of the target object in the work area, and output the geometric coordinates of the target object; S5. The robot arm moves to the precise location of the target object, picks up the target object, and unloads it. The visual positioning-based picking and unloading method according to claim 1, wherein step S1 specifically includes: S11. Form 3D imaging through radar scanning of target objects in coarse positioning vision equipment; S12. Obtain the 2D image of the target object through the 2D camera in the coarse positioning vision device. The visual positioning-based picking and unloading method according to claim 2, wherein step S2 specifically includes: Fusion of 3D imaging point cloud data and 2D image data, identifies the 3D position distribution of a single target item, divides the work area according to specified rules, calculates and outputs the three-dimensional coordinates of the geometric center of a single work area, and uses this as the robot arm's location The rough positioning position of the material. The visual positioning-based picking and unloading method according to claim 3, characterized in that the specified rules for dividing the work area include: Divide a workspace according to a single item; divide it into a workspace according to a group of multiple items; or divide it into a workspace according to the set area size. The visual positioning-based picking and unloading method according to claim 1, wherein step S3 specifically includes: Set the photographing distance between the 3D structured light camera in the precision positioning vision equipment and the target object. The picking end of the robot arm moves to the geometric center coordinates of the target object in a single work area and maintains the set photographing distance from the target object. The 3D structure A light camera captures an image of the target. The visual positioning-based picking and unloading method according to claim 5, wherein step S4 specifically includes: The 3D structured light camera identifies the precise contour and relative distance of the target object, and calculates and outputs the precise three-dimensional coordinates of the material taken. The visual positioning-based picking and unloading method according to claim 1, wherein step S5 specifically includes: The conveyor belt swings to the position of the target object to be picked up. The picking end of the robot arm grabs the target object and drags it to the conveyor belt. The conveyor belt transports the target object and completes unloading. The visual positioning-based picking and unloading method according to claim 1, characterized in that before executing step S1, it further includes: Unify the positions of the radar and the robot arm in the coarse positioning vision equipment. The visual calibration content includes: the coordinate system origins of the radar and the robot arm are unified; the directions of the X/Y/Z/θ axes in the coordinate system are unified; and the radar , the actual mapping relationship between the robot arm coordinates and the space is unified. The visual positioning-based picking and unloading method according to claim 1, characterized in that before executing the method, it further includes: Visual training for the built-in programs of coarse positioning vision equipment and fine positioning vision equipment: input a large amount of picture data to learn to recognize the outline of the target object. The picture data includes a single category of target images or multiple categories of target images. A visual positioning-based material picking and unloading device, characterized by executing the visual positioning-based picking and unloading method according to any one of claims 1 to 9, including Rough positioning vision equipment is used to roughly position and image targets within the field of view and divide the work area; Precision positioning vision equipment, used for precise positioning and imaging of targets in the work area; The robot arm is equipped with a picking end, moves to the vicinity of the target object to be picked based on the results of rough positioning, acquires the target object and performs a dragging action based on the results of fine positioning; The conveyor belt swings to the location of the target object to be retrieved based on the precise positioning results, receives the target object dragged out by the robot arm, and transports the target object for unloading.

Images