JPH04311312A - Device of guiding harvester of fruits and vegetables - Google Patents

Abstract

PURPOSE:To surely harvest targets to be reaped while limiting loss of action to the minimum by controlling the guiding direction of harvester by the positional correlation between an image corresponding to the color of the targets to be reaped and a highlight zone thereof. CONSTITUTION:Position of harvest grasping part is corrected by the change of direction of a vacuum pad depending upon targets to be reaped and harvesting action is carried out. When targets to be reaped are not adsorbed, deviation of position between the central part of a zone F1 corresponding to the color of targets F to be reaped and a highlight zone F2 is obtained by treatment of pickup image. A control means introduces the harvest grasping part from a direction shifted to the opposite side to the deviation of position into the highlight zone F2.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to a hand equipped with a vacuum pad for suctioning an object to be harvested, the direction of which can be changed with respect to a main body part, and which is biased to return to a set direction, and the main body of the vacuum pad. a detection means for detecting the orientation with respect to the harvesting object; an imaging means for taking an image of the harvesting object; and a specific color area corresponding to the color of the harvesting object by processing the information of the imaging means, and a brightness within the specific color area. an image processing means for determining a highlight area where is larger than a set value;
The vacuum pad is caused to approach the highlight area of the harvesting target based on the information of the image processing means, and when the harvesting target cannot be picked up by the approach, the vacuum pad is set based on the information of the detection means. The present invention relates to a guidance device for a fruit and vegetable harvesting machine, which is provided with a control means for reapproaching the machine by correcting its position in a direction in which the direction has been changed.

0002

[Prior Art] In the above-mentioned approach, when a vacuum pad comes into contact with an object to be harvested such as fruit, if there is an error in the guiding direction, the direction of the vacuum pad changes because it comes into contact with the edge of the vacuum pad. will be done. Therefore, by detecting the error in the guidance direction based on this information and correcting the position of the hand, it is possible to correct the guidance direction in the re-approach.

0003

According to the above-mentioned prior art, re-guidance can be started quickly without performing re-imaging processing or the like in re-approaching, so the efficiency of harvesting work can be improved. However, if part of the target to be harvested is blocked by leaves, etc., even if the guiding direction is accurate, the leaves will get in the way and the target cannot be harvested properly, making it difficult to harvest the target. There was a possibility that it would not be possible. SUMMARY OF THE INVENTION An object of the present invention is to provide a guidance device for a fruit and vegetable harvester that eliminates the above-mentioned conventional drawbacks and can reliably harvest objects to be harvested without reducing the efficiency of harvesting operations.

0004

[Means for Solving the Problems] In order to achieve this object, the characteristic structure of the guidance device for the fruit and vegetable harvesting machine according to the present invention is as follows:
If the harvesting target cannot be picked up by the re-approach, the image processing means corrects the positional deviation of the highlight area with respect to the center or center of gravity of the specific color area in a specific direction based on the re-imaging information of the imaging unit. and the control means is configured to cause the hand to approach the highlight area again from a direction in which the hand is deviated to a side opposite to the positional deviation.

[0005]

[Operation] If the target to be harvested is not obscured by leaves or the like, it is sufficient to approach the target directly with the hand. Therefore, it is thought that accurate harvesting can be achieved by detecting the error in the guidance direction from the orientation of the vacuum pad, correcting the position of the hand, and re-approaching the hand. On the other hand,
If part of the harvest is blocked by leaves, etc., it is necessary to avoid the leaves etc. when harvesting. By the way, if a part of the area is blocked by leaves or the like, the highlight area will be shifted in the direction of the leaves or the like from the center or center of gravity of the specific color area in a specific direction. Therefore, if the object to be harvested cannot be picked up by re-approaching, it is possible to perform the harvesting operation without being obstructed by leaves etc. by approaching from the direction opposite to the positional deviation.

[0006]

Effects of the Invention: By detecting the error in the guidance direction from the orientation of the vacuum pad and re-approaching, the re-approach can be quickly executed. Also, only when the harvesting target cannot be picked up by re-approaching, the harvesting operation is performed while avoiding leaves etc. based on the re-imaging information. Therefore, it has been possible to obtain a guidance device for a fruit and vegetable harvesting machine that can reliably harvest the target while minimizing a decrease in the efficiency of harvesting work.

[0007]

Embodiments Hereinafter, embodiments in which the present invention is applied to a guidance device for a fruit harvesting machine will be described with reference to the drawings. As shown in FIG. 6, a boom 2 is attached to a vehicle body V equipped with a pair of left and right running wheels 1 at the front and rear so that it can be driven up and down and rotated, and an auxiliary boom 3 is attached to the tip of the boom 2 in a horizontal direction. A working manipulator 4 is attached to the tip of the auxiliary boom 3 so as to be swingable and equipped with a hand H for harvesting fruit. In addition, in the figure,
5 is a hydraulic cylinder for lifting and lowering the boom, 6 is a hydraulic cylinder for swinging the boom, and 7 is a hydraulic cylinder for swinging the auxiliary boom.

The working manipulator 4 consists of a multi-jointed telescopic arm 8 and a hand H attached to the tip of the telescopic arm 8. The telescoping arm 8 is rotated around the vertical axis Y by the electric motor 9a, swung around the horizontal axis X by the electric motor 9b, and extended and retracted by the electric motor 9c. It is configured as follows.

To explain the hand H, as shown in FIGS. 7 and 8, there is a vacuum pad 10 at its tip for sucking the fruit F to be harvested, and a suction port 10a of the vacuum pad 10. A trapping part case 12 for covering the adsorbed fruit F, and the trapping part case 12
A handle cutting device 13 is provided which cuts the handle part of the fruit F taken into the fruit while holding it in the vertical direction.

To further explain the vacuum pad 10, the vacuum pad 10 is formed into a bellows shape, and is attached to the ventilation pipe 11 serving as the main body so that the direction can be changed freely and the vacuum pad 10 is biased to return forward. ing. Further, three air cylinders 21 connected to the suction port 10a at the tip via wires 22, and three proximity switches S4 are attached to the ventilation pipe 11. The air cylinders 21 and the proximity switches S4 are alternately arranged at intervals of 60 degrees on the same circumference. Also, the suction port 10a
protrusions 10b at positions facing the proximity switches S4, respectively.
are provided, and the proximity switches S4 detect the respective protrusions 10b, thereby individually detecting the backward movement of the suction port 10a with respect to the ventilation pipe 11. In other words, the proximity switch S4 corresponds to a detection means for detecting the orientation of the vacuum pad 10 with respect to the ventilation pipe 11. By the way, by moving the three air cylinders 21 in and out at the same time, the ventilation pipe 11 can be opened with the suction port 10a facing forward.
You can move in and out of the area. Further, by retiring one or two air cylinders 21 and extending the other air cylinders 21, the orientation of the vacuum pad 10 with respect to the ventilation pipe 11 can be changed. In the figure, S1 is a color image sensor as an imaging means for capturing an image of the fruit F located in the direction in which the working hand H is facing, and S2 is a color image sensor as the working hand H approaches the target fruit F within a set distance. Both sensors S1 and S2 are provided inside the vacuum pad 10 so as not to interfere with approach and fruit harvesting. Further, 15 is a lighting device for illuminating the front side of the working hand H with a set light amount in synchronization with the imaging processing of the image sensor S1, and generally a so-called strobe light emitting device is used. .

The catcher case 12 includes a proximal end portion 12a that is externally supported by the ventilation pipe 11 so as to be slidable in the front-back direction of the hand, and an upper cover 12 that is fixed to the upper portion of the distal end side of the proximal end portion 12a.
b, and a lower cover 12c that is also fixed to the lower part of the tip side.
It consists of Then, until the vacuum pad 10 adsorbs the fruit F, the hand is retired to the rear side so as not to get in the way of the approach, and after the vacuum pad 10 has adsorbed the fruit F, it is set so as to cover the adsorbed fruit F. It is designed to protrude toward the front of the hand. A shutter 25 is provided at the lower rear side of the lower cover 12c, and can be opened and closed by an air cylinder 23 for opening and closing. Further, the shutter 25 is provided with a lever 24 made of resin that can freely swing around the horizontal axis, and a proximity switch S5 that detects the swing of the lever 24. Only the lower part of the lever 24 is made of metal to urge it back to the upright position, and when the fruit F whose handle has been cut falls onto the shutter 25, it is swung to the laid down position. The proximity switch S
5 is turned on when the lever 24 is in the prostrate position. With that,
Cutting of the stalk of fruit F can be detected.

The handle cutting device 13 includes a clipper-shaped blade 13a that cuts the handle of the fruit F, and a handle support member 13b that presses and supports the handle of the fruit F from below on the blade 13a.

However, although not described in detail, the hand H is equipped with various pneumatic actuators for harvesting operation, such as an actuator for moving the trap case 12 in and out, and an actuator for driving the handle cutting device 13. , a negative pressure sensor S3 for detecting that the fruit F has been adsorbed on the vacuum pad 10, a sensor for detecting the exit/exit position of the trap case 12,
Various sensors such as a sensor for detecting the driving state of the handle cutting device 13 are also provided.

[0014] Adding an explanation to the fruit harvesting procedure,
With the catching part case 12 retired to the rear side of the hand and the suction port 10a of the vacuum pad 10 protruding forward beyond the catching part case 12, the working hand H is positioned at a preset approach start position, In addition, the image sensor S is directed toward the direction in which the fruit F to be harvested is located.
Image processing is performed in step 1. Then, as shown in FIGS. 5A and 5B, based on the information obtained by image processing the captured image information,
A specific color area F1 corresponding to the color of the fruit F, a highlight area F2 whose brightness is greater than a set value within the specific color area F1, and a center of gravity W2 of the highlight area F2 are determined. Hand H towards the center of gravity W2 of this highlight area F2.
will be approached. After determining the approach direction, each electric motor 9a is activated so that the work hand H moves linearly toward the determined approach direction.
. After the fruit F is taken inside, the stalk is cut and harvested. However, after the proximity sensor S2 is activated, if the negative pressure sensor S3 is not activated while the hand H is moved forward by the set distance that the proximity sensor S2 is activated, it is determined that the harvest has failed, and the set distance is Based on the detection information of the proximity switch S4, the position of the hand H is corrected by a predetermined distance in the direction in which the direction of the vacuum pad 10 is changed from the forward direction from the position where the hand H is retreated to the front side, and the hand H is made to approach again. Furthermore, if the fruit F cannot be absorbed by the approach and a change in the direction of the vacuum pad 10 from the forward direction cannot be detected,
And, if fruit F cannot be adsorbed by re-approaching,
The image is taken again from a position where the set distance has been retreated to the front side, and based on the image information, the approach direction is corrected and the approach is made again. To explain the correction of the approach direction, as shown in Fig. 5C,
The center of gravity W1 of the specific color area F1 and the center of gravity W2 of the highlight area F2 are determined, and if there is a positional deviation between the center of gravity W2 of the highlight area F2 and the center of gravity W1 of the specific color area F1, the center of gravity W2 of the highlight area F2 is determined. The direction in which it deviates to the opposite side is the approach direction.

Next, a control configuration for determining the approach order of the fruit F and guiding the hand H toward the fruit F based on detected information such as image information captured by the image sensor S1 will be described. As shown in Figure 1,
A control device 16 using a microcomputer is provided,
The control device 16 includes the sensors S1, S2, S3,
S4 and S5 are connected. Then, the control device 16 controls the air cylinder 21, each of the hydraulic cylinders 5 and 6, based on preset and stored information and various input information.
7 and each electric motor 9a, 9b, 9c. That is, the image processing means 100 and the control means 101 are configured using the control device 16.

Next, the operation of the control device 16 will be explained based on the flowcharts shown in FIGS. 2 to 4. First, the hand H is set at a preset approach start position, and imaging processing is performed with all air cylinders 21 retired.
The captured image information is subjected to image processing. That is, the specific color area F1 of the fruit F in the image, the highlight area F2, the center of gravity W1 of the specific color area F1, and the center of gravity W of the highlight area F2.
Find 2. Then, based on the size of the highlight area F2, the hand H is moved in the direction where the center of gravity W2 of the largest size area is located.
The approach direction of the telescopic arm 8 is determined and the telescopic arm 8 is extended. At this time, all the air cylinders 21 are operated in a protruding manner. After the extension of the telescopic arm 8 is started, based on the information of the proximity sensor S2, it is determined whether or not the hand H has approached the target fruit F to be harvested by the predetermined distance, As the fruit approaches, the extension speed of the telescopic arm 8 is switched to a low speed, the suction operation of the vacuum pad 10 is started, and the hand H is further moved toward the fruit side until the negative pressure sensor S3 is turned on. When the negative pressure sensor S3 turns on and detects fruit adsorption,
The telescoping arm 8 is stopped and the harvesting operation is started. However, if the negative pressure sensor S3 does not operate after the proximity sensor S2 is turned ON and the negative pressure sensor S3 does not operate while moving the set distance, it is determined that the harvest has failed. At this time, if at least one of the proximity switches S4 is ON, the hand H
Once returned to the position where proximity sensor S2 was turned on,
As mentioned above, the position of the hand H is corrected and the hand H is approached again. Furthermore, if none of the proximity switches S4 are turned on, and when the fruit F is re-approached,
If it is not possible to harvest the image, reimaging is performed from a position moved back from the set distance to the front side, and the positional deviation is determined. Then, approach again from the direction opposite to the positional deviation. When the fruit is successfully harvested, the working hand H is returned to the approach start position while holding the fruit F with the cut handle inside the catcher case 12, and the fruit F is discharged. After discharging the harvested fruit F, it is determined whether or not the work is completed. If the work is not completed, the work hand H is reset to the approach start position and image processing is performed, and the remaining fruit F is discharged. The process of determining the approach order for the object and guiding the working hand H toward the determined approach direction is repeated.

[Another Embodiment] In the above embodiment, the present invention was applied to a guidance device for a fruit harvesting working machine. It can be applied to a commercial work machine, and the specific configuration of each part, such as the specific configuration of the hand H, can be changed in various ways. Incidentally, although reference numerals are written in the claims section for convenient comparison with the drawings, the present invention is not limited to the structure shown in the accompanying drawings.

[Brief explanation of drawings]

[Figure 1] Block diagram of control configuration

[Figure 2] Flowchart of control operation

[Figure 3] Flowchart of control operation

[Figure 4] Flowchart of control operation

[Figure 5] Explanatory diagram of image processing

[Figure 6] Schematic side view of harvester

[Figure 7] Side view of the main parts of the hand

[Figure 8] Front view of main parts of hand

[Explanation of symbols]

10 Vacuum pad 11 Main body 100 Image processing means 101 Control means S1 Imaging means S4 Detection means H Hand F1 Specific color area F2 Highlight area W1 Central part or center of gravity

Claims (1)

[Claims] 1. A hand (H) comprising a vacuum pad (10) for adsorbing a harvested object, the direction of which can be changed with respect to a main body (11), and which is biased to return to a set orientation; and the vacuum pad. (10) The main body part (11)
a detection means (S4) for detecting the orientation of the harvest object; an imaging means (S1) for imaging the harvest target;
Process the information in 1) to find a specific color area (F1) corresponding to the color to be harvested and a highlight area (F2) whose brightness is greater than a set value within the specific color area (F1). an image processing means (100); and the image processing means (
100), the vacuum pad (10) is caused to approach the highlight area (F2) for the harvesting target, and if the harvesting target cannot be absorbed by the approach, the information of the detection means (S4) is and a control means (101) for correcting the position of the vacuum pad (10) in a direction in which the direction has been changed from the set direction based on the direction and reapproaching the vacuum pad (10), the guidance device for a fruit and vegetable harvester. If the harvesting target cannot be picked up by re-approaching, the image processing means (100) detects the central part or center of gravity ( W1)
The control means (101) approaches the highlight area (F2) again from a direction in which the hand (H) is deviated to the opposite side to the positional deviation. A guidance device for a fruit and vegetable harvester configured to cause