JPH04311314A - fruit harvesting equipment - Google Patents

Abstract

PURPOSE:To lessen times of pickup treatment and to increase operation efficiency in fruit harvesting for reaping target fruits by specifying position of the target fruits to be reaped based on pickup information of fruit tree. CONSTITUTION:A pickup magnification of pickup means is variably constituted. A zone F1 of specific color corresponding to the color of target fruits F to be reaped is obtained by treating pickup information prepared by a small pickup magnification, only a zone containing the zone F1 of specific color is imaged by a large pickup magnification, a highlight zone F2 having a higher brightness than a set value in the zone F1 of specific color is obtained and position of the target fruits F to be reaped is specified based on the position of the highlight zone F2.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fruit harvesting method for specifying the position of a fruit to be harvested based on image information taken by an imaging means on a fruit tree and harvesting the fruit to be harvested.

0002

[Prior Art] When specifying the position of a fruit to be harvested, it is necessary to find a highlight area whose brightness is greater than a set value within a specific color area corresponding to the color of the fruit to be harvested. It is necessary to take an image at the imaging magnification. Conventionally, the imaging magnification of the imaging means has been fixed.

0003

[Problems to be Solved by the Invention] When the imaging magnification is large, the imaging range becomes narrow, so as shown in FIG. 10, it takes many times (20 times in this case) to image the entire fruit tree. Furthermore, since both the area where fruit exists and the area where fruit does not exist are imaged at a large imaging magnification, there is a risk that a lot of unnecessary imaging may occur. Therefore, in the above-mentioned conventional technology, there is a risk that the number of imaging processes will increase and the work efficiency will decrease. An object of the present invention is to eliminate the above-mentioned conventional drawbacks and provide a fruit harvesting method with high work efficiency.

0004

[Means for Solving the Problems] In order to achieve this object, the characteristic configuration of the fruit harvesting method according to the present invention is that the imaging magnification of the imaging means is configured to be freely changeable, and images obtained at a small imaging magnification are obtained. The information is processed to find a specific color area corresponding to the color of the fruit to be harvested, and only the area that includes the specific color area is imaged at a high imaging magnification, so that the brightness within the specific color area is lower than the set value. Searching for a large highlight area,
The present invention is configured to specify the position of the fruit to be harvested based on the position of the highlighted area.

[0005]

[Operation] First, to image the entire fruit tree, the image is taken at a small imaging magnification, as shown by the solid line in FIG. The imaging magnification at this time only needs to be such that a specific color area can be determined. Further, as shown by the dotted line in FIG. 5, since only the area including the specific color area is imaged at a high imaging magnification, there is no need to image an area where no fruit exists.

[0006]

[Effects of the Invention] The number of times the entire fruit tree is imaged can be reduced, and unnecessary images can be eliminated, so the amount of image sensing processing can be reduced. As a result, the work efficiency of fruit harvesting can be improved.

[0007]

Embodiments Hereinafter, embodiments in which the present invention is applied to a fruit harvesting machine will be described based on the drawings. As shown in FIG. 7, a vehicle body V includes a pair of left and right running wheels 1 at the front and rear.
A working manipulator is provided with a boom 2 that can be driven up and down and rotated, an auxiliary boom 3 that can be driven to swing in the horizontal direction, and a hand H for fruit harvesting. 4 is the auxiliary boom 3
attached to the tip of the In addition, in the figure, 5 is a hydraulic cylinder for raising and lowering the boom, 6 is a hydraulic cylinder for boom rotation,
7 is a hydraulic cylinder for swinging the auxiliary boom.

The working manipulator 4 is composed of a multi-jointed telescopic arm 8 and a hand H as a catching section attached to the tip of the telescopic arm 8. The telescoping arm 8 is rotated around the vertical axis Y by the electric motor 9a, swung around the horizontal axis X by the electric motor 9b, and extended and retracted by the electric motor 9c. It is configured as follows.

To explain the hand H, as shown in FIGS. 8 and 9, there is a vacuum pad 10 at its tip for sucking the fruit F to be harvested, and a suction port 10a of the vacuum pad 10. A trapping part case 12 for covering the adsorbed fruit F, and the trapping part case 12
A handle cutting device 13 is provided which cuts the handle part of the fruit F taken into the fruit while holding it in the vertical direction.

To further explain the vacuum pad 10, the vacuum pad 10 is formed into a bellows shape, and is attached to the ventilation pipe 11 serving as the main body so that the direction can be changed freely and the vacuum pad 10 is biased to return forward. ing. Further, three air cylinders 21 connected to the suction port 10a at the tip via wires 22, and three proximity switches S4 are attached to the ventilation pipe 11. The air cylinders 21 and the proximity switches S4 are alternately arranged at intervals of 60 degrees on the same circumference. Also, the suction port 10a
protrusions 10b at positions facing the proximity switches S4, respectively.
is provided. In other words, the proximity switch S4 detects the backward movement of the suction port 10a with respect to the ventilation pipe 11 by detecting the respective projections 10b. By the way, by moving the three air cylinders 21 in and out at the same time, they can be moved in and out of the ventilation pipe 11 with the suction port 10a facing forward. Further, by retiring one or two air cylinders 21 and extending the other air cylinders 21, the orientation of the vacuum pad 10 with respect to the ventilation pipe 11 can be changed. Inside the vacuum pad 10, there is a color image sensor S1 serving as an image capturing means for capturing an image of the fruit F located in the direction in which the hand H is facing, and a color image sensor S1 as an imaging means for capturing an image of the fruit F located in the direction in which the hand H is facing. A reflective proximity sensor S2 using infrared light is provided to detect this. A zoom lens 18 is used for the image sensor S1, and the imaging magnification can be changed. Furthermore, 1
Reference numeral 5 denotes a lighting device for illuminating the front side of the working hand H with a set amount of light in synchronization with the imaging process of the image sensor S1, and generally a so-called strobe light emitting device is used.

The catcher case 12 includes a proximal end portion 12a that is externally supported by the ventilation pipe 11 so as to be slidable in the front-back direction of the hand, and an upper cover 12 that is fixed to the upper portion of the distal end side of the proximal end portion 12a.
b, and a lower cover 12c that is also fixed to the lower part of the tip side.
It consists of Then, until the vacuum pad 10 adsorbs the fruit F, the hand is retired to the rear side so as not to get in the way of the approach, and after the vacuum pad 10 has adsorbed the fruit F, it is set so as to cover the adsorbed fruit F. It is designed to protrude toward the front of the hand. A shutter 25 is provided at the lower rear side of the lower cover 12c, and can be opened and closed by an air cylinder 23 for opening and closing. Further, the shutter 25 is provided with a lever 24 made of resin that can freely swing around the horizontal axis, and a proximity switch S5 that detects the swing of the lever 24. Only the lower part of the lever 24 is made of metal to urge it back to the upright position, and when the fruit F whose handle has been cut falls onto the shutter 25, it is swung to the laid down position. The proximity switch S
5 is turned on when the lever 24 is in the prostrate position. With that,
Cutting of the stalk of fruit F can be detected.

The handle cutting device 13 includes a clipper-shaped blade 13a that cuts the handle of the fruit F, and a handle support member 13b that presses and supports the handle of the fruit F from below on the blade 13a.

Although not described in detail, the hand H is equipped with various pneumatic actuators for harvesting operation, such as an actuator for moving the trap case 12 in and out, and an actuator for driving the handle cutting device 13. , various sensors such as a negative pressure sensor S3 that detects that the fruit F is adsorbed to the vacuum pad 10, a sensor that detects the exit/removal position of the trap case 12, and a sensor that detects the driving state of the handle cutting device 13. A category will be established.

[0014] Adding an explanation to the fruit harvesting procedure,
Position the hand H at a predetermined approach start position, and point the image sensor S1 in the direction where the fruit tree is located.
As shown by the solid line in FIG. 6(A), imaging processing is performed at a small imaging magnification. Then, based on information obtained by image processing the captured image information, a specific color region F1 corresponding to the color of the fruit F is determined, as shown in FIG. 6(B). Next, Figure 6(A)
As shown by the dotted line, only the area including the specific color area F1 is imaged at a large imaging magnification. Based on the captured image information, as shown in FIG. 6(C), a specific color area F1, a highlight area F2, and a center of gravity W2 of the highlight area F2 of the fruit F in the image are determined. Then, the direction in which the center of gravity W2 of the highlight area F2 of the fruit F to approach is located is defined as the approach direction. After determining the approach direction, each electric motor 9a, 9b, 9c is operated so that the hand H moves linearly in the determined approach direction, and the proximity sensor S2 detects the fruit F. Accordingly, the vacuum pad 10 is actuated to attract the fruit F to be harvested, and the catching part case 12 is protruded to take the fruit F into its inside, and then the handle is cut and the fruit F is harvested. . However, after the proximity sensor S2 is activated, if the negative pressure sensor S3 is not activated while the hand H is moved forward by the set distance that the proximity sensor S2 is activated, it is determined that the harvest has failed, and the set distance is Based on the detection information of the proximity switch S4, the position of the hand H is corrected by a predetermined distance in the direction in which the vacuum pad 10 has changed its direction from the forward direction, and the hand H is reapproached from the position where the hand H is moved back toward the front side. It has become.

Next, a control configuration for determining the position of the fruit F and guiding the hand H toward the fruit F based on detected information such as image information captured by the image sensor S1 will be described. As shown in FIG. 1, a control device 16 using a microcomputer is provided, and each of the sensors S1, S2, S3, S4, S5 is connected to the control device 16.
is connected. Then, the control device 16 controls the driving motor 18a of the zoom lens 18 and the respective hydraulic cylinders 5 based on preset and stored information and various input information.
, 6, 7, the electric motors 9a, 9b, 9c, and the air cylinder 21.

Next, the operation of the control device 16 will be explained based on the flowcharts shown in FIGS. 2 to 4. First, the hand H is set at a predetermined approach start position, and an image is captured at a small imaging magnification with all air cylinders 21 retired, and the captured image information is image-processed. That is, find the specific color area F1 of each of the n fruits F in the image, find the number n of fruits F based on the number of specific color areas F1,
Determine approach order. Next, only the area including the specific color area F1 is imaged at a high imaging magnification, and the center of gravity W2 of the highlight area F2 is determined. Then, the approach direction of the hand H is determined and the telescopic arm 8 is extended so that the hand H is moved in the direction in which the center of gravity W2 of the fruit F to approach is located. At this time, all the air cylinders 21 are operated in a protruding manner. After the extension of the telescopic arm 8 is started, based on the information of the proximity sensor S2, it is determined whether or not the hand H has approached the target fruit F to be harvested by the predetermined distance, As the fruit approaches, the extension speed of the telescopic arm 8 is switched to a low speed, the suction operation of the vacuum pad 10 is started, and the hand H is further moved toward the fruit side until the negative pressure sensor S3 is turned on. When the negative pressure sensor S3 turns on and detects fruit adsorption, the telescopic arm 8 is stopped and the harvesting operation is started. However, after the proximity sensor S2 is turned on and while moving the set distance, the negative pressure sensor S3
If it does not operate, it is determined that the harvest has failed. At this time, if at least one of the proximity switches S4 is turned on, the hand H is returned to the position where the proximity sensor S2 is turned on, and as described above, the position of the hand H is corrected and the approach is made again. Let it happen. Proximity switch S
If none of 4 are turned on, it is determined that harvesting is impossible and hand H is returned to the approach starting position. When the fruit can be harvested by the first approach or the second approach, the working hand H is returned to the approach start position while holding the fruit F with the cut stalk inside the trap case 12, and the fruit F is harvested. It will be discharged. After the harvested fruits F are discharged, it is determined whether or not there are any fruits F left based on the number n, and the above operation is repeated until there are no more fruits F left. When the fruit F runs out, it is determined whether the work is finished or not, and if the work is not finished, the hand H is set to a new approach start position. The above-mentioned processes are then repeated until the work is completed.

[Another Embodiment] In the above embodiment, the image sensor S1 is used as the imaging means, but the specific configuration of each part can be changed in various ways, such as using a television camera. Incidentally, although reference numerals are written in the claims section for convenient comparison with the drawings, the present invention is not limited to the structure shown in the accompanying drawings.

[Brief explanation of drawings]

[Figure 1] Block diagram of control configuration

[Figure 2] Flowchart of control operation

[Figure 3] Flowchart of control operation

[Figure 4] Flowchart of control operation

[Figure 5] Explanatory diagram of the imaging procedure

[Figure 6] Explanatory diagram of image processing

[Figure 7] Schematic side view of harvester

[Figure 8] Side view of the main parts of the hand

[Figure 9] Front view of main parts of hand

[Fig. 10] Explanatory diagram of conventional imaging procedure

[Explanation of symbols]

S1 Imaging means F1 Specific color area F2 Highlight area

Claims (1)

[Claims] 1. A fruit harvesting method in which the position of a fruit to be harvested is specified based on imaging information of an imaging means (S1) for a fruit tree, and the fruit to be harvested is harvested, the imaging magnification of the imaging means (S1) being is configured to be changeable, and processes the imaging information obtained at a small imaging magnification to obtain a specific color area (F1) corresponding to the color of the fruit to be harvested, and only the area that includes the specific color area (F1) is processed. is imaged at a large imaging magnification to find a highlight area (F2) whose brightness is greater than the set value within the specific color area (F1), and based on the position of the highlight area (F2), A fruit harvesting method configured to locate fruit to be harvested.