JPH0646403B2 - Crop recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention is, for example, for obtaining the control information for guiding the work manipulator by detecting the positional relationship of the work target crop with respect to the crop working machine. More specifically, the present invention relates to a crop recognition device that obtains crop position information based on a specific color corresponding to a recognized crop from image information captured by an imaging device.

[Conventional technology]

The crop recognition device of this type, for example, binarizes the captured image information based on the color of the fruit to be recognized as a crop,
Except for information of objects other than the recognition target, a means for detecting the position information of the recognition target based on the binarized image information, such as the size and the position of the center of gravity, is used, so that a plurality of crops are densely gathered. In case of growing or overlapping and appearing as one lump, in order to binarize based on the color,
The lump will be recognized as one crop.
Since the picked-up image is binarized based on color, it is very difficult to separate the multiple crops that have become this block later, and it is not easy to use only the binarized image by color. Multiple crops could not be separated and identified.

Therefore, if the center of gravity position is used as a control parameter for the guiding direction of the manipulator, for example, based on the image information obtained by binarizing the plurality of crop lumps, the position of the center of gravity of the actual work target crop is different. This leads to inconvenience, resulting in malfunction.

Therefore, the recognition target is imaged as a grayscale image corresponding to only the brightness change, and only the portion having a large brightness change is extracted from the grayscale image information and binarized, and the edge of the brightness change is detected to detect one of the same color. It is conceivable to use a method that separates multiple crops to be recognized into one lump, one by one, but in this case, the captured image contains unnecessary information such as branches and leaves other than the crops. Therefore, if the entire grayscale image information is binarized, it is not possible to identify where the image information corresponding to the target crop is. Further, since there is a lot of processing information for edge detection, there is a disadvantage that the processing time becomes long.

[Problems to be Solved by the Invention]

Therefore, the means for binarizing the picked-up image information based on the color information thereof, or the means for binarizing the grayscale image information based on the change of the brightness thereof, each of which recognizes a plurality of recognition targets one by one. It was impossible to easily obtain the binarized image information in the separated state.

Further, when the grayscale image information is subjected to the edge detection processing over the entire captured image information as described above, there is a problem that the processing time becomes a problem and it becomes difficult to put into practical use. Again 2
It is conceivable to separate the recognition target that appears as one block by using both the image information of the binarized color image information and the grayscale image information, but the edge detection process cannot be omitted, and the device configuration becomes complicated. Moreover, even if the binarized image information separated into each one is obtained, it is necessary to further detect each representative position of the binarized image, for example, the position of the center of gravity, and perform the process for obtaining the position information. There is.

The present invention has been made in view of the above circumstances, and provides a crop recognition device that can easily obtain position information for each crop of a plurality of crops that appear to be one lump that are dense or overlapping. With the goal.

[Means for Solving the Problems]

The crop recognition device according to the present invention is an illumination unit that illuminates an imaging target, a unit that extracts grayscale image information in a range corresponding to the specific color from image information captured when illuminated by the illumination unit, and the grayscale image information Each of the peak position detecting means for detecting the peak position of brightness is provided.

The operation and effect of this characteristic configuration are as follows.

[Work]

Since crops such as mandarin oranges and apples are generally spherical in shape, when this recognition target is illuminated, the position seen as the apex of the spherical shape is in the state where it is most brightly reflected.
Since this brightest position exists for each of a plurality of target crops, this brightest position can be treated as representative position information of each recognition target crop.

Therefore, if the image information captured by illuminating the recognition target is masked with a specific color corresponding to only the target crop and only the grayscale image information within the same coordinate range as the masked specific color is extracted, the grayscale The image information is information on the brightness of only the recognition target. Then, when the brightness peak position of the grayscale image information is detected, this detected position corresponds to the position seen as the apex of the spherical shape, and the detected peak position itself represents position information representing each recognition target crop. It will be.

〔The invention's effect〕

That is, by using the specific color as a mask, by extracting only the grayscale image information within the same coordinate range as this specific color, the specific color corresponding to the fruit excluding unnecessary information in advance and corresponding to the grayscale change Image information is obtained. Then, by detecting where the peak position of the brightness of the grayscale image information in the specified range is, a plurality of recognition target crops can be seen as a single lump in a dense or overlapping state. Even if there is, it is possible to easily obtain location information in the state where the crops are separated from each other. Moreover, in order to obtain the position information of each recognition target, only the process of detecting the peak position of the brightness of the grayscale image information of the specific color range is performed. There is no need to perform complicated processing, and the device configuration can be simplified. Therefore, it has become possible to quickly and easily separate and identify a plurality of crops one by one. In addition, since the grayscale image information other than the crop is completely removed by the specific color, the grayscale image information does not include unnecessary noise components other than the recognition target, and therefore the reliability of the recognition result is also large. It has improved.

〔Example〕

 Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a crop recognition device that obtains position information corresponding to each target crop from image information taken outdoors, and particularly recognizes warm-colored fruits such as oranges and apples. By subtracting the color signal (B) of the blue component from the color signal (R) of the red component that constitutes the color image, the branches other than the fruit color,
Binary color image information (F ₁ ) corresponding only to the fruit color (C) obtained by removing the color components corresponding to the background object such as leaves and sky
Extracting, by masking a luminance signal (Y) in this binary color image information (F _1), gray-scale image of the specific color range corresponding only to the fruit color (C) (S ₁₎ to extract the , Its brightness peak position
By obtaining (P), it is configured to obtain positional information in a state in which each of a plurality of overlapping fruits is separated.

That is, the NTSC color video signal (S ₀ ) output as the image information (see FIG. 8A) captured by the color video camera (1) as the image capturing means is the NTS format.
By the C decoder (2), the vertical synchronizing signal (VD), the horizontal synchronizing signal (HD), the red signal (R), the blue signal (B), and the luminance signal as the grayscale image information ( Y), and the color separation circuit (3) calculates the difference (RB) between the red signal (R) and the blue signal (B) and compares it with the set threshold (Cref) to determine the color of the target fruit. It is converted into the binarized color image information (F ₁ ) (see FIG. 8B) which is the binarized image information of the color corresponding to only (C).

The luminance signal (Y) changes to the fruit color (C) while the binarized color image information (F ₁ ) is at “H” level by using the binarized color image information (F ₁ ) as a control signal. By passing through the gate circuit (4) that opens only the corresponding section, 2
The value of the target fruit color is used as a mask with the valued color image information (F ₁ ).
Extracted as grayscale image information (S ₁ ) corresponding to (C) only,
128 × 1 with 8-bit / pixel resolution by A / D converter (5)
Digital grayscale image information (S ₂ ) consisting of 28 pixels / 1 screen
(See FIG. 8C), and is temporarily stored in the image memory (7) via the data buffer (6).

Processing for obtaining the binarized color image information (F ₁ ) and grayscale image information
The process of A / D converting (S ₁ ) and storing it in the image memory (7) as digital grayscale image information (S ₂ ) for one screen is performed by a vertical synchronization signal (VD) and a horizontal synchronization signal (HD). The processing operation is controlled by the timing circuit (8) so as to be performed in parallel in synchronism with each change timing. In addition, the strobe device (10) as an illuminating means emits light in synchronization with the image pickup operation by the camera (1),
A strobe control circuit (9) activated by a control signal from the timing circuit (8) controls the light emission timing and its light emission intensity.

Further, the digital grayscale image information (S ₂ ) stored in the image memory (7) is converted into a composite video signal by the CRT controller (11) and the TV signal generator (12) and displayed on the monitor TV (13). It is done.

Then, the digital grayscale image information (S ₂ ) is, by the control processor (CPU ₁ ) and the numerical calculation processor (CPU ₂ ),
The brightness change at each coordinate point (x, y) is calculated as a two-dimensional histogram, and the coordinates (Xj, Yi) of the peak position (P) where the brightness (F ₂ ) is maximum are detected.

Then, the information on the number of recognized fruits and their position coordinates is transmitted as a recognition result to the host computer (CPU ₀ ) via the interface device (15), and the entire operation is controlled. In FIG. 1, (16) is a temporary storage of the operation programs of the processors (CPU ₁ ) and (CPU ₂ ) and calculation data, and the host computer (CPU ₀ ).
It is a memory for storing various data and the like exchanged with.

Hereinafter, the configuration of each unit will be described in detail.

As shown in FIG. 4, the color separation circuit (3) controls its operation by a timing controller (30) so as to operate in synchronization with the horizontal synchronizing signal (HD) and the vertical synchronizing signal (VD). It is configured to be performed. Then, the difference (RB) between the color signal (R) of the red component and the color signal (B) of the blue component output from the decoder (2) is calculated by the differential amplifier (31), and the output signal is The gain is set to a predetermined level by the video amplifier (32) whose gain is set by the gain setting resistor (R ₁ ), and is compared by the comparator (33) with the set threshold value (Cref) set by the resistor (R ₂ ). It is converted into the binary color image information (F ₁ ) of the level. By the way, the threshold
(Cref) will be set in advance based on the color (C) of the recognition target fruit, the brightness of the imaging state, and the like.

As shown in FIG. 5, the strobe control circuit (9) has two one-shot multivibrators (VREQ) activated by a trigger signal (VREQ) corresponding to the start position of the blanking period of the vertical synchronizing signal (VD). 90), (91).

Then, as shown in FIG. 6, after the elapse of a predetermined time (t ₁ ) delayed by the first multivibrator (90), the next multivibrator (91) is activated and its output time (t ₂ ) is emitted. The photo coupler (93) for transmitting a light emission signal to the strobe device (10) is turned on so that the strobe device (10) emits light according to the intensity. That is, the strobe device (10) automatically emits light at a predetermined illuminance in synchronization with the image pickup of one screen by the camera (1) and during the blanking period of the vertical synchronizing signal (VD). Of.

The overall operation will be described below based on the flowchart shown in FIG.

That is, when the power is turned on, the control processor (CPU
₁ ) initializes the contents of each memory (7), (14), (16), operation data, etc., and holds itself in a waiting state for receiving a start signal from the host computer (CPU ₀ ). The host computer (CPU ₀ ) confirms that the control processor (CPU ₁ ) is self-holding, sets a start flag for image processing, and notifies the control processor (CPU ₁ ) of the start of processing. The control processor (CPU ₁ ) waits for the end of processing.

The control processor (CPU ₁ ), when the start flag is set, starts the imaging process by the camera (1),
The image information (S ₀ ) for one field is fetched, the grayscale image information (S ₂ ) masked with the binarized color image information (F ₁ ) of the color difference signal (RB) is read, and the peak position of the brightness ( Coordinates of P)
Then, (Xj, Yi) is calculated, and the position information of the recognition target is processed to enter the standby state. The number of peak positions (P) is information corresponding to the number of recognition targets.

On the other hand, the host computer (CPU ₀ ) is the control processor (C
After confirming that it is in the standby state (PU ₁ ), read the recognition result (the number of recognized fruits and their position coordinates),
By resetting the start flag, the control processor (C
Reboot PU ₁ ). Then, the above operation is repeated a necessary number of times.

Next, based on the flow chart shown in FIG. 2, the coordinates (Xj, Yi) of the peak position (P) of the brightness of the digital grayscale image information (S ₂ ) are obtained, and each of the plurality of fruits is individually determined. An operation for obtaining position information for only one fruit in the separated and identified state will be described.

That is, when the start flag is set, the gate circuit (4) and the A / D converter (5) which are opened and closed by the binarized color image information (F ₁ ) output from the color separation circuit (3). The luminance signal (Y) for one field is input to the image memory (7) as the digital grayscale image information (S ₂ ) masked with the specific color (C) corresponding to the fruit via.

Then, as shown in FIG. 9, the brightness at each coordinate point of all pixels of the grayscale image information (S ₂ ), that is, the brightness value (V)
That is, the two-dimensional histogram is obtained, and the coordinates (Xj, Yi) of the peak position (P) are detected as position information representing the recognition target.

As shown in FIG. 10, the process of obtaining the peak position (P) of the brightness value (V) of the grayscale image information (S ₂ ) includes, as shown in FIG. Adjacent 8 pixels (a ~ d, f ~
The brightness difference for i) is calculated, and the brightness of the pixel of interest (e) is
If it is larger than any of the 8 adjacent pixels (a to d, f to i), the peak position of the pixel of interest (e) is set to the peak position of brightness by sequentially performing processing for each pixel.
The coordinates (Xj, Yi) of (P) are detected.

After that, the value of the position coordinates (Xj, Yi) of the detected peak position (P) and the number thereof are transmitted to the host computer (CPU ₀ ) as the position information of the recognized fruit.

In short, the present invention is a simple process of obtaining the peak position of the brightness of the grayscale image information masked with a specific color, and the position information of a plurality of recognition target fruits, each of which is in a state of being separated. Can be obtained at. Moreover, there is no need to perform the conventional processing of detecting the edge point of the grayscale change or obtaining the position of the center of gravity of the recognition area from the binarized image information, which enables high-speed processing.
The device configuration can also be simplified. It should be noted that, if a means for masking the luminance signal with a specific color and preliminarily extracting the grayscale image information of only the specific color range as in the above embodiment is used, the signal preprocessing can be performed in real time with a simple configuration. At the same time, the process for obtaining the position information of the recognition target is only the process for obtaining the peak position of the brightness,
High-speed processing has become possible regardless of the simple device configuration.

[Brief description of drawings]

The drawings show an embodiment of the crop recognizing device according to the present invention, and FIG. 1 is a block diagram showing the overall configuration of the crop recognizing device, and FIG.
FIG. 4 is a flow chart showing the operation of the crop recognition device, FIG. 3 is a flow chart showing the relationship between the control processor of the crop recognition device and the host computer, FIG. 4 is a block diagram showing the configuration of the color separation circuit, and FIG. 5 is a gate. 6 is a time chart showing the operation of the circuit, FIG. 6 is a circuit diagram showing the configuration of the strobe control circuit, and FIG. 7 is a time chart showing the operation.
8 (a), (b), and (c) are explanatory diagrams of image information, FIG. 9 is an explanatory diagram of brightness change, and FIG. 10 is an explanatory diagram of a peak position detection mask. (1) …… Imaging means, (10) …… Illumination means, (C) …… Specific color,
(S ₀ ) …… Sensing image information, (S ₁ ) …… Gray image information of specific color range, (P) …… Brightness peak position

Claims (1)

[Claims] 1. A crop recognition device for obtaining position information of a crop based on a specific color (C) corresponding to a recognized crop from image information (S ₀ ) picked up by the imaging means (1). Illuminating means for illuminating (10), means for extracting the grayscale image information (S ₁ ) in the range corresponding to the specific color (C) from the image information (S ₀ ) captured during illumination by the illuminating means (10), Grayscale image information
A crop recognition device provided with respective peak position detecting means for detecting the peak position (P) of the brightness from (S ₁ ).