JPS629210A - Fruit recognizing device - Google Patents

Abstract

PURPOSE:To enable to perform the treatment to separate and extract plural fruits at high speed with simple constitution by inputting the variable density image information removing in advance the variable density image information in the zone excepting the object fruit color to an edge image detecting means. CONSTITUTION:A color separating circuit 3 separates and extracts the bivalent color image information F1 in the zone corresponding to an object fruit color from the color image information among the image pickedup image information S0 by a color video camera 1. An edge image detecting means separates and extracts the bivalent edge information corresponding to the part where its lightness variation is larger than the variable density image information Y among the informations S0, then, with the information F1 as a switch control signal, controls the switching of a gate circuit 4 to extract the variable density image information S1 in the zone corresponding to the fruit color and inputs the information S1 to the edge image detecting means. Since therefore this information S1 does not include the unnecessary information excepting the object fruit a high speed processing is enabled because of the processing informa tion quantity being less.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is directed to, for example, detecting the positional relationship of a fruit to be worked with respect to a fruit work machine and obtaining control information for guiding a work manipulator. A fruit recognition device used as means etc., specifically, a color binarization means for separating and extracting binarized color image information of a region corresponding to the color of the fruit to be recognized from color image information of the image information captured by the imaging means; edge image detection means for separating and extracting binarized edge image information corresponding to a portion of the captured image information in which the change in brightness is larger than the grayscale image information; and detecting the binarized edge image from the binarized color image information. The present invention relates to a fruit recognition device that is provided with a subtraction means for subtracting image information and obtains binarized image information in which a region corresponding to a fruit to be recognized is separated and extracted from the captured image information.

[Conventional technology]

This type of fruit recognition device described above is capable of detecting a state in which each fruit is separated even when multiple fruits are crowded or overlapped, so that they appear as a single lump. This makes it possible to obtain binarized image information.

By the way, in order to obtain the binarized edge image information by the edge image detection means, it is conceivable to input the grayscale image information of the captured image information as it is to the edge image detection means.

[Problem that the invention seeks to solve]

However, if the grayscale image information of the captured image information is used as is to obtain the binarized edge image information, the processing is time-consuming and has the disadvantage of making it difficult to perform fruit recognition at high speed. .

In other words, the edge image detection means, for example, performs differential processing on the grayscale image information of the captured image information to determine the value and direction of the brightness change, and then extracts noise information such as isolated edge points from the determined information. After removal, binarized image information is obtained that can be recognized as a continuous line (however, it may have a certain width) with areas with large brightness changes as boundaries, and the process involves arithmetic processing of two-dimensional information. Therefore, as the amount of information to be processed increases, the processing time becomes exponentially longer.

Incidentally, if the processing time required for fruit recognition becomes long, this will lead to disadvantages, such as making it difficult to efficiently perform work on fruits such as fruit harvesting when used to control fruit working machines, for example. Therefore, it is desired to be able to perform fruit recognition as quickly as possible.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to provide binarized image information that separates and extracts areas in which a plurality of fruits that are crowded or overlap each other are separated one by one. The key point is that the processing can be performed at high speed with a simple configuration.

[Means for solving problems]

The fruit recognition device according to the present invention is characterized by controlling the opening/closing of the binarized color image information as an opening/closing control signal.
A gate means is provided for masking the grayscale image information with a fruit color and extracts grayscale image information of a region corresponding to the fruit color from among the grayscale image information, and the grayscale image information extracted by the gate means is The present invention is configured to allow input to the edge image detection means, and its functions and effects are as follows.

[For production]

In other words, 2 based on the specific color corresponding only to the J target fruit.
By masking the grayscale image information of the captured image information with the sulfurized binary color image information, the grayscale image information in the same area as the binary color image information is extracted, and the grayscale image information outside the target fruit color primary is extracted. The grayscale image information from which the grayscale image information of the area has been removed in advance is input to the edge image detection means.

〔Effect of the invention〕

Therefore, the grayscale image information masked with the binary color image information does not include unnecessary information other than the fruit to be recognized, and the amount of information processed by the edge image detection means can be significantly reduced. In other words, since the amount of information to be processed is small, bulk processing can be performed.

In addition, since image information other than fruit is removed in advance before image processing, in the binarization process of grayscale image information,
Unnecessary noise components have been significantly reduced, and the reliability of the separation and extraction results of binarized image information has also been greatly improved.

Furthermore, since the grayscale image information and the binary color image information obtained by masking the grayscale image information of the captured image information with the binary color image information can be obtained by real-time processing, the time required for this masking process is This does not affect the processing time required for recognition.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

FIG. 1 is a block diagram of a fruit recognition device that obtains binarized image information in which each target fruit is separated from image information captured outdoors. It is configured as a device for recognizing fruits of the same species, and the color signal (R) of the red component that makes up the color image
By subtracting the color signal (B) of the bluer component, only the fruit color (C) obtained by removing color components corresponding to background objects such as branches, leaves, and the sky other than the fruit color is supported. From the binarized image information (F1) and the edge image information (F2) obtained by binarizing the grayscale image information (S1) in the range corresponding only to the color (C) of this fruit, multiple fruits that appear to overlap are obtained. The configuration is such that binarized image information (F3) is obtained in which each of the images is separated.

That is, a color video camera (1) as an imaging means
The NTSC type 2 color video signal (SO) output as image information (see Figure 8 (a)) is processed by the NTSC decoder (2) into a vertical synchronizing signal (VD
), horizontal sync signal (HD), red signal (R
), a blue signal (B), and a luminance signal (Y) as grayscale image information, and a color separation circuit (3) serving as a color binarization means separates the red signal (R) and blue signal. Calculate the difference (R-B) between the signals (B) and set the set darkness value (C
ref), and the binarized color image information (F3
) (see FIG. 8(n)).

As shown in FIG. 5, the luminance signal (Y) uses the binarized color image information (F1) as a control signal while the binarized color image information (F3) is at "H" level, that is. Gate circuit that opens only in the section corresponding to fruit color (C) (
4), binary color image information (F3
) as a mask, it is extracted as grayscale image information (S1) corresponding only to the color (C) of the target fruit, and is processed by the A/D converter (5) at a resolution of 8 bits/pixel at 128
It is converted into digital grayscale image information (S2) (see FIG. 8 (c)) consisting of 128 pixels/one screen, and is temporarily stored in the first image memory (7) via the data buffer (6).

The process of obtaining the binarized color image information (F1) and the AID conversion of the gray scale image information (S1) are carried out to obtain the first image memory (
7) The process of storing one screen's worth of digital grayscale image information (S) is performed simultaneously and in parallel in synchronization with each change timing of the vertical synchronization signal (VD) and horizontal synchronization signal (HD). , the processing operation is controlled by a timing circuit (8).The timing circuit (8) is configured to control the processing operation so that a strobe device (10) serving as an illumination means emits light in synchronization with the imaging operation by the camera (1). A strobe control circuit (9) activated by a control signal from a circuit (8) is configured to control the light emission timing and the light emission intensity.

Further, the digital grayscale image information (S2) stored in the first image memory (7) is converted into a composite video signal by a CRT controller (11) and a TV signal generator (12), and is sent to a monitor television (13). It is displayed by .

Then, the digital grayscale image information (S2) is subjected to arithmetic processing such as noise removal and contour extraction by a control processor (CPU, ) and a numerical calculation processor (CPU2), and is converted into a binary value as edge image information. It is converted into binary edge image information (h) (see FIG. 8 (=)) and temporarily stored in the second image memory (14).

Next, based on the gradation value of the gradation image information (S2) within the range corresponding to the fruit color (C) stored in the first image memory (7), those whose values are positive are selected from the 2nd image memory (7). It is restored as digitized color image information (F3), and its binarized color image information (
By subtracting the binarized edge image information (F2) stored in the second image memory (14) from F3), the binarized image information (F3) in which each fruit is separated is obtained (
(see FIG. 8(N)), and calculates the second image memory (1).
4). Therefore, binarized color image information (F3)
The gradation image information (S2) masked with , by itself, has the same image information as the binary color image information (F1) in the part where the brightness information is not zero. ), the 24M color image information (F1) can be easily restored from the masked grayscale image information (S2) without having to separately store the image information (F1) itself. There is no need to provide memory.

Then, based on the binarized image information @ (R3) in which each fruit is separated, the number of recognized fruits, their coordinate positions, etc. are calculated, and the interface device (15
), the recognition results are transmitted to the host computer (CPU0), and the overall operation is controlled. In Fig. 1, (16) indicates each of the above-mentioned processes (
Temporarily stores operating programs and calculation data for CPUO, (CPtJz), and
) is a memory for storing various data exchanged between

The configuration of each part will be explained in detail below.

As shown in FIG. 4, the color separation circuit (3) has its operation controlled by a timing controller (30) so that it operates in synchronization with the horizontal synchronization signal (HD) and the vertical synchronization signal (VD). It is configured so that Then, the difference (R-B) between the red component color signal (R) and the blue component color signal (B) output from the decoder (2) is calculated by a differential amplifier (31), and the output signal is is amplified to a predetermined level by a video amplifier (32) whose amplification degree is set by a gain setting resistor (RI), and compared with a set value (Cref) set by a resistor (R2) by a comparator (33). It is converted into TTL level binary color image information (F1). By the way, the darkness value (Cref) is set in advance based on the color (C) of the fruit to be recognized, the brightness of the imaging state, and the like.

The strobe control circuit (9), as shown in FIG.
two one-shot multi-by-breaks (90) activated by a trigger signal (VREQ) corresponding to the start position of the blanking period of the vertical synchronization signal (VD);
(91).

Then, as shown in Fig. 7, after a predetermined time (t3) delayed by the first multivibrator (90) has elapsed, the next multivibrator (91) is activated, and its output time (t2) is equal to the luminescence intensity. Correspondingly, the strobe device (
10) is configured to turn on a photocoupler (92) that transmits a light emission signal to the strobe device (10) so that the strobe device (10) emits light.

That is, the strobe device (10) synchronizes with the imaging of 100 planes by the camera (1), and the vertical synchronization signal (
It is automatically emitted at a predetermined illuminance each time during the retrace period of VD).

The overall operation will be explained below based on the flowchart shown in FIG.

That is, when the power is turned on, the control processor (C
PUI) is for each memory (7), (14), (1
6), and initializes the contents and operation data name, etc., and holds the host computer (CPU) in a self-hold state while waiting to receive a start signal from the host computer (CPU (1)). After confirming that the image processing is being held, a start flag for image processing is set, the start of processing is notified to the control processor (CPUI), and the process waits for the control processor (CPU3) to complete the processing.

When the start flag is set, the control processor (CPU, ``) starts image capturing processing by the camera (1), takes in image information (S0) for one field, and converts the color difference signal ( The gray scale image information (S2) masked with the binarized color image information (F1) of R-8) is read and subjected to labeling processing for binarization and processing such as calculating the center of gravity position of the recognition target, and then returns to a standby state. Become.

On the other hand, the host computer (CPUO) confirms that the control processor (CPUO) is in a standby state, reads out the calculation result, resets the start flag, and restarts the control processor (CPUI). Then, repeat the above operations as many times as necessary.

Next, based on the flowchart shown in FIG. 2, the digital gradation image information (s2) is binarized, and the binarized image information (s2) corresponding to only the fruit in which each of the plurality of fruits has been separated ( The operation to obtain F3) will be explained.

That is, when the start flag is set, the binary color signal (p1) output from the color separation circuit (3)
The luminance signal (Y) for one field is passed through the gate circuit (4) which is opened and closed by the gate circuit (4) and the A/D converter (5), and the brightness signal (Y) for one field is masked with the corresponding color (C) of the fruit. (S2) and is input to the first image memory (7).

Then, using the masks shown in Fig. 9 (a') and ([j), the brightness change of the central pixel U, i) for the three pixels on the top, bottom, left and right is differentiated to a binary value based on the method of rsobel 0peretorJ. become After that, the area with a small area is erased (the value is set to "O"), the remaining area of ``1'' is thinned, the edge image information (F) of the grayscale image is extracted, and the second image memory is 16).

The edge image information (F2) is converted into edge image information connected by continuous lines by performing line thinning processing. Incidentally, as this thinning process, for example, the 10th
As shown in figure (a), as in the case of the differential processing,
This is performed by calculating the relationship between the pixel of interest (indicated by a circle in the figure) and the vertically and horizontally adjacent pixels (indicated by an asterisk in the figure). In other words, as shown in Figure 10 (0),
The number of connections (Nc) at the pixel of interest (X0) whose value is "1' is the value (r
It is represented by the following formula using (x + ) to f (X8) ).

however, j = (1, 3, 5, 7), X9 = XI.

Therefore, NC(X6) takes a value of O to 4, but when it is 1, Xo can be erased. (Connectivity is maintained even if deleted) However, it represents the terminal part of a thin line ('a width 1 line) as shown in Figure 10 (C), (D), (N), and (F). In such cases, do not delete it. Then, by sequentially erasing the erasable parts of each pixel, it is possible to make the line thinner.

Then, this thinned edge image information (F2)
Separation 2 in which each of the clustered fruits is separated by subtracting from the binarized color image information (F1).
Convert to digitized image information (F3).

Next, using the mask shown in FIG. 9 (8), the separation 2
The center pixel of interest (j,
i) is converted into binarized image information in which multiple fruits are completely separated by performing image contraction processing based on the state of each pixel that is adjacent to each other in the upper, lower, left, and right directions. In other words, the pixel of interest (j
+ Any of the pixels adjacent to the top, bottom, left and right of i) is “
0", the pixel of interest (j, i) is erased as O", and if all three pixels on the top, bottom, left and right are "1", the pixel of interest (j, i) is set to "1". It is processed in this way.

Thereafter, by performing so-called labeling processing on the binarized image information (F3), the number of recognized fruits, their sizes, etc. are calculated, the center of gravity position of each recognized fruit is calculated, and the results are sent to the host. It is transmitted to the computer (CPU0).

[Another example]

In the above embodiment, the binarized color image information (F3) itself is not stored, but is restored as necessary from the grayscale image information (S2) masked with this binarized color image information (F3). However, an image memory for separately storing the binarized color image information (F3) may be provided. In this case, a first image memory (7) that stores the grayscale image information (S2)
The stored image information may be sequentially processed and converted into separated binary image information (F3). In addition, in any case, the required image memory is 2 in addition to the image memory for calculation.
This is smaller than the case where both valued color image information (F3) and gradation image information (S2) are stored.

[Brief explanation of the drawing]

The drawings show an embodiment of the fruit recognition device according to the present invention.
The figure is a block diagram showing the overall configuration of the fruit recognition device, FIG. 2 is a flowchart showing the operation of the fruit recognition device, FIG. 3 is a flowchart showing the relationship between the control processor of the fruit recognition device and the host computer, and FIG. FIG. 5 is a block diagram showing the configuration of the color separation circuit, FIG. 5 is a time chart showing the operation of the gate circuit, FIG. 6 is a circuit diagram showing the configuration of the strobe control circuit, FIG. 7 is a time chart showing its operation, and FIG. Figures (a) to (d) are explanatory diagrams of image information, and Figure 9 (a)
, (a), (/1) is an explanatory diagram of an image processing mask. FIGS. 10A to 10F are explanatory diagrams of the thinning process. (1)...imaging means, (3)...color binarization means, (4)...gate means, (C)...
... Fruit color, (s0) ... Captured image information, (Y
)... Grayscale image information of the captured image, (S1)...
...Shade image information of fruit color range, (PI)...
...binarized image information of fruit color, (F2), old...edge image information of grayscale image, (F3)...separated binary image information.

Claims (1)

[Claims] Color binarization means (for separating and extracting binarized color image information (F_1) of a region corresponding to the recognition target fruit color (C) from the color image information of the image information (S_0) captured by the imaging means (1); 3) an edge image detection means for separating and extracting binarized edge image information (F_2) corresponding to a portion of the captured image information (S_0) in which a change in brightness is larger than that of the grayscale image information (Y); Binarized color image information (F_1
) is provided with a subtraction means for subtracting the binarized edge image information (F_2) from the captured image information (S_0) to obtain binarized image information (F_3) in which a region corresponding to the fruit to be recognized is separated and extracted from the captured image information (S_0). A fruit recognition device that obtains the above-mentioned 2.
By controlling opening/closing of the digitized color image information (F_1) as an opening/closing control signal, the gradation image information (Y) is masked with the fruit color (C), and the fruit color of the gradation image information (Y) is masked. Grayscale image information (S_
1), and is configured to input gray scale image information (S_1) extracted by the gate means (4) to the edge image detection means.