JPH05265546A - Crop string detecting device - Google Patents

Abstract

(57) [Abstract] [Purpose] To secure the detection accuracy when calculating the line segment corresponding to the crop row based on the color image information. A color type image pickup means S ₁ for picking up a predetermined range of field scenes including a plurality of crops T arranged in a line, and color image information by the color type image pickup means S ₁ ,
Pixel extraction means 10 for extracting the pixel Ta corresponding to the crop T
0 and a calculation means 101 for obtaining a line segment L connecting the pixels Ta based on the pixel information extracted by the pixel extraction means 100, and a lightness signal Y obtained by the color type image pickup means S ₁ is predetermined. For a pixel Tak having a value larger than the value Yd, the pixel extraction by the pixel extraction unit 100 is prohibited by the pixel extraction prohibition unit 102, and the calculation unit 101 sets only the pixel Tan whose pixel extraction is not prohibited as the calculation target. It is characterized in that it is configured.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is based on color image pickup means for picking up an image of a field scene in a predetermined range including a plurality of crops arranged in rows, and color image information by the color image pickup means. The present invention relates to a crop row detecting device provided with a pixel extracting means for extracting pixels corresponding to the crop and a computing means for obtaining a line segment connecting the pixels based on the pixel information extracted by the pixel extracting means.

[0002]

2. Description of the Related Art A crop row detecting device of the above type has a structure in which, when a seedling as a crop is planted in a field in set intervals at a set unit, such as a rice planting machine, the machines are aligned in the machine traveling direction. In order to obtain control information for automatic traveling along the seedling row, the line segment connecting the pixels extracted corresponding to the seedling row on the imaging screen is approximated to a straight line or a curve and the seedling row that has already been planted for the machine position The control information is obtained by detecting the position and direction of the.

In order to extract pixels corresponding to the seedling row, the three primary color signals R, G, B included in the color image information from the seedling and the color image information from the mud surface or the water surface other than the seedling are used. Utilizing the fact that the ratios of the signal components of the three primary color signals R, G, B included are different, for example, that the seedling portion contains a large amount of green signal G but the other red signal R and blue signal B are small. Then, these three primary color signals R, G, B are color-calculated by a predetermined calculation formula to determine whether or not the pixel corresponds to the seedling row.

[0004]

However, the above prior art does not take into consideration that the image pickup performance of the color type image pickup means such as a CCD camera is deteriorated when the brightness of the image pickup object is excessive. It was In other words, the three primary color signals R, G, B obtained from the color type image pickup means have an appropriate color balance when the brightness of the image pickup object becomes excessive, as when direct reflection light from the water surface is picked up. The noise components mixed in the color signals R, G, and B increase as they are broken. Therefore, if the color calculation process is performed as it is when the brightness of the imaged object is excessive, an error occurs in the seedling row extraction, and an error also occurs in the line segment calculated from these extracted pixels, which causes the work vehicle to be cropped. Accurate steering control information may not be obtained when the vehicle automatically travels along the line.

The present invention has been made in view of the above circumstances, and an object thereof is to obtain a line segment corresponding to a crop row by arithmetic processing based on color image information from a color type image pickup means. To secure the detection accuracy of.

[0006]

According to a characteristic configuration of a crop line detecting device according to the present invention, for a pixel whose brightness signal obtained by the color type image pickup means is larger than a predetermined value, the pixel by the pixel extracting means is used. Pixel extraction prohibiting means for prohibiting extraction is provided, and the computing means is configured to target only pixels extracted by the pixel extracting means and not prohibited by the pixel extraction inhibiting means. There is a point.

[0007]

According to the characterizing feature of the present invention, a pixel whose brightness signal is larger than a predetermined value, for example, the brightness signal is excessive as in the case of reflected light from the water surface, the color balance is disturbed and the noise component is reduced. The large pixels are prohibited from being extracted as the pixels corresponding to the crop, and the pixels whose brightness signal is smaller than the predetermined value are extracted as the pixels corresponding to the crop. Then, when obtaining a line segment connecting pixels corresponding to crops, the pixels for which the above-mentioned pixel extraction is prohibited are excluded from the calculation target, and only the other pixels are targeted for the calculation, so that they correspond to the crop row. Therefore, the detection accuracy of the approximated line segment that is obtained can be secured.

[0008]

Therefore, the accuracy of detecting the approximate line segment corresponding to the crop row can be ensured, and the information can be effectively used as the steering control information when the work vehicle is automatically traveling.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a device for detecting the position of a seedling row planted in a field by a rice transplanter will be described below with reference to the drawings.

As shown in FIGS. 7 and 8, a seedling planting device 2 is provided to be movable up and down behind a machine body V in which both front wheels 1F and rear wheels 1R are steerable, and the seedlings are planted. In the device 2, a plurality of planted seedlings T as a plurality of crops arranged in rows are planted. A color type image sensor S ₁ as a color type image pickup means for picking up an image of a field in a predetermined range including these seedlings T is provided on the front side of the machine body V.

Explaining the mounting structure of the image sensor S ₁ , the base member is fixed to the seedling planting device 2 located at the rear of the machine V, and the support member 4 extending laterally to the front of the machine is provided. Image sensor S on the tip of
₁ is attached to the machine body V so as to capture an image of a row of planted seedlings adjacent to the machine body laterally outward from obliquely above. That is, in a state where the machine body V is properly along the row of the plurality of planted seedlings T arranged in the machine body traveling direction, the line segment L corresponding to the planted seedlings Tn adjacent to the unplanted side region is The image sensor S ₁ is arranged in such a state that it coincides with a traveling reference line La that passes through the center of the image pickup visual field in the front-rear direction. The seedling planting device 2 is installed on the mud surface of the field via the float 13, so that even if there is pitching or rolling of the machine body V, the effect thereof is removed as much as possible. The image pickup posture with respect to is stabilized.

A plurality of work strokes from one end side to the other end side of the field are set in a state of being arranged parallel to the machine lateral width direction. In each work stroke, based on the image information of the image sensor S _1. The steering control is carried out so that the vehicle automatically travels along the already planted seedling row. However, as it reaches the end of one work stroke, it is automatically turned in a state where the direction is changed by 180 degrees toward the start end of the next work stroke adjacent to the work stroke.

Therefore, the traveling direction of the machine body V with respect to the field is reversed every time the vehicle travels one stroke, and the position of the planted seedlings T with respect to the machine body V is laterally reversed.
One image sensor S ₁ is provided on each of the left and right sides of the machine body V, and the sensor on the side to be used is switched between the left and right for each stroke. In addition, in FIG. 7, the position of the planted seedling T is
Since it is on the right side of the airframe V, the image sensor S ₁ on the right side is used.

Explaining the structure of the machine body V, as shown in FIG. 1, the output of the engine E is transmitted to each of the front wheel 1F and the rear wheel 1R via a transmission device 5, and the transmission device 5 operates. as the operation state of the speed change operation state corresponds to a preset speed, the shifting state detection potentiometer R ₃ provided, and, on the basis of the detection information of the shifting state detecting potentiometer R _3, shift The electric motor 6 is driven.

Further, the front wheel 1F and the rear wheel 1R are constructed so that power steering is individually operated by hydraulic cylinders 7F and 7R, respectively, and steering angle detecting potentiometers R ₁ and R interlocked with the steering operation of the wheels. _The electromagnetically operated control valves 8F and 8R for operating the hydraulic cylinders 7F and 7R are driven so that the detected steering angle by ₂ becomes the target steering angle.

Therefore, a parallel steering system for steering the front wheels 1F and the rear wheels 1R at the same phase and at the same angle,
It is possible to selectively use three types of steering types: a four-wheel steering type in which the front wheels 1F and the rear wheels 1R are steered in opposite phases and at the same angle, and a two-wheel steering type in which only the front wheels 1F are turned. Is becoming

However, when the steering operation is automatically performed based on the image information of the image sensor S ₁ , the two-wheel steering type is used, and when one work process is finished and the next work process is performed, The four-wheel steering type and the parallel steering type are used. In FIG. 1, S ₂ is a distance sensor for detecting the traveling distance based on the output speed of the transmission 5.

Next, a control configuration for approximating the line segment L corresponding to the boundary between the unplanted side seedling row and the already planted side seedling row based on the image information of the image sensor S ₁ will be described. ..

As shown in FIG. 1, the image sensor S
₁ is configured to output the three primary color information R, G, B and the lightness signal Y separately, and from the green information G including the color component of the seedling T to the blue information B not including the color component of the seedling T. The pixel T corresponding to the seedling T is obtained by subtracting and binarizing.
It is configured to extract a.

In addition, a subtracter 9 for subtracting the blue color information B from the green color information G in the form of an analog signal,
The comparator 10 for binarizing the output of the subtractor 9 based on a preset threshold value corresponding to the color of the seedling T and outputting the binarized information corresponding to the pixel Ta, by the image sensor S ₁ . The brightness signal Y obtained is a predetermined value Yd.
An image memory 11 that stores the output signal of the comparator 10 as image information corresponding to a preset pixel density (32 × 32 pixels / one screen) for a pixel Tan excluding a pixel Tak that is larger than A microcomputer that obtains information that approximates a line segment L connecting these pixels Tan to a straight line or a curve based on the pixel information corresponding to the seedling T stored in the image memory 11 and that controls traveling based on the information. Each of the usage control devices 12 is provided.

That is, the subtractor 9 and the comparator 10 make the pixel T corresponding to the color of the seedling T as a crop.
It corresponds to the pixel extracting means 100 for extracting a, and using the control device 12, the pixel Tak for which the lightness signal Y obtained by the image sensor S ₁ is larger than a predetermined value Yd. Extraction means 100
Pixel extraction prohibiting means 102 for prohibiting pixel extraction by the pixel extracting means 100 is configured, and the pixel extracting means 100 extracts the pixel using the control device 12 and the image memory 11.
In addition, the arithmetic means 101 is configured to obtain the line segment L connecting these pixels Tan by approximating it to a straight line or a curve, with only the pixels Tan that are not prohibited by the pixel extraction prohibiting means 102 as the object of calculation. ..

Next, based on the flowchart shown in FIG. 2, the operation of the control device 12 will be described, and the configuration of each unit will be described in detail.

Every time the machine body V travels a set distance or every set time, the image pickup process by the image sensor S ₁ is executed, and the seedling T is set by the pixel extracting means 100 and the pixel extracting prohibiting means 102. , And only the pixel Tan whose brightness signal Y is equal to or less than the predetermined value Yd is extracted.

To explain the extraction of the pixel Tan, since the color of the seedling T is green, focusing on the green information G of the three primary color information of the field image,
The value corresponding to the pixel which imaged the seedling T becomes larger than the value of the pixel which imaged other parts, such as the mud surface of a field. However, water surface W
Since the natural light is almost totally reflected, the reflected light contains all color components. Therefore, the value of the green color information G corresponding to the pixel of which the water surface W is imaged is larger than the value of the pixel of which the other portion is imaged, like the pixel of which the seedling T is imaged.

Focusing on the blue color information B of the three primary color information obtained by capturing the image of the field, the seedling T is greenish and the mud surface is brownish or grayish. The value of the blue component contained in is low. However, since the reflected light from the water surface W contains all color components, the value of the pixel which imaged the water surface W becomes large.

Therefore, when the blue color information B is subtracted from the green color information G, only the value of the pixel corresponding to the seedling T becomes larger than the pixel value of the other part, and the subtracted value is obtained. If the binarization is performed based on the set threshold value, pixels can be extracted corresponding to only the seedling T. However, if the lightness signal Y is excessive, such as direct reflected light from the water surface W, the image pickup performance of the image sensor S ₁ deteriorates and the color balance of the three primary color signals R, G, B is proper. Since the deviation from the state (see FIG. 4) and the mixing amount of the noise component increase, the pixel extraction is prohibited for the pixel Tak whose brightness signal Y is larger than the predetermined value Yd, and only the other pixels Tan are extracted. (See FIGS. 3A and 3B). In the figure, eight show a state in which the pixel Ta ₁ to Ta ₈ is extracted, the eight pixels in the Ta ₁ through T
a ₈ is actually the pixel density (32 × 32 pixels / 1
It may be one pixel in a screen), or may be one or more pixels.

Next, the extracted pixels Ta _{1 to} Ta _{8 are} extracted.
The line segment L connecting the two is approximated to a straight line or a curved line by the calculation means 101. Here, the line segment L is calculated as a straight line by a Hough transform process, which is a conversion unit for linear approximation.

[0028] If described Hough transform, as shown in FIG. 6, the x axis passing through the center of the imaging field of view of the image sensor S ₁ as a reference line in the polar coordinate system, each pixel Ta
_A plurality of straight lines passing through _{1 to} Ta ₈ are set to the origin θ, that is, the center of the imaging visual field, with inclinations θ preset in a plurality of stages in the range of 0 to 180 degrees with respect to the x-axis based on the following formula (i). It is obtained as a combination with the corresponding distance ρ from the center of the screen. ρ = y · sin θ + x · cos θ …… (i)

After repeating the process of adding a two-dimensional histogram for counting the frequency of each obtained straight line until the value of the inclination θ set in the above-mentioned plural stages reaches 180 degrees for one pixel. , Each of the pixels Ta _{1 to} T
The frequencies of a plurality of types of straight lines passing through a ₈ will be counted for every pixel.

When the counting of the frequency of the straight line for each of the pixels Ta _{1 to} Ta ₈ is completed, the combination of the inclination θ and the distance ρ that has the maximum frequency is obtained from the value added to the two-dimensional histogram. One straight line Lx having the maximum frequency (see FIG. 6) is determined, and the straight line Lx is
The line segment L connecting the planted seedlings Tn adjacent to the unplanted side region on the image pickup surface of the image sensor S ₁ is obtained as linearly approximated information.

Next, the straight line Lx on the image pickup surface is stored in advance by the memory information of the shape and size of the image pickup field A of the image sensor S ₁ measured on the ground surface, and the image pickup surface through which the straight line Lx having the maximum frequency passes. Based on the pixel positions a, b, and c at the positions (see FIG. 5 and FIG. 6). That is, as shown in FIG. 5, a straight line L on the ground surface that is set as a value of an inclination ψ with respect to a traveling reference line La that passes through the center of the imaging visual field A in the widthwise direction in the front-rear direction and a position δ in the widthwise direction. Will be converted into information.

To further explain, the front and rear positions (y = 16 and y =) of the imaging visual field A in the direction intersecting the straight line L corresponding to the line segment connecting the planted seedlings Tn adjacent to the unplanted side region.
-16) two side lengths l ₁ and l ₃₂ , the center of the screen (x = 1
6, the pixel position where y = 0), the length l ₁₆ in the lateral width direction of the imaging visual field A, and the distance h between the two front and rear sides.
Each of the above is measured in advance and stored in the control device 12.

Then, the straight line Lx on the imaging plane is
X-coordinate values X ₁ and X ₃₂ of positions a and b (positions where y = 16 and y = −16) of pixels intersecting the x-axis corresponding to two sides at the front and rear positions of the imaging visual field A, The value X ₁₆ of the x coordinate of the image position c at which the straight line Lx intersects the x axis passing through the center of the screen is obtained from the following equation (ii) which is a modification of the above equation (i). Xi = ([rho] -Yi * sin [theta]) / cos [theta] (ii) where Yi is 16,0, -16 respectively.

Then, based on the coordinate values on the x-axis obtained by the above equation (ii), the position δ in the lateral direction with respect to the traveling reference line La is calculated from the following equations (iii) and (iv).
And the slope ψ, the values of the calculated position δ and the slope ψ are
It is calculated as the position information of the straight line L corresponding to the above-mentioned seedling row on the ground surface.

[0035]

[Equation 1]

That is, the information obtained by approximating the line segment connecting the pixels Ta _{1 to} Ta ₈ selected and selected by the pixel extraction means 100 and the pixel extraction prohibition means 102 using the Hough transform as a straight line L is obtained. The processing to be performed corresponds to the calculation means 101.

Therefore, in the steering control for automatically traveling the machine body V along the row of the planted seedlings T arranged in the machine body traveling direction, the inclination ψ of the straight line L with respect to the traveling reference line La and the lateral width direction are The steering operation is performed in a two-wheel steering manner so that the position δ and the position δ both approach zero.

The steering control will be described from the values of the inclination ψ of the straight line L with respect to the traveling reference line La and the position δ in the lateral width direction, and the current steering angle φ of the front wheels 1F. The target steering angle θf of the front wheel 1F is set based on the following equation (v), and the current steering angle φ detected by the front wheel steering angle detection potentiometer R ₁ is set to the target steering angle φf. The control valve 8F of the front wheel hydraulic cylinder 7F is driven so as to be maintained within the set dead zone with respect to the orientation angle θf. θf = K ₁ δ + K ₂ ψ + K ₃ Φ (v) Incidentally, K ₁ , K ₂ and K ₃ are constants preset according to the steering characteristics.

The turn control for reaching the end of one work stroke and turning toward the beginning of the next work stroke will be described. The travel distance detected by the distance sensor S ₂ is one work stroke. Although it will not be described in detail, when the set distance set corresponding to the length is exceeded, it is detected from the image information of the image pickup means S _{1 that} the end of the work stroke is reached. Then, when it is determined that the end of the work process has been reached, the planting work by the seedling planting device 2 is interrupted, the two-wheel steering system is switched to the four-wheel steering system, and the maximum cutting is performed during the set time. By maintaining the angle, the direction is changed 180 degrees to the next work stroke side, and then the parallel steering type is changed to maintain the maximum turning angle for the set time.
The position will be corrected in the lateral direction of the aircraft for the next work stroke, and the turn will be ended. After the end of the turn, the two-wheel steering system is restored, and the steering control in the next work stroke is restarted.

[Other Embodiment] In the above embodiment, the color image sensor S ₁ is used as the color image pickup means for picking up the field scene, and the blue information B is subtracted from the green information G to be based on the set threshold value. The case where the pixel Ta corresponding to the seedling T is extracted by performing binarization by means of an example has been described. However, in addition to this, all three primary color signals R, G, and B are used and their ratio is An area within a set ratio range corresponding to the color of the seedling T may be extracted as the pixel Ta. More specifically, in the equation p * R + q * G + r * B, p, q, and r, which are the coefficients of the three primary color signals R, G, B, respectively.
The value of is set to an appropriate value, and the pixel corresponding to the seedling T is extracted depending on whether or not the value of the above expression is larger than the set threshold value. Incidentally, in the above embodiment, p = 0, q = 1, r =-
This corresponds to the case of 1.

Further, in the above embodiment, the support member 4 having the image sensor S ₁ attached to the front end is fixed to the seedling planting device 2 located at the rear of the machine body V at the base end, and the front end is fixed. Although it is configured to extend laterally to the front of the machine body, a structure may be used in which the support member 4 is fixed to a position on the front side of the machine body V.

Further, in the above embodiment, the case where the line segment L connecting the pixels Ta corresponding to the crop T is linearly approximated by using the Hough transform has been exemplified, but, for example, the least square method or the like is used. It is also possible to obtain information obtained by linear approximation or curve approximation, and the specific configuration of the arithmetic means 101 is as follows.
Various changes can be made.

Further, in the above embodiment, the present invention is applied to a device for automatically traveling a rice transplanter along a seedling row planted in a field, and an approximate line segment corresponding to the detected crop row. Although the information is exemplified as being configured to be used as control information for steering control, the present invention can be applied to an apparatus for detecting information on approximate line segments corresponding to various crop rows. Therefore, the specific configuration of each part such as the type of crop and the configuration of the traveling system of the machine body, and the usage pattern of the information of the approximate line segment corresponding to the detected crop sequence can be variously changed.

It should be noted that reference numerals are given in the claims for convenience of comparison with the drawings, but the present invention is not limited to the configurations of the accompanying drawings by the entry.

[Brief description of drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a flowchart of control operation.

FIG. 3 is an explanatory diagram of image processing.

FIG. 4 is a graph illustrating the imaging characteristics of a color imaging unit.

FIG. 5 is an explanatory diagram showing a relationship between a traveling direction of the body of the imaging field of view and an approximate straight line.

FIG. 6 is an explanatory diagram of Hough transform.

FIG. 7 is a schematic plan view of a rice transplanter.

FIG. 8 is a schematic side view of the same.

[Explanation of symbols]

L line segment S ₁ color type imaging means T crop Ta pixel Tak pixel Tan pixel Y brightness signal Yd predetermined value 100 pixel extraction means 101 calculation means 102 pixel extraction prohibition means

Claims (1)

[Claims] 1. A color type image pickup means (S ₁ ) for picking up an image of a field scene in a predetermined range including a plurality of crops (T) arranged in rows.
And the pixel (T) corresponding to the crop (T) on the basis of the color image information by the color type image pickup means (S ₁ ).
pixel extraction means (100) for extracting a), and calculation means (101) for obtaining a line segment (L) connecting the pixels (Ta) based on the pixel information extracted by the pixel extraction means (100). In which the brightness signal (Y) obtained by the color type image pickup means (S ₁ ) is larger than a predetermined value (Yd) (Ta).
Regarding k), a pixel extraction prohibiting means (102) for prohibiting pixel extraction by the pixel extracting means (100) is provided, and the computing means (101) is extracted by the pixel extracting means (100), and Pixels (Ta) that are not prohibited by the pixel extraction prohibiting means (102).
A crop row detection device configured to operate only n).