JPS63253204A - Imaging type boundary detection device - Google Patents

Abstract

PURPOSE:To decrease the number of picture elements to be processed and to speed up image processing by finding the inclination of the brightness varying direction of each quantized picture element of image information and setting the inclination range as a border detection range. CONSTITUTION:This detector is provided with potentiometers R1 and R2 for steering position detection which detect the steering positions of front and rear wheels 1 and 2 and a potentiometer R3 for speed change position detection which detects the operation state of a transmission 8. A controller 11 allows a vehicle body to travel automatically along the border based on pieces of detection information of the potentiometers and the image pickup information of an image sensor according to information on a straight line corresponding to the border and the detection information of an engine speed sensor S2. The border is detected by finding the inclination of the brightness varying direction of each quantized picture element of the image pickup information of the image sensor S1 and setting the inclination range as the detection range.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an imaging type boundary detection device that detects a straight line corresponding to the boundary between an untreated work site and a treated work site.

(Prior Art) This type of imaging type boundary detection device described above is used to detect control information for automatically driving a work vehicle along a boundary, and is used to detect control information for automatically driving a work vehicle along a boundary. Taking advantage of the fact that the brightness of the processed work area appears different, by processing the captured image information, we can draw a straight line that passes through areas with large brightness changes between the untreated work area and the treated work area. This is so that it is detected as information corresponding to boundaries.

Conventionally, the work area is imaged in two dimensions, and from the image information, the differential value of brightness in one direction that intersects the boundary is calculated for each quantized pixel arranged in the secondary direction. , Based on the differential value, pixels with large brightness changes are extracted by binarization processing, and using Hough transform processing, for each extracted pixel, the direction passing through that pixel and the set angle Find a large number of straight lines, each with a different direction, and choose the one with the highest frequency among all the straight lines found,
It was designed to be detected as information indicating a straight line corresponding to the boundary.

[Problem that the invention seeks to solve]

In the above conventional configuration, all binarized pixels are subjected to Hough transform processing, and in Hough transform processing, for each pixel to be processed, a large number of straight lines passing through that pixel are Due to the need to find the straight line that has the highest frequency among the large number of straight lines that have been found, the amount of calculation processing in image processing becomes extremely large. It was difficult to speed up the process.

By the way, the information on the detected straight line is used as control information for automatically driving the work vehicle along the boundary between the untreated work area and the treated work area, so the traveling speed of the work vehicle may be affected. It is necessary to process images at corresponding processing speeds, and faster image processing is desired.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to speed up image processing by reducing the amount of arithmetic processing.

[Means for solving problems]

The characteristic configuration of the imaging-type boundary detection device according to the present invention includes an imaging means for imaging a work area in two-dimensional directions, and imaging information of the imaging means to detect two orthogonal directions of each quantized pixel arranged in the two-dimensional direction. Differentiating means for determining the differential value of each brightness;
A slope calculating means for calculating a slope corresponding to the maximum brightness change direction from the obtained differential values in each of the two orthogonal directions, and a brightness change is large and set based on the obtained differential value and the slope. A pixel extraction means for extracting pixels within the slope range, a straight line calculation means for calculating a straight line orthogonal to the slope of each pixel extracted by the pixel extraction means, and a maximum of the straight lines found by the straight line calculation means. The present invention is provided with straight line extraction means for extracting frequencies, and their functions and effects are as follows.

[For production]

That is, for each quantized pixel of image information captured in two-dimensional directions, the slope of the maximum brightness change direction is determined, and only those pixels whose slope is within the set range and whose brightness changes are large are determined. Then, for each extracted pixel, find a straight line perpendicular to the previously found slope, and of those found straight lines, choose the one with the highest frequency that corresponds to the boundary. It is extracted as a straight line.

Incidentally, the reason why a pixel whose slope is within a set range can be specified as a target pixel for boundary detection is because the boundary detection device of the present invention is applied to steering control of a work vehicle, and therefore there is no difference between the boundary and the work vehicle. This is because the slope of will not deviate significantly.

〔Effect of the invention〕

Therefore, the slope of the direction of brightness change for each quantized pixel of image information captured in two-dimensional directions is calculated before the process of calculating a straight line passing through each pixel, and the range of the slope crosses the boundary. Only pixels that are within the detection range and have a large brightness change are extracted as pixels for detecting a straight line, so the number of pixels to be targeted when performing the process of finding a straight line can be significantly reduced. . Furthermore, the process of finding a straight line now requires only finding one straight line that is perpendicular to the previously found slope, and the amount of calculations required to find a straight line has been reduced from the amount of calculation required for finding a straight line, compared to the conventional method of finding many straight lines that pass through each pixel. This can be significantly reduced compared to those that require a straight line. Moreover, the amount of calculation required to find the maximum frequency can be significantly reduced compared to the conventional method. As a result, it is possible to speed up image processing as a whole, making it extremely useful for use in automatic driving of work vehicles.

〔Example〕

Embodiments of the present invention will be briefly described below with reference to the drawings.

As shown in Fig. 7, a pair of left and right front and rear wheels (1).

(2) are provided so that the steering can be operated freely, and a lawn mower (3) is suspended from the lower abdomen of the vehicle body (V) so as to be movable up and down. The work vehicle used is configured.

An image sensor (Sl) serving as an imaging means for capturing an image of the work area in two-dimensional directions is provided so as to image the work area on the front side with respect to the vehicle body (V).

To explain the imaging field of view of the image sensor (Sl), the vehicle body (V) is aligned in an appropriate state with respect to the boundary (L) between the untreated work area (B) and the treated work area (C). In this state, the boundary (L) is positioned at the center of the field of view in the width direction.

To explain the structure of the vehicle body (V), as shown in FIG. 6, there is a hydraulic cylinder (4) for steering that operates the front and rear wheels (1) and (2) separately.

(5), and control valves (6), (7) for it
The engine (E)
The transmission motor (9) is operatively connected to the transmission arm (10) of the transmission (8).

Further, in order to detect the traveling distance of the vehicle body (V), a rotation speed sensor (S2) that outputs a set number of pulse signals per unit rotation speed is rotationally driven by the output of the transmission (8). It is set up like this.

Steering position detection potentiometers (1) and (2) for detecting the respective steering positions of the front and rear wheels (1), (2)
R1), (Rz) and a shift position detection potentiometer (R3) that detects the operating state of the transmission (8), and the detection information thereof and the image sensor (S
,), the vehicle body (V) moves along the boundary (L) based on the information on the straight line corresponding to the boundary (L) obtained based on the imaging information of A control device (11) using a microcomputer is provided to control the vehicle to run automatically.

In the figure, (H) is a steering wheel for boarding operation, (Ro) is a potentiometer for detecting its operation position, (
12) is a speed change pedal for boarding operation.

However, various means for performing image processing on image information captured by the image sensor (S,) and detecting a straight line corresponding to the boundary (L) are configured using the control device (11). There is.

To explain the automatic driving of the vehicle body (V), as shown in FIG. As one work process, the untreated work area (B) and the treated work area (C) on the side where the vehicle body (V) is along the length of the work process.
The steering will be controlled based on the information detected by the image processing so as to automatically travel along the boundary (11) at the end of one work process.
2) In other words, as the opposite side of the unprocessed work area (B) is reached, the process of automatically turning toward the starting end of the next work process that intersects with the work process is repeated, resulting in a so-called circular cutting style. The lawn mowing work in a predetermined area will be automatically performed.

To explain the operation of the control device (11), as shown in FIG. , a boundary detection process to be described later is performed, and based on the detection information, it is determined whether or not the boundary (L2) on the terminal side is detected.

When the boundary (L2) on the terminal side is detected, the terminal position relative to the current position of the vehicle body (V) is calculated based on the position information on the image, and the front and rear wheels (L2) are moved to the calculated position. By maintaining the steering in the neutral state in 1) and (2), the vehicle is driven straight ahead and then turned toward the next work process based on a turn pattern that has been set and stored in advance.

If the terminal side boundary (L2) is not detected,
In order to automatically travel along the detected stroke side boundary (Ll), the steering control is performed by calculating the deviation with respect to the stroke side boundary (L,).

Then, for example, by determining whether a preset number of strokes has been traveled or whether a preset distance has been reached, it is determined whether the work is complete or not. In this case, the processes after the boundary detection process described above will be repeated. On the other hand, in the case of finishing the work, the vehicle body (V) will be stopped and the entire process will be completed.

To explain the calculation of the deviation with respect to the boundary (L,) on the stroke side, the deviation in the position in the width direction of the vehicle body,
Although there is a deviation in the inclination with respect to the direction of the boundary (Ll), these may be calculated as quantitative values, or only the direction of deviation may be calculated.

Further, to explain the steering control processing, the parallel steering type corrects positional deviations in the width direction of the vehicle body, and the four-wheel steering type corrects directional deviations. However, the steering amount of the front and rear wheels (1) and (2) may be different to correct the positional and orientation deviations at the same time.

Next, the boundary detection process will be described in detail based on the flowchart shown in FIG.

However, the boundary detection processing described below is performed by utilizing the phenomenon that the untreated work area (B) appears darker than the treated work area (C).

That is, as the boundary detection process is started, the image sensor (S) performs the imaging process, and the captured image information is quantized in accordance with a preset pixel density.

Next, as shown in FIG. 3, the pixel to be processed (e
) including 8 adjacent pixels (a = d, fxi)
Using a 3x3 pixel mask that covers the pixels, perform the following (i)
, (ii), the differential value of the brightness change in the X-axis direction and the y-axis direction on the image (SX) , (S
The process of calculating Y) is performed for each pixel arranged in a two-dimensional direction.

5X(x, y) = (c+2f+i) −(a+2
6+g) −...(i) SY(x, y) =
(g+2h+i)-(a+2b+c)-...(i
i) Note that the process of calculating the differential values (SX) and (SY) of brightness changes in the X-axis direction and the y-axis direction corresponds to the differentiating means (100).

Then, as shown in equation (iii) below, by adding the absolute values of the above-mentioned differential values (SX) and (SY), a differential representing the magnitude of the brightness change of each pixel (x, y) is calculated. Find the value (JP).

JP(x,y)=IsXI+lsy+...
(iii) Furthermore, using the following formula (iv), from each of the differential values C3X) and (SY), the slope (corresponding to the direction of maximum brightness change for each pixel (x, y))
BP) is calculated.

0P(x,y)=tan-'(SY/SX)...
(iv) However, the tilt (BP) is defined as an angular change in the clockwise direction with respect to the X-axis.

Note that the process of calculating this slope (BP) is carried out by the slope calculation means (
101).

Then, by performing binarization to extract only pixels whose differential value (JP) is greater than or equal to a preset threshold value, pixels at locations where brightness changes are large are extracted.

Next, as shown in FIG. 4, based on the value of the inclination (BP), pixels whose inclination is within ±30 degrees with respect to the The number of target pixel candidates is reduced in advance.

To explain, as mentioned above, the vehicle body (V) travels each work process along the boundary (L) between the untreated work area (B) and the treated work area (C). Therefore, even if the orientation of the vehicle body (V) or the position in the width direction with respect to the boundary (L) shifts, it will not shift significantly. Therefore, the slope of the direction of maximum brightness change of the pixel to be processed Even if only those for which the angle is within ±30 degrees are to be processed, the boundary (Ll) on the stroke side can be accurately detected.

Then, for each pixel to which the angle restriction has been applied, a straight line that passes through that pixel and is orthogonal to the slope (BP) is determined using Hough transform processing, as shown in equation (v) below, and Take that frequency in a histogram, and from that histogram, find the single straight line that gives the maximum frequency.
The straight line is the boundary (L
, ) is detected as a straight line corresponding to the line.

ρ=x-cosθ+y-sinθ... group (v) Next, based on the value of the slope (BP), the direction near 270 degrees (240 degrees or more) is perpendicular to the boundary (Ll) on the stroke side. 300 degrees) are re-extracted. That is, the terminal side boundary (tz) is in a direction perpendicular to the working stroke side boundary (Ll), and as described above, the vehicle body (V) is located at the boundary (L,).
Since the steering is controlled to travel along the working path, the range of inclination is wide (there is no difference. Therefore, as in the case of detecting the boundary (Ll) in the direction along the work process, By adding angle restrictions and reducing the number of pixels to be processed, we aim to speed up image processing without reducing accuracy.

As shown in FIG. 5, the slope (B
A straight line that is perpendicular to P) and passes through the pixel is obtained by the Hough transform process described above, and from these straight lines, the one that has the highest frequency is extracted, thereby forming the terminal side boundary (L2 ) is detected.

In addition, in each of the process of determining the straight line corresponding to the boundary on the stroke side (Ll) and the process of determining the boundary on the terminal side (L2), if the differential value (JP) is larger than the preset darkness value, The pixel extraction means (102) performs a binarization process to extract only the pixels that are the same, and a process to extract pixels whose slope (BP) is within a set range from the binarized image. It will be configured.

Further, the process of finding a straight line for each pixel using Hough transform processing corresponds to the straight line calculating means (103), and the process of extracting the one with the highest frequency among the found straight lines corresponds to the straight line extracting means (104). ).

Furthermore, in this boundary detection process, the screen used for explanation is a virtual one, and each of the boundaries (Ll),
The straight line corresponding to (tz) is not actually drawn, but the value (ρ
) and the value of the angle (θ) substituted from the slope (BP), which is the value thereof, are used as information corresponding to the detected straight line. Then, in the process of calculating the deviation with respect to the boundary and the calculation of the end position, these values (ρ) are used.

(θ), the actual boundary (L
l) and (tz) will be calculated.

To explain how to determine whether or not the terminal side boundary (L2) has been detected, since the aforementioned boundary detection process is performed for each set distance, the terminal side boundary (L2) detected in the current boundary detection process is The position of L2) is the position of the terminal side boundary (L2) detected in the previous boundary detection process,
By determining whether the detected straight line is near the position where the set distance is added in the longitudinal direction of the vehicle body, it is determined whether the detected straight line is the boundary (Lx) on the terminal side.

By the way, since the image sensor (Sl) is installed so as to image the work area on the front side of the vehicle body from diagonally above, each boundary (L) obtained from the captured image information is
The positional changes of l) and (Lt) on the screen are inversely proportional to the distance to the actual positions of the boundaries (Ll) and (LZ) with respect to the vehicle body (V), and become smaller at far distances.

Therefore, in each of the determination of the terminal side boundary (L2), the calculation of the distance to the terminal side boundary (L2), and the calculation of the deviation from the detected stroke side boundary (L,), The detected position is converted into an actual position with respect to the vehicle body (V).

[Another example]

In the above embodiment, the case where the end detection is performed at the same time every time the boundary detection process is performed is illustrated, but for example,
The detection information of the rotation speed sensor (s2) may be used to perform the process only when the vehicle body (V) approaches the end of the working stroke.

Further, in the above embodiment, the pixel extraction means (102) is configured to perform the following operations based on the magnitude of the differential value (JP) obtained by adding the differential value (SX) in the X-axis direction and the differential value (SY) in the y-axis direction. Although the case where the binarization is performed is illustrated, for example, when detecting the boundary (L,) on the travel side, the above-mentioned X Only the axial differential value (SX) may be used. In addition, the terminal side boundary (L2
), the differential value in the y-axis direction (
SY) may be used.

Furthermore, in the above embodiment, the pixel extraction means (102) first binarizes all pixels and then re-extracts pixels within the set range. Before that, pixels within a set range may be extracted.

Further, in the above embodiment, a straight line calculating means (103) for calculating a straight line corresponding to the slope (BP) of each extracted pixel is
), the case in which Hough transform processing is used has been exemplified, but the specific configuration of the linear calculation means (103) can be modified in various ways.

Further, in the above embodiment, the present invention is used as a means for automatically driving a lawn mowing work vehicle, but
The present invention can be applied as means for detecting the boundaries of various work areas, and the specific configuration of each part can be modified in various ways.

Incidentally, although reference numerals are written in the claims section for convenience of reference to the drawings, the present invention is not limited to the structure of the attached drawings.

[Brief explanation of the drawing]

The drawings show an embodiment of the imaging type boundary detection device according to the present invention, and FIG. 1 is a flowchart of boundary detection processing, FIG. 2 is a flowchart showing an outline of travel control of a work vehicle, and FIG. 3 is a flowchart of differential processing. 4 and 5 are explanatory diagrams of boundaries, FIG. 6 is a block diagram showing a control configuration, FIG. 7 is an overall side view of the working vehicle, and FIG. 8 is an explanatory diagram of the working area. CB)...Untreated work area, (C)...
Treated work area, (L)... Boundary, (S,)...
...imaging means, (SX), (SY)...
・Differential value, (BP)... Slope, (100)...
... Differentiation means, (101) ... Slope calculation means, (102) ... Pixel extraction means, (103)
. . . Straight line calculation means, (104) . . . Straight line extraction means.

Claims (1)

[Claims] Boundary (L) between untreated working area (B) and treated working area (C)
) is an imaging-type boundary detection device that detects straight lines corresponding to
S_1), a differentiating means (1) that calculates the differential values (SX) and (SY) of brightness in two orthogonal directions of each quantized pixel arranged in a two-dimensional direction from the imaging information of the imaging means (S_1).
00), the obtained differential values in each of the two orthogonal directions (SX
), (SY), a slope calculating means (101) for calculating a slope (BP) corresponding to the maximum brightness change direction, based on the calculated differential value (SX), (SY) and the slope (BP). A pixel extraction means (102) extracts pixels with a large brightness change and within a set slope range, and the slope (BP) of each pixel extracted by the pixel extraction means (102).
), a straight line calculating means (103) for calculating a straight line orthogonal to the straight line calculating means (103), and a straight line extracting means (10
4) An imaging type boundary detection device comprising each of the above.