JPH10124687A - Road white line detection method and road white line detection device - Google Patents

Abstract

(57) [Summary] [Problem] To accurately recognize a road white line from a vehicle traveling on a road by image processing. SOLUTION: A road image is inputted from a road image photographing means 11, and an image having only a low spatial frequency component is generated by a blur image generating means 12. The area dividing means 13 divides the image according to the change in luminance, and the road area detecting means 14 detects the road area using the knowledge about the road. The edge image extracting means 15 generates an image of only a component having a higher spatial frequency than the road image, and the contour point estimating means 16 detects a candidate point corresponding to a white line of the road. The road contour point extracting means 17 detects a contour point of a road white line in the road area, and the curve detecting means 18 detects the road white line as a curve.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road white line detecting device for recognizing a white line on a road on which an automobile runs by image processing.

[0002]

2. Description of the Related Art Conventionally, a road white line detecting device is disclosed in
No. 5,341,104 is known. FIG. 9 shows a conventional road white line detection device. The white line detection device captures a road image ahead of the vehicle by the image input unit 1 and extracts edge points where the brightness changes, such as a white line or a road edge, based on a signal from the image input unit 1 by the preprocessing unit 2.

A window is set in the vicinity of the straight line equation at the time of the previous calculation by the straight line adapting means 3, the coordinates of a plurality of edge points in each of the set windows are measured, and a plurality of straight line equations are obtained by straight line approximation. . FIG.
Shows an example of an input image and a straight line formula.

A straight line / vanishing point determining means 4 passes through the x, y coordinates of the vanishing point and the vanishing point so that the sum of the squares of the errors between the straight line formulas is minimized based on the plurality of obtained linear equations. An amount corresponding to the inclination of each straight line is estimated, and the result is set as the current result. Here, a plurality of rain markers (white lines) are parallel, and the fact that the lane width and the lane marker width do not fluctuate greatly with time is used. The vehicle moves fast in the vertical direction, but the fast movement in the horizontal direction does not. By assuming that there is no line, based on the obtained linear equation, a temporal and spatial smoothing is performed in consideration of the above assumption to obtain a straight line and a vanishing point.

Next, an edge point corresponding to the lane marker is extracted by the edge point tracking means 5 based on the result of the preprocessing means 2 and the detection result of the straight line fitting means 3, and the curve fitting means 6 is used.
By applying a curve equation to the edge points extracted by the above, the curvature of the curve is detected, and the smoothing means 7 smoothes the result of the curve fitting means, thereby detecting a white line.

[0006]

However, in such a conventional road white line detecting device, curve fitting is performed by the least square method using only the edge points of the lane markers. Therefore, since all points where the brightness changes are detected as edge points, there is a disadvantage that noise is easily affected and it is difficult to accurately detect a white line.

SUMMARY OF THE INVENTION The present invention has been made in view of such a conventional problem, and corrects not only edge data but also low-frequency components of a road image by correcting the low-frequency components.
An object is to enable detection of an accurate white line.

[0008]

According to the first aspect of the present invention, a blurred image is generated by generating only data having a low spatial frequency from road image data, and the blurred image is generated in the blurred image generating step. Using the blurred image data, the image is divided into regions having substantially the same brightness, and a region including a predetermined position of the image is detected as a road region from the region divided data from the region dividing step. An edge image is extracted by detecting only components having high spatial frequencies independent of each other and not including a spatial frequency component generated in the blurred image generation step, and is estimated to correspond to a road white line from high frequency component data from the edge image extraction step. The estimated contour points to be estimated are estimated, and the contour point estimation step included in the road area detected by the road area detection step is performed. Estimation contour points obtained road white line only is extracted, using the white line edge points extracted by the contour point extracting step and detecting a white line.

According to a second aspect of the present invention, a road image photographing means for photographing a road ahead of a vehicle, and data of only low frequency components having a low spatial frequency among the road image data from the road image photographing means are generated. A blurred image generating unit, an area dividing unit that divides each of the regions having substantially the same luminance using image data obtained from the blurred image generating unit, and a method of determining an image from the area divided data from the area dividing unit. A road area detecting means for detecting an area including a position as a road area; an edge image extracting means for detecting a high frequency component having a high spatial frequency in the road image data from the road image photographing means; A contour point estimating means for extracting an estimated contour point estimated to correspond to a road white line from the high frequency component data; and a road area detected by the road area detecting means. Contour point extracting means for selecting only an estimated contour point of a road white line obtained from the contour point estimating means included in the above, and detecting a white line as a curve by a polynomial using the white line contour points extracted from the contour point extracting means And a curve detecting means.

[0010] In the invention according to claim 3 of the present application, the contour point extracting means includes the contour point estimating means which is included in the road area detected by the road area detecting means and is within a predetermined range from an end of the road area. The method is characterized by selecting an estimated contour point from the means.

In the invention according to claim 4 of the present application, the curve detecting means detects a spline curve passing through each white line contour point,
A plurality of spline interpolation formulas are calculated from a part of the white line contour points, and a curve is calculated by correcting a coefficient value of the spline curve based on a maximum coefficient value of the spline interpolation formula. .

According to the present invention having the above features,
A road image is photographed to generate a blur image having a low spatial frequency, and a region is divided from the blur image to detect a road region.
Then, separately from this, an edge image having a high spatial frequency is extracted from the road image, and a contour point of an area where a white line is considered to be drawn is extracted from the road image. Then, a white line on the road is detected based on the extracted contour points.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION

(Embodiment 1) FIG. 1 is a block diagram showing a road white line detecting apparatus according to an embodiment of the present invention. This road white line detecting device includes a road image photographing unit 11, a blur image generating unit 1
2, a region dividing means 13, a road area detecting means 14, an edge image extracting means 15, a contour point estimating means 16, a road contour point extracting means 17, and a curve detecting means 18.

The road image photographing means 11 is for inputting a road image in front of the vehicle, and uses a TV camera. Assuming that the TV camera is arranged substantially at the center in front of the vehicle, the bottom center R of the image obtained by the imaging means 11 includes a road as shown in FIG.
The blurred image generating means 12 generates an image of only a component having a low spatial frequency using the luminance data from the road image photographing means 11, and includes a two-dimensional low-pass filter (LP).
F). The area dividing means 13 uses the low-frequency image data from the blurred image generating means 12 and divides the image at once into areas having substantially equal luminance values from the luminance distribution. The road area detecting means 14 detects a road area by merging the areas divided by the luminance distribution of the low-frequency image using the knowledge about the road.

Here, the knowledge about the road means that the area including the point R at the center of the bottom of the image taken by the road image pickup means 11 is a road.
It is also possible to include that the vanishing point where the left and right white lines intersect does not change abruptly with time, or that the occupation of pixels on the road is almost constant. Accordingly, the road is defined as an area including at least the point R, or an adjacent area is added to the area.

On the other hand, the edge image extracting means 15 generates an image of only a component having a high spatial frequency using the luminance data from the road image photographing means 11, and is constituted by a two-dimensional high-pass filter (HPF). . The blurred image generating means 12 and the edge image extracting means 15 respectively separate the low frequency and the high frequency from the images captured by the road image capturing means 11 so that the selected frequencies do not overlap. The contour point estimating means 16 uses the high frequency image data from the edge image extracting means 15 to detect an estimated contour point which is a white line candidate corresponding to a white line of a road. The road contour point extracting means 17 detects the contour points of the road white line using the road area data detected by the road area detecting means 14 and the estimated contour points detected by the contour point estimating means 16. The curve detecting means 18 detects a road white line as a curve using a spline curve by using the contour points of the road white line from the road contour point extracting means.

Next, the operation of the thus configured road white line detecting apparatus will be described with reference to the image examples of FIGS. 2 and 3 and the flowchart of FIG. FIG. 2A is a luminance image from the image input unit. Now, in step S1,
The image data from the road image capturing means 11 is put in a two-dimensional low-pass filter (LPF), and the two-dimensional low-pass filter (LPF) is used as shown in FIG.
Radius ω = centered on the origin in the dimensional frequency plane (ω1, ω2)
Create low-frequency image data of ω0 or less. FIG. 2B is a diagram showing the blurred image thus obtained. Next, in step S2, a histogram based on luminance is created using the low-frequency image data. FIG. 6 shows an example of the luminance value histogram thus created. By the way, in FIG. 2A, a point R at the center of the bottom surface of the image reliably reflects the road, assuming that the TV camera is mounted substantially at the center in front of the vehicle. Therefore, when the luminance value at the point R is read from the luminance histogram, it can be estimated that the area including the luminance is a road. Therefore, in step S3, an inflection point of a frequency change is searched for in the direction of higher luminance and lower luminance around the position in the luminance 0 histogram of the luminance value of the central point A at the bottom of the image. In step S4, the range of the luminance value surrounded by the inflection point in the luminance image is obtained. FIG.
(C) shows an image thus divided into a plurality of regions. The above-mentioned knowledge about the road is added to the area shown in FIG. 2D as a road area. Here step S
Steps S2 and S3 achieve the function of the area dividing means 13 for dividing the blurred image into areas each having a small change in luminance.
5 achieves the function of the road area detecting means 14.

On the other hand, in step S5, the image data from the road image pickup means 11 is put into a two-dimensional high-pass filter (HPF), and the origin is centered on a two-dimensional frequency plane (ω1, ω2) as shown in FIG. , High-frequency image data larger than the radius ω = ω3 is created. FIG. 3E shows an edge image obtained from the luminance image. Next, step S6
In FIG. 7, an estimated contour point which is a candidate for a white line of a road exceeding a predetermined threshold th in the left-right direction from the center of the edge image is searched upward from the bottom of the image. In step S7, the estimated supplementary contour points that are not included in the road area determined in step S4 are excluded, and the estimated contour points that are inside the outer edge of the road area by a predetermined distance L or more are excluded. For example, in FIG. 2D, only a region within a predetermined distance L from the edge of the road region is set as a detection target, and a central portion of the road and a region other than the road are ignored. Here, the threshold value L is set to a value that gradually decreases as the distance from the host vehicle increases, that is, as the distance from the host vehicle increases. In this way, the edge of the region to be detected can be extracted as shown in FIG. Here, step S5 achieves the function of the edge extracting means 15, and step S6 achieves the function of the contour point estimating means 16 for searching for an estimated contour point. Step S7 achieves the function of the contour point extracting means 17 for extracting the contour points of the white line based on the road data and the data of the estimated contour points.

Next, in step S8, spline curves are calculated for each of the left and right using the remaining left and right contour points, for example, all N points. Note that the horizontal axis of the image is x and the vertical axis is y. The spline curve is expressed by the following equation.

[0020] ^{y = a m xm + am-} 1 xm-1 + ··· + a0 and regions of only ± alpha around the respective coefficients obtained a make the Hough space. The Hough space is an m-order space formed by the coefficients am to a0. FIG. 8 is a diagram illustrating an example of the Hough space when m = 2.

Next, the white line contour points corresponding to the left white line have values on the screen, for example, (x0, y0), (x1, y1), (x2,
y2) Many can be obtained as in. Assuming that the number is N, an arbitrary n (N> n) is selected from the N, and an approximate expression based on the above expression is obtained as a spline interpolation expression passing through the n points. An approximate polynomial is similarly obtained for the right white line.

For example, when m = 2, the above-mentioned polynomial is as follows. y = a ₂ x 2 + a 1 x + a 0 The coefficients a 0, a 1, and a 2 thus obtained fall into any one of the small areas obtained by dividing the area on the Hough space of ± α into a large number. Therefore, the number of times that data enters one of these small areas is A, and the number of repetitions is k.
That is, in step 10, 1 is added to the accumulator A of the corresponding small area on the Hough space defined in step 8 corresponding to the coefficient of this polynomial, and the number of repetitions k
Is also added to 1. If k is equal to or smaller than the upper limit value kmax of the number of repetitions in step S11, step S9 is repeated, and another n out of N contour points corresponding to the white line are selected, and a spline interpolation formula is similarly obtained. If it exceeds kmax, the process proceeds to step S12, where the accumulator A having the largest number is selected from the accumulators A in the Hough space, and the corresponding polynomial coefficient is obtained. In step S13, a polynomial formed by the coefficient is used as a formula on the road white line image. This curve detection is performed for the left and right white lines.

In this way, the left and right white line equations are calculated as shown in FIG. 3 (g). This equation is a curve showing the vehicle-side edge of the white line drawn on the road. Here, steps S8 to S13 achieve the function of the curve detecting means 18 for detecting a curve based on the contour points.

As described above, a low-frequency image and a high-frequency image having no spatial frequency overlap are created from the same luminance image data, and the road area and road white line candidate contour points are independently detected from each image. Highly accurate road white line detection can be realized.

(Embodiment 2) FIG. 11 is a block diagram showing a road white line detecting apparatus according to an embodiment of the present invention.
The road white line detecting device includes a road image photographing unit 111, a blurred image generating unit 112, a region dividing unit 113, an upper road region detecting unit 114, an edge image extracting unit 115, a contour point estimating unit 116, and a bottom road contour point extracting unit 117. , And lane setting means 118.

The road image photographing means 111 inputs a road image ahead of the vehicle, and uses a TV camera.
Assuming that the TV camera is arranged substantially at the center in front of the vehicle, the bottom center R of the image obtained by the imaging means 111 includes a road as shown in FIG. The blur image generating means 112 is a road image photographing means 111
This is to generate an image of only a component having a low spatial frequency using the luminance data from, and is constituted by a two-dimensional low-pass filter (LPF). The area dividing means 113 uses the low-frequency image data from the blurred image generating means 112 to collectively divide the image into areas having substantially equal luminance values from the luminance distribution.

The upper road area detecting means 114 detects the lane area by merging the areas divided by the luminance distribution of the low frequency image from the upper part of the image using the knowledge about the road. Here, the knowledge about the road means that the area including the point R at the center of the bottom surface of the image captured by the road image capturing unit 111 is a road, and in addition, the vanishing point where the left and right white lines intersect. May not include an abrupt change with time, or the size of a road occupying a substantially constant pixel. Accordingly, the road is defined as an area including at least the point R, or an adjacent area is added to the area.

On the other hand, the edge image extracting means 115 generates only an image having a high spatial frequency component using the luminance data from the road image photographing means 111, and is constituted by a two-dimensional high-pass filter (HPF). . The blurred image generating means 112 and the edge image extracting means 115 separate the low frequency and the high frequency, respectively, from the images captured by the road image capturing means 111 so that the selected frequencies do not overlap. The contour point estimating means 116 uses the high frequency image data from the edge image extracting means 115 to detect an estimated contour point which is a white line candidate corresponding to a white line of a road. The bottom road contour point extracting means 117 detects a road white line contour point candidate upward from the bottom of the image using the estimated contour points detected by the contour point estimating means 116 to detect a lane area. Traveling lane setting means 118
The lane area detected by the upper road area detecting means 114 and the lane area detected by the bottom road contour point extracting means 117 are combined and detected as one lane.

Next, the operation of the thus configured road white line detecting apparatus will be described with reference to the image examples of FIGS. 2 and 3 and the flowchart of FIG. FIG. 2A is a luminance image from the image input unit. In step S1, the image data from the road image capturing unit 111 is put in a two-dimensional low-pass filter (LPF), and a radius ω centering on the origin is set on a two-dimensional frequency plane (ω1, ω2) as shown in FIG. = Low-frequency image data of ω0 or less is created. FIG.
(B) is a diagram showing the blurred image thus obtained. Next, in step S2, a histogram based on luminance is created using the low-frequency image data. FIG. 6 shows an example of the luminance value histogram thus created.

In FIG. 2 (a), a point R at the center of the bottom of the image, assuming that the TV camera is mounted substantially at the center in front of the vehicle, certainly reflects the road. Therefore, when the luminance value at the point R is read from the luminance histogram, it can be estimated that the area including the luminance is a road. Therefore, in step S3, an inflection point of a frequency change is searched from the center of the brightness value of the brightness value of the center point A at the bottom of the image toward higher and lower brightness values. In step S4, the range of the luminance value surrounded by the inflection point in the luminance image is obtained.

FIG. 2C shows an image thus divided into a plurality of areas. The above-mentioned knowledge about the road is added to the area shown in FIG. Here, steps S2 and S3 achieve the function of the area dividing means 113 for dividing the blurred image into areas having small luminance changes, and step S5 achieves the function of the upper road area detecting means 114.

On the other hand, in step S5, the image data from the road image pickup means 111 is put into a two-dimensional high-pass filter (HPF), and the origin is centered on a two-dimensional frequency plane (ω1, ω2) as shown in FIG. , High-frequency image data larger than the radius ω = ω3 is created. FIG. 3E shows an edge image obtained from the luminance image.

Next, in step S6, a threshold value th is set for each region divided upward from the bottom of the image as shown in FIG. In step S7, a point exceeding the threshold th in each area is detected, and a distance and a direction from the previously obtained point are calculated. In step S8, if the change in the distance is larger than ΔL and the change in the direction is larger than ΔΘ, it is considered that the point is not a road contour point, and the process returns to step S7 to continue the search. In step S9, a region between the left and right road contour points detected in step S8 and sandwiched by these lines is defined as a bottom road lane region.

In step S10, the upper road lane area detected in step S4 and the bottom road lane area detected in step S9 are merged into one lane while allowing overlap.

As described above, the lane is detected by extracting the area at the top of the image, and the lane is detected at the bottom of the image by the edge data.

[0036]

As described above in detail, according to the present invention, the contour points of the road area and the road white line candidate are detected independently from the low-frequency image and the high-frequency image having no spatial frequency overlap. Therefore, compared to the conventional example in which the contour point is detected only from the high-frequency image, the effect of noise is less likely to be obtained, and the effect that the white line can be detected with high accuracy can be obtained.

Further, by changing the detection method for each place, the lane can be detected even in a bad environment such as rainy nighttime.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a road white line detection device according to an embodiment of the present invention;

FIG. 2 is a view showing an input image and a process of the image processing (part 1);

FIG. 3 is a diagram showing a process of image processing (part 2);

FIG. 4 is a processing flowchart of road white line detection in the embodiment.

FIG. 5 is a diagram showing an example of a spatial frequency of a generated image and an image signal to be processed in the embodiment.

FIG. 6 is a diagram showing an example of a luminance histogram generated in this embodiment.

FIG. 7 is a diagram illustrating an example of searching for an outline point of a white line in an image including only a component having a high spatial frequency in the embodiment;

FIG. 8 is a diagram showing a region of coefficients of a Hough space and a spline curve;

FIG. 9 is a block diagram illustrating an example of a conventional road white line detection device.

FIG. 10 is a diagram showing an input image with a conventional road white line detection device.

FIG. 11 is a block diagram showing a road white line detection device according to an embodiment of the present invention.

FIG. 12 is a flowchart of the operation of the road white line detection device.

FIG. 13 is a diagram showing a method of dividing an image and setting a threshold value for each region.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Road image photographing means 12 Blurred image generating means 13 Area dividing means 14 Road area detecting means 15 Edge image extracting means 16 Contour point estimating means 17 Road contour point extracting means 18 Curve detecting means

Claims (5)

[Claims] 1. A blur image is generated by generating data of only a low spatial frequency component from road image data, and the brightness is substantially the same using the blur image data generated in the blur image generating step. Divided into each area of
A region including a predetermined position of the image is detected as a road region from the region division data from the region division step, and a high spatial frequency independent of the road image data that does not include the spatial frequency component generated in the blurred image generation step is included. An edge image is extracted by detecting only the component, an estimated contour point estimated to correspond to a road white line is estimated from the high-frequency component data from the edge image extraction step, and a road area detected by the road area detection step is detected. Extracting only the estimated contour points of the road white line obtained from the contour point estimating step, and detecting the white line using the white line contour points extracted in the contour point extracting step. Method. 2. A road image photographing means for photographing a road ahead of a vehicle, a blur image generating means for generating data of only a low frequency component of a low spatial frequency in the road image data from the road image photographing means, A region dividing unit that divides the image data into regions each having substantially the same luminance using image data obtained from the image generating unit; and a region including a predetermined position of the image from the region divided data from the region dividing unit as a road region. Road area detecting means for detecting, edge image extracting means for detecting a high frequency component having a high spatial frequency in the road image data from the road image photographing means, and corresponding to a road white line from high frequency component data from the edge image extracting means. A contour point estimating means for extracting an estimated contour point to be estimated; and the contour point estimating means included in the road area detected by the road area detecting means. Contour point extracting means for selecting only the estimated contour points of the road white line obtained from the determining means, and curve detecting means for detecting a white line as a polynomial curve using the white line contour points extracted from the contour point extracting means, A road white line detection device, comprising: 3. The contour point extracting means according to claim 1, wherein said contour point extracting means includes an estimated contour point included in the road area detected by said road area detecting means and within a predetermined range from an end of the road area. 3. The road white line detection device according to claim 2, wherein the road white line is selected. 4. The curve detecting means detects a spline curve passing through each white line contour point, calculates a plurality of spline interpolation formulas from a part of the white line contour point, and based on a maximum coefficient value of the spline interpolation formula. 3. The road white line detection device according to claim 2, wherein the curve is calculated by correcting the coefficient value of the spline curve. 5. The road area detecting step is characterized in that a lane area is detected by merging areas divided by a luminance distribution of a low-frequency image from the upper part of the image using knowledge about roads. The road white line detection method according to claim 1.