JPH08320997A - Vehicle lane recognition device, obstacle detection device, road departure notification device - Google Patents

Abstract

(57) [Abstract] [Purpose] To quickly and accurately determine whether the lane markers on the left and right of the road are broken lines or solid lines, and improve the recognition accuracy of the driving lane of the host vehicle. [Structure] The edge point count maximum value of an edge component representing a straight line component is measured from each of the left and right image information in front of the vehicle, and based on this maximum value, a broken line is drawn when the maximum value periodically changes. It is recognized as a lane marker, and when it is a steady constant value, it is recognized as a solid lane marker. The lane of travel of the host vehicle is recognized based on the recognition results of the lane markers on the left and right of the image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device mounted on a vehicle for recognizing whether the lane in which the vehicle is traveling is a traveling lane, an overtaking lane, a lane at a road end, or the like, and a traveling lane in front of the vehicle. The present invention also relates to a device that detects an obstacle on the adjacent lane that may change the lane of the own vehicle or an interrupting vehicle, and a device that predicts and notifies the possibility that the vehicle may deviate from the road against the driver's will.

[0002]

2. Description of the Related Art By recognizing the types of lane markers on both sides of the lane of the host vehicle, it is possible to recognize the lane of the host vehicle and to change the lanes ahead of the host vehicle and the lane of the host vehicle or the interrupted vehicle. Japanese Patent Application Laid-Open No. 5-108147 and Japanese Patent Application Laid-Open No. 4-108147 disclose a vehicle traveling lane recognition device necessary for detecting obstacles on a lane and predicting and notifying road deviations due to vague driving, drowsy driving, and the like. It is possible to apply a conventional traveling road recognition device as disclosed in Japanese Patent No. 184603.

In the former case, in order to accurately recognize the broken lane marker, the image input by the camera is
Set the reference position for the lane marker, detect whether a white line exists at this reference position, and if there is a white line, calculate the feature amount indicating the position of the white line and calculate the next white line from that feature amount. The existing area is estimated, and the same processing is performed in the estimated area. The broken line lane marker is recognized by sequentially repeating this operation. The latter is to set a window on the image input from the camera and change the setting position of the window according to the vehicle speed so that the white line part exists in the window, and change the white line recognition area according to the vehicle speed. By doing so, the traveling lane shape is recognized.

A device for recognizing the traveling lane of the own vehicle can be considered as an application of such a road recognition device for recognizing a lane marker.

[0005]

However, in such a conventional vehicle lane recognition device for a vehicle, the processing operation for recognizing the broken lane marker is complicated. In addition, when there is a preceding vehicle in front of the vehicle and the lane marker is blocked or the road has a curve with a small radius of curvature, the lane marker cannot be obtained as image information. Since it is not possible to recognize in advance whether or not there is a lane recognition device for a conventional vehicle,
Complex processing to improve the detection accuracy of the broken line lane marker will also be performed on the solid line lane marker,
There is a problem that the processing time becomes long and the detection accuracy of the solid lane marker is rather deteriorated.

The present invention has been made by paying attention to such a conventional problem, and the lane marker can be surely grasped by an image, the lane marker can be recognized by a simple processing operation, and the traveling lane can be recognized accurately at high speed. An object is to provide a vehicle lane recognition device for a vehicle. Another object of the present invention is to provide a highly reliable obstacle detection device and road deviation notification device using this device.

[0007]

Therefore, in the vehicle lane recognition apparatus according to the invention as set forth in claim 1, as shown in FIG. 1, an image pickup means A for picking up an image of a road ahead of the own vehicle,
On the left and right sides of the lower end of the image picked up by the image pickup means A, windows each having a height corresponding to a length less than the total length of the set of the wired section and the wireless section of the broken line lane marker are set. Window setting means B, edge point extraction means C for extracting a point where the image density changes by a predetermined value or more in each window as an edge point, and each window,
An edge point counting means D for counting the number of edge points on all line segments formed by connecting two sides facing each other in the height direction for each line segment, and for each line segment obtained by the edge point counting means D Counting maximum value extraction means E for extracting the maximum value of the edge point count values, and it is determined whether or not the maximum value of the edge point count values for each image that is continuously obtained is periodically increased or decreased. A periodicity determining means F, and a stationarity determining means G for determining whether or not the maximum edge point count value for each image is a constant value determined by the window height without changing.
When the periodicity determining unit F determines that the maximum value has periodicity, it is a broken line lane marker, and the stationarity determining unit G determines that the maximum value has stationarity. A lane marker recognizing means H for recognizing the lane marker as a solid line lane marker and a traveling lane recognizing means I for recognizing the traveling lane of the own vehicle based on the lane marker recognition results for the left and right windows.

According to the second aspect of the present invention, as shown in FIG. 2, the periodicity determining means F changes the maximum edge point count value in proportion to the increase / decrease in the vehicle speed detected by the vehicle speed detecting means J. The maximum count value change state detecting means F'for detecting whether or not the maximum value is present is detected, and the maximum value periodicity is determined based on the detection result of the maximum count value change state detecting means F '. Good.

Further, in the obstacle detecting device for a vehicle according to the third aspect of the present invention, as shown in FIG.
An obstacle detection unit K for detecting an obstacle in front of the vehicle is added to the vehicle traveling lane recognition device described above, and an obstacle search of the obstacle detection unit K is performed based on a lane marker recognition result of the vehicle traveling lane recognition device. The area is limited to the front of the host vehicle and the side of the broken lane marker.

Further, in the vehicle road departure notification device according to the invention of claim 4, as shown in FIG. 4, the vehicle lane recognition device for vehicle of claim 1 or 2 detects a blinker operation for detecting a blinker operation. The means L, the lane marker crossing prediction means M for predicting whether or not the own vehicle crosses the solid line lane marker, and the blinker operation detecting means L and the lane marker crossing prediction means M, when the own vehicle crosses the solid line lane marker without blinker operation. And a notifying unit N for notifying when the time is predicted.

According to the fifth aspect of the present invention, the lane marker crossing prediction means M is a lane marker for which the own vehicle is a solid line lane marker based on the detection result from the lateral speed detection means for detecting the lateral speed of the vehicle. It is a configuration for predicting whether or not to cross.

[0012]

According to the first aspect of the present invention, the maximum value of the edge point count value of each window changes cyclically based on the traveling road image data in front of the vehicle captured by the image capturing means A. It is determined whether or not it is a constant value, and if it is changing periodically, it is recognized as a broken line lane marker, and if it is constantly a constant value, it is recognized as a solid line lane marker. Based on this lane marker recognition result, the distinction between the driving lane of the own vehicle and the adjacent portion is recognized, and whether the own lane is the driving lane at the left end, the passing lane at the center, or the passing lane at the right end, etc. recognize. Then, since the window is set at the lower end of the image, which corresponds to the immediately preceding portion of the own vehicle, the lane marker can be detected as a substantially straight line and there is no possibility of being blocked by the preceding vehicle or the like.

Further, in the invention described in claim 2,
The maximum value of the edge point count value in the window shows a change proportional to the vehicle speed, and as the vehicle speed increases, the cycle of the change becomes shorter (frequency becomes higher), and when decelerating, the cycle becomes longer (frequency becomes lower). ), The cycle is substantially constant (frequency does not change) when traveling at a constant speed. Therefore, it is possible to accurately determine the broken line by determining whether the maximum edge point count value is in a changing state corresponding to the detected vehicle speed. It becomes possible to recognize the lane marker.

According to the invention described in claim 3, claim 1
Alternatively, the vehicle lane recognition device for vehicle according to 2 and the obstacle detection device are combined to limit the search area for the obstacle by the obstacle detection device to the front of the own vehicle where the obstacle is likely to exist and the side of the broken lane marker. To do. This saves unnecessary time for unnecessary data processing in obstacle detection,
Obstacle detection processing time is short and highly reliable obstacle detection is possible.

According to the invention of claim 4, the invention according to claim 1
Alternatively, the solid line lane marker side recognized by the vehicle lane recognizing device described in 2 is notified when it is predicted that the host vehicle will cross without operating the blinker.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 5 is a system configuration diagram of the first embodiment of the vehicle lane recognition device according to the present invention. In FIG. 5, the apparatus according to the present embodiment is provided with a camera 11 as an image pickup means for photographing a traveling direction of a vehicle, for example, a traveling path in front of the vehicle, and the camera 11.
Based on the image signal obtained from 11, the image processing device 12 having functions such as window setting means, edge point extracting means, edge point counting means, and counting maximum value extracting means, which will be described later, and the speed of the host vehicle. A vehicle speed sensor 13 as a vehicle speed detecting means for detecting, based on each information signal from the image processing device 12 and the vehicle speed sensor 13, the left and right lane markers of the lane in which the vehicle is traveling are a solid line lane marker or a broken line lane marker. And a microcomputer 14 for identifying and executing the operation of recognizing the traveling lane of the own vehicle.

The camera 11 is fixed, for example, at a position near the rear-view mirror so that the road ahead of the vehicle can be photographed. Next, the processing operation of the image processing device 12 will be described. The image processing apparatus 12 continuously inputs the image data from the camera 11 every Δt seconds as 256 gradation monochrome grayscale image information in which one frame is 512 × 480 pixels. Δt is generally 33 msec. As shown in FIG. 6, the input image data for one frame has pixels such as G (x, y) for each pixel with the horizontal direction as the X coordinate and the vertical direction as the Y coordinate with the upper left corner of the screen as the origin. The specified coordinates are assigned.

FIG. 6 and FIG. 7 are explanatory diagrams regarding the setting of the window. Here, YL indicates the Y coordinate at which the vanishing point exists. The image shows the case where the left boundary line of the traveling lane is the solid line lane marker 20 and the right boundary line is the broken line lane marker 21. The windows 22 and 23 are the camera as shown
It is set to the left and right position of the lower end of the image determined by the angle of view and the installation height of 11. The lane marker in front of the vehicle is always a substantially straight line regardless of the shape of the road. Therefore, if the window is set at this position, the lane marker is always present in the window as long as the vehicle runs in the lane. This position is a position where a reliable and clear lane marker image can be obtained even if there is a vehicle ahead, unless the inter-vehicle distance is extremely short at an extremely low speed.

The height of the left and right windows 22 and 23 is
The broken line lane marker 21 is set to a height corresponding to a length less than the total length of the length of the wired portion and the length of the wireless portion, and in the figure, corresponds to a length slightly shorter than the length of the wired portion. It is high. Here, the actual length of the lane marker and the height on the image data have a relationship as shown in FIG. 8. From the focal length f of the camera 11 and the height H _{0 of the} set position, the following equation (1) is used. The converted length Z on the actual road surface is represented by y pixels on the image data.

Y = f · (H ₀ / Z) (1) In the following description, the length on the road plane and the height on the image data are used. Mutually convertible. Next, the edge point extraction processing will be described. In the edge point extraction process, the input image G (x,
For the y), the SOBEL operator is used to obtain the first-order derivative in the X direction, and the value Sx (x, y) is used as the density value of G (x, y). Next, the threshold value Cs and Sx (x, y) are compared,
It is determined that the pixel of Sx (x, y) <Cs is not an edge point.

Here, the edge point means a point where the brightness changes abruptly. The threshold Cs is determined by the brightness of the input image. For example, when the contrast of the entire input image is weak, Cs is made small, and when the contrast of the entire input image is strong, Cs is made large.
Since the SOBEL operator is a general method for edge extraction, detailed description will be omitted here.

Next, the counting process and maximum value extracting process of the edge points extracted according to FIGS. 9 and 10 will be described. FIG. 9 shows the windows 22 and 23 shown in FIGS. 6 and 7, and in the image coordinate system, the upper base is (x = xu _i (i =
0-n), y = yu), and the lower bottom is (x = xd _j (j = 0-
m) and y = yd = yu + H), which is a trapezoid parallel to the X axis.

With respect to the edge image obtained by the edge point extraction process, variables i, j and Pmax are initialized in step 50, and then in step 51, upper end (xu _i , yu) and lower end (xd).
The sum p of the density values of the pixels on the virtual straight line in the window connecting _j , yd) is calculated. Next, in step 52, the density sum p
Is compared with the maximum value Pmax of the past concentration sum, and p is P
If it is larger than max, Pmax, xu, x in step 53
Update the coordinate value of d. Then, according to step 54, this process is performed up to j = 0 to m, and the same process is performed according to steps 55 to 56 for each of i = 0 to n.
Finally, at step 57, the density sum p becomes maximum (xu,
xd) and the maximum value Pmax of the concentration sum are calculated by a microcomputer.
Output to 14. The value (xu, xd) obtained here is the X coordinate value of the end point of the detection line, and Pmax is the maximum edge point count value. This value Pmax is also an index showing the accuracy of the detection line.

Here, how the maximum edge point count value Pmax changes with respect to the broken line lane marker and the solid line lane marker will be described with reference to FIGS. 6 to 14. When the time between frames for capturing images is Δt and the traveling speed of the vehicle is v, the image of the first frame is as shown in FIG.
The image of the next frame after t is as shown in FIG. Therefore, in the window 22 on the side of the solid line lane marker, since the lane marker is not discontinued in the window 22 and is in the same state as the image of FIG. 6, the maximum edge point count value Pmax is
It does not change at a value equal to the height of the window. On the other hand, in the window 23 on the side of the broken lane marker, in FIG. 7, there is a wired part (white line part) and a wireless part (part without white line).
Pmax in the image of the frame in FIG. 7 is smaller than Pmax in the image of the frame in FIG. 6 in which only the wired portion (white line portion) is inside the window 23.

FIG. 11 shows the road coordinate system in FIG. 6 and FIG.
6 shows the relationship between the broken line lane marker and the window, and shows the result of projecting the window 23 of FIGS. 6 and 7 on the road. Here, the length of the wired portion of the broken line lane marker is L ₁ , the length of the wireless portion is L ₂ , and the length corresponding to the height H of the window in the image is h. In the figure, the wired section 8
0 and 81 are the broken line lane markers 21 in the case of FIG.
Reference numerals 3 and 84 indicate the broken line lane marker 21 in the case of FIG. 7, reference numerals 82 and 85 indicate the window 23 projected on the road in each case, and L ₀ is the end point on the front side of the wired portion as the reference position. The initial position of the hour window.

In this case, considering the length ratio such that the front end point of the wired section is 0 and the front end point of the next wired section is 1, a set of the wired section and the wireless section of the broken line lane marker is considered. The ratio k of the length of the lower end of the window from the end point 0 to the total length is expressed by the following equation (2) in the window 82 and the following equation (3) in the window 85. k = L ₀ / (L ₁ + L ₂ ) ... (2) k = (L ₀ + vΔt) / (L ₁ + L ₂ ) ... (3) When the vehicle travels at the speed v, this window The general formula of the ratio k of the lower end points is as the following formula (4).

K = (L ₀ + vnΔt) / (L ₁ + L ₂ ) −int (L ₀ + vnΔt) / (L ₁ + L ₂ ) ... (3) A set of the wired part and the wireless part determined in this way The position of the window lower end point in the broken line lane marker of the length L ₁ + L ₂ consisting of parts is to obtain the length T of the wired part of the lane marker that enters the window corresponding to the maximum edge point count value Pmax in the window 23. I understand.

Specifically, the length T of the wire portion of the broken line lane marker existing in the window is roughly divided into three cases according to the relationship of L ₁ , L ₂ and h, and each of the following equations (5) to () depending on the value of k. It is determined as in equation 16). However,
Here, it is assumed that the general condition of L ₁ ≦ L ₂ is satisfied. In case of h ≦ L ₁ ≦ L ₂ 0 ≦ k ≦ (L ₁ −h) / (L ₁ + L ₂ ) → T = h (5) (L ₁ −h) / (L ₁ + L ₂ ) <k ≦ L ₁ / (L ₁ + L ₂ ) → T = L ₁ −k (L ₁ + L ₂ ) ... (6) L ₁ / (L ₁ + L ₂ ) <k ≦ (L ₁ + L ₂ −h) / (L ₁ + L ₂ ) → T = 0 (7) (L ₁ + L ₂ −h) / (L ₁ + L ₂ ) <k <1 → T = h− (L ₁ + L ₂ ) + k (L ₁ + L ₂ ) ・ (8) When L ₁ <h ≦ L2 0 ≦ k ≦ L ₁ / (L ₁ + L ₂ ) → T = L ₁ −k (L ₁ + L ₂ ) ... (9) L ₁ / (L ₁ + L ₂ ) <k ≦ (L ₁ + L ₂ −h) / (L ₁ + L ₂ ) → T = 0 ... (10) (L ₁ + L ₂ −h) / (L ₁ + L ₂ ) <K ≦ (2L ₁ + L ₂ −h) / (L ₁ + L ₂ ) → T = h− (L ₁ + L ₂ ) + k (L ₁ + L ₂ ) ·· (11) (2L ₁ + L ₂ −h) / _{(L 1 + L 2) <} k <1 → T = L 1 ·· _{(12) L 1 ≦ L2 ≦} h < In the case of _{L 1 + L 2 0 ≦ k} ≦ (L 1 + L 2 -h) / (L 1 + L 2) → T = L 1 -k (L 1 + L 2) ··・ (13) (L ₁ + L ₂ −h) / (L ₁ + L ₂ ) <k ≦ L ₁ / (L ₁ + L ₂ ) → T = h−L ₂ (14) L ₁ / (L ₁ + L ₂ ) <k ≦ (2L ₁ + L ₂ −h) / (L ₁ + L ₂ ) → T = h− (L ₁ + L ₂ ) + k (L ₁ + L ₂ ) ・ ・ (15) (2L ₁ + L ₂ −) h) / (L ₁ + L ₂ ) <k <1 → T = L ₁ (16) FIGS. 12 to 14 show the initial position of the window by the formula (4) as L _0.
12 shows changes in the length T (corresponding to the maximum Pmax of the edge point count value) of the wire portion in the window in each of the above cases when = 0 and n = 0. Is h ≦ L ₁ ≦ L2, FIG. 13 shows L ₁ <h
In the case of ≦ L2, FIG. 14 shows the case of L ₁ ≦ L2 ≦ h <L ₁ + L ₂ . In any case, the length T of the wire portion, that is, the maximum edge point count value has an accurate periodicity.

Since the periodicity is easier to detect as the amplitude is larger, the window height is generally set to L ₁
It is desirable to set so that <h ≦ L2 (in the case of FIG. 13). On expressways, etc., h = 10m. The figure
Since 12 to 14 qualitatively show the change of T, the maximum and minimum steady intervals are long in the figure, but the difference between the length of the window and the wired part is 5 m, which is relatively large. Even if the maximum value and the minimum value are steady, the running speed is 30 km / h.
It takes 0.6 seconds at 100 km / h and only 0.18 seconds at 100 km / h, and in reality it is more sharp.

When the lane marker is a solid lane marker, the length T of the wire portion in the window does not change, and
As shown by the dotted line in FIGS. 12 to 14, it becomes constant at a value corresponding to the length of the window. Next, the driving lane recognition processing operation in the apparatus of this embodiment will be described with reference to the flowcharts of FIGS. 15 and 16.

First, in step 110, the image processing device 12
The maximum value Pmax of the edge point count value obtained in (3) is taken in. In step 120, it is judged whether or not Pmax is taken in for the first time, and if it is the first time, in step 130 Pmax is set to the maximum value Tmax and the minimum value Tmin of the periodic change. This is a temporary placement because L ₁ and L ₂ are generally unknown, and thereafter updated as Pmax changes, as will be described later.

From the second time of fetching Pmax, the processing from step 140 onward is performed. In step 140, it is determined from the signal from the vehicle speed sensor 13 whether or not the vehicle is traveling. When the car is not running, there is no change in the image and Pma
Since x also does not change, the periodicity cannot be detected, so the routine proceeds to step 610, where Pmax is set to the latest value PL and the counter value n is counted up.

If the vehicle is running, step 150
The stationarity determination value Tclim is set with. As shown in FIGS. 12 to 14, in the case of the broken line lane marker, the maximum value duration tmax of Pmax is represented by the following equation (17), and the minimum value duration tmin is represented by the following equation (18). . _{tmax = | L 1 -h | /} v ··· (17) tmin = | L 2 -h | / v ··· (18) Therefore, L _1, L ₂ different from and by road, the processing for each Δt Since the stationarity determination value Tclim corresponds to all cases due to the above, the value is set to a value larger than the value obtained by dividing these values by Δt.

In step 160, it is determined whether PL representing the maximum count value in the image of the previous frame and the maximum count value Pmax of the image captured this time are equal. This is to detect that the maximum value of the count of the edge points in the window is either the maximum value or the minimum value of the broken line lane marker or the constant value of the solid line lane marker.
When Pmax = PL is one of the above cases, P
If max ≠ PL, Pmax
Indicates that is changing.

When Pmax = PL, go to step 170.
The maximum possible value of Pmax in the set window.
Determine whether the maximum value Tmax is equal to the height H of the window.
Set. Here, in the case of a broken line lane marker, h ≦ L₁
≤L₂In the case of, Tmax has already been updated and becomes a constant value.
If YES, the determination in step 170 is YES. h ≦ L
₁≤L₂Even in case of, when Tmax is being updated, it is judged as NO.
Become. Also, h ≦ L₁≤L₂Other than, that is, L₁≤
h ≦ L₂And L₁≤L₂≦ h <L₁+ L₂In case of
In the case of the broken line lane marker, the maximum value Tma that Pmax can take
x is the height H of the window as shown in FIGS. 13 and 14.
Less than (corresponding to the window length h on the road)
Line length L₁It becomes the size equivalent to
In the case of power, the maximum count value Pmax is always H,
A completely different value can be obtained for the line marker and the solid line lane marker.
It

If NO in step 170, it is determined in step 180 whether H = Pmax. Where H =
In the case of Pmax, there is only the case of the solid line lane marker, and the detected straight line in the window is judged to be the solid line lane marker in step 270. In the case of H ≠ Pmax, it is the maximum value Tmax or the minimum value Tmin of the broken line lane marker, and in the case of the maximum value Tmax, the counting maximum value Pmax always decreases as the vehicle advances,
In the case of the minimum value Tmin, it always increases as the vehicle advances. Therefore, in step 240, it is judged whether or not Pmax = Tmax.
When Pmax = Tmax, in step 260, the increase / decrease direction flag TU is set to 2 which means decrease of Pmax, and Pmax ≠
If it is Tmax, the increase / decrease direction flag TU is set to P in step 250.
Set to 1, which means increase max.

When h ≦ L ₁ ≦ L ₂ and when it is determined that Pmax = H in step 170, it is determined in step 190 whether Pmax which is fetched this time is equal to the maximum value Tmax of the broken line lane marker. Tmax is initially 130
However, after that, it is updated by the change of Pmax and Tmax = H. If Pmax = Tmax, the number of consecutive frame acquisitions Tc is counted in step 200, it is determined in step 210 whether or not the count value Tc ≧ Tclim has continued, and if it continues, it is determined in step 270 that it is a solid line lane marker. .

When the count value Tc of consecutive frames where Pmax is equal to Tmax is less than Tclim, it may be the maximum value of the broken line lane marker. Therefore, the increase / decrease direction flag TU is set to 2 (decrease) in step 220. Also, step 19
When Pmax and Tmax are not equal to 0, the maximum value Pmax of the image of the current frame is the minimum value of the broken line lane marker, and therefore the increase / decrease direction flag TU is set to 1 (increase) in step 230.

When the PL indicating the maximum count value of the previous frame and the maximum count value Pmax of the current frame are not equal in step 160, the frame number count value Tc is cleared in step 300, and the increase / decrease direction flag TU is set in step 310. It is determined whether it has been done. If the increase / decrease direction flag TU is not set, it is determined in step 320 whether the maximum edge point count value Pmax is decreasing or increasing from the PL of the previous frame. If it is decreasing, the increasing / decreasing direction is determined in step 330. The flag TU is set to 2 (decrease), and if it is increasing, it is set to 1 (increase) in step 360. If it is decreasing, Pma is set in steps 340 to 350.
It is determined whether or not x is smaller than the current minimum value Tmin, and if smaller, Tmin is replaced with the current Pmax and updated. vice versa,
If it is increasing, it is determined in steps 370 to 380 whether Pmax is larger than the current maximum value Tmax.
Replace max with Pmax this time and update.

If Pmax is not equal to PL indicating the maximum value of the previous frame and the increase / decrease direction flag TU has already been set, the increase / decrease direction flag TU indicating the direction in which the maximum value Pmax changes in step 500 is set to 1 (increase). ) Or not. If the flag TU is 1 (increase), P in step 560
It is determined whether max is an increase, and if it is an increase, it is recognized in step 570 that the detection line in this window is a broken line lane marker. Then, step 580-
At 590, it is determined whether or not Pmax is larger than the current maximum value Tmax, and if larger, the current Pmax is replaced with Tmax and updated. When it is determined in step 560 that Pmax has not increased despite the increase / decrease direction flag TU being 1 (increase), it is determined that Pmax has exceeded the maximum value and has started to decrease.
At 600, the increase / decrease direction flag TU is set to 2 (decrease).
However, since it is considered that Pmax fluctuates for some reason in this direction change, the lane marker is not determined.

In step 500, the increase / decrease direction flag TU is set to 1
If it is not (increase), that is, if it is 2 (decrease), it is judged in step 510 whether or not Pmax is smaller than PL, and if it is decrease, in step 520, the detection line in this window is a broken line. Recognize as a lane marker. Then, in steps 530 to 540, Pmax is the current minimum value Tmin.
It is judged whether it is smaller or not.
Replace with n and update. If it is determined in step 510 that Pmax has not decreased despite the increase / decrease direction flag TU being 2 (decrease), it is determined that Pmax has exceeded the minimum value and has increased, and the increase / decrease direction flag TU is set to 1 in step 550. Set to (Increase). However, since it is considered that Pmax fluctuates for some reason also in this direction change, the lane marker is not determined.

After all the processings of steps 110 to 600 described above are completed, Pmax is set to P in step 610.
L is set, and the process of counting up the count value n of the capture counter is performed. The above processing operation is independently executed for each of the right window and the left window, and recognizes whether the left and right lane markers are solid lines or broken lines.

In this way, after the recognition operation for determining whether the left and right lane markers of the host vehicle are solid lines or broken lines, the traveling lane in which the host vehicle is traveling is recognized from the recognition result. For example, if the left side is a solid lane marker and the right side is a dashed lane marker, the leftmost lane of a road with two or more lanes, the left and right lane markers are three or more lanes inside, and the left is a dashed lane marker and the right is a solid lane marker. In the case of a right lane marker for roads with two or more lanes, and for both left and right lane markers, the driving lane of the host vehicle is recognized, such as a road with one lane on each side.

As described above, in the apparatus of this embodiment, the maximum value of the count of the edge points on the straight line in the window is calculated from the image information of a small part of the front part of the own vehicle, which is a substantially straight line with a small change in the lane marker. However, in continuous input images, it is determined whether the lane marker that divides the driving lane of the host vehicle is a broken line or a solid line, by determining whether the lane marker has a periodicity that increases or decreases, or has a constant value that corresponds to the height of the window. Is recognized, and the traveling lane of the own vehicle is recognized from the recognition result.

Therefore, there is no problem that the lane marker is blocked by the presence of a preceding vehicle or the like, and there is no problem that the lane cannot be detected even with a curve having a small radius of curvature. It can be recognized, and the traveling lane of the own vehicle can be recognized. Also,
The type of lane marker can be easily recognized in advance, and the complicated process for detecting the broken line lane marker with high accuracy and the process for detecting the solid line lane marker can be appropriately applied, respectively, which shortens the processing time and improves the overall detection accuracy. It is possible to improve.

Next, a second embodiment corresponding to the invention described in claim 2 will be described. Here, the system configuration is similar to that of the first embodiment shown in FIG. 5, and the description thereof is omitted. The lane marker recognition processing operation is also the same as that of the first embodiment except for the periodicity detection processing operation, and therefore description thereof is omitted. In the broken line lane marker, the change of the maximum edge point count value Pmax in the window is calculated by the equation (4).
It is determined by the value of. The value of k is the traveling speed v of the vehicle.
Is a parameter changing value, and in the change of Pmax shown in FIGS. 12 to 14, durations tmax and tmin of a constant interval of the maximum value and the minimum value and the expressions (6) and (8),
The frequency of the periodicity of the change in Pmax, such as the amount of change in the increase / decrease portion determined by the equations (9), (11), (13), and (15), is determined. The amount of change in the increased / decreased portion is the change in the number of images corresponding to the length of vΔt.

Therefore, the traveling speed v that determines the frequency of this change is input from the vehicle speed sensor 13, and the input traveling speed v
By detecting that Pmax is changing at the frequency corresponding to, it is possible to further increase the certainty of recognition of the broken lane marker. The processing operation of the second embodiment is shown in FIG.
It will be described with reference to the flowcharts of FIGS. 17 and 18.

In this processing flowchart, the processing operations from step 110 to step 610 are the same as those in the first embodiment. In the second embodiment, before the broken line recognition operation of steps 520 and 570, a step of determining whether or not the difference between Pmax and PL is the number of images VΔt corresponding to the length of vΔt
700 and step 710 are provided, and when this relationship is not established, the broken line recognition is not performed, and the process proceeds to steps 530 and 580 of the updating process of the respective maximum value Tmax and minimum value Tmin.

By performing such processing operation, the traveling speed v, which is a phenomenon applicable to the case of the broken line lane marker.
The maximum edge point count value P in the window is proportional to the increase or decrease of
It is possible to know whether the change amount of max, that is, the changing frequency of Pmax is increasing or decreasing, or whether the changing frequency of Pmax is constant when the traveling speed v is constant. Therefore, the second
According to the apparatus of the embodiment, in addition to the effect of the apparatus of the first embodiment,
The reliability of recognition of the broken line lane marker is increased, and the traveling lane of the own vehicle can be recognized at higher speed and more accurately.

Next, an embodiment of an obstacle detecting device using the above-mentioned traveling lane recognition device, which is the invention described in claim 3, will be described. FIG. 19 shows the system configuration of an embodiment of the obstacle detection device. In Figure 19, laser radar
15 measures the distance to the object based on the time until the radar beam is emitted forward and the reflected beam from the object existing in front is received. Here, the laser radar 15 is of a multi-beam type in which, for example, three radar beams are arranged with different beam emission angles, and the radar beams are sequentially emitted, and are installed near the front grill or the like. A scanning method in which one radar beam is periodically rotated in the horizontal direction may be used.

The microcomputer 14 is an image processing device.
The lane recognition processing operation is executed based on the information from 12 and the laser radar 15 is also controlled to execute the obstacle detection operation processing. In such an obstacle detection device, based on the result of the traveling lane recognition of the own vehicle by the traveling lane recognition device, the obstacle search direction by the laser radar 15 is set to the front of the own vehicle where another vehicle is likely to exist and the lane marker side lane on the broken line. Limited to

FIG. 20 shows an example in which the host vehicle 31 is traveling in the right lane on a two-lane road, and FIG. 21 is an example in which the vehicle is traveling in the left lane. In FIG. 20, the left side of the host vehicle 31 is a broken line lane marker, and the right side is a solid line lane marker.Therefore, as shown by hatching, the central beam for searching the front area of the host vehicle 31 and the dashed lane marker side lane, namely, The presence of obstacles is detected by the left beam searching the left lane area.

On the contrary, in FIG. 21, as indicated by hatching in the drawing, the central beam for searching the front area of the own vehicle 31 and the right beam for searching the lane marker side lane of the broken line, that is, the right lane area are obstacles. Detect the presence. 22 and 23 show the case of a curved road, and even in the case of a curved road, the central beam and the beam on the side of the broken line lane marker are searched for the minimum necessary area for detecting an obstacle as shown by the hatching in the figure. can do.

In this way, if the obstacle search area is limited to the front side of the vehicle and the lane marker side of the broken line based on the recognition result of the traveling lane, it is not necessary to input the obstacle detection information from all three beams. It is possible to recognize the situation in front of the lane where the vehicle 31 may change lanes and the presence of a vehicle that may interrupt the front of the own vehicle 31 with high speed and high accuracy by searching only the necessary area. A possible obstacle detection device can be realized.

Next, a description will be given of an embodiment of a vehicle road departure notification device using the above-described traveling lane recognition device, which is the invention described in claim 4. FIG. 24 shows the system configuration of an embodiment of the road departure notification device. In FIG. 24, the lateral G sensor 16 measures the lateral velocity component of the vehicle. The winker device 17 outputs a winker operation signal to the microcomputer 14 when the winker is operated. The notification device 18 is a microcomputer 14
The output from the device notifies the driver. The lateral velocity component may be detected by detecting the amount of change in the position of the lane in the image data, and in this case, the lateral G sensor 16 is not necessary.

In the road departure notification device, the vehicle is detected from the position of the own vehicle in the lane which can be detected from the position of the lane on the image in the traveling lane recognition device and the lateral velocity component detected by the lateral G sensor 16. Predicts in the microcomputer 14 whether there is a possibility of crossing the lane marker. Then, when the winker operation signal is not input from the winker device 17, it is predicted that the own vehicle may cross the right or left lane marker, and when the lane marker on the side predicted to cross is the solid line lane marker, it is notified. The device 18 notifies.

This makes it possible to prevent the vehicle from deviating from the driving lane. Incidentally, the traveling lane recognition device of the present invention, other than the application to the obstacle detection device and the road departure notification device,
When the right lane marker is a solid line, it may be applied to a headlight system that cuts only the light distribution of that part. The vehicle lane recognition device for vehicles, the vehicle obstacle detection device, the vehicle road deviation notification device, and the like configured based on the device can improve reliability.

[0058]

As described above, according to the first aspect of the present invention, the maximum number of edge points on the detected straight line in the window is counted based on the image information of the front part of the own vehicle in which the lane marker is a substantially straight line. Whether the lane marker that divides the driving lane of the host vehicle is a broken line by discriminating whether the value changes from moment to moment in the input image and periodically changes or has a constant value corresponding to the window height. Whether or not the vehicle is a solid line is recognized, and the traveling lane of the own vehicle is recognized from the recognition result.

Therefore, there is no problem that the lane marker is shielded by the presence of a preceding vehicle or the like, and there is no problem that the lane cannot be detected even with a curve having a small radius of curvature. By recognizing, the traveling lane of the own vehicle can be recognized. In addition, the type of lane marker can be easily recognized in advance, and the complicated process for detecting the broken line lane marker with high accuracy and the process for detecting the solid line lane marker can be appropriately applied to each of them, thereby shortening the processing time and detecting the whole. The accuracy can be improved.

According to the second aspect of the present invention, the broken line lane marker is determined by detecting that the maximum edge point measurement value in the window increases or decreases by a value proportional to the traveling speed of the vehicle. Therefore, the broken line lane marker can be recognized at high speed and accurately, and the reliability of the recognition of the traveling lane can be further improved. According to the third aspect of the present invention, the obstacle detection device may be combined with the traveling lane recognition device, and the obstacle search area may approach another vehicle based on the recognition result of the traveling lane recognition device. Since it is limited to the front of the vehicle and the side of the broken lane marker that is the lane area that occurs, unnecessary data processing time can be saved, obstacle detection processing time can be shortened, and objects other than the driving lane are detected as obstacles. It is possible to realize a highly reliable vehicle obstacle detection device without doing so.

Further, according to the invention of claim 4, a blinker operation detection function, a vehicle lateral speed detection function and a notification function are added to the traveling lane recognition device, and the lane marker recognition result in the traveling lane recognition device is used. By notifying the driver when the vehicle is predicted to cross the solid line lane marker without notifying the driver, the driver can be provided with a highly reliable road departure notification device.

The traveling lane recognition device, the vehicular obstacle detection device, the vehicular road deviation notification device, and the like based on the recognition device can improve reliability.

[Brief description of drawings]

FIG. 1 is a block diagram of the invention according to claim 1.

FIG. 2 is a block diagram of the invention according to claim 2;

FIG. 3 is a block diagram of the invention according to claim 3;

FIG. 4 is a block diagram of the invention according to claim 4;

FIG. 5 is a system configuration diagram showing a first embodiment of the vehicle lane recognition device for a vehicle of the present invention.

FIG. 6 is an explanatory diagram of window setting according to the first embodiment.

FIG. 7 is an explanatory diagram of window setting according to the first embodiment.

FIG. 8 is a diagram illustrating a correspondence relationship between a length on image data and a length on an actual road.

FIG. 9 is an explanatory diagram of edge point count maximum value detection processing of each window.

FIG. 10 is a flowchart of the edge point count maximum value detection processing of each window.

FIG. 11 is an explanatory diagram regarding a change in the positional relationship between the lane marker and the window.

FIG. 12 is an explanatory diagram regarding a change state of the maximum edge point count value.

FIG. 13 is another explanatory diagram regarding a change state of the maximum edge point count value.

FIG. 14 is another explanatory diagram regarding a change state of the maximum edge point count value.

FIG. 15 is a flowchart illustrating a driving lane recognition operation according to the first embodiment.

FIG. 16 is a flowchart that follows FIG.

FIG. 17 is a flowchart illustrating a driving lane recognition operation according to the second embodiment.

FIG. 18 is a flowchart following FIG.

FIG. 19 is a system configuration diagram showing an embodiment of a vehicle obstacle detection device of the present invention.

FIG. 20 is an explanatory diagram of an obstacle search area in the case of a straight road according to the above embodiment.

FIG. 21 is another explanatory diagram of the obstacle search area in the case of the straight road according to the embodiment.

FIG. 22 is an explanatory diagram of an obstacle search area in the case of a curved road according to the above embodiment.

FIG. 23 is another explanatory diagram of the obstacle search area in the case of the curved road according to the embodiment.

FIG. 24 is a system configuration diagram showing an embodiment of a vehicle road departure notification device of the present invention.

[Explanation of symbols]

 11 camera 12 image processing device 13 vehicle speed sensor 14 microcomputer 15 laser radar 16 lateral G sensor 17 blinker device 18 notification device

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁶ Identification number Internal reference number FI Technical display location G08G 1/09 G05D 1/02 K H04N 7/18 G06F 15/62 380 // G05D 1/02 9061 -5H 15/70 330G

Claims (5)

[Claims] 1. A total of a length of a wire portion and a length of a wireless portion of a set of broken line lane markers on the left and right of a lower end portion of an image captured by the image capturing means, and an image capturing means for capturing an image of a road ahead of the host vehicle. Window setting means for setting each window having a height corresponding to a length less than the length; edge point extracting means for extracting, as an edge point, a point at which the image density in each window changes by a predetermined value or more; In each window, an edge point counting means for counting the number of edge points on all line segments formed by connecting two sides facing each other in the height direction for each line segment, and each line segment obtained by the edge point counting means A counting maximum value extracting means for extracting the maximum value of the edge point count values for each image, and determining whether or not the maximum value of the edge point count values for each image obtained continuously is increasing or decreasing periodically. Periodicity determination hand And a stationarity determination unit that determines whether or not the maximum value of the edge point count value for each image is a constant value determined by the window height without changing, and the maximum value is determined by the periodicity determination unit. A lane marker recognizing unit that recognizes a broken line lane marker when it is determined to have periodicity, and a solid line lane marker when the maximum value is determined to have continuity by the stationarity determining unit. A lane recognition device for a vehicle, comprising: a lane recognition unit that recognizes a lane of the host vehicle based on a result of lane marker recognition for each of the left and right windows. 2. The periodicity determining means has a maximum count value change state detecting means for detecting whether or not the maximum edge point count value is changing in proportion to an increase or decrease in the vehicle speed detected by the vehicle speed detecting means. 2. The vehicle traveling lane recognition device according to claim 1, wherein the presence or absence of periodicity of the maximum value is determined based on the detection result of the count maximum value change state detecting means. 3. The vehicle traveling lane recognition device according to claim 1, further comprising obstacle detection means for detecting an obstacle in front of the vehicle, and the lane marker recognition result of the vehicle traveling lane recognition device is used to detect the lane marker. An obstacle detection device for a vehicle, wherein the obstacle detection area of the obstacle detection means is limited to the front of the vehicle and the side of the broken lane marker. 4. The vehicle driving lane recognition device according to claim 1, further comprising a winker operation detection means for detecting a winker operation, and a lane marker crossing prediction means for predicting whether or not the own vehicle crosses a solid line lane marker. A vehicle road deviation notification device comprising: a blinker operation detection means and a lane marker crossing prediction means, and a notification means for giving a notification when the own vehicle is predicted to cross a solid lane marker without a blinker operation. 5. The lane marker crossing prediction means is configured to predict whether or not the vehicle crosses the solid lane marker based on a detection result from a lateral speed detection means for detecting a lateral speed of the vehicle. 4. The vehicle road deviation notification device according to 4.