JP3473296B2 - In-vehicle driving environment recognition device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle-mounted traveling environment recognition device, and more particularly to a vehicle-mounted traveling environment recognition device that can contribute to accurate vehicle position detection.

[0002]

2. Description of the Related Art As conventional on-vehicle traveling environment recognition devices, there are an "automatic target detection method" described in JP-A-6-36033 and a "vehicle traveling environment recognition device" described in JP-A-7-244717. It has been reported. First, in the conventional “automatic target detection method”, by setting a low threshold level used when binarizing an image signal from an image sensor having multiple pixels, a target image located at a long distance can be detected. Weak signals can be detected quickly, and
Even if a target image is formed between pixels of the image sensor, the target signal can be enhanced by performing the local spatial correlation processing and the frame correlation processing.

In addition, a conventional "vehicle-mounted traveling environment recognition device"
In that case, by integrating the brightness data obtained by shooting, then obtaining the change amount of this data and detecting the part with a large change amount as the white line position, transient noise components etc. It is integrated and smoothed so as not to show a large value. As a result, it is possible to reduce the risk of erroneously detecting a noise component as a white line, and to reliably detect the position of the white line.

[0004]

However, in the conventional vehicle-mounted traveling environment recognition device for removing the influence of noise, shadow, etc., is it necessary to use an image subjected to integration processing?
Since there is no judgment as to whether to use the input image, integration processing is performed even when an image that is less affected by noise and shadows is input, so the position of the preceding vehicle is detected using the image with blurred edges. As a result, there is a problem that an error is always included in the vehicle position detection.

Further, when such an integration process is performed, there is a problem that the edge of the image to be imaged is always blurred when the imaged object is moving or when the host vehicle is moving. there were. The present invention has been made in view of the above, and an object thereof is to provide a vehicle-mounted traveling environment recognition device that can contribute to accurate vehicle position detection.

[0006]

The invention according to claim 1 is
In order to solve the above-mentioned problems, an image pickup means for picking up an image of a forward running path of the own vehicle, a moving distance calculation means for calculating a moving distance of the own vehicle, and a front image picked up by the image pickup means for each preset moving distance. Based on the input means up to the previous time and the input image up to this time, and the integration means that generates the integrated image by adding Input image evaluation means for evaluating whether or not the input image of is suitable for the characteristic region extraction processing, and based on the evaluation result from the input image evaluation means,
Input image switching means for switching between outputting an integrated image and outputting an input image for the characteristic area extraction processing, and characteristic area extraction for extracting a characteristic area of a road ahead according to the output from the input image switching means. The gist is to have means. In order to solve the above-mentioned problems, the input image evaluation means uses the input image of the area where the detection target is detected as a reference image based on the characteristic area extraction result from the characteristic area extraction means. With respect to the reference image extracting means for extracting and the input image captured this time, a comparison image having a wider range than the reference image extracted in the previous process is set, and the correlation value between the comparison image and the reference image is set. ratio
A correlation value calculating means for calculating in the entire area of the comparison image ,
The input image switching means is the highest of the calculated correlation values.
The gist is to select the input image when the large value exceeds a predetermined threshold value, and to select the integral image when the large value is less than or equal to the threshold value. In order to solve the above-mentioned problems, the invention according to claim 3 further has white line detecting means for detecting a road white line based on an integrated image from the integrating means, and the characteristic region extracting means is the white line detecting means. The gist is to extract the preceding vehicle on the front lane based on the detected position of the road white line and the image output from the input image switching means.

[0007]

According to the present invention as set forth in claim 1, the forward running path of the vehicle is imaged by the image pickup means, and the moving distance of the host vehicle is calculated by the moving distance calculating means. Next, the front image captured by the image capturing unit is output for each preset moving distance , and integrated.
The means is to add this forward image to the integral image,
Generates a new integral image. Here, based on the detection result of the characteristic region of the front road up to the previous time, and the input image up to the previous time and the input image up to this time, the input image evaluation means determines whether or not the input image of this time is suitable for the characteristic region extraction processing. Based on the evaluation result, the input image switching means switches between outputting the integrated image and the input image for the characteristic region extraction processing, and the front road is output according to the output from the input image switching means. Since the characteristic region of the input image is extracted by the characteristic region extraction means, the optimum resource used for the characteristic region extraction processing can be switched according to the state of the input image of this time, and as a result, the input image of this time is appropriate. In the case of resources, it is possible to accurately extract the characteristic region of the road ahead.

According to the second aspect of the present invention, the reference image extracting means extracts, as the reference image, the input image of the area where the detection target is detected based on the characteristic area detection result from the characteristic area extracting means. Then, by calculating the correlation value between the extracted reference image and the comparison image with respect to the input image in the subsequent processing cycle by the correlation value calculation means, in the input image switching means, when the calculated correlation value is high. Selects the input image and outputs the integrated image when the correlation value is low, so it is possible to switch the optimum resource used for the feature region extraction processing according to the state of the input image this time. , As a result, when the input image of this time is an appropriate resource, especially when the imaging target is moving or the own vehicle is moving, the edge of the image of the imaging target is not blurred, Road ahead It is possible to extract the feature region exactly.

According to the third aspect of the present invention, the white line detecting means detects the road white line based on the integrated image from the integrating means to estimate the road area, and the characteristic area extracting means.
Is the detected position of the road white line and the input image switching means
Based on the image output from the preceding vehicle on the road ahead
Is detected, it is possible to accurately extract the characteristic region of the road ahead.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a diagram showing a system configuration of an in-vehicle traveling environment recognition device 1 according to a first embodiment of the present invention. In FIG. 1, the traveling speed detection unit 3 detects a traveling speed value from the number of rotations of the wheels and outputs it to the integration processing unit 7.

The camera 5 captures an image of the front lane of the vehicle in response to an input command from the integration processing unit 7, and captures the image on the integration processing unit 7, the reference image extraction unit 15, the correlation calculation unit 17, and the input image switching unit 19. The output image. The integration processing unit 7 calculates a moving distance value based on the speed value from the traveling speed detecting unit 3 and the processing cycle, integrates the forward image input from the camera 5 for each predetermined moving distance, and as a result, The obtained integrated image is output to the white line detection unit 9 and the input image switching unit 19.

The white line detection unit 9 detects a white line on the road based on the integrated image from the integration processing unit 7, and outputs the white line position to the preceding vehicle detection unit 11. The preceding vehicle detection unit 11 detects the preceding vehicle position on the front lane based on the white line position from the white line detection unit 9 and the image or the integral image input from the input image switching unit 19, and the result is used as a reference image. Extraction unit 15
Output to.

The input image evaluation unit 13 is a reference image extraction unit 1.
5 and the correlation calculation unit 17 and evaluates whether the current input image input to the camera 5 is suitable for the characteristic region extraction processing. Specifically, the reference image extraction unit 15 extracts the reference image from the input image based on the image from the camera 5 and the position of the preceding vehicle from the preceding vehicle detection unit 11 and outputs it to the correlation calculation unit 17. . The correlation calculation unit 17 calculates a correlation value based on the image from the camera 5 and the reference image from the reference image extraction unit 15, and the result is input image switching unit 19
Output to. The input image switching unit 19 selects and switches between the image input from the camera 5 and the integrated image input from the integration processing unit 7 based on the correlation value from the correlation calculation unit 17, and switches the preceding vehicle detection unit 11 Output to.

Next, the operation of the vehicle-mounted traveling environment recognition device 1 will be described with reference to FIGS. 5 to 8 and the flowcharts shown in FIGS. First, in step S10, the integration processing unit 7 calculates a moving distance value based on the speed value from the traveling speed detecting unit 3 and the processing cycle, and the moving distance value is equal to or greater than a preset certified threshold value. In the case where the above condition occurs, an input command is output to the camera 5 to cause the camera 5 to image the front runway, and the front runway image from the camera 5 is input. At the same time, the calculated moving distance value is cleared to zero.

In step S20, the integration processing section 7 adds the input image to the previously stored previous integrated image. Here, when the integrated image is G and the input image is F, G = G × (1−α) + F × α is calculated. Note that α is a coefficient value set in advance to represent the accumulation from the past, and is set to 0.5, for example. Next, in step S30, the white line detection unit 9
Road white lines are detected using this integral image. Although the method for detecting the left white line has been described in the present embodiment, the same method can also be used for detecting the right white line. Further, the position of the white line is represented by the start point and the end point.

Now, the white line detecting process performed by the white line detecting section 9 will be described with reference to the flowchart of the subroutine shown in FIG. First, in step S210, the SOBEL operator for extracting a vertical edge component is applied to the entire input integrated image of the forward runway to create a vertical edge image composed of a positive edge and a negative edge as shown in FIG. To do. Note that FIG. 5 is a diagram showing an example of a vertical edge image.

In step S220, as shown in FIG. 6, a detection window area is set at a position where the road white line appears on the screen, and the road white line is searched in the window. As the initial position of the window, for example, the position W1 (x1, y1) near the road white line appearing on the screen when traveling straight on an expressway is stored in advance, and this value is read to read the window position WP (xwp, ywp)
And Note that FIG. 6 is a diagram showing an example of window positions for detecting a white line.

Next, in step S230, straight line detection is performed. Now, with reference to the window position set in step S220, the straight line detection process will be described with reference to the flowchart of the subroutine shown in FIG. First, in step S310, a straight-line leading end initial position is set. That is, as shown in FIG. 7, the tip position of the straight line is calculated from the window position WP and the preset width WL of the search region. Therefore, as shown in FIG. 6, the initial position S (xs, ys) of the straight line is xs = xwp + WL / 2 ys = ywp.

Next, in step S320, the straight-line rear end initial position is set. That is, as shown in FIG. 7, the inclination of the white line to be detected is preset from the mounting state of the camera 5. Note that FIG. 7 is a diagram showing an example of variables in a window for detecting a white line. Here, if the detected inclination of the white line is θ ± α, the rear end initial position E (xe, ye) shown in FIG. 6 from the window height h is xe = xs−h × tan (θ−α ) Ye = ys-h.

In step S330, the tip initial position S
A straight line f passing through (xs, ys) and the trailing edge initial position E (xe, ye) is calculated. The straight line f becomes y-ye = mx (x-xe) m = (ys-ye) / (xs-xe). Therefore, as shown in FIG. 8, the straight line on the positive edge side and the straight line on the negative edge side are detected. In step S340, the number of edge points on the edge image through which the straight line f passes is measured. At this time, the numbers of both positive and negative edges are measured separately. The number of edges is stored as the measurement result in association with the leading end position xs and the trailing end position xe.

Next, in step S350, the x coordinate of the rear end of the straight line is moved. That is, for movement of the x coordinate, xe = xe-1 is substituted. In step S360, x is reached up to the lower limit of the rear end.
Move the coordinates. That is, if the rear end position xe is xe ≧ xs−h × tan (θ + α), the process returns to step S330 and is repeated.

On the other hand, the rear end position xe is xe <xs−h × tan (θ + α) In the case of, the process proceeds to step S370.

Next, in step S370, the x coordinate of the tip of the straight line is moved. That is, xs = xs-1 is substituted for the x-coordinate xs of the linear tip used. In step S380, it is determined whether or not the tip has moved to the lower limit. That is, when xs at the linear tip is xs <xwp-WL / 2, the process proceeds to step S390. On the other hand, when xs ≧ xwp−WL / 2, the process returns to step S330 and is repeated.

Next, in step S390, the maximum positive edge position is searched. That is, the tip position having the maximum number of positive edges is searched from the stored tip positions and the number of positive edges. Here, in step S400, it is determined whether or not this maximum number of positive edges is equal to or larger than a threshold value. If the number of searched positive edges is equal to or larger than the preset threshold value, it is determined that there is a white line, and the process proceeds to step S410. On the other hand, if it is less than or equal to this threshold value, the straight line detection processing ends.

In step S410, the straight edge position of the positive edge is determined. That is, the straight line searched as the position of the white line
Position (xps, ys) and rear end position (xpe, y) of
It is determined by substituting the x coordinate of e) . Next, step S4
At 20, the maximum negative edge position is searched. That is, the tip position having the maximum negative edge number is searched from the stored certain tip position and the number of negative edges. In step S430, it is determined whether the number of negative edges is greater than or equal to the threshold value. If the number of retrieved negative edges is equal to or greater than the preset threshold value, it is determined that there is a white line, and the process proceeds to step S440. On the other hand, if it is less than or equal to the threshold value, the straight line detection processing is ended.

Next, in step S440, the linear position of the negative edge is determined. That is, the front end position (xns, ys) and the rear end position (xn) of the straight line retrieved as the negative white line position.
It is determined by substituting the x coordinate of e, ye) . In the above process, the presence or absence of a white line in the detection area is determined, and if there is a white line, the position of the white line on the image is calculated.

Next, returning to the flow chart shown in FIG.
The white line detection process will be described. In step S240, as the determination of whether there is a white line, a positive white line position and negative white line position is calculated, the difference between the previous end position and rear end position is determined whether the following predetermined threshold . Front end position and rear end position
If the difference is less than or equal to the threshold value, there is a white line and the process proceeds to step S250. On the other hand, if it is not less than the threshold value, there is no white line and the process ends.

In step S250, the white line position is calculated. That is, a white line straight line is calculated from the detected positive white line position (xs, ys) and negative white line position (xe, ye) . Next, in step S260, the window position is moved. That is, the window position WP (xwp, yw for performing white line detection)
Substitute xwp = (xs + xe) / 2 into the x coordinate xwp of p). Thereby, even if the position of the white line moves, it can be detected. With that, the white line detection processing ends.

[0029] Referring back to the flowchart shown in FIG. 2, the operation of the vehicle running-environment recognition apparatus 1. In step S40, it is determined whether the white line detection unit 9 has detected a white line. If the road white line is detected, the process proceeds to step S50, and if not detected, the process returns to step S10 to repeat the above-described white line detection process.

In step S50, the reference image extraction unit 15
First, it is determined whether or not a reference image for evaluating the input image is extracted. When the reference image is extracted, the process proceeds to step S90, and when not extracted, the process proceeds to step S60. In step S60, the integrated image is selected as the processing image, and the preceding vehicle in the image is detected. Here, the detection of the vehicle is performed by using the horizontal edge of the area within the detected left and right road white lines. Since the left and right white lines are represented by the left and right front ends and the rear end, the straight lines connecting the front end and the rear end are calculated respectively for the left and right. The vehicle feature is searched for by using the calculated area between the left and right straight lines as a road area.

In the search for vehicle features, horizontal edges are searched from bottom to top of the screen. When the edges equal to or more than the set threshold value continuously extend in the horizontal direction for the length equal to or more than the preset length, it is determined as the edge of the vehicle region. When the search for all the pixels in the white lines on the left and right roads is completed, the horizontal edge determined to be the edge of the vehicle region is determined to have the largest x coordinate value and the smallest x coordinate as the left and right ends of the vehicle, and the center thereof The position and the difference between them are calculated as the width.

In step S70, it is determined whether the vehicle is detected in step S60. When the vehicle is detected, the process proceeds to step S120. On the other hand, if no vehicle is detected, the process proceeds to step S80. In step S80, the reference image extracted by the reference image extracting unit 15 is erased, and there is no reference image. On the other hand, in step S90, the correlation calculation unit 17 calculates the correlation value to determine whether the input image is normally input. That is, a comparative image larger than the reference image region is extracted from the input image of this time using the stored previous vehicle position. It should be noted that how much the comparison image is extracted from the reference image is preset from the state of the camera 5.
Next, the normalized correlation value between the reference image and the comparison image is calculated in the entire area of the comparison image. Here, the maximum value among the calculated correlation values is set as the representative correlation value of the comparison image.

In step S100, the input image switching unit 1
Reference numeral 9 determines whether or not the representative correlation value obtained by the correlation calculation unit 17 is equal to or larger than a preset threshold value, and examines the presence or absence of correlation. If the representative correlation value is equal to or greater than the preset threshold value, it is determined that the image input from the camera 5 is used to detect the vehicle, and the process proceeds to step S110. On the other hand, when the representative correlation value is equal to or less than the threshold value, it is determined that the integrated image input from the integration processing unit 7 is used, and the process proceeds to step S60.

In step S110, the input image switching unit 1
9 selects an image input from the camera 5 as a processed image. Next, the preceding vehicle detection unit 11 detects the preceding vehicle in the image. The method of detecting the preceding vehicle is step S
This is similar to the method shown in 60. Next, step S120.
Then, the reference image extraction unit 15 calculates the reference image area from the detected vehicle position and width in order to determine the position of the reference image. The size of the reference image region is calculated based on the vehicle aspect ratio and the detected width that are stored in advance. In step S130, the reference image extraction unit 15 extracts the reference image based on the image input from the camera 5 based on the calculated position and size of the reference image area.

In this way, the front lane of the vehicle is imaged by the camera 5 and the moving distance of the vehicle is calculated by the integration processing section 7.
Next, the front image captured by the camera 5 is integrated by the integration processing unit 7 for each preset moving distance to generate an integral image. Here, based on the detection result of the characteristic region of the front road up to the previous time and the input image up to the previous time and the input image up to this time, it is determined whether the input image of this time is suitable for the characteristic region extraction processing. In accordance with the output from the input image switching unit 19, the input image switching unit 19 switches whether to output the integrated image or the input image for the characteristic region extraction processing based on the evaluation result. Since the characteristic region of the road ahead is extracted by the preceding vehicle detection unit 11, the optimum resource used for the characteristic region extraction processing can be switched according to the state of the input image this time, and as a result, the input of this time is performed. When the image is an appropriate resource, the characteristic region of the road ahead can be accurately extracted.

The reference image extraction unit 15 extracts the input image of the region where the detection target is detected based on the characteristic region detection result from the preceding vehicle detection unit 11 as the reference image, and the extracted reference image and the next image. By calculating the correlation value with the input image in the subsequent processing cycle by the correlation calculation unit 17, in the input image switching unit 19, when the calculated correlation value is high, the input image in a state in which noise such as a shadow is small On the other hand, when the calculated correlation value is low, it is determined that the input image contains noise such as a shadow, and vehicle position detection is performed using the integral image. Therefore, when the input image is less affected by noise and shadows, the position of the vehicle can be accurately detected using the input image. In particular,
When the imaging target is moving or when the vehicle is moving, the edges of the image of the imaging target are not blurred,
It is possible to accurately extract the characteristic region of the road ahead.

Further, in the white line detecting section 9, the integration processing section 7
The road white line is detected on the basis of the integrated image from Eq. 1 to estimate the road region, and the preceding vehicle detection unit 11 detects the preceding vehicle on the forward road to extract the characteristic region of the forward road. Therefore, the characteristic region of the road ahead can be accurately extracted.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration of an in-vehicle traveling environment recognition device 1 according to a first embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of the vehicle-mounted traveling environment recognition device 1.

FIG. 3 is a subroutine for explaining white line detection processing.

FIG. 4 is a subroutine for explaining straight line detection processing.

FIG. 5 is a diagram showing an example of a vertical edge image.

FIG. 6 is a diagram showing an example of window positions for detecting a white line.

FIG. 7 is a diagram showing an example of variables of a window for detecting a white line.

FIG. 8 is a diagram showing an example of a detected straight line on the positive edge side and a detected straight line on the negative edge side.

[Explanation of symbols] 3 Running speed detector 5 camera 7 Integration processing unit 9 White line detector 11 Leading vehicle detector 13 Input image evaluation section 15 Reference image extraction unit 17 Correlation calculation unit 19 Input image switching section

─────────────────────────────────────────────────── ─── Continued Front Page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G01B 11/00-11/30 102 G01C 3/00-3/32 G06T 1/00-9/40 G08G 1 / 00-9/02 B60R 21/00-21/00 630

Claims (3)

(57) [Claims] 1. A imaging means for imaging the front track of the vehicle, the moving distance calculating means for calculating a moving distance of the vehicle, the output for each moving distance set in advance the forward image captured by the imaging means Based on the detection result of the characteristic region of the front road up to the previous time, the input image up to the previous time, and the input image up to this time. , Input image evaluation means for evaluating whether or not the input image this time is suitable for the characteristic region extraction processing, and whether to output an integral image for the characteristic area extraction processing based on the evaluation result from the input image evaluation means. A vehicle-mounted traveling environment characterized by having an input image switching means for switching whether to output an input image and a characteristic area extracting means for extracting a characteristic area of a road ahead according to an output from the input image switching means. recognition apparatus. 2. The input image evaluation means, based on a characteristic region extraction result from the characteristic region extraction means, a reference image extraction means for extracting an input image of an area in which a detection target is detected as a reference image, and a current imaging For the input image that was set, set a comparative image that is a wider range than the reference image extracted in the previous processing,
The comparison value of the comparison image and the correlation value between the reference image and
Correlation value calculating means for calculating in the entire region, the input image switching means is the maximum of the calculated correlation value.
The in-vehicle traveling according to claim 1, wherein when the large value exceeds a predetermined threshold value , the input image is selected, and when the large value is less than the threshold value, the integral image is selected. Environment recognition device. 3. Based on the integral image from said integrating means
The present invention further comprises white line detection means for detecting a road white line,
The area extracting means is the road detected by the white line detecting means.
The position of the white line and the image output from the input image switching means.
2. The vehicle-mounted traveling environment recognition device according to claim 1 , wherein a preceding vehicle on the road ahead is extracted based on the image .