JPH1040499A - Vehicle external recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simply correct the deviation of a camera parameter due to a pitch angle variation caused by a static change and to accurately detect lane marking by correcting an image pickup pattern during the period of traveling based on a detected traveling state and a coordinate-transformed traveling lane partition line. SOLUTION: In a traveling environment in which partition lines are indicated on a road in parallel, two traveling lane partition lines are practically extracted by using an initialized camera parameter, especially an angle of depression, so as to detect the parallelism of these partition lines. In the device, an output from a CCD camera 10 is sent to a lane marking detection picture processing unit 30 by a control unit 24 and straight component traveling lane partition line (lane marking)} is extracted by edge detection and Hough transformation. A current pitch angle variable against an initialized value can be estimated by reversely finding out such a depression angle of the camera that these two traveling lane partition lines are made parallel, so that a correction value can be found out.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for recognizing the outside world of a vehicle, and more particularly, to an apparatus for recognizing a traveling road from an image taken by a camera (image pickup means) for picking up a road surface in a traveling direction. The present invention relates to an apparatus for correcting a camera parameter with respect to an angle variation.

[0002]

2. Description of the Related Art An external environment recognizing device for a vehicle, in which a road surface in a traveling direction is imaged and a lane marking is recognized, is known from, for example, JP-A-8-30770.

In such an apparatus, an image input from a camera mounted on a vehicle is processed, and when the position of the obtained lane marking (lane marking) is converted into a real plane (lane road) coordinate, the camera and the It is necessary that the positional relationship with the traveling road surface is known. Some values indicating this positional relationship are called camera parameters (the same as the above-described imaging parameters).

Among the camera parameters, the one that seriously affects the above-mentioned coordinate conversion error is an error in the depression angle of the camera, that is, a change in the inclination angle of the camera due to a change in the attitude of the vehicle. When the attitude of the vehicle changes, especially when the pitch angle fluctuates, an error as shown in FIG. 19 occurs when coordinate conversion is performed without correcting the camera depression angle.

[0005] In this specification, the inclination of the vehicle in the front-rear direction with respect to the road surface is referred to as "pitch angle fluctuation of the vehicle". The inclination of the camera (imaging means) in the vehicle front-rear direction is referred to as “camera depression angle”.

The graph shown in FIG. 20 shows a vehicle (four occupants).
Is a measurement of pitch angle fluctuation when the vehicle is traveling straight at about 80 km / h. Specifically, a pitch angle of 0.019 [deg.] (Degrees) measured in a stopped state and two occupants was set as an initial value, and a change amount from the initial value was expressed with respect to a traveling distance [m].

[0007] The pitch angle of the vehicle at the time of data measurement is shifted in the plus direction (state in which the front of the vehicle is lifted) by about 0.35 [deg.] On average from the initial value. By comparison, the pitch angle change due to the shaking of the running vehicle is about ± 0.1 [deg.], Which is smaller than the steady posture change amount of the vehicle due to a change in the number of occupants or the like. .

As described above, the attitude change of the vehicle in the pitch angle direction is roughly divided into the following two. a) Dynamic changes Short-term changes that occur when the vehicle accelerates or decelerates during running, or shakes due to unevenness of the road surface or undulation. Some degree of change from steady state can be measured by accelerometer or pitch rate sensor. b) Static change Steady attitude change of the vehicle due to a change in the number of occupants or a load. Can be measured with an inclinometer or a vehicle height sensor. Even if it is static, it changes every time you run.

Although the detection error of the lane marking is not uniform depending on the contents of the intended automatic driving control, for example, the allowable error of the detected inclination angle of the lane marking with respect to the straight traveling direction is ± 0.5 [deg]. .], ± 0.4 [de
g.] The above change in pitch angle needs to be corrected.

[0010] In addition, an error factor at the time of coordinate conversion may be a road shape (a slope, a bank, or the like). When traveling on an expressway is basically used, the road structure is regulated by laws and regulations. Since the width of the traveling road (lane) is constant and the left and right dividing lines (marking) can be basically assumed to be parallel, correction can also be performed from the image processing result.

With respect to such a pitch angle fluctuation, in the prior art, for example, a vehicle height sensor (ultrasonic distance sensor) is provided at each of the four corners of the vehicle to measure the height with respect to the road surface, and the pitch angle fluctuation is measured. However, the camera parameters are corrected, but the configuration is complicated in that four vehicle height sensors are required, and the cost increases.

For this purpose, as proposed in Japanese Patent Application Laid-Open No. 3-276221, in a vehicle that can be switched between manual driving and automatic driving, the difference between the external recognition result and the recognition result estimated from the manual operation amount is determined. There has been proposed a method for correcting the external world recognition result at the time of automatic traveling so as to eliminate the deviation and the deviation. Japanese Unexamined Patent Publication No. Hei 3-118611 also discloses a configuration in which the image processing start position is changed in accordance with the vibration of the image input means, that is, the image processing start position is moved up and down in accordance with the pitch angle fluctuation.

[0013]

However, the prior art described above mainly focuses on correcting a deviation caused mainly by a dynamic change in pitch angle fluctuation, and therefore the configuration is complicated, for example, the correction is always performed. there were. As described with reference to FIG. 20, a static change such as an increase or decrease in the number of occupants or a loaded cargo is more dominant as a cause of the pitch angle fluctuation. Moreover, while the vehicle is running, the number of occupants and the increase or decrease of the loaded cargo should not change, so that the correction does not need to be performed frequently during the running.

Accordingly, an object of the present invention is to solve the above-mentioned drawbacks of the prior art, and to easily correct a deviation of camera parameters due to a pitch angle fluctuation caused by a static change so that lane marking detection can be accurately performed. Another object of the present invention is to provide a vehicle external recognition device that can be corrected only once during traveling.

Further, the camera is mounted on the vehicle so as to look directly at the traveling direction. However, the center axis of the camera field of view may be shifted with respect to the vehicle running center line due to mounting variations. Such a shift of the pan angle similarly causes a lane marking detection error. On the other hand, if the mounting accuracy is to be increased, the production efficiency must be sacrificed to some extent.

Therefore, another object of the present invention is to solve the above-mentioned drawbacks of the prior art, to improve the lane marking detection accuracy by simply and appropriately correcting the pan angle, and not to reduce the production efficiency. And a vehicle external recognition device.

[0017]

In order to achieve the first object, an external recognition apparatus for a vehicle according to the present invention comprises: an image pickup means for picking up an image of a road surface in a traveling direction; A traveling road lane marking extracting means for extracting a set of lane markings; a coordinate converting means for transforming the extracted lane markings into a real plane based on predetermined imaging parameters; and detecting a traveling state. Traveling state detection means,
And an imaging parameter correction unit that corrects the imaging parameter at least once during traveling based on the detected traveling state and the traveling-path lane marking that has been subjected to the coordinate conversion.

According to a second aspect of the present invention, the image pickup parameter correction means includes a determination means for determining whether or not at least one set of travel path dividing lines subjected to coordinate conversion has a predetermined relationship. When it is determined that the road segmentation lines do not have the predetermined relationship, the imaging parameters are corrected so as to have the predetermined relationship.

According to a third aspect of the present invention, the positional relationship between the imaging means and a traveling road surface, wherein the imaging parameter changes due to a steady change in the attitude of the vehicle including a change in the number of vehicle occupants. The parameters were configured as shown.

According to a fourth aspect of the present invention, the traveling state is:
The vehicle is configured such that the vehicle travels straight ahead in a traveling environment in which the traveling path dividing lines are parallel.

According to a second aspect of the present invention, there is provided an external environment recognition apparatus for a vehicle, comprising: an image pickup device for picking up an image of a road surface in a traveling direction; A coordinate conversion unit for performing coordinate conversion on a real plane, a declination calculation unit for comparing a coordinate system of a captured image of the imaging unit with a reference line, and obtaining a declination with respect to a traveling direction, and An imaging parameter correction unit for correcting an imaging parameter is provided.

According to a sixth aspect of the present invention, the image pickup parameter correcting means includes a notifying means for notifying whether or not the correction has been completed.

[0023]

In the apparatus according to the first aspect, it is possible to easily correct the deviation of the camera parameter due to the pitch angle fluctuation caused by the static change, and to accurately detect the lane marking. In addition, since the correction is performed only once during traveling, the configuration is simple. The details of the correction are as described in claims 2 to 4.

In the apparatus according to the fifth aspect, the pan angle can be easily and properly corrected to improve the detection accuracy of the lane marking, and the pan angle deviation due to the mounting variation can be easily corrected. Since it is possible, it is not necessary to raise the mounting accuracy beyond a predetermined standard, and therefore, the production efficiency does not decrease.

In the device according to the sixth aspect, the driver can recognize whether or not the correction has been completed, and take a necessary measure such as not selecting the automatic operation when the correction is not completed continuously. Can be taken.

[0026]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is an explanatory perspective view showing the entire vehicle external recognition apparatus according to the present invention. In FIG.
Reference numeral 0 denotes a CCD camera (monochrome TV camera) (the above-described image pickup means), and the CCD camera 10 is fixed to a position near a rear-view mirror mounting position above a driver's seat, and allows a single-eye view of the traveling direction of the vehicle.

Reference numeral 12 denotes a yaw rate sensor which is provided near the center of the vehicle interior and detects angular acceleration about the vertical axis (z axis) of the vehicle center of gravity. Further, a vehicle speed sensor 14 composed of a reed switch is provided near a drive shaft (not shown) of the vehicle to detect a traveling speed of the vehicle, and a steering angle sensor 16 is provided near a steering shaft 18 of the vehicle. To detect the steering angle.

A steering angle control motor 20 is mounted on the steering shaft 18, a throttle actuator 22 composed of a pulse motor is mounted on a throttle valve (not shown), and a brake (not shown) is mounted on a brake (not shown). A pressure actuator (not shown) is attached.

The output of the above-described CCD camera 10 and the like is as follows:
It is sent to the control unit 24.

FIG. 2 is a block diagram showing the configuration of the control unit in more detail.

The output of the CCD camera 10 is supplied to the control unit 2
4, the lane marking detection image processing unit 30
, And the extraction of a straight line component (traveling lane marking (lane marking)) is performed by edge detection and Hough transformation. Outputs of the steering angle sensor 16 and the like are sent to a vehicle traveling state detection unit 32 via a processing circuit (not shown), and outputs of a control actuator such as the steering angle control motor 20 are also transmitted through the vehicle control unit 34. It is sent to the vehicle running state detection unit 32.

Lane marking detection image processing unit 3
0, the outputs of the vehicle running state detection unit 32 and the vehicle control unit 34 are input to the comprehensive judgment unit 36. On the other hand, a driver (driver) inputs a command to switch from manual driving to automatic driving through a driver interface 38 (not shown in FIG. 1) provided on a dashboard in front of the driver's seat of the vehicle. Is input to

When the switching command is issued, the comprehensive judgment unit 36 controls the steering angle and the vehicle speed appropriately based on the image input obtained from the CCD camera 10 in a predetermined driving state based on the input information, and controls the braking operation. Perform automatic operation. The automatic driving is, for example, a technique proposed by the present applicant in Japanese Patent Application Laid-Open No. 7-81604. However, since the gist of the present invention lies in the correction of camera parameters, a description thereof will be omitted.

Next, the operation of the vehicle external recognition apparatus according to the present invention will be described with reference to the flowchart of FIG. This is the operation performed by the above-mentioned comprehensive judgment unit 36.

The illustrated program is basically executed once during one running from the start of the engine to the stop of the engine. Specifically, this process is performed when an ignition switch (not shown) is turned on and the output of the vehicle speed sensor 14 reaches a predetermined value from zero.

Since the object of the present invention is to correct camera parameters due to a static change in pitch angle fluctuation,
It is desirable to detect a situation in which a static change in pitch angle variation has occurred during traveling, for example, stop in a parking area, and then perform this processing thereafter. Such a situation may be detected, for example, by providing a means for detecting the opening and closing of the vehicle door, and detecting whether or not the vehicle speed is zero for a predetermined time from the output of the vehicle speed sensor 14.

First, in step S10, the pitch angle variation (deviation) is initially set to zero, and the process proceeds to step S12 to determine whether or not the vehicle is traveling straight ahead at a predetermined speed or higher.

Preferably, in order to reduce the lane marking detection error in the automatic driving, it is determined whether or not the vehicle is traveling in a straight line at a speed equal to or higher than a predetermined speed, not in the automatic driving, but in the manual driving in which a person is operating. . A predetermined speed (for example, 8
0 km / h) is estimated (determined) from the output of the vehicle speed sensor 14 and whether the vehicle is traveling straight is determined from the outputs of the yaw rate sensor 12 and the steering angle sensor 16.

That is, in this device, it is assumed that the environment in which the lane markings such as a highway are prepared is provided.
Based on the assumption that the left and right parting lines of the traveling path are parallel, a static variation in the pitch angle of the vehicle is obtained and corrected. The reason for requesting that the speed be higher than
This is to estimate that the vehicle is traveling on a highway.

When the result in S12 is affirmative, the program proceeds to S14,
A lamp (not shown) provided on a dashboard panel in front of the vehicle driver's seat is turned on to notify (notify) that the correction process is being performed, and the process proceeds to S16 to perform lane marking detection (extraction) image processing. . As described above, this is a process of detecting a straight line component from edge detection, Hough transform, and the like, and detecting (extracting) lane markings (traveling road division lines). The lane marking is used to mean not only a broken line displayed between the driving lanes but also a solid line indicating the road shoulder.

Then, the program proceeds to S18, in which it is determined whether or not left and right lane markings have been detected.
, The detected lane marking is linearly approximated on a real plane (traveling road surface), and a straight line formula of two right and left straight lines and an integrated value (average value) thereof are obtained. Then, the program proceeds to S22, in which it is determined whether or not the process has been repeated a predetermined number of times. If the result is affirmative, the process proceeds to S24, where the integral value is divided by (the number of processing times -1) to calculate the current linear equation.

More specifically, first, the detected lane marking position is converted into coordinates on a real plane using the initial set values of the camera parameters including the camera depression angle, and the left and right two straight lines of the detected lane marking are converted. Find the linear equation on the real plane. FIG. 4 shows the contents of the camera parameters, and FIG. 5 shows the approximate straight lines of the detected left and right lane markings. Then, the lane marking detection, the conversion into the coordinates on the real plane, and the linear equation calculation process are repeatedly performed a predetermined number of times, and an integral value of two straight lines as shown in FIG. 6 is obtained. Each of the above integral values is divided by (the number of processing times -1) to obtain a current linear equation.

That is, as shown in FIGS. 5 and 6, two straight lines on the left and right are L1 and L2, and a linear equation on a real plane is L1: y = A1.x + B1 L2: y = A2.x + B2 , And the integral value is obtained for each of the slope A and the intercept B of each straight line.

To explain this integration, if the lane marking detection image processing is performed on an input image at constant intervals, the resulting linear data of two straight lines on the left and right will be treated as equidistant dividing points. Conceivable. Therefore, the integrated value of the data of each linear equation is given an approximate value by an interpolation formula represented by the Newton-Cotes formula.

As an example, the slope A1 of the straight line L1 will be described. By the (n + 1) times processing, the data A1 (0), A1 (1), A1 (2), A1 (3), A1 (4),... A1 (n-1),
It is assumed that A1 (n) is obtained (n: even number). Section of integration (a, b)
Is (0, n). Using the simplest trapezoidal rule in the Newton-Cotes formula, the approximate value S of the integral is given as follows. S = h (A1 (0) / 2 + A1 (1) + A1 (2) + ... + A1 (n-1) +
A1 (n) / 2) (However, h = (b-a) / n)

If the Simpson's rule is used, an approximate value S of the integral in the same case is given as follows. S = (h / 3) (A1 (0) + 4A1 (1) + 2A1 (2) + 4A1 (3) + 2A1 (4) +
... 4A1 (n-1) + A1 (n)) (However, h
= (B−a) / n)

The current slope A1 is obtained as follows by dividing the approximate value S of the integral thus obtained by m (= the number of processing times-1). A1 = S / m

Subsequently, the program proceeds to S26, in which it is determined whether or not the obtained two straight lines are parallel (whether or not the above-mentioned predetermined relationship is established).
That is, the sandwich angle θ (shown in FIG. 7) between the two left and right straight lines L1 and L2 is obtained as θ = atan (A1) −atan (A2), and the obtained sandwich angle θ is compared with a threshold value which is appropriately set. Do it by doing.

More specifically, when the obtained included angle θ is smaller than the threshold value, the two straight lines are regarded as parallel. The threshold value is ± 0.1 from the requirement for the automatic operation control described above.
[Deg.].

As described above, in the driving environment where the lane markings are displayed in parallel on the road, two camera parameters are set by default so that the parallelism of the lane markings can be detected. The current pitch angle change amount (deviation) with respect to the initial setting value is obtained by extracting the travel path dividing lines of
Was estimated, and the correction amount was calculated accordingly.

When the result in S26 is NO, the program proceeds to S28,
A camera depression angle that makes the two detected straight lines parallel (that is, the predetermined relationship described above) is obtained, a current pitch angle change amount with respect to an initial setting value is estimated, and camera parameters are corrected.

To explain this, the linear equation on the real plane of the two right and left straight lines L1 and L2 obtained by integration is as follows: L1: y = A1.x + B1 L2: y = A2.x + B2 , The center line y = A.x + B in the camera coordinate system of L1 and L2 as shown in FIG.

The center line is a straight line having a slope A = tan ((atan (A1) + atan (A2)) / 2) and an intercept B = (B1 + B2) / 2.

Subsequently, the two straight lines L1 and L2 are converted into image (imaging plane) coordinates. As shown in FIG. 8, the linear equation on the image (imaging plane) coordinates is L1 ': yd = Ad1.xd + Bd1 L2': yd = Ad2.xd + Bd2.

Lines L1 ', L on image (imaging plane) coordinates
When 2 ′ is converted again into the real plane (camera coordinate system), a camera depression angle (camera parameter correction value) is determined such that the inclinations A1 ″ and A2 ″ become equal to the previously obtained inclination A of the center line. If it is difficult to obtain a camera depression angle such that A1 "and A2" are both equal to A, A1 "and A1"
The camera depression angles that are equal to A for each of 2 ”may be separately obtained and averaged. Further, when L1 is a solid line displayed on the road shoulder side and L2 is a broken line, A1”
May be used with a camera depression angle that is equal to A.

In this operation, more specifically, as shown in FIG. 9, the processing area is changed by an amount corresponding to the pitch angle change amount.
The image may be translated in the vertical direction. That is,
Other camera parameter correction methods include the CCD camera 1
Various measures such as physically changing the mounting position of 0 and changing the coordinate conversion calculation value can be considered, but correction may be made by adjusting the processing area as shown in the figure.

Then, the process proceeds to S30, where the specified number of times (for example, when the lane marking detection image processing time is 66 ms, 5
Times), and if not affirmed, S12
And the above processing is repeated using the corrected camera parameters.

When it is determined in S26 that the two straight lines are parallel, the process proceeds to S32, in which another lamp (not shown) provided on the dashboard panel is turned on, and a notification (notification) that the correction is completed is made. Go and exit the program. Also S
If it is determined in step 30 that the number of times has exceeded the specified number, the process proceeds to step S34, in which another rank is lit, a notification (notification) indicating that correction was not possible (correction not completed) is performed, and the process returns to step S10.

When the result in S12 is negative, the program proceeds to S36, in which another lamp is turned on to notify (notify) that the traveling condition is not correctable, and the program returns to S10.
When it is determined in S18 that the left and right lane markings have not been detected, the process proceeds to S38, in which it is determined whether or not the number has exceeded a specified number of times (for example, five times).
Returning to S2, if affirmative, proceed to S40 to S3
The same process as in No. 4 is performed, and the process returns to S10.

As described above, the illustrated program is basically started at the beginning of traveling or the like, and is executed only once during the travel from engine start to stop.

With the above-described configuration, it is possible to easily correct a deviation of camera parameters due to a pitch angle fluctuation caused by a static change and to accurately detect lane marking. In addition, since the correction is performed only once during traveling, the configuration is simple.

Further, since the completion of the correction is notified (notified), the driver can recognize whether or not the correction has been completed. If the correction is not completed continuously, the driver does not select the automatic operation. Necessary measures can be taken.

Next, the correction of the pan angle of the camera parameter will be described.

The CCD camera 10 is attached with the central axis of the vehicle's visual field and the straight traveling direction of the vehicle so that the traveling direction is directly seen in front of the vehicle. Specifically, the coordinate system includes a camera coordinate system (shown in FIG. 10) in which the center of the field of view of the CCD camera 10 is the Xr axis and the focal position of the lens is the coordinate origin, and a straight forward direction of the vehicle is an Xr-car axis. A coordinate system (referred to as a "vehicle local coordinate system" shown in FIG. 11) having a predetermined position (such as the position of the center of gravity) of the vehicle as a coordinate origin.

Then, when performing coordinate transformation on a real plane,
To be precise, as shown in FIG. 12, the determination is performed with reference to the local coordinate system of the vehicle. FIG. 13 shows the conversion of the points at that time, and FIG.
4 shows the details of the straight line conversion.

Therefore, the CCD camera 10 is mounted so that the vertical axis (X-axis) of the camera coordinate system and the vehicle local coordinate coincide with each other. However, in actuality, the CCD camera 10 is displaced from the mounting variation and the like, as shown in FIG. It is likely that the detection lane marking in the camera coordinate system always has a certain declination. The pan angle correction of the camera parameter is a work of correcting such a shift of the central axis of the camera field of view with respect to the straight traveling direction of the vehicle and making the X axes of both coordinate systems coincide as shown in FIG.

FIG. 17 is a flowchart showing the operation.

Once the pan angle correction is performed, it is not necessary to repeat the pan angle correction unless the camera is attached / detached. Therefore, the illustrated program is performed, for example, at the time of inspection after manufacturing, or at the time of periodic inspection after traveling a predetermined distance. Therefore, the pan angle correction is a process performed prior to the depression angle correction described above with reference to the flowchart of FIG. Since the pan angle correction is similar to the depression angle correction described above, the description will be briefly omitted.

In the following, the pan angle correction amount Pan is initially set to zero in S100, and the flow advances to S102 to perform simulated lane marking detection (extraction) image processing. This is to detect (extract) a simulated lane marking (specifically, two parallel straight lines similar to the preceding lane marking) displayed on the road surface of the inspection factory in order to confirm the straight traveling direction of the vehicle. Processing.

Then, the program proceeds to S104, in which it is determined whether or not left and right lane markings have been detected.
Proceeding to step 06, the detection lane marking is linearly approximated on the real plane, and the declination between the center line of the two right and left straight lines (the above-mentioned reference line) and the vertical axis of the real plane coordinates and its integral value are obtained. It is determined whether or not the process has been repeated a predetermined number of times.

That is, as shown in FIG. 18, the detected simulated lane marking position is converted into coordinates on a real plane using the initially set camera parameters.
n = 0). Subsequently, the center lines of the two left and right straight lines of the detected simulated lane marking are obtained on a real plane, and then the deviation between the obtained center line and the vertical axis (X axis) of the vehicle local coordinate system is obtained. The method of obtaining the center line is the same as the above-described depression angle correction, as shown in the figure.

The above processing is repeated and performed a predetermined number of times to obtain the integral value of the argument (the argument is positive when the counterclockwise direction is turned).
The integral value is divided by (the number of processing times -1) to obtain the current argument. Subsequently, the flow proceeds to S112 to judge whether or not the declination obtained is smaller than a threshold value (about ± 0.1 [deg.] For the same reason as described above). The angle is added to the pan angle correction amount Pan (initial setting value = 0) to correct the pan angle correction amount Pan (subtraction is made when a negative argument is obtained).

Next, the process proceeds to S116, and unless it is determined that the specified number of times has been exceeded, the process immediately returns to S102. If it is determined that the specified number of times has been exceeded, the process proceeds to S118 to perform the same processing as in the above-described flowchart of FIG. A lamp (not shown) provided on the dashboard is turned on to notify (notify) that the correction could not be performed (correction has not been completed), and the program ends. In this case, by notifying that the correction has not been completed, it is possible for the inspector to take measures such as checking whether there is any abnormality in the apparatus.

If it is determined in step S112 that the argument is less than the threshold value, the process proceeds to step S120, in which the lamp is turned on to notify (notify) the completion of the pan angle correction, and the program ends.

If the result in S104 is NO, S12 is reached.
The process proceeds to step S2 to determine whether the number of times has exceeded the specified number. If the result is negative, the process returns to step S102.
Then, the same processing as in S118 is performed, and the program is terminated. At this time, the inspection staff may check the device as needed.

With the above configuration, it is possible to easily and properly correct the pan angle and improve the detection accuracy of lane marking. Further, since the pan angle deviation due to the mounting variation can be easily corrected, it is not necessary to increase the mounting accuracy beyond a predetermined standard, and therefore, the production efficiency is not reduced.

In the above description, the center line of the lane marking is used as the reference line. However, the present invention is not limited to this, and a white line may be displayed at the inspection factory and used as the reference line. Further, although the reference line is detected by running the vehicle, the road surface may be projected forward via a projector or the like while the vehicle is stationary, and the reference line therein may be detected.

Although the present invention has been described with respect to both camera parameter correction and pan angle correction based on pitch angle fluctuation, pan angle correction is not necessarily required to achieve the first object, and the second angle Needless to say, correction of camera parameters is not essential when achieving the purpose.

Further, although the present invention has been described with respect to a device capable of switching between manual operation and automatic operation, it goes without saying that the type of automatic operation does not matter. However, as described above, the allowable error of the camera parameter differs depending on the type of the automatic driving, and accordingly, the accuracy of correction by correction differs.

Further, the configuration in which one CCD camera is used for monocular vision is taken as an example, but a configuration in which two cameras are used for binocular vision may be used.

[0082]

According to the first aspect of the present invention, lane marking detection can be accurately performed by simply correcting a camera parameter shift due to a pitch angle change caused by a static change. In addition, since the correction is performed only once during traveling, the configuration is simple.

In the apparatus according to the fifth aspect, it is possible to easily and properly correct the pan angle to improve the detection accuracy of the lane marking, and to easily correct the pan angle deviation due to the mounting variation. Since it is possible, it is not necessary to increase the mounting accuracy beyond a predetermined standard, and therefore, the production efficiency does not decrease.

[Brief description of the drawings]

FIG. 1 is an explanatory perspective view showing an entirety of a vehicle external recognition apparatus according to the present invention.

FIG. 2 is a functional block diagram illustrating details of a processing unit illustrated in FIG. 1;

FIG. 3 is a flowchart showing the operation of the apparatus in FIG. 1;

FIG. 4 is an explanatory diagram showing camera parameters used in the flow chart of FIG. 3;

FIG. 5 is an explanatory diagram showing approximate straight lines of lane markings detected in the flowchart of FIG. 3;

6 is an explanatory diagram showing a straight line of lane marking obtained after integration in the flow chart of FIG. 3;

FIG. 7 is an explanatory diagram showing an operation of obtaining a sandwich angle between two straight lines required to determine whether or not the two straight lines obtained in the flowchart of FIG. 3 are parallel;

FIG. 8 is an explanatory diagram showing an operation for obtaining a camera depression angle such that two straight lines detected in the flowchart of FIG. 3 are parallel to each other.

FIG. 9 is an explanatory diagram showing an example of a camera parameter correction method in the flow chart of FIG. 3;

FIG. 10 is an explanatory diagram showing a coordinate system of the CCD camera shown in FIG.

FIG. 11 is an explanatory diagram illustrating a vehicle local coordinate system in which a vertical direction (X-axis) is defined as a direction in which the vehicle travels straight ahead, and a predetermined position of the vehicle is defined as a coordinate origin.

12 is an explanatory diagram showing an operation when the camera coordinate system of FIG. 10 is converted to the vehicle local coordinate system of FIG. 11;

FIG. 13 is an explanatory diagram showing in detail the transformation from the camera coordinate system of FIG. 12 to the vehicle local coordinate system with respect to the transformation of points;

14 is an explanatory diagram showing in detail the transformation from the camera coordinate system of FIG. 12 to the vehicle local coordinate system with respect to the transformation of a straight line.

FIG. 15 is an explanatory diagram showing a pan angle shift indicating a shift of a center axis of a camera visual field with respect to a traveling direction of a vehicle.

FIG. 16 is an explanatory diagram showing pan angle correction in which the vertical axis (X-axis) of the camera coordinate system and the vehicle local coordinate system are made to coincide with each other, and the shift of the center axis of the camera visual field with respect to the traveling direction of the vehicle is corrected.

FIG. 17 is a flowchart specifically showing a pan angle correction operation shown in FIG. 16;

FIG. 18 is an explanatory diagram showing a calculation operation of an argument in the flowchart of FIG. 17;

FIG. 19 is an explanatory diagram showing the influence of vehicle pitch angle fluctuation on lane marking detection.

FIG. 20 is a graph showing an analysis result of vehicle pitch angle fluctuation.

[Explanation of symbols]

 Reference Signs List 10 CCD camera (imaging means) 12 Yaw rate sensor 14 Vehicle speed sensor 16 Steering angle sensor 24 Control unit 30 Total judgment unit

Continued on the front page (51) Int.Cl. ⁶ Identification number Agency reference number FI Technical display location H04N 7/18 G06F 15/70 330G

Claims (6)

[Claims] 1. A method according to claim 1, Imaging means for imaging a road surface in the traveling direction; b. Traveling road lane marking extracting means for extracting at least one set of lane markings from the captured image; c. Coordinate conversion means for performing coordinate conversion on the real plane on the basis of a predetermined imaging parameter, based on a predetermined imaging parameter; d. Traveling state detecting means for detecting a traveling state; and e. An external environment recognizing device for a vehicle, comprising: an imaging parameter correction unit configured to correct the imaging parameter at least once during traveling based on the detected traveling state and the coordinate-transformed traveling road dividing line. 2. The method according to claim 1, wherein the imaging parameter correction unit includes: f. Determining means for determining whether or not at least one set of travel path division lines subjected to the coordinate conversion is in a predetermined relation; The apparatus according to claim 1, wherein the imaging parameter is corrected so as to be as follows. 3. The imaging parameter is a parameter indicating a positional relationship between the imaging unit and a traveling road surface, which changes due to a steady change in the attitude of the vehicle including a change in the number of occupants of the vehicle. The external recognition apparatus for a vehicle according to claim 1 or 2, wherein: 4. The external environment of a vehicle according to claim 1, wherein the traveling state is a state in which the vehicle travels straight in a traveling environment in which the lane markings are parallel. Recognition device. G. Imaging means for imaging a road surface in the traveling direction; h. Coordinate conversion means for performing coordinate conversion of the captured image on a real plane based on predetermined imaging parameters; i. Declination calculation means for comparing the coordinate system of the image picked up by the image pickup means with a reference line and obtaining a declination with respect to the traveling direction; and j. An external parameter recognizing device for a vehicle, comprising: an image capturing parameter correcting means for correcting the image capturing parameter based on the obtained declination. 6. The imaging parameter correction unit, comprising: k. 6. The external environment recognition device for a vehicle according to claim 1, further comprising a notification unit configured to notify whether or not the correction has been completed.