JP2004103018A - Vehicle periphery monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring device capable of detecting a movement of an object existing around a vehicle more accurately, determining a possibility of collision, and appropriately issuing a warning to a driver.
SOLUTION: From images obtained by two infrared cameras 1R, 1L, a position of an object around a vehicle in a real space is detected using a coordinate system (X, Y, Z) fixed to the vehicle. When the ratio between the distance between the vehicle and the object (coordinates in the Z direction) and the coordinates of the object in the horizontal direction (X direction) at 90 ° with respect to the traveling direction of the vehicle is substantially constant, It is determined that an object may collide.
[Selection diagram] Fig. 1

Description

According to the present invention, a collision with a large animal such as a deer or a bear affects the running of the vehicle. Therefore, in order to avoid the collision, there is a possibility that the vehicle will collide with an image obtained by an imaging unit mounted on the vehicle. The present invention relates to a surroundings monitoring device that monitors the surroundings of a vehicle to detect an object.

A vehicle is equipped with two CCD cameras, and a distance between an object and the vehicle is detected based on a difference between images obtained from the two cameras, that is, parallax, and a pedestrian 30 to 60 m ahead of the vehicle is detected. A crossing object detecting device configured as described above is conventionally known (Patent Document 1).
JP-A-9-226490

However, the above-described conventional apparatus uses the direction of the optical flow of the object detected based on the image obtained by the camera as it is to determine the possibility of collision, and thus the relative distance between the host vehicle and the object. There is a problem that the determination accuracy is reduced depending on the vehicle speed and the vehicle speed. That is, for example, when the vehicle speed is higher than the speed of the moving object, the optical flow on the image corresponding to the moving object that is actually moving toward the center of the road is captured as a vector directed to the outside of the road. Erroneous determination may occur.

The present invention has been made by paying attention to this point, and it is intended to more accurately detect the movement of an object existing around the vehicle, determine the possibility of collision, and more appropriately issue a warning to the driver. It is an object of the present invention to provide a vehicle periphery monitoring device capable of performing the following.

In order to achieve the above object, an invention according to claim 1 is a vehicle periphery monitoring device that detects an object existing around the vehicle from an image obtained by an imaging unit mounted on the vehicle. Relative position detecting means for detecting the relative position of the object with respect to the vehicle from the obtained image as position data, and the actual position of the object based on the position data of the object detected by the relative position detecting means. Real space position calculating means for calculating a position in space; a distance (Zv (N-1)) to the object calculated by the real space position calculating means; When the ratio (Xv (N-1) / Zv (N-1)) with respect to the horizontal coordinate (Xv (N-1)) at 90 degrees is substantially constant, the object is Colliding with a vehicle Characterized in that it comprises a determination means that there is a potential.

According to a second aspect of the present invention, in the vehicle surroundings monitoring apparatus according to the first aspect, the determination unit performs the determination using a collision determination condition (Equation (12)) corresponding to a vehicle width of the vehicle. It is characterized by performing.

撮 像 It is desirable that the imaging means be two infrared cameras capable of detecting infrared rays. Thus, it is possible to easily extract an animal, a running vehicle, and the like during night driving, which is difficult for the driver to confirm.

The imaging unit is two television cameras that detect infrared light or visible light, and based on the position of the target image included in the output image of one camera, the output image of the other camera corresponding to the target image. A search area for searching for an object image included in the search area is set, the corresponding target image is identified by performing a correlation operation in the search area, and from the vehicle from the parallax between the target image and the corresponding target image, Calculate the distance to the object. Preferably, the correlation operation is performed using a grayscale signal including halftone information. By using a grayscale signal instead of a binarized signal, a more accurate correlation operation can be performed, and it is possible to avoid erroneously specifying a corresponding target image.

(4) An object moving in the output image of the imaging means is tracked using a signal obtained by binarizing the gray scale signal. In this case, it is desirable that the tracking target be recognized by run-length encoded data. As a result, it is possible to reduce the data amount of the tracked object and save the memory capacity.

Further, it is desirable to determine the identity of the tracking target object based on the barycentric position coordinates, the area, and the aspect ratio of the circumscribed rectangle. This makes it possible to accurately determine the identity of the objects.
When it is determined that the possibility of collision with the target object is high by the determination means, warning means for issuing a warning to the driver, the warning means, when the driver is performing a brake operation, When the acceleration due to the brake operation is larger than a predetermined threshold, it is desirable not to issue an alarm. If the driver has already noticed the target object and has performed an appropriate brake operation, no warning is issued, so that the driver is not bothered.

According to the first aspect of the present invention, when the ratio between the distance to the object and the coordinates of the object in the horizontal direction at 90 degrees to the traveling direction of the own vehicle is substantially constant, It is determined that the target object may collide with the host vehicle. Therefore, it is possible to determine the possibility of collision for an object that moves so as to cross the traveling direction of the vehicle.

According to the second aspect of the present invention, the possibility of collision is determined using the collision determination condition according to the vehicle width of the host vehicle. Therefore, the possibility of collision is more accurately determined, and an unnecessary warning is issued. Can be prevented.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. This device detects two infrared cameras 1R and 1L capable of detecting far infrared rays and a yaw rate of the vehicle. A yaw rate sensor 5, a vehicle speed sensor 6 for detecting a traveling speed (vehicle speed) VCAR of the vehicle, a brake sensor 7 for detecting an operation amount of a brake, and image data obtained by these cameras 1R and 1L. An image processing unit 2 that detects an object such as an animal in front of the vehicle and issues an alarm when the possibility of collision is high, a speaker 3 that issues an alarm by voice, and an image obtained by the camera 1R or 1L. And a head-up display (hereinafter, referred to as “HUD”) 4 for causing the driver to recognize an object having a high possibility of collision.

As shown in FIG. 2, the cameras 1R and 1L are arranged in front of the vehicle 10 at positions substantially symmetrical with respect to the lateral center axis of the vehicle 10, and the optical axes of the two cameras 1R and 1L are They are fixed so that they are parallel to each other and have the same height from the road surface. The infrared cameras 1R and 1L have such a characteristic that the higher the temperature of the target, the higher the output signal level (the higher the luminance).

The image processing unit 2 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (Central Processing Unit) that performs various arithmetic processes, (Random Access Memory) used to store data, ROM (Read Only Memory) that stores programs, tables, maps, and the like executed by the CPU, an output circuit that outputs a driving signal of the speaker 3, a display signal of the HUD 4, and the like. The output signals of the cameras 1R and 1L and the sensors 5 to 7 are converted into digital signals and input to the CPU.
As shown in FIG. 2, the HUD 4 is provided so that a screen 4a is displayed at a position in front of the driver in a front window of the vehicle 10.

FIG. 3 is a flowchart showing the procedure of processing in the image processing unit 2. First, the output signals of the cameras 1R and 1L are A / D converted and stored in the image memory (steps S11, S12 and S13). The image stored in the image memory is a gray scale image including luminance information. FIGS. 5A and 5B are diagrams for explaining a grayscale image (a right image is obtained by the camera 1R and a left image is obtained by the camera 1L) obtained by the cameras 1R and 1L, respectively. The hatched area is an area of intermediate gradation (gray), and an area surrounded by a thick solid line is an area of an object having a high luminance level (at high temperature) and displayed as white on the screen (hereinafter, “ High brightness area "). In the right image and the left image, the horizontal position of the same object on the screen is shifted, so that the distance to the object can be calculated from the shift (parallax).

In step S14 in FIG. 3, the right image is used as a reference image, and the image signal is binarized, that is, an area brighter than an experimentally determined luminance threshold ITH is set to “1” (white), and a dark area is set to “0”. (Black). FIG. 6 shows an image obtained by binarizing the image shown in FIG. This figure shows that the hatched area is black, and the high-luminance area surrounded by the thick solid line is white.

In the following step S15, a process of converting the binarized image data into run-length data is performed. FIG. 7A is a diagram for explaining this, and in this diagram, the regions that have become white due to binarization are shown as lines L1 to L8 at the pixel level. Each of the lines L1 to L8 has a width of one pixel in the y direction, and is actually arranged without a gap in the y direction, but is shown apart for the sake of explanation. The lines L1 to L8 have a length of 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixels, 8 pixels, and 8 pixels in the x direction, respectively. The run-length data indicates the lines L1 to L8 as the coordinates of the start point of each line (the left end point of each line) and the length (the number of pixels) from the start point to the end point (the right end point of each line). It is shown. For example, since the line L3 is composed of three pixels (x3, y5), (x4, y5) and (x5, y5), the run length data is (x3, y5, 3).

In steps S16 and S17, as shown in FIG. 7B, a process of extracting the target object is performed by labeling the target object. That is, of the lines L1 to L8 converted into run-length data, the lines L1 to L3 having a portion overlapping in the y-direction are regarded as one object 1, the lines L4 to L8 are regarded as one object 2, and the run-length data To the object labels 1 and 2. By this processing, for example, the high luminance areas shown in FIG.

In step S18, as shown in FIG. 7C, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle indicated by the broken line are calculated. The area S is calculated by integrating the length of the run-length data for the same object, and the coordinates of the center of gravity G are bisected in the x coordinate and the y direction of the line that bisects the area S in the x direction. The aspect ratio APECT is calculated as the y coordinate of the line, and the aspect ratio APECT is calculated as the ratio Dy / Dx between Dy and Dx shown in FIG. The position of the center of gravity G may be replaced by the position of the center of gravity of a circumscribed rectangle.

In step S19, the tracking of the object is performed between times, that is, the same object is recognized in each sampling cycle. The time at which the time t as the analog amount is discretized at the sampling period is k, and when the objects 1 and 2 are extracted at time k as shown in FIG. 8A, the object extracted at time (k + 1) The identity of the objects 3, 4 and the objects 1, 2 is determined. Specifically, when the following identity determination conditions 1) to 3) are satisfied, it is determined that the objects 1 and 2 are the same as the objects 3 and 4, and the objects 3 and 4 are respectively 1 and 2. By changing the label to 2, the time tracking is performed.

1) The coordinates of the position of the center of gravity of the object i (= 1, 2) on the image at the time k are (xi (k), yi (k)), and the object j (= 3, 3) at the time (k + 1). When the coordinates of the position of the center of gravity on the image of 4) are (xj (k + 1), yj (k + 1)),
| Xj (k + 1) -xi (k) | <Δx
| Yj (k + 1) -yi (k) | <Δy
That. Here, Δx and Δy are allowable values of the moving amount on the image in the x direction and the y direction, respectively.

2) The area of the object i (= 1, 2) on the image at time k is Si (k), and the area of the object j (= 3, 4) on the image at time (k + 1) is Sj ( k + 1),
Sj (k + 1) / Si (k) <1 ± ΔS
That. Here, ΔS is an allowable value of the area change.

3) The aspect ratio of the circumscribed rectangle of the object i (= 1, 2) at time k is assumed to be ASPECTi (k), and the aspect ratio of the circumscribed rectangle of the object j (= 3, 4) at time (k + 1) is ASPECTj ( k + 1),
ASPECTj (k + 1) / ASPECTi (k) <1 ± ΔASPECT
That. Here, ΔASPECT is an allowable value of the aspect ratio change.

8A and 8B, each object has a large size on the image, but the objects 1 and 3 satisfy the above-described identity determination condition, and the objects 2 and 3 have the same size. 4 satisfies the above-mentioned identity determination condition, the objects 3 and 4 are recognized as the objects 1 and 2, respectively. The position coordinates (of the center of gravity) of each object recognized in this way are stored in the memory as time-series position data, and are used in subsequent arithmetic processing.
The processes in steps S14 to S19 described above are executed for the binarized reference image (the right image in the present embodiment).

In step S20 of FIG. 3, the vehicle speed VCAR detected by the vehicle speed sensor 6 and the yaw rate YR detected by the yaw rate sensor 5 are read, and the yaw rate YR is time-integrated to obtain the turning angle θr of the host vehicle 10 (see FIG. 14). Is calculated.

On the other hand, in steps S31 to S33, processing for calculating the distance z between the target object and the host vehicle 10 is performed in parallel with the processing in steps S19 and S20. Since this calculation requires a longer time than steps S19 and S20, it is executed in a cycle longer than steps S19 and S20 (for example, about three times the execution cycle of steps S11 to S20).

In step S31, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the search image R1) is obtained from the right image as shown in FIG. , The entire area surrounded by the circumscribed rectangle is set as a search image). In a succeeding step S32, a search area for searching for an image corresponding to the search image (hereinafter referred to as “corresponding image”) is set from the left image, and a correlation operation is executed to extract a corresponding image. Specifically, as shown in FIG. 9B, a search area R2 is set in the left image according to each vertex coordinate of the search image R1, and the height of the correlation with the search image R1 in the search area R2. Is calculated by the following equation (1), and a region where the total value C (a, b) is minimum is extracted as a corresponding image. This correlation operation is performed using a grayscale image instead of a binary image. When there is past position data for the same object, a region R2a (shown by a broken line in FIG. 9B) smaller than the search region R2 is set as a search region based on the position data.

Here, IR (m, n) is a luminance value at the position of coordinates (m, n) in the search image R1 shown in FIG. 10, and IL (a + m−M, b + n−N) is a value within the search area. The luminance value at the position of the coordinates (m, n) in the local region R3 having the same shape as the search image R1 with the coordinates (a, b) as the base point. The position of the corresponding image is specified by changing the coordinates (a, b) of the base point and obtaining the position where the luminance difference total value C (a, b) is minimized.

As shown in FIG. 11, the search image R1 and the corresponding image R4 corresponding to the target object are extracted by the processing in step S32, and therefore, in step S33, the position of the center of gravity of the search image R1, the image center line LCTR, And the distance dL (number of pixels) between the position of the center of gravity of the corresponding image R4 and the image center line LCTR, and the distance dR (the number of pixels) is applied to the following equation (2) to obtain the distance between the host vehicle 10 and the object. Calculate z.

Here, B is the base line length, that is, the distance in the horizontal direction (x direction) between the center position of the image sensor 11R of the camera 1R and the center position of the image sensor 11L of the camera 1L as shown in FIG. (Axis interval), F is the focal length of the lenses 12R and 12L, p is the pixel interval in the imaging elements 11R and 11L, and Δd (= dR + dL) is the amount of parallax.

In step S21, the coordinates (x, y) in the image and the distance z calculated by the equation (2) are applied to the following equation (3) to convert them into real space coordinates (X, Y, Z). Here, the real space coordinates (X, Y, Z) are, as shown in FIG. 13 (a), the origin O, which is the position of the middle point of the mounting positions of the cameras 1R and 1L (the position fixed to the host vehicle 10). The coordinates in the image are defined as x in the horizontal direction and y in the vertical direction with the center of the image as the origin, as shown in FIG.

Here, (xc, yc) represents the coordinates (x, y) on the right image with the center of the real space O and the center of the image based on the relative positional relationship between the mounting position of the camera 1R and the real space origin O. Are converted to the coordinates in the virtual image in which is matched. F is the ratio between the focal length F and the pixel interval p.

In step S22, a turning angle correction for correcting a positional shift on the image due to the turning of the host vehicle 10 is performed. As shown in FIG. 14, when the host vehicle turns, for example, to the left by a turning angle θr during the period from time k to (k + 1), on the image obtained by the camera, as shown in FIG. Therefore, this is a process for correcting this. Specifically, the correction coordinates (Xr, Yr, Zr) are calculated by applying the real space coordinates (X, Y, Z) to the following equation (4). The calculated real space position data (Xr, Yr, Zr) is stored in the memory in association with each object. In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).

In step S23, as shown in FIG. 16, for the same object, N real space position data (for example, about N = 10) obtained after turning angle correction and obtained in the period of ΔT, that is, time-series data, An approximate straight line LMV corresponding to a relative movement vector between the object and the host vehicle 10 is obtained. Specifically, assuming that a direction vector L = (lx, ly, lz) (| L | = 1) indicating the direction of the approximate straight line LMV, a straight line represented by the following equation (5) is obtained.

Here, u is a parameter that takes an arbitrary value, and Xav, Yav, and Zav are the average value of the X coordinate, the average value of the Y coordinate, and the average value of the Z coordinate of the real space position data sequence, respectively. The equation (5) becomes the following equation (5a) if the parameter u is eliminated.
(X-Xav) / lx = (Y-Yav) / ly = (Z-Zav) / lz
… (5a)

FIG. 16 is a diagram for explaining the approximate straight line LMV, in which P (0), P (1), P (2),..., P (N-2), P (N-1) The time series data after the turning angle correction is shown, and the approximate straight line LMV passes through the average position coordinates Pav (= (Xav, Yav, Zav)) of the time series data, and is the average value of the square of the distance from each data point. Is obtained as a straight line that minimizes. Here, the numerical value in parentheses attached to P indicating the coordinates of each data point indicates that the larger the value is, the more past the data is. For example, P (0) indicates the latest position coordinates, P (1) indicates the position coordinates one sample cycle ago, and P (2) indicates the position coordinates two sample cycles ago. The same applies to D (j), X (j), Y (j), Z (j) and the like in the following description.

More specifically, a vector D (j) = (DX (j), DY (j), DZ (j) from the average position coordinates Pav to the coordinates P (0) to P (N-1) of each data point. ) = (X (j) -Xav, Y (j) -Yav, Z (j) -Zav) and the inner product s of the direction vector L are calculated by the following equation (6), and the variance of the inner product s is the maximum. A direction vector L = (lx, ly, lz) is obtained.
s = lx.DX (j) + ly.DY (j) + lz.DZ (j) (6)

The variance-covariance matrix V of the coordinates of each data point is represented by the following equation (7). Since the eigenvalue σ of this matrix corresponds to the variance of the inner product s, the largest of the three eigenvalues calculated from this matrix is The eigenvector corresponding to the eigenvalue is the direction vector L to be obtained. In order to calculate the eigenvalues and eigenvectors from the matrix of equation (7), a method known as the Jacobi method (for example, shown in “Numerical Calculation Handbook” (Ohm)) is used.

Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and the position coordinates P (N−1) = (N−1) samples before (time ΔT before) (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV. Specifically, by applying the Z coordinates Z (0) and Z (N-1) to the above equation (5a), that is, by using the following equation (8), the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)).

相 対 A relative movement vector is obtained as a vector from the position coordinates Pv (N−1) calculated by the equation (8) to Pv (0). In this way, the influence of the position detection error is reduced by calculating the approximate straight line that approximates the relative movement trajectory of the target object with respect to the vehicle 10 from the plurality (N) of data within the monitoring period ΔT. As a result, it is possible to more accurately predict the possibility of collision with the object.

Returning to FIG. 3, in step S24, the possibility of collision with the detected object is determined, and an alarm determination process (FIG. 4) for issuing an alarm when the possibility is high is executed.
In step S41, the relative velocity Vs in the Z direction is calculated by the following equation (9). When the following equations (10) and (11) are satisfied, it is determined that there is a possibility of a collision, and the process proceeds to step S42. When (10) and / or Expression (11) are not satisfied, this processing ends.
Vs = (Zv (N−1) −Zv (0)) / ΔT (9)
Zv (0) / Vs ≦ T (10)
| Yv (0) | ≦ H (11)

Here, Zv (0) is the latest distance detection value (v is added to indicate that the data has been corrected by the approximate straight line LMV, but the Z coordinate is the same value as before the correction). Yes, Zv (N-1) is the distance detection value before the time ΔT. T is a margin time, which is intended to determine the possibility of collision by the time T before the predicted collision time, and is set to, for example, about 2 to 5 seconds. H is a predetermined height that defines a range in the Y direction, that is, the height direction, and is set to, for example, about twice the height of the host vehicle 10.
FIG. 17 shows the relationship of Expression (10), where the coordinates corresponding to the detected relative speed Vs and the distance Zv (0) are in the hatched area, and | Yv (0) | When ≦ H, the determination in step S42 and subsequent steps is performed.

FIG. 18 shows an area that can be monitored by the cameras 1R and 1L as an outer triangular area AR0 indicated by a thick solid line, and furthermore, areas AR1, AR2 and AR1 in the area AR0 that are closer to the host vehicle 10 than Z1 = Vs × T. AR3 is an alarm determination area. Here, the area AR1 is an area corresponding to a range obtained by adding a margin β (for example, about 50 to 100 cm) to both sides of the vehicle width α of the host vehicle 10, in other words, both sides of the lateral center axis of the host vehicle 10. Is a region having a width of (α / 2 + β), and if the target object continues to exist as it is, the possibility of collision is extremely high, and is referred to as an approach determination region. The areas AR2 and AR3 are areas in which the absolute value of the X coordinate is greater than the approach determination area (outside the approach determination area in the lateral direction). For an object in this area, an intrusion collision determination described later is performed. This is called an intrusion determination area. These regions have a predetermined height H in the Y direction as shown in the above equation (11).

The answer to step S41 is affirmative (YES) when the object is present in any of the approach determination area AR1 or the intrusion determination areas AR2, AR3.
In a succeeding step S42, it is determined whether or not the object is in the approach determination area AR1. When the answer is affirmative (YES), the process proceeds to step S44 immediately, while when the answer is negative (NO), In step S43, an intrusion collision determination is performed. Specifically, the latest x coordinate xc (0) on the image (c is added to indicate that the coordinate has been corrected so that the center position of the image coincides with the real space origin O as described above. Is determined) and the difference between the x coordinate xc (N−1) before the time ΔT satisfies the following expression (12), and if so, it is determined that the possibility of collision is high.

As shown in FIG. 19, when there is an animal 20 traveling from a direction substantially 90 ° with respect to the traveling direction of the vehicle 10, Xv (N−1) / Zv (N−1) = Xv (0). / Zv (0), in other words, when the ratio Vp / Vs = Xv (N-1) / Zv (N-1) of the animal's speed Vp and the relative speed Vs, the host vehicle 10 and the animal 20 The viewing azimuth θd is constant, and the possibility of collision is high. Equation (12) determines this possibility in consideration of the vehicle width α of the host vehicle 10. Hereinafter, the derivation method of Expression (12) will be described with reference to FIG.

A straight line passing through the latest position coordinates of the object 20 and the position coordinates before the time ΔT, that is, the approximate straight line LMV and the XY plane (a plane including the X axis and the Y axis, that is, a line corresponding to the tip of the vehicle 10 (X axis ), And the X coordinate of the intersection with the plane perpendicular to the traveling direction of the vehicle 10) is XCL, the condition for the occurrence of a collision in consideration of the vehicle width α is given by the following equation (13).

　この式にＺ＝０，Ｘ＝ＸＣＬを代入してＸＣＬを求めると下記式（１５）のようになる。

−α / 2 ≦ XCL ≦ α / 2 (13)
On the other hand, a straight line obtained by projecting the approximate straight line LMV on the XZ plane is given by the following equation (14).

When X = 0 and X = XCL are substituted into this equation to obtain XCL, the following equation (15) is obtained.

Further, since the real space coordinates X and the coordinates xc on the image have the relationship shown in the above equation (3),
Xv (0) = xc (0) × Zv (0) / f (16)
Xv (N−1) = xc (N−1) × Zv (N−1) / f (17)
When these are applied to the equation (15), the intersection X coordinate XCL is given by the following equation (18). By substituting this into equation (13) and rearranging, the condition of equation (12) is obtained.

Returning to FIG. 4, when it is determined in step S43 that the possibility of collision is high, the process proceeds to step S44, and when it is determined that the collision is low, the process ends.
In step S44, an alarm output determination, that is, a determination as to whether or not to output an alarm is performed as follows. First, it is determined from the output of the brake sensor 7 whether or not the driver of the host vehicle 10 is performing a brake operation. If the driver has not performed the brake operation, the process immediately proceeds to step S45 to output an alarm. When the brake operation is being performed, the acceleration Gs generated thereby (the deceleration direction is assumed to be positive) is calculated. When the acceleration Gs is equal to or less than the predetermined threshold GTH, the process proceeds to step S45, while Gx> GTH. If, it is determined that the collision is avoided by the brake operation, and the process ends. Thereby, when an appropriate brake operation is being performed, it is possible to prevent a warning from being issued and not to add unnecessary trouble to the driver.

　ステップＳ４５では、スピーカ３を介して音声による警報を発するとともに、図２１に示すようにＨＵＤ４により例えばカメラ１Ｒにより得られる画像を画面４ａに表示し、接近してくる対象物を強調表示する（例えば枠で囲んで強調する）する。図２１（ａ）はＨＵＤの画面４ａの表示がない状態を示し、同図（ｂ）が画面４ａが表示された状態を示す。このようにすることにより、衝突の可能性の高い対象物を運転者が確実に認識することができる。 The predetermined threshold GTH is determined as in the following equation (19). This is a value corresponding to a condition that the host vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) when the brake acceleration Gs is maintained as it is.

In step S45, an alarm by voice is issued via the speaker 3, and an image obtained by, for example, the camera 1R is displayed on the screen 4a by the HUD 4, as shown in FIG. (Enclosed in a frame to emphasize). FIG. 21A shows a state where the screen 4a of the HUD is not displayed, and FIG. 21B shows a state where the screen 4a is displayed. By doing so, the driver can reliably recognize an object having a high possibility of collision.

As described above, in the present embodiment, the position of the target in the real space is calculated based on a plurality of time-series position data of the same target, and the movement vector is calculated from the position of the real space. The possibility of collision between the target object and the host vehicle 10 is determined based on the movement vector calculated as above, so that erroneous determination unlike the conventional apparatus does not occur, and the accuracy of determining the possibility of collision is determined. Can be improved.
Further, an approximate straight line LMV approximating the relative movement locus of the target object is obtained, the position coordinates are corrected so that the detected position is located on this approximate straight line, and the movement vector is obtained from the corrected position coordinates. The influence of the error in the detection position can be reduced, and the possibility of collision can be determined more accurately.
In addition, since the collision is determined in consideration of the vehicle width α, it is possible to accurately determine the possibility of the collision and prevent the unnecessary warning from being issued.

In the present embodiment, the image processing unit 2 constitutes a part of a relative position detecting unit, a real space position calculating unit, a determining unit, and a warning unit. More specifically, steps S14 to S19 in FIG. 3 correspond to relative position detecting means, steps S20 to S23 correspond to real space position calculating means, and steps S41 to S44 in FIG. 4 correspond to determining means. Step S45, the speaker 3 and the HUD 4 in FIG.

The present invention is not limited to the embodiment described above, and various modifications are possible. For example, in the present embodiment, an infrared camera is used as the image pickup means. However, for example, a television camera that can detect only ordinary visible light may be used as shown in Japanese Patent Application Laid-Open No. 9-226490. However, the use of an infrared camera can simplify the process of extracting an animal, a running vehicle, and the like, and can realize even an arithmetic device having a relatively low arithmetic capability.
Further, in the above-described embodiment, an example in which the front of the vehicle is monitored has been described, but any direction such as the rear of the vehicle may be monitored.

1 is a block diagram illustrating a configuration of a periphery monitoring device according to an embodiment of the present invention. FIG. 2 is a diagram for explaining a mounting position of the camera shown in FIG. 1. 3 is a flowchart illustrating a procedure of processing in the image processing unit in FIG. 1. FIG. 4 is a flowchart showing details of an alarm determination process in FIG. 3. FIG. 3 is a diagram showing a grayscale image obtained by an infrared camera, with hatching applied to a halftone portion. FIG. 3 is a diagram showing hatched black areas for explaining an image obtained by binarizing a grayscale image. FIG. 7 is a diagram for explaining a conversion process to run-length data and labeling. It is a figure for explaining tracking between objects of time. FIG. 4 is a diagram for explaining a search image in a right image and a search region set in a left image. FIG. 9 is a diagram for explaining a correlation calculation process for a search area. FIG. 9 is a diagram for explaining a method of calculating parallax. FIG. 9 is a diagram for explaining a method of calculating a distance from parallax. It is a figure showing a coordinate system in this embodiment. It is a figure for explaining turning angle correction. FIG. 4 is a diagram illustrating a shift in the position of an object on an image caused by turning of the vehicle. FIG. 9 is a diagram for explaining a calculation method of a relative movement vector. It is a figure for explaining conditions which perform alarm judgment. It is a figure for explaining area division ahead of vehicles. It is a figure for explaining the case where a collision is easy to occur. FIG. 4 is a diagram for explaining a method of determining an intrusion alarm according to the width of a vehicle. It is a figure for explaining a display on a head up display.

Explanation of reference numerals

1R, 1L infrared camera (imaging means)
2 Image processing unit (relative position detecting means, real space position calculating means, determining means, alarm means)
3 Speaker (alarm means)
4 Head-up display (alarm means)
5 Yaw rate sensor 6 Vehicle speed sensor 7 Brake sensor

Claims (2)

In a vehicle periphery monitoring device that detects an object existing around the vehicle from an image obtained by an imaging unit mounted on the vehicle,
Relative position detection means for detecting the relative position of the object with respect to the vehicle from the image obtained by the imaging means as position data,
A real space position calculating unit that calculates a position of the object in a real space based on position data of the object, detected by the relative position detecting unit;
When the ratio between the distance to the object calculated by the real space position calculating means and the coordinates of the object in the horizontal direction at 90 degrees to the traveling direction of the vehicle is substantially constant, Determining means for determining that there is a possibility that the object collides with the vehicle. The apparatus according to claim 1, wherein the determination unit performs the determination using a collision determination condition corresponding to a width of the vehicle.

Images