JPH05280974A - Inter-vehicle distance detector - Google Patents

Abstract

(57) [Abstract] [Purpose] To obtain a highly reliable inter-vehicle distance detection device with a short image processing time. [Structure] An image of an object 5 is formed on image sensors 3 and 4 via lenses 1 and 2. Image sensor 3, 4
The image signals from are stored in the memories 8 and 9. CPU1
0 sets a window in the memory 8 and compares the image signal in this window with the image signal in the memory 9 to detect the distance. The CPU 10 stores the image signal in the window in the window memory 11 and compares the image signal with the image signal in the image comparison area in the memory 8 to update the window. This image comparison area is an area obtained by extending the position of the previous window horizontally and / or vertically.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an inter-vehicle distance detecting device for an automobile which detects an inter-vehicle distance by using an image sensor.

[0002]

2. Description of the Related Art Conventionally, an inter-vehicle distance detecting device using an image sensor is well known, for example, Japanese Examined Patent Publication No. 63-4.
There is a 6363 publication. FIG. 10 is a block diagram of an apparatus which is an example thereof. In the figure, the left and right optical systems are arranged with a base line length L, and each optical system includes a lens 1,
2 and image sensors 3 and 4 arranged at a focal distance f from the lenses 1 and 2.

An image of the object 5 at a distance R from the lens surface is formed on the image sensors 3 and 4 by the lenses 1 and 2. Image signals obtained from the image sensors 3 and 4 are converted into digital signals by the A / D converters 6 and 7, respectively, and stored in memories 8 and 9. The image signals stored in the memories 8 and 9 are subjected to image processing in the CPU 10 and the distance to the object 5 is calculated.

The CPU 10 first reads the pixel signals a11 and b11 corresponding to the upper left ends of the image sensors 3 and 4 from the memories 8 and 9, and the absolute value of the difference c11 = | a11.
-B11 | Next, the pixel signals a12 and b12 located one right from the left end are read out, and the absolute value c1 of the difference therebetween is read.
2 = | a12-b12 | is calculated. This operation is sequentially performed for all pixels on the screen, and the calculation results are added together to obtain s0 = Σc1i.

Next, a pixel signal a11 corresponding to the upper left end of the image sensor 3 and a pixel signal b12 located one right from the upper left end of the image sensor 4 are read out, and c21 = | a1
1-b12 | is calculated, and the integrated value s1 = Σc2i of the absolute value of the difference is obtained in the same manner as above. Hereinafter, for the pixel signal corresponding to the image sensor 4, the integrated value si of the absolute values of the pixel signal differences when the pixels are sequentially shifted to the right by one pixel is obtained, and if the minimum value is sn, the left and right images are n. It means that the pixels are misaligned. If the pixel pitch is p,
The amount of left / right deviation is n * p, and the object 5
The distance R to is given by the equation R = (f * L) / (n * p).

An image tracking device using an image signal from a video camera is well known, for example, Japanese Patent Publication No. 61-61.
There is a 57756 publication. FIG. 11 is a block diagram of an apparatus which is an example thereof. In the figure, the object 5 in front of
The image signal a (ij) obtained when the image is taken by the video camera 1A is converted into a digital signal by the A / D converter 6 and stored in the memory 8A, and the image signal b (ij) obtained after a predetermined time is Similarly, it is stored in the other memory 8B.

In the CPU 10, the window {a
(Ij)} is set, image processing for correlating the image signals a (ij) and b (ij) stored in the memories 8A and 8B is performed, and the window is moved to the position having the highest correlation. The image is tracked by doing so. The actual correlation calculation is performed on the image signal a (ij) of the memory 8A and the image signal b (ij) of the memory 8B by the same procedure as in the distance calculation: Σ | a (ij) -b (ij). By updating the window to a position where | is minimized, image tracking by the window becomes possible.

[0008]

The conventional distance detecting device has a problem that it cannot be applied to a system such as an inter-vehicle distance warning device because the calculation time is long because the images are compared on the entire screen when the distance is calculated. It was Also, when applied to a system with the image tracking function of the preceding vehicle,
It was necessary to shorten the processing time. Further, when the image tracking device of FIG. 11 is applied to a tracking traveling system of an automobile, the window gradually shifts when a disturbance having a high contrast ratio enters the object other than the tracking target object. It is necessary for the driver to check the vehicle on the TV monitor to correct the deviation, and the cumbersome operation and the visibility are problems in safety.

The present invention has been made in order to solve the above problems, and shortens the processing time, obtains highly reliable inter-vehicle distance information, and is further affected by disturbance. It is an object of the present invention to provide an inter-vehicle distance detection device that displays highly reliable tracking information in a small amount in an easy-to-understand manner and is also applicable to an obstacle warning device and an automatic tracking device.

[0010]

The inter-vehicle distance detecting apparatus of the present invention compares the reference image signal in the window memory with the image signal stored in the memory in which the window is set after a predetermined time, thereby performing the window processing. The image comparison area in the window updating means for updating is a cross shape extending vertically and / or horizontally from the position corresponding to the previous window image, or a vertically long type or a horizontally long type.

Further, in the image display during the tracking running, the brightness of the image inside the window is made different from the brightness of the image outside the window.

[0012]

According to the inter-vehicle distance detecting apparatus of the present invention, the distance measuring function is speeded up by reducing the number of pixels of one screen used for image comparison, and the image comparison area in the window updating means at the time of tracking is the cross shape, Since one of the vertically long type and the horizontally long type is selected according to the specification, the tracking function can be speeded up by reducing the image comparison area compared to the full screen search. Therefore, once the driver sets the window so as to include the image of the target object with a switch, etc., the position of the window automatically moves according to the movement of the target object as long as it is within the visual field of the device, The distance to the object is obtained.

Further, since the images inside and outside the window are made to have different brightness so that the image of the object inside the window can be easily confirmed during tracking, the safety during the tracking operation is improved.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same reference numerals indicate the same parts.

Example 1. In FIG. 1, reference numerals 1 and 2 denote lenses that are arranged apart from each other by a base line length L and that constitute left and right optical systems for capturing an image of an object 5 that is a preceding vehicle ahead of the own vehicle,
Reference numerals 3 and 4 correspond to the lenses 1 and 2, respectively.
A two-dimensional image sensor 5 arranged at a focal distance f of 1 is an object which is a preceding vehicle.

Reference numerals 6 and 7 are A / D converters for converting the analog signals from the image sensors 3 and 4 into digital signals, and 8 and 9 are memories for storing the digital signals of the A / D converters 6 and 7. Is a CPU. 11 is a window memory for storing image signals in the window of the memory 8, 1
Reference numeral 2 is a window forming device for forming a window for image tracking, and 13 is a display device for displaying an image captured by the lower image sensor 3.

The CPU 10 initializes a window by a signal from the window forming device 12, controls the memories 8 and 9, the window memory 11 and the display device 13, and performs various calculations such as inter-vehicle distance calculation processing and window update processing. Is to do.

Next, the operation of the first embodiment will be described mainly with reference to FIG. The image of the object 5 is formed via the lenses 1 and 2 on the image sensors 3 and 4 which are installed with their optical axes aligned with the lenses 1 and 2. The outputs of the image sensors 3 and 4 are converted into digital signals at a predetermined sampling period via the A / D converters 6 and 7, and the data are stored in the memories 8 and 9, respectively.

The image signals stored in the memories 8 and 9 are C
After being sent to the PU 10 and the display device 13, the CPU 10 performs image processing, and then calculates the inter-vehicle distance by the principle of triangulation,
The display device 13 displays the calculated inter-vehicle distance together with the image signal of the memory 8. The driver is the display device 13
By operating the window forming device 12 while watching the front image, the window is set to a certain portion on the screen, that is, an area including the image of the object 5.

Since the CPU 10 calculates the image signal in time series when updating the window, the image signal surrounded by the window among the image signals in the memory 8 is stored in the window memory 11 as the reference image signal. And CPU
In the next sampling, 10 compares the image signal of a predetermined image comparison area among the image signals stored in the memory 8 with the reference image signal in the window memory 11. And
The CPU 10 detects the position of the image in the image comparison area of the memory 8 that best matches the image in the window memory 11, sets the window at that position, and updates it.

Regarding the distance measuring process, the number of pixels at the time of image comparison is reduced from one screen to the inside of the window, so that the processing time is greatly shortened and the speed of the apparatus is increased.
The image comparison method and the distance calculation formula are the same as those in the conventional example, and since they have been described in detail in the conventional example, they are omitted here.

FIG. 2 shows the reference image signal in the window 21 in the memory 8 at t = t ₀ and also shows the image comparison area 21A in the tracking process at t = t ₁ which is the next sampling time. There is. Next, the window updating process will be described in more detail with reference to FIG.

First, at t = t ₀ , the CPU 10 stores the image signal of the memory 8 corresponding to the window 21 surrounding the image of the object 5 in the window memory 11 as a reference image signal. Next, the image signal of the image comparison area 21A in the image signals of the memory 8 obtained at t = t ₁
A new window is selected by image comparison with the reference image signal of the window memory 11. The image comparison area 21A in FIG. 2 is extended by y bits on both sides in the vertical direction of the corresponding position of the window 21 at t = t ₀ and is extended by x bits on both sides in the horizontal direction, that is, as shown in FIG. The shape of the image is the cross shape, and the position of the image that best matches the previous window 21 among them is determined to be the new window.

FIG. 3A shows the window 21 when t = t _0.
T extended by y bits on both sides of the corresponding position in the vertical direction
3A shows the vertically long image comparison area 21A at t = t ₁ and FIG.
(B) shows a horizontally long image comparison area 21A at t = t ₁ extended by x bits on both sides in the horizontal direction of the corresponding position of the window 21 at t = t ₀ . The image comparison area 21A shown in FIG. 3 can also be used for the window updating process as in FIG.

By using the image signal in the window as a reference as described above, the number of pixels in one screen used for image comparison can be reduced, and the processing time can be greatly shortened. , If the driver first sets the window to include the preceding vehicle, as long as the preceding vehicle is within a wide field of view corresponding to the full screen, then the movement of the preceding vehicle is checked while checking the left-right symmetry of the image signal in the window. The position of the window is automatically updated in accordance with the distance measurement, so that the reliability is improved, and once the window is set, the inter-vehicle distance to the preceding vehicle is automatically obtained thereafter.

In this embodiment, the pair of optical systems are arranged on the left and right, but a window may be set in the image of the video camera above or below in the vertical direction, or the memory for setting the window may be the memory 8. Alternatively, the memory 9 may be used. The image comparison area may be selected from the three types shown in FIGS. 2 and 3 according to the specifications of the apparatus.

As described above, since it is decided to extend the image in the horizontal and / or vertical direction with respect to the window position at t = t ₀ to make the image comparison area as small as possible, the image at the time of updating the window. Processing time becomes short.

Example 2. The overall configuration of the inter-vehicle distance detecting device is the same as that of FIG. 1 except for the window updating process by the CPU 10, and therefore FIG. 1 is incorporated in the second embodiment. The other operations except the window updating procedure are the same as those described in the first embodiment, and therefore the description thereof is omitted.

Next, the procedure for updating the window will be described. Figure 4 shows a window 21 enclosing the image of the image signal 20 and the object 5 in the memory 8 o'clock t = t _0, 5 t = t ₀
An image signal 22 of the memory 8 at + Δt (= t ₁ ), an image comparison area 23 in the window tracking processing, and a new window 24 showing an area of an image that best matches the image of the window 21 of FIG. 4 are shown. 6 shows an image comparison area 25 in the left-right symmetry correction in the vicinity of the new window 24 of FIG. 5 and a new window 26 showing an image area that best matches the image of the window 21 of FIG. 4, and FIG. It is a figure for demonstrating the calculation method at the time.

FIG. 8 is a schematic flow chart for explaining the processing procedure of the second embodiment. The window updating procedure will be described below with reference to FIGS. 1 and 4 to 8.

The CPU 10 first obtains the inter-vehicle distance at t = t ₀ (step S1), and stores the image signal of the window 21 in FIG. 4 in the window memory 11 (step S).
2). After that, when t = t ₀ + Δt, the image signals newly sampled from the image sensors 3 and 4 are stored in the memories 8 and 9 (step S3). Next, the CPU 10
Is an area in the memory 8 corresponding to the window 21 shown in FIG. 4 and an area in the vicinity thereof as an image comparison area 23 as shown in FIG. 5, and an image comparison in the memory 8 is performed with respect to a reference image signal in the window memory 11. While the image signal in the area 23 is sequentially shifted pixel by pixel, the sum of absolute values of the signal difference between the right and left pixels is calculated to obtain the minimum value. That is, the position of the image in the image comparison area 23 that best matches the image in the window 21 is sequentially shifted pixel by pixel in the image comparison area 23, and the new window 24 is obtained (step S4).

Thereafter, as shown in FIG. 6, the image 2 in the memory 8 is
Among the two, the window image with the most symmetry is obtained in the symmetric image comparison area 25 including the area of the window 24 and the area in the vicinity thereof, and the window image is set as the new window 26 (step S5). Hereinafter, returning to step S1, t = t ₀ + Δ
Distance measurement processing at time t is performed.

Next, the left-right symmetry correction in step S5 will be described with reference to FIGS. In the image comparison area 25 of FIG. 6, in the image {ei, j} of k rows and m columns having the same size as the previous window 21, the image signal e11 at the left end and the image at the right end of the first row as shown in FIG. The absolute value of the difference between the signals e1m is Δe11, the next e12 at the left end and the next e at the right end of the first row.
The absolute value of the difference between 1 and m-1 is Δe12, and similarly, Δei
j (i is up to k, j is up to m / 2) is determined, and the image having the smallest sum ΣΔeij in the image comparison area 25 in FIG. 6 is set as the new window 26.

By adopting the above algorithm, even if a disturbance with a high contrast ratio such as a streetlight, a white line, or an interrupting vehicle enters the window, the window is updated (step S4) by the above-mentioned left-right symmetry correction (step S5). Since the window shift in () is corrected, the reliability of tracking performance is improved.

FIG. 9 shows the tracking state of the display device 13 of this embodiment. The brightness inside the window is left unchanged from the video output of the camera, and the brightness outside the window is compared with the average brightness of the image inside the window. It is darkened by about 10 to 20%. For this reason, the driver can see what is being tracked at present without carefully staring at the display device 13, so that the safety during the tracking run, which is apt to be inadvertently forward, is improved. In the present embodiment, the distance display is displayed with a constant brightness, but in consideration of visibility, the brightness may be the same as that in the window, or the brightness inside and outside the window may be reversed. Good.

In this embodiment, the pair of optical systems are arranged horizontally, but it is possible to arrange them vertically even in one direction. Further, in the present embodiment, the window data of the memory 8 is stored in the window memory 11 (step S2), but immediately after the distance measurement (step S1), the memory 8 and the window memory 11 are switched as the image writing memory. Writing (step S3) may be performed.

[0037]

As described above, in the inter-vehicle distance detecting device according to the present invention, the image comparison area is set to be as small as possible in each sampling cycle and the image comparison is performed while updating the image comparison area. Computation time is significantly shortened and reliability is improved as compared with the case of
Furthermore, once the window is set, as long as it is within the field of view of the device, the window automatically moves according to the movement of the target object, and the distance to the target object can be measured. It can also be applied to inter-vehicle distance warning systems and automatic tracking systems.

Further, since the position of the window is automatically corrected after the left-right symmetry correction according to the movement of the object, the tracking accuracy is remarkably improved. Further, since the display during the tracking traveling is changed in the display brightness inside and outside the window, the visibility is improved, and the tracking type inter-vehicle distance detecting device with higher safety can be obtained.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an inter-vehicle distance detecting device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a window position at t = t ₀ and an image comparison area at t = t ₁ .

FIG. 3 is an explanatory diagram showing another image comparison area at t = t ₁ .

FIG. 4 is an explanatory diagram showing window positions and the like at t = t ₀ according to another embodiment.

FIG. 5 is an explanatory diagram showing an image comparison area and a new window.

FIG. 6 is an explanatory diagram for explaining left-right symmetry correction.

FIG. 7 is an explanatory diagram of an image signal for explaining a calculation method of left-right symmetry correction.

FIG. 8 is a schematic flowchart for explaining a processing procedure of an apparatus according to another embodiment.

FIG. 9 is a diagram showing a screen state during tracking of a display device according to another embodiment.

FIG. 10 is a diagram showing a configuration of a conventional inter-vehicle distance detecting device.

FIG. 11 is a diagram showing a configuration of an image tracking device using a conventional image sensor.

[Explanation of symbols]

 1, 2 lens 3, 4 image sensor 5 object 6, 7 A / D converter 8, 9 memory 10 CPU 11 window memory 12 window forming device 13 display device 21, 24, 26 window 21A, 23, 25 image comparison area

Claims (2)

[Claims] 1. A two-dimensional image sensor, a pair of substantially right and left optical systems which are arranged at a predetermined distance from each other and form an image of an object on the image sensor, and output from each of the image sensors. A pair of memories for storing image signals, window setting means for setting a window in one of the image signals stored in the memory, a window memory for storing a reference image signal in the window, and a window memory in the window. A distance calculating means for calculating a distance to the object by the principle of triangulation by performing comparison processing while shifting the other image signal using the image signal as a reference image signal, the reference image signal in the window memory and the window. The window that updates the window by comparing the image signals stored in the memory of A dough updating means, and the image comparison area in the window updating means extends vertically and horizontally from a position corresponding to the previous window image, or a vertically long type or horizontally long type extending only in one direction. An inter-vehicle distance detecting device characterized by: 2. A two-dimensional image sensor, a pair of left and right optical systems which are arranged at a predetermined distance from each other and form an image of an object on the image sensor, and output from each of the image sensors. A pair of memories for storing image signals, window setting means for setting a window in one of the image signals stored in the memory, a window memory for storing a reference image signal in the window, and a window memory in the window. Distance calculation means for calculating a distance to the object by the principle of triangulation by performing comparison processing while shifting the other image signal using the image signal as a reference image signal, an image in a memory in which the window is set, and The display means for displaying the calculated distance, the reference image signal in the window memory and the memory in which the window is set Window updating means for updating the window by comparing the image signals stored after a predetermined time, and left-right symmetry correcting means for correcting the window to an image position having left-right symmetry in the vicinity of the window updated by the window updating means. An inter-vehicle distance detecting apparatus, characterized in that, in the image display during tracking traveling by the display means, the brightness of the image in the window and the brightness of the image in the outer peripheral portion of the window are made different from each other.