JPH0954898A - Vehicle periphery monitoring device - Google Patents

Abstract

(57) [Abstract] [PROBLEMS] To provide a vehicle surroundings monitoring device which is capable of recognizing a moving position of an obstacle which has been recognized or has disappeared within the imaging range, and which has improved usability. To do. An obstacle detection means (41a) for detecting an obstacle from image data in a vehicle periphery monitoring device for monitoring the periphery of the vehicle based on image data from an image pickup means (1) installed in the vehicle, Obstacle position acquisition means (4) for acquiring position information of the obstacle detected by the obstacle detection means.
1b), traveling state information output means (2) for detecting and outputting traveling state information indicating a traveling state of the vehicle, and a moving position of the obstacle detected based on the traveling state information and the positional information of the obstacle. Obstacle Position Prediction Means (41c)
And a notifying means (7) for notifying the driver of the moving position of the obstacle predicted by the obstacle position predicting means.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device, and more particularly, it is effectively applied to monitor the periphery of a vehicle such as an automobile to assist the driver's safety confirmation in driving the vehicle. The present invention relates to a vehicle surroundings monitoring device.

[0002]

2. Description of the Related Art Conventionally, as a device for monitoring the periphery of a vehicle such as an automobile, there has been a device having a configuration shown in FIG. In FIG. 1, reference numeral 1 denotes an image pickup unit for picking up an image around the vehicle
Reference numeral 1 is a steering angle detection unit that detects a steering angle of a steering wheel or a steering wheel and outputs turning information. Reference numeral 3 is a storage unit that holds an image signal obtained from the imaging unit 1 or vehicle shape information. A data processing unit configured by a computer that operates according to a program stored in advance, 5 is a display unit having a display device 51, and 6 is a voice instruction unit for generating an alarm sound or voice guidance.

The rudder angle detector 21 is provided with a steering wheel rudder angle sensor for detecting the steering wheel rudder angle or a steering angle detection sensor for detecting the steering angle of the steered wheels (neither is shown), and outputs turning information of the vehicle. There is. An A / D converter 21a for converting the input signal into a digital signal and outputting the digital signal is attached to the steering angle detection unit 21, and outputs the digitally converted steering angle information to the data processing unit 4.

The storage unit 3 has a frame memory 31 for temporarily storing an image signal obtained from the image pickup unit 1 and a reference data memory 32 for preliminarily holding image data in the case of a flat ground with no obstacles as reference data. And a vehicle data memory 33 that holds the current position data of the vehicle, the two-dimensional or three-dimensional shape data of the vehicle, and the vehicle reference position data that is the mounting position of the imaging unit 1 in the vehicle as vehicle information. .

The data processing unit 4 has a ROM 42 in which an operation program is stored, a CPU 41 which operates according to this operation program, and a RAM 43 which temporarily stores information required when the CPU 41 operates. The display unit 5 displays the relative position of the object with respect to the vehicle or displays a message to the driver based on the display image signal output from the CPU 41. The voice instruction unit 6 includes a speaker 61 and a drive circuit 62 that generates a buzzer signal or a voice signal based on a control signal from the CPU 41 and outputs the voice signal to the speaker 61.

The image pickup unit 1 may be, for example, one using laser light or a stereo type. As shown in FIG. 15A, the laser light source uses a laser light source 12, a laser light source drive device 11 for driving the laser light source 12, and a laser light input from the laser light source 12, and the input laser light is used. It has a pattern light projection element 13 for projecting the bright spot matrix 13a on the monitoring area by light and a CCD camera 14 for picking up an image of the bright spot matrix 13b once projected on the monitoring area.

In this configuration, the image picked up by the CCD camera 14, that is, the bright spot matrix 13b disturbed by obstacles in the monitoring area is temporarily held in the frame memory 31 as image information. Then, the data processing unit 4 compares the image information held in the frame memory 31 with the reference data held in the reference data memory 32 to obtain the shape of the obstacle and the relative position information with respect to the imaging unit 1. get.

On the other hand, the stereo type is shown in FIG.
As shown in FIG. 3, the right CCD camera 14R and the left CCD camera 1 are arranged so as to be separated from each other by a predetermined distance dxa.
It has two 4L CCD cameras. Along with this,
Two frame memories, a right frame memory 31R and a left frame memory 31L, are provided.

In this configuration, the right CCD camera 1
The image captured by 4R is stored in the right frame memory 31R as right image information, and the image captured by the left CCD camera 14L is stored as left image information in the left frame memory 3
Store in 1L. Then, the data processing unit 4, for example, performs horizontal differentiation of the left-side image information stored in the frame memory 31L, detects an edge of the obtained object such as an obstacle, and uses this edge point as a trigger point, that is, a reference point. As, the edge point corresponding to the edge point is acquired from the right side image information stored in the other frame memory 31R. Then, the position of the object is detected by the difference between the positions of the edge points of the left and right images thus obtained on the pixel, that is, the parallax.

As shown in FIG. 16, for example, such an image pickup unit 1 is attached at an angle θ with respect to the normal line of the ground in the substantially central portion of the vehicle 100 on the upper rear side. As a result, the image pickup unit 1 detects an obstacle or the like in the rear of the vehicle 100, more specifically, in the monitoring area 100a near the rear as shown in FIG. The mounting position of the image pickup unit 1 is stored in the vehicle data memory 33 as the vehicle position reference information described above.

The data processing unit 4 uses the obstacle information recognized as described above and vehicle information such as vehicle current position data, vehicle shape data and vehicle reference position data stored in the vehicle data memory 33. The relative position between the vehicle and the obstacle is calculated, and the positional relationship between the vehicle and the obstacle is displayed on the display device 51 of the display unit 5 based on the calculated relative position. Further, the data processing unit 4 is the steering angle detection unit 2
The route of the vehicle is predicted based on the turning information output from the vehicle, and when it is predicted that the detected obstacle collides with the vehicle, a voice signal is output to the voice instruction unit 6. Based on the voice signal output from the data processing unit 4, the voice instruction means 6 issues a warning such as a buzzer sound or voice guidance to notify the driver of the collision in advance.

[0012]

In the above-mentioned conventional apparatus, as described with reference to FIG.
An obstacle inside the vehicle is detected, and a collision with a vehicle is predicted with respect to the obstacle existing in the monitoring area 100a. Therefore, in this conventional device, the obstacle is in this monitoring area 10
As long as it exists within 0a, it is effective because the relative position of the vehicle and the obstacle can be grasped.

However, when the obstacle is once out of the monitoring area 100a, no consideration is given to the obstacle outside the monitoring area, and it is impossible to predict a collision. there were.

For example, when the vehicle 100 moves backward with the steering wheel largely turned, that is, when the steered wheels (front wheels) have a relatively large steering angle, as shown in FIG. 17, for example, as shown in FIG. The obstacle 200 existing in the monitoring area 100a (P1) in
When the vehicle 100 moves backward at the steering angle θ ₁ and moves along the locus indicated by the arrow to the position P2 (the position indicated by the alternate long and short dash line in the figure), the vehicle 100 deviates from the monitoring area 100a (P2) at the position P2. As a result, the data processing unit 4 excludes the obstacle 200 from the monitoring target. When the vehicle 100 further moves backward and reaches the position P3 (the position shown by the solid line in the figure), the obstacle 200 is located on the right front side surface portion of the vehicle 100 and collides with the steered wheels. At this time, since the data processing unit 4 has already excluded the obstacle 200 from the monitoring target, the alarm is not issued although there is a risk of collision.

Further, an obstacle 200 is formed on the side surface of the vehicle 100.
Is located, and when the vehicle 100 retreats from this state while turning, as shown in FIG. 18, for example, at the position P4 (position shown by a dotted line in the figure), the monitoring area 100a
The obstacle 200 existing in (P4) moves out of the monitoring area as the vehicle 100 moves backward, and when the vehicle 100 further moves backward and reaches the position P5 (the position shown by the solid line in the figure), the obstacle 200 Is located on the right side surface of the vehicle 100.

From this state, when the vehicle 100 is moved backward with the steering angle of the steered wheels set to θ ₂ by operating the steering wheel, the vehicle 100 moves along the locus shown by the arrow in the figure, and the obstacle 200 moves to the right front. Collide with the side. Even in this case, the data processing unit 4 detects that the obstacle 200 is in the monitoring area 10.
Since the obstacle 200 is removed from the monitoring target at the time when the obstacle 200 is deviated from 0a, no alarm is issued although there is a risk of collision.

Further, the obstacle is very small,
When this obstacle is located in the gap between the detection lines of the CCD sensor (not shown) that constitutes the CCD camera 14 (14R, 14L), the obstacle cannot be detected and the obstacle is in the monitoring area. Even if the obstacle exists within 100a, the obstacle is repeatedly detected and disappeared, and the visibility is impaired.

The above-mentioned problems are caused by the CCD camera 1.
The number is not limited to 4, but may occur when a video camera (not shown) is used. That is, needless to say, when the obstacle is out of the imaging area of the video camera, when the obstacle is very small, the obstacle is located between the scanning lines of the video camera. In this case, the detection and disappearance of the obstacle will be repeated.

Therefore, in view of the above-mentioned conventional problems, the present invention can know the position of an obstacle recognized even if it is outside the monitoring area, and there is a risk of collision with a vehicle. If there is, it is an object of the invention to provide a vehicle periphery monitoring device which notifies the driver of the situation and can prevent a collision or the like in advance, and which has improved usability and safety. Another object of the present invention is to provide a vehicle periphery monitoring device that can recognize the moving position of an obstacle that has disappeared in the monitoring area, improve the visibility, and improve the usability.

[0020]

As shown in the basic configuration diagram of FIG. 1, a vehicle periphery monitoring device made according to the present invention to solve the above-mentioned problems has an image pickup means (1) installed in a vehicle.
In a vehicle periphery monitoring device for monitoring the periphery of a vehicle on the basis of image data from an obstacle, an obstacle detecting means (41a) for detecting an obstacle from the image data, and an obstacle detected by the obstacle detecting means (41a). Obstacle position acquisition means (41b) for acquiring position information of an object, traveling state information output means (2) for detecting and outputting traveling state information indicating a traveling state of the vehicle, the traveling state information and the obstacle. Driving the obstacle moving position predicted by the obstacle position predicting means (41c) for predicting the detected moving position of the obstacle based on the position information of the obstacle and the moving position of the obstacle predicted by the obstacle position predicting means (41c). And a notification means (7) for notifying the person. (Claim 1)

In the above-mentioned structure of the present invention, the image pickup means acquires an image of the periphery of the vehicle as image data, and the obstacle detection means detects an obstacle from the image data. The obstacle position acquisition means acquires the position information of the detected obstacle, and the obstacle position prediction means acquires the traveling state information indicating the traveling state of the vehicle from the traveling state information output means and the traveling state information. And the moving position of the obstacle is predicted based on the obstacle position information acquired by the obstacle position acquisition means. The notification means displays an obstacle at the moving position predicted by the obstacle position prediction means.

That is, in the structure of claim 1, the moving position of the obstacle detected based on the traveling state information of the vehicle is predicted, and the obstacle is displayed at the predicted moving position. Since it is configured, it is possible to perform control based on the predicted movement position, and it is possible to provide a convenient vehicle periphery monitoring device.

The obstacle position predicting means (41c)
When the obstacle disappears from the imaging range of the imaging unit (1), the moving position of the obstructed obstacle is predicted, and the notification unit (7) predicts the disappeared obstacle. It is characterized by notifying the moved position. (Claim 2)

In the structure of claim 2, the obstacle targeted by the obstacle position predicting means and the notifying means is an obstacle disappeared from the image pickup range of the image pickup means. Therefore,
In the structure of the second aspect, it is possible to know the moving position of the obstacle disappeared from the imaging range, and it is possible to provide the vehicle periphery monitoring device which is improved in safety and is easy to use.

The obstacle position acquisition means (41b)
When the obstacle disappears outside the imaging range of the imaging means (1) and disappears, the information regarding the disappeared obstacle is acquired as out-of-range obstacle information, and the acquired out-of-range obstacle information is included in the range. External obstacle information holding means (3
4), and the obstacle position prediction means (41c)
Is characterized by predicting a movement position for an obstructed obstacle based on the out-of-range obstacle information. (Claim 3)

In the structure of claim 3, the position information of the obstacle acquired by the obstacle position acquiring means is an obstacle disappeared by moving out of the range of the image pickup means, and the position information of the obstacle is The out-of-range obstacle information holding means holds the out-of-range obstacle information. The obstacle position predicting means predicts the moving position of the disappeared obstacle based on the out-of-range obstacle information. Therefore, in the configuration of the third aspect, it is possible to know the moving position of an obstacle that has moved to the outside of the image pickup range by the image pickup means and disappeared, and to provide a convenient vehicle periphery monitoring device with improved safety. be able to.

The obstacle position acquisition means (41b)
Acquires position information of the obstacle within the imaging range by the imaging means (1) as in-range obstacle information, and the obstacle position prediction means (41c) detects an obstacle based on the in-range obstacle information. The moving position of the obstacle is predicted, and the predicted moving position of the obstacle is stored in the in-range obstacle moving position holding means (35).
Is characterized in that, when the obstacle disappears within the imaging range of the imaging means (1), the predicted moving position of the obstacle is notified about the disappeared obstacle. (Claim 4)

In the structure of the above-mentioned claim 4, the position information of the obstacle acquired by the obstacle position acquiring means is the position information of the obstacle existing within the image capturing range of the image capturing means, and the obstacle position predicting means comprises: The moving position of the obstacle is predicted based on the position information of the obstacle existing in the imaging range and the traveling state information. The notification means notifies the driver of the moving position predicted by the obstacle position prediction means with respect to the obstacle disappeared within the imaging range. Therefore, in the structure according to the fourth aspect, it is possible to know the moving position of an obstacle that has disappeared within the image capturing range by the image capturing means, and to provide a convenient vehicle periphery monitoring device with improved safety. it can.

The obstacle position acquisition means (41b)
Periodically acquires the in-range obstacle information, and the obstacle position prediction means (41c) moves the obstacle in each cycle based on the in-range obstacle information and the traveling state information. It is characterized in that the position is predicted and the predicted moving position of the obstacle is stored in the in-range obstacle moving position holding means (35). (Claim 5)

In the structure of the fifth aspect, the obstacle position obtaining means periodically obtains the position information (obstacle information within range) of the obstacle existing within the image pickup range of the image pickup means. The obstacle position predicting means predicts the acquired moving position of the obstacle for each cycle, and stores the predicted moving position of the obstacle in the in-range obstacle moving position holding means. Then, the notification means notifies the driver of the moving position predicted by the obstacle position prediction means regarding the obstacle disappeared within the imaging range. Therefore, according to the structure of the fifth aspect, it is possible to immediately know the moving position of the obstacle disappeared within the image pickup range by the image pickup means, and to provide a convenient vehicle periphery monitoring device with improved safety. can do.

Further, it has a vehicle route predicting means (41d) for predicting the route of the vehicle, and the notifying means (7) has the vehicle route predicted by the vehicle route predicting means (41d) and the obstacle. It is characterized by notifying the driver of the moving position of the obstacle predicted by the position predicting means. (Claim 6)

In the structure of claim 6, the notifying means notifies the driver of the vehicle path predicted by the vehicle path predicting means and the moving position of the obstacle predicted by the obstacle position predicting means. It is possible to know the existence of obstacles on the route in which the vehicle is predicted to travel, even for obstacles that have disappeared from the imaging range, and to provide a user-friendly vehicle periphery monitoring device with improved safety. it can.

The vehicle route prediction means (41d)
Is characterized by recognizing the turning direction of the vehicle based on the image information from the image pickup means (1) and predicting the course of the vehicle based on the recognized turning direction.
(Claim 7)

In the structure of claim 7, the vehicle course predicting means recognizes the turning direction of the vehicle on the basis of the image information from the image pickup means and predicts the course of the vehicle from the recognized turning direction. The device is not required and the structure of the device can be simplified.

Further, the vehicle route prediction means (41d)
Is characterized in that the turning direction of the vehicle is recognized based on the moving direction and the moving amount of the pixel of interest in the image holding means (31) which holds the image signal from the image pickup means (1). (Claim 8)

In the above structure of the present invention, the vehicle route prediction means refers to the image information held in the image holding means for holding the image information from the image pickup means, and detects the moving direction and the moving direction of the pixel of interest in the image information. The amount of movement is detected. Then, the moving direction of the vehicle is recognized based on the detected moving direction and the moving amount, and the course of the vehicle is predicted from the recognized moving direction. Therefore, according to this configuration, when predicting the route of the vehicle, only the point of interest needs to be processed, and the calculation steps required for the process can be simplified.

The vehicle route prediction means (41d)
Is characterized by predicting the route of the vehicle based on the traveling state information. (Claim 9)

In the above structure of the present invention, the vehicle route predicting means predicts the route of the vehicle on the basis of the traveling state information. Therefore, the prediction of the route of the vehicle and the prediction of the moving position of the obstacle are common information. The device configuration can be simplified.

Further, the running state information output means (2)
Has a traveling distance detecting means (24) for detecting a traveling distance of the vehicle, and a turning direction detecting means (21) for detecting a turning direction of the vehicle and outputting the detected turning direction as turning information. It has a feature. (Claim 10)

In the above configuration of the present invention, the traveling state information output means includes traveling distance information from the traveling distance detecting means for detecting the traveling distance of the vehicle and information about the turning direction from the turning direction detecting means provided in the vehicle. And are output as traveling state information. Since the traveling distance information and the turning direction information can be read directly from the vehicle, the processing can be performed at high speed when predicting the course of the vehicle.

The turning direction detecting means (21) is
Steering wheel steering angle sensor (22) for detecting the steering wheel steering angle,
One of the steering angle sensors (23) for detecting the steering angle of the steered wheels. (Claim 11)

In the eleventh aspect of the invention, the turning direction detecting means is constituted as a steering wheel angle sensor for detecting the steering angle of the steering wheel or a steering angle sensor for detecting the steering angle of the steered wheels. The information of can be surely grasped.

Further, based on the vehicle path predicted by the vehicle path predicting means (41d) and the moving position of the obstacle predicted by the obstacle position predicting means, a collision between the vehicle and the obstacle is detected. Collision prediction means for prediction (41
e), the notification means (7) is characterized in that when the collision prediction means (41e) predicts a collision, the notification means (7) notifies the collision. (Claim 12)

According to the twelfth aspect of the present invention, the collision predicting means predicts whether or not the vehicle will collide with the obstacle based on the predicted vehicle course and the predicted movement position of the obstacle. When it is predicted, the notification means notifies that the collision has occurred. Therefore, according to this configuration, the driver can know in advance the possibility of collision between the vehicle and the obstacle when the vehicle moves in the current state, can prevent the collision in advance, and can improve safety. It is possible to provide a vehicle surroundings monitoring device having an improved performance.

The notification means (7) is at least one of a display means (5) for displaying the relative distance between the vehicle and the obstacle and a message, or a voice instruction means (6) for generating a buzzer sound or a voice. It is characterized by being. (Claim 13)

In the structure of the above-mentioned claim 13, the notifying means is one or both of a display means for displaying a relative distance between the vehicle and the obstacle and a message, or a voice instruction means for generating a buzzer sound or a voice. The positional relationship between the vehicle and the obstacle can be transmitted by the driver's eyes or hearing. In particular, when notifying a driver of a danger such as a collision, it is possible to more clearly notify that fact by operating both.

Further, the image pickup means (1) is mounted behind the vehicle so as to image the rear area of the vehicle. (Claim 14)

In the above structure of the fourteenth aspect, since the image pickup means is attached to the rear of the vehicle so as to image the rear area of the vehicle, the driver drives an obstacle in the rear vicinity of the vehicle which is difficult for the driver to visually recognize. Can be recognized by the person.

[0049]

Embodiments of the present invention will be described below with reference to the drawings. First, a first embodiment of a vehicle periphery monitoring device to which the present invention is applied will be described.
FIG. 2 is a block diagram illustrating the configuration of the vehicle periphery monitoring device according to the present invention. In FIG. 2, the same parts as those described above are designated by the same reference numerals. That is, 1 is an imaging unit, 2 is a detection unit having a steering angle detection unit 21, 3 is a storage unit, 4 is a data processing unit, and 7 is a notification unit including a display unit 5 and a voice instruction unit 6.

The image pickup section 1 functions as an image pickup means, and FIG.
Either one using the laser beam described in 5 (a) or the stereo type described in FIG. 15 (b) is appropriately selected and used. Since the image pickup unit 1 is the same as the conventional one, its explanation is omitted. Note that, as described above, a video camera can be applied to the image pickup unit 1 in addition to the CCD camera.

The detecting section 2 comprises a steering angle detecting section 21 and a traveling distance detecting section 24. The steering angle detector 21 is
Either one or both of the steering wheel steering angle sensor 22 and the steering angle sensor 23 are used. These steering wheel steering angle sensor 22 and steering angle sensor 23 are
It is attached to a steering mechanism that steers 00.

FIG. 3A is a view showing an example of how the steering wheel steering angle sensor 22 is attached. Reference numeral 110 is a steering mechanism, and 120 is a wheel (only the outer peripheral surface of the wheel is shown). The steering mechanism 110 is attached to a steering wheel 111 rotated by a driver, a main shaft 112 having an upper end attached to the center of the steering wheel 111, and a lower end of the main shaft 112 via a joint 118a. An intermediate shaft 113, a pinion 114 having a tooth surface formed on the peripheral surface thereof and connected to the lower end of the intermediate shaft 113 via a joint 118b, and a rack 115 having a tooth surface meshing with the pinion 114, This rack 1
15 has an intermediate arm 116 joined to both ends (only the right end is shown in the figure) via a joint 118c, and a knuckle arm 117 joined to an outer end of the intermediate arm 116 via a joint 118d. are doing.
The wheel 120 is rotatably supported by a support portion 117a at the outer end of the knuckle arm 117.

In such a steering mechanism 110, when the driver rotates the steering wheel 111, the rotation of the steering wheel 111 is sequentially transmitted to the main shaft 112 and the intermediate shaft 113, whereby the pinion 114.
Rotates. Then, as the pinion 114 rotates, the rack 115 meshing with the pinion 114 moves in its extension direction (left-right direction in the drawing). The movement of the rack 115 is transmitted to the intermediate arm 116 and the knuckle arm 117. Then, the knuckle arm 117 is at the steering center C.
To rotate the wheel 120 supported by the supporting portion 117a at an angle indicated by a solid line and a dotted line.

The steering wheel steering angle sensor 22 is attached to a member that rotates as the steering wheel 111 is rotated, such as the pinion 114 described above. The steering wheel steering angle sensor 22 illustrated in the same figure is composed of an optical sensor 22a having two pairs of light emitting and receiving elements (not shown), and a drum member 22b in which a reflective surface and a non-reflective surface are alternately formed on the peripheral surface thereof. Has been done. The drum member 22b is attached to the pinion 114 so as to rotate together with the pinion 114. The optical sensor 22a is the drum member 22b.
The pairs of light emitting / receiving elements are arranged so as to be opposed to the peripheral surface of the drum member 22b and are slightly displaced with respect to the rotation direction of the drum member 22b.

When the driver rotates the handle 111 to rotate the pinion 114, the drum member 22b also rotates. As the drum member 22b rotates, the reflective surface and the non-reflective surface pass one after another in front of the optical sensor 22a, and the optical sensor 22a outputs a pulse signal corresponding to the rotation. At this time, since the respective pairs of the light emitting and receiving element pairs of the optical sensor 22a are slightly displaced, the optical sensor 22a
Outputs a pulse signal whose phase is shifted. The phase shift state of this pulse signal is determined by the drum member 22.
It depends on the rotation direction of b. Therefore, the steering wheel steering angle sensor 22 grasps the operation amount (rotation amount) of the steering wheel 111 by counting the number of pulses of the pulse signal from the optical sensor 22a, and the steering wheel 11 from the phase shift state.
The rotation direction of 1, that is, the turning direction of the vehicle is grasped.

The steering angle sensor 23 is attached to the knuckle arm 117 of the steering mechanism 110 [see FIG. 3 (a)] as shown in FIG. 3 (b), for example. The steering angle sensor 23 includes a roller member 23a and a pulse generator 23b that generates a pulse signal based on the rotation of the roller member 23a. The roller member 23a is arranged so as to come into contact with the turning center C of the knuckle arm 117, and the roller member 23 also rotates itself when the turning center C of the knuckle arm 117 rotates. The pulse generator 23b
For example, it is configured as a so-called photo interrupter (neither is shown) composed of a slit plate that rotates together with the roller member 23a and a light emitting / receiving element pair arranged on this slit plate, and it is configured according to the rotation of the roller member 23a. It is configured to output a pulse signal. The steering angle sensor 23 also grasps the operation amount and the rotating direction of the handle 111 based on this pulse signal.

The pulse signal from the steering wheel steering angle sensor 22 or the steering angle sensor 23 is supplied to the CPU 41 of the data processing unit 4 as turning information. Although the steering wheel angle sensor 22 or the steering angle sensor 23 using an optical sensor is exemplified here, the steering angle sensors 22 and 23 are not limited to the exemplified ones, and the steering angle of the steering wheel 111 is not limited thereto. Alternatively, any device that detects the steering angle of the steered wheels (wheel 120) may be used.

The traveling distance detector 24 is a mechanism for detecting the traveling distance of the vehicle, and is attached to the transmission shaft of the vehicle. The mileage detector 24 includes, for example, a magnet attached to the transmission shaft, a Hall element disposed adjacent to the magnet, and a comparison voltage variable according to the output from the Hall element at one terminal thereof. (For example, a + terminal) and a comparator having a reference voltage supplied to the other terminal (for example, a-terminal).

In such a traveling detector 24, the magnet rotates as the transmission shaft rotates, and applies a magnetic field of N, S, N, S, ... To the Hall element. As a result, the Hall element induces a predetermined voltage according to the applied magnetic field, and increases or decreases the comparison voltage supplied to one terminal of the comparator. The comparator outputs "H" when the comparison voltage supplied to one terminal is lower than the reference voltage supplied to the other terminal, and outputs "L" voltage when the comparison voltage is higher than the reference voltage. Therefore, the comparator 1
Outputs traveling pulses of "H" and "L" corresponding to the rotation of the magnet. That is, the traveling detector 24 is configured to output a pulse signal each time the vehicle travels a unit distance.

The storage unit 3 is the image pickup unit 1 described in the conventional apparatus.
In addition to the frame memory 31 for storing the image signal from the vehicle, the reference data memory 32 for holding the image data in the case of a flat ground without obstacles, and the vehicle data memory 33 for holding the vehicle shape data and the like, Monitoring area 100a
It has an out-of-range obstacle memory 34 which holds the data regarding the obstacles out of the range as the out-of-range obstacle data.

Similar to the conventional apparatus, the data processing unit 4 has a ROM 42 in which an operation program is stored, a CPU 41 which operates according to this operation program, and a RAM 43 which temporarily stores information required when the CPU 41 operates. ing. Further, the notifying unit 7 including the display unit 5 and the voice instructing unit 6 is not significantly different in configuration from the conventional device. However, the display unit 5 is provided with a switch 52 for switching and displaying the image captured by the image capturing unit 1 and the image generated by the data processing unit.

Next, the operation of the first embodiment having the above configuration will be described. In the present embodiment, the data processing unit 4 executes the processing of the flowchart shown in FIG. 4 to perform the operation of calculating the relative position of the vehicle and the obstacle and the alarm operation for avoiding the collision.

First, in step S110, it is determined whether or not this measurement operation, that is, the processing of this flowchart is the first time. If it is determined in step S110 that this is the first process, it is not necessary to determine whether or not the obstacle has moved outside the monitoring area 100a of the image pickup means 1, and the process proceeds to step S120. Then, if it is determined in step S110 that this is not the first time, the process proceeds to step S111.

In step S111, the moving position of the vehicle is calculated, that is, the moving amount of the vehicle such as the moving distance and the moving direction of the vehicle within the period from the previous measuring operation to the current measuring operation is calculated. In this step S111,
The traveling distance of the vehicle is acquired based on the pulse signal from the traveling detector 24, and the moving direction of the vehicle is acquired based on the turning information from the steering wheel steering angle sensor 22 or the steering angle sensor 23 described in FIG. . Then, the traveling distance information and the turning information are retained (stored in the RAM 43) as the relative movement amount of the vehicle.

In step S111, the traveling detector 8
The relative movement amount of the vehicle is calculated based on the traveling distance information from the steering wheel steering angle sensor 22 (the steering angle sensor 23) and the relative movement amount of the vehicle is calculated as shown in FIG. The image information from 1 may be used for the calculation.

That is, as shown in FIG. 5A, an arbitrary index object B is set in the image information stored in the frame memory 31, and the coordinates B ₁ of the point of interest in this index object B are acquired. The coordinates of this point of interest move to the coordinate B _{2 as} shown in FIG. 5B as the vehicle moves, and move to the coordinate B _{3 as} shown in FIG. 5C as the vehicle moves further. .

Then, the point of interest is coordinate B ₁ , coordinate B ₂ ,
The relative movement amount of the vehicle may be calculated based on the movement amounts dx ₁ and dy ₁ and the movement amounts dx ₂ and dy ₂ in the frame memory 31 when moving to the coordinate B ₃ . In this case, the height of the target point is known by the three-dimensional position calculation, and the above-described calculation is executed based on the target point whose height is known.

In addition, a vehicle speed sensor for detecting the speed of the vehicle is used, and the vehicle speed detected by this vehicle speed sensor and the elapsed time from the previous measurement operation to the current measurement operation are integrated to determine the vehicle speed. Calculate the travel distance, and add this travel distance information to
The relative movement amount of the vehicle may be acquired in consideration of the turning information from the steering wheel steering angle sensor 22 and the like.

The calculation of the relative movement amount of the vehicle in step S111 is performed by the vehicle 1 as shown in the schematic diagram of FIG.
No. 00 mounting position P100 (x100, y1) of the image pickup means 1
It is convenient that the processing can be speeded up by processing 00) as the reference point and using the movement amount of the reference point. In this case, in the collision prediction step described later, the vehicle information held in the vehicle data memory 33 of the storage unit 3, that is, the reference point P100, the right rear end point P100a, the left rear end point P100.
The position and shape of the vehicle are grasped and the collision is predicted by superimposing the respective data of b, the right front end point P100c and the left front end point P100d on the reference point P100.

When the relative movement amount of the vehicle is calculated in step S111 as described above, the process proceeds to step S112. In this step S112, the position of the obstacle at the time of the current measurement operation is estimated for the obstacle detected up to the previous time based on the relative movement amount of the vehicle calculated in step S111. This is performed by assuming that the obstacle is stationary and adding the relative movement amount of the vehicle calculated in step S111 to the position coordinate of the obstacle at the time of the previous measurement operation.

In the obstacle position estimating operation in step S112 as well, if the center of gravity of the obstacle 200 is set as the reference point P200 and the moving coordinate value of the reference point P200 is processed as shown in FIG. 6, the processing speed is increased. be able to.
And when the reference points are used in this way, in the collision prediction step described later as in the case of the previous vehicle data,
The shape data P200r of the obstacle 200 is used as the reference point P2.
The position and the shape of the obstacle are grasped and the collision is predicted by superimposing it on 00.

Then, in the following step S113, the obstacle estimated in step S112 is the monitoring area 100a.
It is determined whether it is within the range or out of the range. This determination is made for each obstacle in the monitoring area 100a. Then, regarding the obstacle determined to be “out of range”, the position information and the shape information thereof are stored in the out-of-range obstacle memory 43 of the storage unit 3. The determination of "inside the range" or "outside the range" in step S113 can be speeded up by configuring the center of gravity of the obstacle as a reference point to determine whether or not the reference point is within the monitoring area 100a. it can.

That is, as shown in the schematic diagram of FIG. 6, the center of gravity of the obstacle 200 is at the reference point P200 at the time of the previous measurement operation, and the center of gravity of the obstacle 100 is changed by the movement of the vehicle 100. When the obstacle 200 is moved to the position P200, the center of gravity of the obstacle 200 is in the monitoring area 100a when the obstacle 200 is at the position P200.
Is determined to be “in range”. And the obstacle 200 is P2
When it is at the position of 00a, its center of gravity is the monitoring area 100a.
Obstacle 200 is “out of range” because it is located further out
Is determined. As described above, by using the center of gravity of the obstacle 200 as a criterion for the determination, the arithmetic processing can be simplified.

The above steps S111 to S11
3, it is determined whether or not the obstacle is within the monitoring area 100a, and for the obstacle determined to be outside the range of the monitoring area 100a, its position information and shape information are stored in the out-of-range obstacle memory 43 of the storage unit 3. Hold. And step S1
Move to 20.

In this step S120, an obstacle is detected at the vehicle position at the time of this measurement. That is, in this step S120, the presence or absence of an obstacle existing in the monitoring area 100a at this time is detected by the image pickup unit 1 (see FIG. 15). And step S12
In 1, the acquisition operation of shape information such as a two-dimensional shape or a three-dimensional shape and relative position information with respect to the imaging unit 1 are calculated for each of the obstacles detected in step S120. In the calculation of the relative position information in step S121, for example, the center of gravity of the obstacle is calculated based on the shape information of the obstacle, and the relative position between the center of gravity and the imaging unit 1 is calculated.

Then, in step S122, the relative position information and the shape information of the obstacle calculated in step S121 are stored. The relative position information and the shape information are held in the RAM 43 of the data processing unit 4. Through the processes of steps S120 to S122, the obstacle in the monitoring area 100a is detected, and the relative position between the detected obstacle and the vehicle (imaging unit 1) is calculated.

Then, the process proceeds to step S130. In step S130, the course of the vehicle is predicted. That is, in this step S130, the steering angle detection unit 21 (the steering wheel steering angle sensor 22, the steering angle sensor 2 described with reference to FIG. 3 is described.
The turning angle of the vehicle is calculated based on the turning information from 3), and the predicted course of the vehicle is calculated based on the turning angle.

Regarding the processing of step S130, it goes without saying that the course of the vehicle can be calculated by image processing on the data stored in the frame memory 31, as in step S111 described above.

Then, in the following step S131, the positional information and the shape information regarding the obstacle "out of range" stored in the above step S113, and the above step S122.
The position information and the shape information regarding the obstacle in the monitoring area 100a (“within range”) stored in step S1
Based on the predicted route of the vehicle calculated in step 31, the relative position between the "in-range" and "outside range" obstacles and the vehicle on the predicted route is calculated, and the obstacle is calculated based on the calculated relative position. And the vehicle collide (there is a risk of collision).

In the next step S140, the above step S
Based on the determination result of 131, it is determined whether or not to issue an alarm. That is, if it is determined in step S131 that "no collision" occurs, then step S1
The process moves to 10 and the next measurement operation is performed. On the other hand, if it is determined in step S131 that "collision (risk)", the process proceeds to step S150.

In step S150, the CPU 4
1 sends a drive signal to the display device 51 of the display unit 5 so as to display a message of “collision”,
A voice guidance or a voice signal for generating a buzzer sound is transmitted to the drive circuit 62 of the voice instruction unit 6 to notify the driver of the “collision”. The display device of the display unit 5 displays a message in response to a drive signal or a voice signal from the CPU 41, and the speaker 61 of the voice instruction unit 6 generates a voice or a buzzer sound.

Incidentally, this step S140-step S
In 150, when there is a risk of collision between the obstacle and the vehicle, a warning message or a warning sound (voice guidance,
Although a configuration for generating a buzzer sound has been illustrated, the display device 5 constantly displays obstacles "in range" and "outside range".
Alternatively, the driver may recognize the relative distance to the vehicle.

Next, the actual processing based on the above steps will be described with reference to FIG. When the vehicle 100 is at the position P4 (shown by the dotted line in the figure),
The obstacle 200 exists in the monitoring area 100a of the vehicle 100. Therefore, at this position P4, the obstacle 200 is treated as “in range”. Then, the mounting position P100 (P4) of the imaging unit 1 and the obstacle 200 in the vehicle 100
The relative distance SP (P4) to the center of gravity P200 and DX (P4) and DY (P4) indicating the positional relationship between the two are calculated and stored as relative position data.

At this position P4, since the steering angle of the steered wheels (front wheels) of the vehicle is the steering angle θ ₀ parallel to the side surface of the vehicle 100, the vehicle 100 goes straight forward and moves backward. C
PU43 is the vehicle data of the vehicle 100 and this obstacle 200
It is determined whether or not the vehicle 100 and the obstacle 200 collide with each other by taking into consideration the shape data. In this case, as is clear from the figure, the two do not collide, so no alarm is issued.

When the vehicle moves to the position P5 (indicated by the solid line in the figure), the obstacle 200 is in the monitoring area 1
It is off 00a. Therefore, at this position P5, the obstacle 200 is treated as being “out of range”, and its position information and shape information are stored in the out-of-range obstacle memory 3 of the storage unit 3.
4 is held. Then, the mounting position P100 (P5) of the imaging unit 1 and the center of gravity P of the obstacle 200 in the vehicle 100
A relative distance SP (P5) with respect to 200 and DX (P5) and DY (P5) indicating the positional relationship between the two are calculated, and these are stored as relative position data.

At this position P5, since the steering angle of the steered wheels (front wheels) of the vehicle is the steering angle θ ₂ , the vehicle 100 moves backward in an arc on the locus shown by the arrow in the figure. CPU4
Reference numeral 3 determines whether or not the vehicle 100 and the obstacle 200 collide with each other by taking the vehicle data of the vehicle 100 and the shape data of the obstacle 200 into consideration. In this case, as is clear from the figure, the obstacle 200 is located at the right front end portion P100 of the vehicle 100.
Since it exists within the range of the locus of c, in this case, the CPU 43
Sends a signal to the notification unit 7 so as to generate an alarm. Thus, in this embodiment, the monitoring area 100
Since the relative position with respect to the vehicle is calculated even for obstacles outside the range of a, if there is a risk of collision with the vehicle even if the obstacle is outside the monitoring range, the driver is notified of it. .

In the first embodiment described above, the moving position can be known even when the obstacle detected within the image pickup range of the CCD camera 14 moves out of the monitoring area due to the relative movement with the vehicle. Next, a second embodiment will be described in which the moving position of a relatively small obstacle that repeats detection and disappearance within the imaging range of the CCD camera 14 can be known.

FIG. 7 is a block diagram for explaining the configuration of the second embodiment. In FIG. 7, the same parts as those described above are designated by the same reference numerals. The second embodiment has a structural difference in that the storage unit 3 is provided with an in-range obstacle movement position memory 35. The in-range obstacle moving position memory 35 is provided in place of the out-of-range obstacle memory 34 in the first embodiment.

Further, the image pickup unit 1 in the second embodiment
Is a stereo type (see FIG. 15B) configured by a pair of video cameras, and a pair of left and right frame memories 31 are also provided. Accordingly, the road surface image can be deleted from the parallax between the left and right images, and the edge image of the image (that is, the object) having a height can be acquired. Note that other configurations in the second embodiment are the same as the configurations described in the first embodiment described above, and therefore description thereof will be omitted.

Then, the in-range obstacle moving position memory 35
8, the three holding areas, that is, the first holding area 35a, the second holding area 35b, and the third holding area 3 are shown in FIG.
5c.

Of these holding areas, the first holding area 35
The area a is an area where the three-dimensional position data of all feature points of the image captured by the image capturing unit 1, that is, the obstacle existing in the monitoring area is stored.

The second holding area 35b is an area for storing three-dimensional position data of the feature points of the obstacle at the time when a predetermined time has elapsed. This second holding area 3
The data stored in 5b is the vehicle detected based on the three-dimensional position data of the feature points of the obstacle stored in the first holding area 35a and the transmission interval of the pulse signal output from the traveling distance detection unit 24. Traveling speed information and the steering angle detection unit 21
The three-dimensional position data of the feature point of the obstacle predicted to exist at the time when the predetermined time has elapsed, which is predicted based on the vehicle turnover information detected by.

In addition, the third holding area 35c is predicted based on the three-dimensional position data of the feature points of the obstacle stored in the second holding area 35b, the traveling speed information of the vehicle, and the turning information of the vehicle. The three-dimensional position data of the feature points of the obstacle that is predicted to exist after the predetermined time has elapsed.

Next, the operation of the second embodiment having the above configuration will be described. In the present embodiment, the data processing unit 4 executes the processing of the flowchart shown in FIG. 9 to perform the operation of calculating the relative position of the vehicle and the obstacle and the alarm operation for avoiding the collision.

First, in step S200, the first holding area 35a and the second
The contents held in the holding area 35b and the third holding area 35c are reset. In step S201, the cycle counter is reset. The cycle counter counts the operation cycle of the device and is set in the RAM 43 of the data processing unit 4. The time count of the cycle counter in this embodiment is defined as the time required to process an image of one frame. Then, in step S202, the counting operation by the cycle counter is started.

In step S210, the image in the monitoring area 100a (see FIG. 16) captured by the image capturing section 1 is captured in the frame memory 31. In step S211,
The steering wheel steering angle information from the steering angle detection unit 21 and the transmission interval of the pulse signal output from the travel distance detection unit 24, that is, the vehicle speed information is acquired.

In step S212, obstacle detection processing is performed. This obstacle detection process is a process performed on the captured image captured in step S210.
In this process, first, a road surface image (that is, an image of height “0”) is removed, and a differential process in the horizontal direction and / or the vertical direction is performed on the image from which the road surface image is removed. , An edge image representing the appearance of the object is acquired.

Then, from the acquired edge image, the three-dimensional coordinate data of the feature points indicating the shape of the object (that is, the obstacle) existing in the monitoring area 100a is acquired as the obstacle data. In step S213, the above step S2
Three-dimensional coordinate data of feature points acquired in 12 (detection result)
The first holding area 35 of the obstacle movement position memory 35 within the range
Store it in a.

In step S220, the contents held in the second holding area 35b of the obstacle movement position memory 35 within range are referred to, and it is determined whether or not data is stored in the second holding area 35b. Then, if the data is stored (Y), the process proceeds to step S221, and if the data is not stored (N), the process proceeds to step S225.

In step S221, the contents held in the third holding area 35c of the in-range obstacle moving position memory 35 are referred to, and it is determined whether or not data is stored in the third holding area 35c. Then, if the data is stored (Y), the process proceeds to step S222, and if the data is not stored (N), the process proceeds to step S224.

In step S222, the first holding area 35a and the second holding area 3 of the in-range obstacle moving position memory 35 are stored.
The data stored in 5b and the third holding area 35c is displayed on the display device 51 of the display unit 5.

More specifically, the three-dimensional position data (first holding area 35a) of the feature points of the obstacle acquired from the image picked up in the current operation cycle, the feature point of the obstacle acquired one operation cycle before. Three-dimensional position data of the obstacle predicted from the three-dimensional position data of the obstacle (second holding area 35b), three-dimensional position data of the obstacle predicted from the three-dimensional position data of the obstacle acquired two operation cycles (first 3 holding area 35c) is displayed. When the process of step S222 is completed, the process proceeds to step S223, and the process waits until the count value of the cycle counter reaches the specified time and stops.

On the other hand, no data is stored in the third holding area 35c of the in-range obstacle moving position memory 35 (S2
If “N” in step 21) and the process proceeds to step S224,
In this step S224, the first holding area 35a (data of the current operation cycle) and the second holding area 35b (data predicted from the data of one operation cycle before) of the in-range obstacle movement position memory 35 are displayed on the display device 51. To display. Then, the process proceeds to step S223, and waits until the count value of the above-described cycle counter reaches the specified time and stops.

Data is not stored in the second holding area 35c of the in-range obstacle moving position memory 35 (S2).
If “N” in step 20) and the process proceeds to step S225,
In step S225, only the first holding area 35a (data of the current operation cycle) of the in-range obstacle moving position memory 35 is displayed on the display device 51. Then, the process shifts to step S226 to stop the counting of the cycle counter described above, and then the process shifts to step S232.

When the process of step S223 is completed and the process proceeds to step S230, the position of the obstacle in the next operation cycle is predicted in step S230. That is, in this step S230, the vehicle at the next operation cycle is based on the steering wheel steering angle information from the steering angle detection section 21, the vehicle speed information from the travel distance detection section 24 and the operation cycle defined by the cycle counter (RAM 43). The relative movement amount of the obstacle is calculated, and the movement position of the obstacle (three-dimensional coordinate data of the characteristic point) at the time of the next operation cycle is calculated for the data stored in the second holding area 35b of the in-range obstacle movement position memory 35. Calculate (predict)

In a succeeding step S231, the moving position of the obstacle calculated in the above step S230 is stored in the third holding area 35c of the in-range obstacle moving position memory 35.

In step S232, the position of the obstacle in the next operation cycle is predicted for the obstacle acquired in the current operation cycle. That is, in this step S232,
The operation of step S230 is performed on the data stored in the first holding area 35a of the in-range obstacle moving position memory 35. In subsequent step S233, the moving position of the obstacle calculated in step S232 is stored in the second holding area 35b of the in-range obstacle moving position memory 35. When the process of step S233 is completed, the process proceeds to step S201, and the operation in the next cycle is executed.

Next, the actual processing based on the above steps will be described with reference to FIGS. FIG.
0 is a schematic diagram showing the stored contents in the in-range obstacle moving position memory 35, FIG. 11 is a schematic diagram for explaining the relative positional relationship between the vehicle 100 and the obstacle 200A, and FIG. 12 is a video camera of the image pickup unit 1. FIG. 13 is a schematic diagram for explaining the relationship between scanning lines and an obstacle imaged by this video camera, and FIG. 13 is a schematic diagram for explaining how the obstacle 200A is displayed by the display device 51.

As shown in FIG. 11, the vehicle 100 (t1) that is stopped at the time of the operation cycle t1 starts moving rearward immediately after the operation cycle t1 elapses, and the operation cycle after the predetermined operation cycle t elapses. Consider a case where an image is acquired at time t2 (corresponding to the position indicated by reference numeral 100 (t2)).

In this case, first, when the vehicle stops before the operation cycle t1, the stored contents of the in-range obstacle moving position memory 35 are reset (S200). Then, the count value of the cycle counter is reset and the count by the cycle counter is started (S201, S202). And
The image capturing unit 1 captures an image of the inside of the monitoring area 100a, and the image data is taken into the frame memory 31 as image data at the time of the operation cycle t1 (S210). At the same time, steering wheel steering angle information from the steering angle detecting unit 21 and traveling The transmission interval of the pulse signal output from the distance detection unit 24, that is, the vehicle speed information is taken into the RAM 43 (S211). The obstacle 200A at this point is, as shown in FIG.
It exists on the scanning line of the video camera and is in a state where it can be detected by this video camera.

Subsequently, detection of an obstacle, that is, creation of an edge image of an object having a height from image data taken into the frame memory 31 and feature points (three-dimensional position coordinate data, hereinafter referred to as feature point data) based on this edge image. Say)
Is acquired (S212). Then, as illustrated, when only the obstacle 200A exists in the monitoring area 100a, the feature point data of the obstacle 200A is acquired. Further, when a large number of obstacles exist, the feature point data of all the existing obstacles are acquired.

The acquired feature point data is stored in the first holding area 35a of the in-range obstacle moving position memory 35 (S213). In this case, since it is the first operation cycle, the acquired feature point data is stored as the feature point data P ₁₁ in the first holding area 35a as shown in the operation cycle t1 in FIG.

Subsequently, the obstacle moving position memory 35 within the range
It is determined whether or not past feature point data is stored in the second holding area 35b and the third holding area 35c (S).
220, S221). In this case, the first holding area 35
Since the feature point data is stored only in a (S220
"N"), an obstacle based on the feature point data stored in the first holding area 35a is displayed on the display device 51 (S225, see FIG. 13A).

Then, the process waits until the count value of the operation cycle counter reaches the specified time (S226), and next to the characteristic point data stored in the first holding area 35a by the arithmetic operation using the steering wheel steering angle information and the vehicle speed information. The movement position at the time of the operation cycle is predicted (S232), and the predicted movement position is stored as the characteristic point data P ₁₂ in the second holding area 35b (S233, see the operation cycle t2 in FIG. 10).

Then, the cycle counter is reset and counting is started again (S201, S20).
2), the image at the operation cycle t2, the steering angle of the steering wheel, and the vehicle speed are acquired (S210, S211). The obstacle 200A at the time of the operation cycle t2 exists between the scanning lines of the video camera as shown by the dotted line in FIG. 12, and cannot be detected (disappeared).

[0116] Subsequently, and retrieving feature data of the captured image data from the obstacle (S212), the range in the obstacle movement position memory 3 the acquired feature data P ₂₁
5 in the first holding area 35a (S213, FIG.
0 operation cycle t2).

Subsequently, the obstacle moving position memory 35 within the range
It is determined whether or not past feature point data is stored in the second holding area 35b and the third holding area 35c (S22).
0, S221). In this case, the first holding area 35
Since the characteristic point data is stored in the a and the second holding area 35b ("N" in S221), an obstacle based on the characteristic point data stored in the first holding area 35a and the second holding area 35b is detected. It is displayed on the display device 51 (S224).

At this time, this obstacle 200A (t2)
, Which disappears in the captured image as described above, can be displayed based on the feature point data held in the second holding area 35b (see FIG. 13B).

Then, the process waits until the count value of the operation cycle counter reaches the specified time (S223), and the characteristic point data P ₁₂ stored in the second holding area 35b by the arithmetic operation using the steering wheel steering angle information and the vehicle speed information. The moving position at the time of the next operation cycle of (S230) is predicted, and the predicted moving position is stored in the third holding area 35c as the feature point data P ₁₃ (S221, operation cycle t in FIG. 10).
3).

Further, a similar movement position predicting operation is first described.
The feature point data P ₂₁ stored in the holding area 35a is processed (S232), and the predicted movement position is stored in the second holding area 35b as the feature point data P ₂₂ (S23).
3). After that, the process of shifting to the operation cycle t3 is repeatedly executed.

By continuously executing such processing, as shown in FIG. 10, the in-range obstacle moving position memory 35 stores the feature point data P ₁₁ based on the picked-up image (current operation cycle). , P ₂₁ , P _31, ... Feature point data P ₁₂ , P ₂₂ , P _32, ...
-, and, since the second operation cycle before feature point data P ₁₃ based on the captured image, P _23, P ₃₃ · · · are stored, as illustrated, detects obstacle in the captured image of the video camera, Even when the disappearance is repeated, the obstacle can be continuously captured.

Also in the present embodiment described above, the turning direction of the vehicle may be detected based on the moving amount of the pixel of interest stored in the frame memory 31 as in the first embodiment. Good. Similarly, based on the relative distance between the obstacle and the vehicle, when there is a risk of collision between the two, the driver may be notified of the fact via a notification means.

The basic configuration of the present invention and this embodiment have the following correspondence. That is, the image pickup means in the basic configuration of the present invention corresponds to the image pickup section 1 in the present embodiment, the traveling state information output means corresponds to the detection section 2, and the notification means corresponds to the notification section 7. The turning direction detecting means corresponds to the steering angle detecting section 21, the display means corresponds to the display section 5, and the voice instruction means corresponds to the voice instruction section 6. The out-of-range obstacle information holding means corresponds to the out-of-range obstacle memory 34, the in-range obstacle moving position holding means corresponds to the in-range obstacle moving position memory 35, and the image holding means corresponds to the frame memory 31. There is.

The obstacle detecting means 41a corresponds to step S120 (first embodiment) and step S212 (second embodiment) in the flow chart, and the obstacle position acquiring means (41b) carries out step S121 (first embodiment). Embodiment) and step S212 (second embodiment). The obstacle position predicting means 41c operates in step S130.
(1st Embodiment) and step S230 and S232.
This corresponds to (first embodiment). The vehicle course prediction means 41d corresponds to step S130 (first embodiment).

[0125]

As described in detail above, the present invention has the following effects. That is, since the moving position of the obstacle detected based on the traveling state information of the vehicle is predicted and the obstacle is displayed at the predicted moving position, the control based on the predicted moving position is performed. Therefore, it is possible to provide a convenient vehicle surroundings monitoring device.

Further, since the obstacle targeted by the obstacle position predicting means and the notifying means is the obstacle disappeared from the image pickup range of the image pickup means, the moving position of the obstacle disappeared from the image pickup range can be known. Therefore, it is possible to provide a vehicle surroundings monitoring device with improved safety and ease of use.

Further, the position information of the obstacle acquired by the obstacle position acquisition means is regarded as an obstacle disappeared by moving it out of the range of the image pickup means, and the obstacle position prediction means determines the information of the obstacle (outside the range). Since the moving position of the disappeared obstacle is predicted based on the obstacle information), it is possible to know the moving position of the obstacle that has disappeared after moving out of the imaging range by the image pickup means, which is improved in usability. A good vehicle periphery monitoring device can be provided.

Further, the position information of the obstacle acquired by the obstacle position acquisition means is used as the position information of the obstacle existing in the image pickup range of the image pickup means, and the obstacle position prediction means and the position information of this obstacle are traveled. Based on the state information, to predict the moving position of the obstacle, the notifying means, with respect to the obstacle disappeared in the imaging range, to notify the driver of the moving position predicted by the obstacle position predicting means, It is possible to know the moving position of an obstacle that has disappeared within the imaging range by the imaging means, and it is possible to provide a user-friendly vehicle periphery monitoring device with improved safety.

Further, the obstacle position acquisition means periodically acquires the position information (obstacle information within the range) of the obstacle existing within the image pickup range of the image pickup means, and the obstacle position predicting means makes it possible to obtain it every cycle. , The acquired moving position of the obstacle is predicted, and the predicted moving position of the obstacle is stored in the obstacle moving position holding means within the range, and the notifying means relates to the obstacle disappeared within the imaging range. Since the moving position predicted by the position predicting means is notified to the driver, it is possible to immediately know the moving position of the obstacle disappeared within the image capturing range by the image capturing means, which is a vehicle with improved safety and ease of use. A peripheral monitoring device can be provided.

Since the notifying means notifies the driver of the route of the vehicle predicted by the vehicle route predicting means and the moving position of the obstacle predicted by the obstacle position predicting means,
It is possible to know the existence of obstacles on the route in which the vehicle is predicted to travel, even for obstacles that have disappeared from the imaging range, and to provide a user-friendly vehicle periphery monitoring device with improved safety. it can.

Further, the vehicle course predicting means recognizes the turning direction of the vehicle based on the image information from the image pickup means and predicts the course of the vehicle from the recognized turning direction.
No special device is required, and the structure of the device can be simplified.

The vehicle route prediction means refers to the image information held in the image holding means for holding the image information from the image pickup means, and detects the moving direction and the moving amount of the pixel of interest in this image information. Then, since the moving direction of the vehicle is recognized based on the detected moving direction and the detected moving direction and the course of the vehicle is predicted from the recognized moving direction, only the point of interest is processed when predicting the course of the vehicle. Since it only has to be performed, the calculation steps required for processing can be simplified.

Further, since the vehicle route predicting means predicts the route of the vehicle based on the traveling state information, it is possible to predict the route of the vehicle and the moving position of the obstacle with common information. The device configuration can be simplified.

The running state information output means uses the running distance information from the running distance detecting means for detecting the running distance of the vehicle and the turning direction information from the turning direction detecting means provided in the vehicle as the running state information. Since it is output, the traveling distance information and the turning direction information can be read directly from the vehicle, and the processing can be performed at high speed when predicting the course of the vehicle.

Further, since the turning direction detecting means is constituted as a steering wheel angle sensor for detecting the steering angle of the steering wheel or a steering angle sensor for detecting the steering angle of the steered wheels, the information on the turning direction of the vehicle can be reliably grasped. can do.

Further, the collision predicting means predicts whether or not the vehicle will collide with the obstacle based on the predicted vehicle course and the predicted movement position of the obstacle, and informs when the collision is predicted. Since the means informs of the collision, the driver can know in advance the possibility of collision between the vehicle and the obstacle when the vehicle moves in the current state, and the collision can be prevented in advance. It is possible to provide a vehicle periphery monitoring device with improved safety.

Since the notification means is a display means for displaying the relative distance between the vehicle and the obstacle and a message, or one or both of voice instruction means for generating a buzzer sound or a voice, the driver's vision or The positional relationship between the vehicle and the obstacle can be transmitted by hearing.

[Brief description of drawings]

FIG. 1 is a basic configuration diagram of the present invention.

FIG. 2 is a block diagram illustrating the configuration of the first embodiment.

FIG. 3 is a diagram illustrating a steering angle detection unit 21.

FIG. 4 is a flowchart illustrating the operation of the first embodiment.

5 is a diagram illustrating a method of calculating a relative movement amount of a vehicle from image information stored in a frame memory 31. FIG.

FIG. 6 is a schematic diagram illustrating an operation of estimating the position of an obstacle and an operation of determining an area where an obstacle exists.

FIG. 7 is a block diagram illustrating a configuration of a second embodiment.

FIG. 8 is a diagram illustrating a configuration of an in-range obstacle moving position memory 35.

FIG. 9 is a flowchart illustrating the operation of the second embodiment.

FIG. 10 is a schematic diagram showing stored contents in an in-range obstacle moving position memory 35.

FIG. 11 is a schematic diagram illustrating a relative positional relationship between a vehicle 100 and an obstacle 200A.

FIG. 12 is a schematic diagram illustrating a relationship between a scanning line of a video camera which constitutes the imaging unit 1 and an obstacle imaged by the video camera.

FIG. 13 is a schematic diagram illustrating how an obstacle 200A is displayed by a display device 51.

FIG. 14 is a block diagram illustrating a configuration of a conventional device.

FIG. 15 is a diagram illustrating a configuration of an image pickup unit 1.

FIG. 16 is a diagram illustrating an attachment mode of the image pickup unit 1.

FIG. 17 is a diagram illustrating the relationship between the movement of the vehicle 100 and the obstacle 200.

FIG. 18 is a diagram illustrating the relationship between the movement of the vehicle 100 and the obstacle 200.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Imaging part 2 Detecting part 21 Snake angle detecting part 24 Running distance detecting part 3 Storage part 31 Frame memory 34 Out-of-range obstacle memory 35 In-range obstacle moving position memory 4 Data processing part 41 CPU 42 ROM 43 RAM 5 Display part 6 Voice instruction unit 7 Notification unit 100 Vehicle 200, 200A Obstacle

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication location G08G 1/04 G08G 1/04 C (72) Inventor Kazuyuki Sasaki 1500 Onjuku, Susono-shi, Shizuoka Yazaki Sogo Co., Ltd. In the company

Claims (14)

[Claims] 1. A vehicle periphery monitoring device for monitoring the periphery of a vehicle based on image data from an image pickup means installed in the vehicle, the obstacle detecting means for detecting an obstacle from the image data, and the obstacle detection. Obstacle position acquisition means for acquiring position information of the obstacle detected by means, traveling state information output means for detecting and outputting traveling state information indicating the traveling state of the vehicle, the traveling state information and the obstacle Obstacle position prediction means for predicting the detected moving position of the obstacle based on the position information, and notifying means for notifying the driver of the moving position of the obstacle predicted by the obstacle position predicting means. And a vehicle surroundings monitoring device. 2. The obstacle position predicting means predicts a moving position of the disappeared obstacle when the obstacle disappears from an imaging range of the imaging means, and the notifying means disappears. The vehicle periphery monitoring device according to claim 1, wherein the predicted movement position of the obstacle is notified. 3. The obstacle position acquisition means acquires, when the obstacle is out of the imaging range of the imaging means and disappears, information about the disappeared obstacle as out-of-range obstacle information and the acquisition. The out-of-range obstacle information is held in the out-of-range obstacle information holding means, and the obstacle position predicting means predicts a moving position for the disappeared obstacle based on the out-of-range obstacle information. The vehicle periphery monitoring device according to claim 1 or 2. 4. The obstacle position acquisition means acquires position information of the obstacle within the imaging range of the imaging means as in-range obstacle information, and the obstacle position prediction means is in the range obstacle. The moving position with respect to the obstacle is predicted based on the information, and the predicted moving position of the obstacle is stored in the in-range obstacle moving position holding means, and the notifying means causes the obstacle to fall within the imaging range of the imaging means. The vehicle periphery monitoring device according to claim 1, wherein, when the obstacle is disappeared, the predicted movement position of the obstacle is announced. 5. The obstacle position obtaining means periodically obtains the in-range obstacle information, and the obstacle position predicting means obtains the in-range obstacle information and traveling state information for each cycle. 5. The vehicle periphery monitoring device according to claim 4, wherein the moving position of the obstacle is predicted based on the above, and the predicted moving position of the obstacle is stored in the in-range obstacle moving position holding means. 6. A vehicle route predicting means for predicting a route of the vehicle, wherein the notifying means predicts the route of the vehicle predicted by the vehicle route predicting means and the obstacle predicted by the obstacle position predicting means. The vehicle periphery monitoring device according to any one of claims 3 to 5, wherein the driver is notified of the moving position of the vehicle. 7. The vehicle course predicting means recognizes a turning direction of the vehicle based on the image information from the imaging means and predicts a course of the vehicle based on the recognized turning direction. Item 6. The vehicle periphery monitoring device according to item 6. 8. The vehicle route prediction means recognizes a turning direction of the vehicle based on a moving direction and a moving amount of a pixel of interest in an image holding means for holding an image signal from the image pickup means. The vehicle periphery monitoring device according to claim 7. 9. The vehicle periphery monitoring device according to claim 6, wherein the vehicle route prediction means predicts the route of the vehicle based on the traveling state information. 10. The running state information output means detects a running distance of the vehicle, and a turning direction detecting means that detects a turning direction of the vehicle and outputs the detected turning direction as turning information. The vehicle periphery monitoring device according to any one of claims 1 to 9, further comprising: 11. The vehicle according to claim 10, wherein the turning direction detecting means is one of a steering wheel angle sensor that detects a steering wheel steering angle and a steering angle sensor that detects a steering angle of steered wheels. Perimeter monitoring device. 12. A collision for predicting a collision between the vehicle and the obstacle based on a vehicle path predicted by the vehicle path prediction means and a moving position of the obstacle predicted by the obstacle position prediction means. 7. A predicting unit, wherein the notifying unit notifies the collision when the collision predicting unit predicts the collision.
11. The vehicle periphery monitoring device according to any one of 11. 13. The notification means is a display means for displaying a relative distance between a vehicle and an obstacle and a message, or
The vehicle periphery monitoring device according to any one of claims 1 to 12, which is at least one of voice instruction means for generating a buzzer sound or a voice. 14. The vehicle periphery monitoring device according to claim 1, wherein the image pickup unit is attached to a rear side of the vehicle so as to pick up an image of a rear region of the vehicle.