JP2017003531A - Object detection apparatus, object removal operation control system, object detection method, and object detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress erroneous detection and omission of detection of a detection object. SOLUTION: A detection target within the illumination range of the light source light is detected based on the amount of light received by the light receiving means 206 when the light source light is illuminated by the light source 202 that periodically repeats irradiation and non-irradiation of the light source light. In the object detection device for detection, the amount of light received by the light receiving means when the light from the light source is not illuminated by the light source and the amount of light received during non-illumination are the period prior to the time of lighting immediately before the time of non-illumination corresponding to the amount of light received during non-illumination. The amount of light received during illumination and the amount of light received during illumination at least one of the amount of light received during illumination in a period after the time of illumination immediately after the time of non-illumination corresponding to the amount of light received during non-illumination are within the illumination range. The detection processing means 102A for executing the detection process for determining whether or not a predetermined condition for detecting the existence status of the detection target in the above conditions is satisfied is provided. [Selection diagram] Fig. 2

Description

  The present invention relates to an object detection device, an object removal operation control system, an object detection method, and an object detection program.

  2. Description of the Related Art Conventionally, an object detection device that performs detection processing for detecting a detection target by imaging a detection target within an illumination range illuminated by light from a light source using a light receiving unit such as an image sensor is known. ing.

  For example, in Patent Document 1, reflected light from a windshield (light transmissive member) of an automobile illuminated by light source light emitted from a light source is received by a raindrop detection region of an image sensor and imaged. A raindrop detection device that performs raindrop detection processing for detecting attached raindrops is disclosed. This raindrop detection device periodically repeats turning on (irradiating) and turning off (non-irradiating) the light source light to suppress the influence of disturbance light other than the light source light that lowers the detection accuracy of the raindrop, and images when the light source is turned on. The raindrop detection process is performed using difference information between the turned-on image and the off-image captured when the light source is turned off. And when raindrops are detected by this raindrop detection process, control, such as operating a wiper, is implemented. According to Patent Document 1, since the off-time image shows only disturbance light, the difference information obtained by subtracting the off-time image from the on-time image is obtained by excluding the disturbance light component from the on-time image. . By performing raindrop detection processing using such difference information, raindrop detection processing that suppresses the influence of ambient light is made possible.

  Further, when the raindrop detection process is performed using the difference information, the disturbance light component differs between the on-time image and the off-time image if the disturbance light changes greatly after a short time. As a result, the difference information does not appropriately exclude disturbance light components from the on-light image, and in the raindrop detection process, erroneous detection of raindrops or missed detection of raindrops may be caused. For this reason, the raindrop detection device disclosed in Patent Document 1 calculates a difference value between two unlit images captured before and after the on-image, and when the unlit image difference value exceeds a threshold value Then, it is determined that the disturbance light changes greatly after a lapse of a short time, and the raindrop detection process using the difference information is stopped.

  In general, an image sensor is provided with an automatic correction function such as an automatic exposure correction function and a level correction function. When such an automatic correction function is performed, even if the received light amount of the image sensor is the same, the image sensor The output value of the sensor changes. Therefore, even if the raindrops are attached in exactly the same state, the output value of the image sensor for the lighting image will be different due to the automatic correction function. However, the output value of the image when it is turned off may not change substantially even by the automatic correction function. For this reason, in the raindrop detection device disclosed in Patent Document 1, even when the raindrop adhesion state is exactly the same, the automatic correction function changes the difference information between the on-image and the off-image, thereby causing an error in the raindrop. This may cause detection or raindrop detection omission. In addition, at this time, since the difference value between the two off-images captured before and after the on-image is almost zero, the raindrop detection process is not stopped.

  This problem is not limited to detecting the presence state of the detection object using the difference information, but at least using the ambient light and the automatic correction function when detecting the presence state of the detection object using the received light amount during illumination. This can occur in the same manner when taking measures such as grasping erroneous detection or detection omission based on the amount of light received during non-illumination and stopping the detection process.

  In order to solve the above-described problem, the present invention illuminates the light source light based on the amount of light received by the light receiving unit when the light source light is illuminated by the light source that periodically repeats irradiation and non-irradiation of the light source light. In the object detection apparatus for detecting a detection target within a range, the light reception amount received by the light receiving unit when the light source light is not illuminated by the light source and immediately before the non-illumination corresponding to the light reception amount during the non-illumination The amount of light received during illumination in the previous period from the time of illumination and the amount of light received during illumination corresponding to the amount of light received during non-illumination and at least one of the light reception amounts during illumination after the illumination immediately after non-illumination And a detection processing means for executing a detection process for determining whether or not a predetermined condition for detecting the presence state of the detection target in the illumination range is satisfied.

  According to the present invention, there is an excellent effect that it is possible to suppress erroneous detection and detection omission of a detection target.

It is a schematic diagram which shows schematic structure of the vehicle equipment control system in Embodiment 1. FIG. It is explanatory drawing which shows schematic structure of the imaging part and image analysis unit in the same vehicle equipment control system. It is explanatory drawing for demonstrating the optical system of the imaging unit. It is the perspective view which showed typically schematic structure of the imaging unit. (A) is a side view which shows the imaging unit of the state attached to the vehicle whose inclination angle of a windshield with respect to a horizontal surface is 22 degrees. (B) is explanatory drawing which shows the optical system of the imaging unit in case the raindrop has not adhered in the state of the figure (a). (C) is explanatory drawing which shows the optical system of an imaging unit in case the raindrop has adhered in the state of the figure (a). It is a perspective view which shows the reflective deflection | deviation prism of the imaging unit. It is a graph which shows the filter characteristic of the cut filter applicable to the picked-up image data for raindrop detection. It is a graph which shows the filter characteristic of the band pass filter applicable to the picked-up image data for raindrop detection. It is a front view of the front | former stage filter provided in the optical filter of the imaging part. It is explanatory drawing which shows the image example of the captured image data of the imaging part. It is a model enlarged view when the optical filter and image sensor of the imaging part are seen from a direction orthogonal to the light transmission direction. It is explanatory drawing which shows the area | region division pattern of the polarizing filter layer and spectral filter layer of the same optical filter. 5 is an explanatory diagram of raindrop detection processing in Embodiment 1. FIG. 6 is an explanatory diagram showing a relationship between an imaging frame and raindrop detection in Embodiment 1. FIG. 4 is a flowchart showing a flow of raindrop detection processing in the first embodiment. It is a graph which shows the example which the output signal output from the image sensor at the time of lighting changed by the black level correction function. It is a graph which shows the example which the output signal output from an image sensor at the time of light extinction fluctuated by the black level correction function. 10 is a flowchart illustrating an example of raindrop detection processing according to Modification 1. It is explanatory drawing about the various light rays in connection with the raindrop detection in Embodiment 2. FIG. 6 is a flowchart showing a flow of raindrop detection processing in the second embodiment. 10 is a flowchart showing a flow of raindrop detection processing in Modification 2. 10 is a flowchart showing a flow of raindrop detection processing in Modification 3.

Embodiment 1
Hereinafter, the object detection device according to the present invention is applied to an attached matter detection device used in an in-vehicle device control system as a mobile device control system that controls a target device mounted on a vehicle such as an automobile that is a mobile body. An embodiment (hereinafter, this embodiment is referred to as “Embodiment 1”) will be described.
Note that the object detection device according to the present invention can be widely used in a system that executes various processes and controls using the detection result of the detection target, and other movements such as a vehicle other than an automobile, a ship, and an aircraft. The present invention can also be used for a control system for various mobile equipment mounted on the body or a control system for a processing apparatus such as FA (factory automation) that is not equipment mounted on the mobile body.

FIG. 1 is a schematic diagram illustrating a schematic configuration of an in-vehicle device control system according to the first embodiment.
This in-vehicle device control system uses a picked-up image data obtained by picking up an area around the own vehicle (especially a forward area in the traveling direction) by an image pickup unit mounted on the own vehicle 100 such as an automobile, and distributes the headlamp light. Control, wiper drive control, and other in-vehicle devices are controlled.

  The imaging unit provided in the in-vehicle device control system according to the first embodiment is provided in the imaging unit 101, and images an area in the traveling direction of the traveling host vehicle 100 as an imaging region. The windshield 105 is installed near the room mirror. The captured image data captured by the imaging unit of the imaging unit 101 is input to the image analysis unit 102. The image analysis unit 102 includes a CPU, a RAM, and the like, analyzes captured image data transmitted from the imaging unit, and determines the position (direction and distance) of another vehicle existing ahead of the host vehicle 100 in the captured image data. Calculation, detection of an adhering substance (detection target object) such as raindrops or foreign matters adhering to the windshield 105, or detection of a white line (partition line) on the road surface existing in the imaging region. In the detection of other vehicles, a preceding vehicle traveling in the same traveling direction as the own vehicle 100 is detected by identifying the tail lamp of the other vehicle, and traveling in the opposite direction to the own vehicle 100 by identifying the headlamp of the other vehicle. An oncoming vehicle is detected.

  The calculation result of the image analysis unit 102 is sent to the headlamp control unit 103. For example, the headlamp control unit 103 generates a control signal for controlling the headlamp 104 that is an in-vehicle device of the host vehicle 100 from the position data of the other vehicle calculated by the image analysis unit 102. Specifically, for example, while avoiding that the strong light of the headlamp of the own vehicle 100 is incident on the driver of the preceding vehicle or the oncoming vehicle, the driver of the other vehicle is prevented from being dazzled. The switching of the high beam and the low beam of the headlamp 104 is controlled, and partial shading control of the headlamp 104 is performed so that the driver's visibility can be secured.

  The calculation result of the image analysis unit 102 is also sent to the wiper control unit 106. The wiper control unit 106 controls a wiper 107 serving as an object removing unit, and removes attached matter (detection target) such as raindrops and foreign matters attached to the windshield 105 of the host vehicle 100. The wiper control unit 106 receives the attached matter detection result detected by the image analysis unit 102 and generates a control signal for controlling the wiper 107. When the control signal generated by the wiper control unit 106 is sent to the wiper 107, the wiper 107 is operated to ensure the visibility of the driver of the host vehicle 100.

  The calculation result of the image analysis unit 102 is also sent to the vehicle travel control unit 108. Based on the white line detection result detected by the image analysis unit 102, the vehicle travel control unit 108 warns the driver of the host vehicle 100 when the host vehicle 100 is out of the lane area defined by the white line. Notification is performed, and driving support control such as controlling the steering wheel and brake of the host vehicle is performed. In addition, the vehicle travel control unit 108 notifies the driver of the host vehicle 100 of a warning when the distance from the preceding vehicle is close based on the position data of the other vehicle detected by the image analysis unit 102, Carry out driving support control such as controlling the steering wheel and brakes of the vehicle.

FIG. 2 is an explanatory diagram illustrating a schematic configuration of the imaging unit 200 provided in the imaging unit 101 and the image analysis unit 102 constituting the attached matter detection apparatus.
The imaging unit 200 mainly includes an imaging lens 204, an optical filter 205, a sensor substrate 207 including an image sensor 206 as a light receiving unit in which light receiving elements are two-dimensionally arranged, and analog electric power output from the sensor substrate 207. The signal processing unit 208 generates and outputs captured image data obtained by converting a signal (the amount of light received by each light receiving element on the image sensor 206) into a digital electric signal. In the first embodiment, the light source unit 202 as a light source device is mounted on the sensor substrate 207. The light source unit 202 is disposed on the inner wall surface (one surface) side of the windshield 105 as a light-transmitting member, and the adhered matter (hereinafter, the adhered matter is mainly raindrops) attached to the outer wall surface (the other surface). A case will be described as an example.).

  In the first embodiment, the imaging unit 101 is arranged so that the optical axis of the imaging lens 204 matches the horizontal direction. However, the present invention is not limited to this, and the horizontal direction (X direction in FIG. 2) is used as a reference. It may be an example directed to a specific direction. For example, the imaging lens 204 includes a plurality of lenses and has a focal point set farther than the position of the windshield 105. The focal position of the imaging lens 204 can be set, for example, between infinity or infinity and the windshield 105.

  The image sensor 206 includes a plurality of two-dimensionally arranged light receiving elements that receive light transmitted through a cover glass that protects the sensor surface, and has a function of photoelectrically converting incident light for each light receiving element (imaging pixel). In the drawings and the like to be described later, each pixel of the image sensor 206 is illustrated in a simplified manner. However, the image sensor 206 is actually composed of about several hundred thousand pixels arranged two-dimensionally. As the image sensor 206, for example, a CCD (Charge Coupled Device) that reads all the imaging pixels simultaneously by exposing all the imaging pixels simultaneously (global shutter), or a signal of each imaging pixel exposed by line exposure (rolling shutter). Is an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) or the like, and a photodiode can be used as the light receiving element thereof.

  The signal processing unit 208 generates captured image data obtained by converting an analog electrical signal (the amount of light incident on each light receiving element of the image sensor 206) that is photoelectrically converted by the image sensor 206 and output from the sensor substrate 207 into a digital electrical signal. Output function. The signal processing unit 208 is electrically connected to the image analysis unit 102. When an electric signal (analog signal) is input from the image sensor 206 via the sensor substrate 207, the signal processing unit 208 determines the brightness (luminance) of each imaging pixel on the image sensor 206 from the input electric signal. A digital signal (captured image data) is generated. Then, the captured image data is output to the subsequent image analysis unit 102 together with the horizontal / vertical synchronization signal of the image.

  The image sensor 206 of the present embodiment includes a plurality of optical blacks (light receiving elements configured so that no light is incident) for performing dark correction, which is a type of level correction function, outside the effective imaging pixel area. Yes. The optical black output signal is sent to the signal processing unit 208 and used to measure the amount of dark current noise. Specifically, the signal processing unit 208 functions as a zero reference setting unit. For example, a signal level obtained by adding a predetermined margin to the average level of the output signal from the optical black is used as a zero reference (pixel value zero) of the captured image data. Set to. As a result, the signal level of the analog electric signal output from the sensor board 207 is lowered as a whole, and those below the signal level corresponding to the zero reference are set to a zero value. An analog electric signal exceeding a signal level corresponding to the zero reference is converted into a digital electric signal having a digital value obtained by dividing the zero reference signal level and the maximum signal level by a predetermined number of gradations. Accordingly, it is possible to avoid a situation in which a light receiving element that does not receive light at all is erroneously detected as receiving light due to a dark current flowing through the light receiving element of the image sensor 206.

  Further, the image analysis unit 102 has a function of controlling the imaging operation of the imaging unit 101 and a function of analyzing captured image data transmitted from the imaging unit 101. The image analysis unit 102 detects an object to be imaged by the image sensor 206 (an object such as another vehicle existing in the front area of the host vehicle) from the captured image data transmitted from the imaging unit 101 by the exposure amount control unit 102C as an exposure amount control unit. ) And an automatic exposure correction function for setting an optimal exposure amount (exposure time in the first embodiment) for each imaging target of the image sensor 206. Further, the image analysis unit 102 has a function of controlling the light emission timing of the light source unit 202 by the light source control unit 102B as a light source control unit in conjunction with the acquisition of the image signal from the signal processing unit 208. In addition, the image analysis unit 102 has a function of detecting information related to a road surface condition, a road sign, and the like from captured image data transmitted from the imaging unit 101. Further, the image analysis unit 102 has a function of calculating the position, direction, distance, and the like of another vehicle existing ahead of the host vehicle from the captured image data transmitted from the imaging unit 101.

  Further, the image analysis unit 102 has a function of detecting the state of the windshield 105 (attachment of raindrops, freezing, cloudiness, etc.) by the detection processing unit 102A. When detecting the attachment of raindrops, the detection processing unit 102A uses the captured image data transmitted from the imaging unit 101 and calculates the amount of raindrops on the windshield 105 according to the luminance value (pixel value) of the captured image data. To do. Further, as will be described in detail later, the image analysis unit 102 includes a storage unit 102D for temporarily storing captured image data transmitted from the imaging unit 101 or results of various processes using the captured image data. I have.

FIG. 3 is an explanatory diagram for explaining an optical system of the imaging unit 101 according to the first embodiment.
The light source unit 202 according to the first embodiment irradiates light source light (illumination light) for detecting a deposit (raindrop, freezing, cloudiness, etc.) attached on the windshield 105. The light source unit 202 of the first embodiment includes a plurality of LEDs as the light source, and emits light source light emitted from each LED to the outside through an optical system such as a collimator lens. By providing a plurality of light sources in this manner, the detection area for detecting the deposit on the windshield 105 is broadened compared to the case where there is a single light source, and the detection accuracy of the deposit on the windshield 105 is improved. improves.

  In the first embodiment, since the LEDs are mounted on the same sensor substrate 207 as the image sensor 206, the number of substrates can be reduced as compared with the case where they are mounted on separate substrates, and low cost can be realized. Further, the LED is arranged in one or more rows along the Y direction in FIG. 3 so that, as will be described later, the windshield is projected below the image area where the image of the vehicle front area is projected. It is possible to make the illumination for capturing an image uniform. In the first embodiment, the light source unit 202 is mounted on the same sensor substrate 207 as the image sensor 206, but may be mounted on a different substrate from the image sensor 206.

  The light source unit 202 is disposed on the sensor substrate 207 so that the optical axis direction of the light emitted from the light source unit 202 and the optical axis direction of the imaging lens 204 have a predetermined angle in advance. Further, the light source unit 202 is arranged on the windshield 105 illuminated by the light emitted from the light source unit 202 so that the illumination range is within the field angle range (viewing angle range) of the imaging lens 204. ing. As the emission wavelength of the light source unit 202, it is preferable to avoid visible light so as not to dazzle the driver or pedestrian of the oncoming vehicle. For example, the wavelength is longer than the visible light, and the light receiving sensitivity of the image sensor 206 extends. A range (for example, an infrared light wavelength range of about 800 to 1000 nm) is used. Drive control such as the light emission timing of the light source unit 202 is performed by the light source control unit 102B of the image analysis unit 102 in conjunction with the acquisition of the image signal from the signal processing unit 208.

  As shown in FIG. 3, the imaging unit 101 according to the first embodiment is provided with a reflection deflection prism 230 including a reflection surface 231 that reflects light from the light source unit 202 and guides the light to the windshield 105. The reflection deflection prism 230 is disposed so that one surface thereof is in close contact with the inner wall surface of the windshield 105 in order to appropriately guide the light from the light source unit 202 into the windshield 105. Specifically, even if the incident angle of light from the light source unit 202 incident on the reflection deflection prism 230 changes within a predetermined range, the light is irradiated from the light source unit 202 and regularly reflected by the reflection surface 231 of the reflection deflection prism 230. Of the light, regular reflection light regularly reflected at a non-attached portion where no raindrop (detection target) on the outer wall surface of the windshield 105 is attached is received by the image sensor 206 so as to maintain the relationship. A reflection deflection prism 230 is attached to the inner wall surface of the windshield 105.

  When the reflection deflecting prism 230 is attached to the inner wall surface of the windshield 105, it is preferable to improve the adhesion by interposing a filler such as a gel made of a translucent material or a sealing material therebetween. Thereby, it is possible to prevent an air layer, bubbles, or the like from intervening between the reflection deflection prism 230 and the windshield 105, and to prevent fogging between them. The refractive index of the filler is preferably an intermediate refractive index between the reflective deflection prism 230 and the windshield 105. This is because the Fresnel reflection loss between the filler and the reflection deflecting prism 230 and between the filler and the windshield 105 can be reduced. Fresnel reflection here refers to reflection that occurs between materials having different refractive indexes.

  As shown in FIG. 3, the reflection deflection prism 230 specularly reflects incident light from the light source unit 202 once on the reflection surface 231, and folds it back toward the inner wall surface of the windshield 105. The folded light is configured to have an incident angle θ (θ ≧ about 42 °) with respect to the outer wall surface of the windshield 105. The appropriate incident angle θ is a critical angle that causes total reflection on the outer wall surface inside the windshield 105 due to a difference in refractive index between air and the outer wall surface of the windshield 105. Therefore, in the first embodiment, when no deposits such as raindrops adhere to the outer wall surface of the windshield 105, the light reflected by the reflection surface of the reflection deflection prism 230 passes through the outer wall surface of the windshield 105. All are reflected without transmitting.

  On the other hand, if deposits such as raindrops (refractive index 1.38) different from air (refractive index 1) adhere on the outer wall surface of the windshield 105, this total reflection condition breaks down, and in places where raindrops adhere Light passes through the outer wall surface of the windshield 105. Therefore, for the non-attached portion where no raindrop is attached to the outer wall surface of the windshield 105, the reflected light is received by the image sensor 206 to become a high-luminance image portion, while the attached portion where the raindrop is attached. Since the amount of reflected light is reduced and the amount of light received by the image sensor 206 is reduced, an image portion with low luminance is obtained. Therefore, on the captured image, the contrast between the raindrop-attached portion and the non-raindrop-attached portion can be obtained.

FIG. 4 is a perspective view schematically showing a schematic configuration of the imaging unit 101 of the first embodiment.
The imaging unit 101 of the first embodiment is mounted with a first module 101A as a first support member that fixedly supports the reflection deflection prism 230 and is fixedly disposed on the inner wall surface of the windshield 105, an image sensor 206, and an LED 211. The sensor module 207, the light guide 215, and the second module 101B as a second support member that fixes and supports the imaging lens 204.

  These modules 101 </ b> A and 101 </ b> B are rotatably connected by a rotation connecting mechanism 240. The rotation coupling mechanism 240 has a rotation shaft 241 extending in a direction orthogonal to both the inclination direction and the vertical direction of the windshield 105 (the front-rear direction in FIG. 3). The one module 101A and the second module 101B can be rotated relative to each other. Such a rotatable configuration is aimed at the imaging direction of the imaging unit 200 of the second module 101B even when the first module 101A is fixedly arranged with respect to the windshield 105 having different inclination angles. This is because it can be directed in a specific direction (horizontal direction in the first embodiment).

  FIG. 5A is a side view showing the imaging unit 101 in a state of being attached to a vehicle in which the windshield 105 has an inclination angle θg of 22 ° with respect to a horizontal plane. FIG. 5B is an explanatory diagram showing the optical system of the imaging unit 101 when no raindrops are attached in the state of FIG. FIG. 5C is an explanatory diagram showing the optical system of the imaging unit 101 when raindrops are attached in the state of FIG.

  The light L1 from the light source unit 202 is specularly reflected by the reflection surface 231 of the reflection deflection prism 230, and the reflected light L2 is transmitted through the inner wall surface of the windshield 105. When raindrops are not attached to the outer wall surface of the windshield 105, the reflected light L2 is totally reflected on the outer wall surface, and this reflected light L3 is transmitted through the inner wall surface of the windshield 105 toward the imaging lens 204. Goes on. On the other hand, when raindrops 203 are attached to the outer wall surface of the windshield 105, the reflected light L2 specularly reflected by the reflecting surface 231 of the reflecting deflection prism 230 is transmitted through the outer wall surface. In such a system, when the angle θg of the windshield 105 changes, the second module 101B is fixed to the inner wall surface of the windshield 105 while maintaining the attitude of the second module 101B (while the imaging direction is fixed in the horizontal direction). The posture of the module 101B is changed, and the reflection deflection prism 230 is rotated around the Y axis in the drawing together with the windshield 105.

  Here, the arrangement relationship between the reflecting surface 231 of the reflecting deflection prism 230 and the outer wall surface of the windshield 105 in the first embodiment is always within the rotation adjustment range of the rotary coupling mechanism 240 at the outer wall surface of the windshield 105. The total reflected light L3 is received by a light receiving region of the image sensor 206 for detecting a windshield state change (hereinafter referred to as “attached matter detecting light receiving region”). Therefore, even if the angle θg of the windshield 105 changes, the total reflected light L3 on the outer wall surface of the windshield 105 is received by the adhering matter detection light receiving area of the image sensor 206, and appropriate raindrop detection can be realized.

  In particular, the arrangement relationship in the first embodiment is such that the corner cube principle is substantially established within the rotation adjustment range of the rotary coupling mechanism 240. Therefore, even if the angle θg of the windshield 105 changes, the angle θ formed between the optical axis direction of the total reflected light L3 on the outer wall surface of the windshield 105 and the horizontal plane is substantially constant. Therefore, it is possible to suppress the fluctuation of the portion through which the optical axis of the total reflected light L3 passes on the outer wall surface of the windshield 105 in the attached light detection region of the image sensor 206, thereby realizing more appropriate raindrop detection. .

  The corner cube principle is established if the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is perpendicular to each other. If the cube principle is established, the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not limited to a perpendicular relationship. Even if the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not perpendicular to each other, the angle of the other surface (incident surface or emission surface) of the reflection deflection prism 230 is adjusted. Thus, even if the inclination angle θg of the windshield 105 changes, the angle θ of the optical axis of the total reflected light L3 toward the imaging lens can be kept substantially constant.

FIG. 6 is a perspective view showing the reflective deflection prism 230 of the first embodiment.
The reflection deflection prism 230 includes an incident surface 233 on which light from the light source unit 202 is incident, a reflecting surface 231 that reflects the light L1 incident from the incident surface 233, and an adhesion surface 232 that is in close contact with the inner wall surface of the windshield 105. And an emission surface 234 that emits the reflected light L3 reflected by the outer wall surface of the windshield 105 toward the imaging unit 200. In the first embodiment, the incident surface 233 and the emission surface 234 are configured to be parallel to each other, but they may be non-parallel surfaces.

  The material of the reflection deflection prism 230 may be any material that transmits at least light from the light source unit 202, and glass, plastic, or the like can be used. Since the light from the light source unit 202 of the first embodiment is infrared light, the reflective deflection prism 230 may be made of a black material that absorbs visible light. By using a material that absorbs visible light, light (such as visible light from outside the vehicle) other than the light from the LED (infrared light) can be prevented from entering the reflective deflection prism 230.

  Further, the reflection deflection prism 230 is formed so that the total reflection condition for totally reflecting the light from the light source unit 202 by the reflection surface 231 is satisfied within the rotation adjustment range of the rotary coupling mechanism 240. In addition, when it is difficult to satisfy the condition of total reflection on the reflection surface 231 within the rotation adjustment range of the rotation coupling mechanism 240, a film such as aluminum is vapor-deposited on the reflection surface 231 of the reflection deflection prism 230. A reflection mirror may be formed.

  In the first embodiment, the reflecting surface 231 is a flat surface, but the reflecting surface may be a concave surface. By using such a concave reflecting surface, it is possible to parallelize the diffused light beam incident on the reflecting surface. This makes it possible to suppress a decrease in illuminance on the windshield 105.

  Here, when imaging the infrared wavelength light from the light source unit 202 reflected by the outer wall surface of the windshield 105 with the imaging unit 200, the image sensor 206 of the imaging unit 200 includes the infrared wavelength light from the light source unit 202. A large amount of disturbance light including infrared wavelength light such as sunlight is also received. Therefore, in order to distinguish the infrared wavelength light from the light source unit 202 from such a large amount of disturbance light, the light emission amount of the light source unit 202 needs to be sufficiently larger than the disturbance light. It is often difficult to use the light source unit 202 with a large light emission amount.

  Therefore, in the first embodiment, for example, a cut filter that cuts light having a wavelength shorter than the light emission wavelength of the light source unit 202 as shown in FIG. 7, or a transmittance peak as shown in FIG. Is configured such that light from the light source unit 202 is received by the image sensor 206 through a bandpass filter that substantially matches the emission wavelength of the light source unit 202. Accordingly, light other than the light emission wavelength of the light source unit 202 can be removed and received, so that the amount of light from the light source unit 202 received by the image sensor 206 is relatively large with respect to disturbance light. As a result, light from the light source unit 202 can be distinguished from disturbance light even if the light source unit 202 does not have a large light emission amount.

  However, in the first embodiment, not only the raindrop 203 on the windshield 105 is detected from the captured image data, but also the preceding vehicle and the oncoming vehicle and the white line are detected. Therefore, if the wavelength band other than the infrared wavelength light emitted by the light source unit 202 is removed from the entire captured image, the image sensor 206 transmits light in the wavelength band necessary for detection of the preceding vehicle and the oncoming vehicle and detection of the white line. The light cannot be received, and this hinders detection. Therefore, in the first embodiment, the image area of the captured image data is used for detecting the raindrop detection image area for detecting the raindrop 203 on the windshield 105, detecting the preceding vehicle and the oncoming vehicle, and detecting the white line. A filter that removes a wavelength band other than the infrared wavelength light emitted by the light source unit 202 only in a portion corresponding to the raindrop detection image region is arranged in the optical filter 205.

FIG. 9 is a front view of the front-stage filter 210 provided in the optical filter 205.
FIG. 10 is an explanatory diagram illustrating an example of captured image data.
The optical filter 205 of the first embodiment has a structure in which a front-stage filter 210 and a rear-stage filter 220 are overlapped in the light transmission direction. As shown in FIG. 9, the front-stage filter 210 includes an infrared light cut filter region 211 and a raindrop detection image region 214 that are arranged at locations corresponding to the upper two-third captured image that is the vehicle detection image region 213. The region is divided into an infrared light transmission filter region 212 arranged at a position corresponding to a lower one third of a captured image. For the infrared light transmission filter region 212, the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. 8 is used.

  The headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line image are often present mainly from the center to the upper part of the captured image, and there is usually an image of the nearest road surface in front of the host vehicle at the lower part of the captured image. It is. Therefore, the information necessary for identifying the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line is concentrated on the upper portion of the captured image, and the information on the lower portion of the captured image is not so important in the identification. Therefore, when both oncoming vehicle and preceding vehicle or white line detection and raindrop detection are performed simultaneously from a single captured image data, the lower portion of the captured image is displayed in the raindrop detection image area 214 as shown in FIG. It is preferable that the upper portion of the remaining captured image is the vehicle detection image region 213 and the pre-filter 210 is divided into regions corresponding to this.

  In the first embodiment, the raindrop detection image area 214 is provided below the vehicle detection image area 213 in the captured image. However, the raindrop detection image area 214 is provided above the vehicle detection image area 213. Or the raindrop detection image area 214 may be provided both above and below the vehicle detection image area 213.

  When the imaging direction of the imaging unit 200 is tilted downward, the hood of the host vehicle may enter the lower part of the imaging area. In this case, sunlight reflected by the bonnet of the host vehicle or the tail lamp of the preceding vehicle becomes disturbance light, which is included in the captured image data, which may cause misidentification of the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line. Become. Even in such a case, in the first embodiment, the cut filter shown in FIG. 7 and the band-pass filter shown in FIG. 8 are arranged at the position corresponding to the lower part of the captured image. The disturbance light such as the tail lamp of the preceding vehicle is removed. Therefore, the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line identification accuracy are improved.

  In the first embodiment, due to the characteristics of the imaging lens 204, the scene in the imaging area and the image on the image sensor 206 are reversed. Therefore, when the lower portion of the captured image is used as the raindrop detection image region 214, the upper portion of the upstream filter 210 of the optical filter 205 is configured by the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. .

  Here, when the preceding vehicle is detected, the preceding vehicle is detected by identifying the tail lamp on the captured image. However, the tail lamp has a smaller amount of light compared to the headlamp of the oncoming vehicle, and disturbance such as street lights. Since there is a lot of light, it is difficult to detect the tail lamp with high accuracy only from mere luminance data. Therefore, it is necessary to identify the tail lamp based on the amount of received red light using spectral information for identifying the tail lamp. Therefore, in the first embodiment, as described later, a red filter or a cyan filter (a filter that transmits only the wavelength band of the tail lamp color) that matches the color of the tail lamp is disposed in the subsequent filter 220 of the optical filter 205, The amount of red light received can be detected.

  However, since each light receiving element constituting the image sensor 206 of the first embodiment has sensitivity to light in the infrared wavelength band, it is obtained when the image sensor 206 receives light including the infrared wavelength band. The captured image is reddish as a whole. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp. Therefore, in the first embodiment, a portion corresponding to the vehicle detection image region 213 in the upstream filter 210 of the optical filter 205 is used as the infrared light cut filter region 211. Thereby, since the infrared wavelength band is excluded from the captured image data portion used for identifying the tail lamp, the identification accuracy of the tail lamp is improved.

FIG. 11 is a schematic enlarged view of a portion of the optical filter 205 and the image sensor 206 corresponding to the vehicle detection image region 213 viewed from a direction orthogonal to the light transmission direction.
As described above, the image sensor 206 is an image sensor using a CCD, a CMOS, or the like, and a photodiode 206A is used as a light receiving element thereof. The photodiodes 206A are two-dimensionally arranged for each pixel, and a microlens 206B is provided on the incident side of each photodiode 206A in order to increase the light collection efficiency of the photodiode 206A. The image sensor 206 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 207.

  An optical filter 205 is disposed close to the surface of the image sensor 206 on the micro lens 206B side. As shown in FIG. 11, the post-filter 220 of the optical filter 205 in the portion corresponding to the vehicle detection image region 213 is formed by sequentially forming a polarizing filter layer 222 and a spectral filter layer 223 on a transparent filter substrate 221. It is what. Each of the polarizing filter layer 222 and the spectral filter layer 223 is divided into regions corresponding to one photodiode 206A on the image sensor 206.

  A configuration may be adopted in which there is a gap between the optical filter 205 and the image sensor 206, but the configuration in which the optical filter 205 is in close contact with the image sensor 206 is different from the polarization filter layer 222 and the spectral filter layer 223 of the optical filter 205. It becomes easy to make the boundary of each area | region and the boundary between photodiode 206A on the image sensor 206 correspond. The optical filter 205 and the image sensor 206 may be bonded with, for example, a UV adhesive, or UV bonding or thermocompression bonding is performed on the four side areas outside the effective pixel while being supported by the spacer outside the effective pixel range used for imaging. Also good.

FIG. 12 is an explanatory diagram showing a region division pattern of the polarizing filter layer 222 and the spectral filter layer 223 of the optical filter 205 in a portion corresponding to the vehicle detection image region 213.
In the polarizing filter layer 222 and the spectral filter layer 223, two types of regions, a first region and a second region, are arranged corresponding to one photodiode 206A on the image sensor 206, respectively. As a result, the amount of light received by each photodiode 206A on the image sensor 206 is determined as polarization information, spectral information, or the like depending on the types of regions of the polarizing filter layer 222 and the spectral filter layer 223 through which the received light is transmitted. Can be acquired. As a result, according to the first embodiment, in the vehicle detection image region 213, the red light vertical polarization component image, the non-spectral vertical polarization component image, and the non-spectral horizontal polarization component image 3 are obtained by one imaging operation. Various types of captured image data can be obtained.

  The vertically polarized component image of red light obtained in this way can be used, for example, for identifying a tail lamp. Since the vertical polarization component image of the red light has the horizontal polarization component S cut off, the red light is horizontal, such as red light reflected on the road surface or red light (reflection light) from the dashboard of the vehicle 100 or the like. It is possible to obtain a red image in which a disturbance factor due to red light having a strong polarization component S is suppressed. Therefore, the recognition rate of the tail lamp is improved by using the vertical polarization component image of the red light for the identification of the tail lamp.

  The non-spectral vertical polarization component image can be used, for example, for identifying a white line or a headlamp of an oncoming vehicle. In the non-spectral horizontal polarization component image, since the horizontal polarization component S is cut off, white light such as a headlamp or a streetlight reflected on the road surface, white light from a dashboard in the interior of the vehicle 100, etc. (reflection light) A non-spectral image in which disturbance factors due to white light with a strong horizontal polarization component S are suppressed can be obtained. Therefore, the recognition rate is improved by using the non-spectral vertically polarized component image for identifying the white line and the headlamp of the oncoming vehicle. In particular, it is generally known that the reflected light from the water surface covering the road surface has a large amount of horizontal polarization component S in a rainy road. Therefore, by using the non-spectral vertical polarization component image for white line identification, it becomes possible to appropriately identify the white line under the water surface in the rainy road, and the recognition rate is improved.

  In addition, if a comparison image using pixel values as pixel values for comparing each pixel value between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image is used, the wetness and moisture of a metal object and road surface in the imaging region are used. The state, the three-dimensional object in the imaging region, and the white line on the rainy road can be identified with high accuracy. As a comparison image used here, for example, a difference image in which a pixel value is a difference value between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image, or a pixel value between these images. Use a ratio image with the ratio as the pixel value, or a differential polarization degree image with the ratio of the difference value of the pixel values between these images to the sum of the pixel values between these images (difference polarization degree) as the pixel value can do.

  On the other hand, for the raindrop detection image region 214, the visible light region is cut by the infrared light transmission filter region 212 in the upstream filter 210 of the optical filter 205, and the wavelength includes the infrared light region that is the emission wavelength of the light source unit 202. An image of the band is taken.

[Raindrop detection processing on the windshield]
Hereinafter, the raindrop detection process according to the first embodiment will be described.
In the first embodiment, for the purpose of performing drive control of the wiper 107 and discharge control of the washer liquid, a process of detecting raindrops that are attached as a detection target is performed. In addition, here, the case where the deposit adhered on the windshield is a raindrop will be described as an example. However, the same applies to deposits such as bird droppings and splashes on the road surface from the adjacent vehicle. is there.

  In the first embodiment, as described above, the illumination light (infrared light) that is irradiated from the light source unit 202 and is incident on the inner wall surface of the windshield 105 from the reflection deflection prism 230 has raindrops on the outer wall surface of the windshield 105. In the non-attached part which has not adhered, regular reflection is performed on the outer wall surface of the windshield 105. The specularly reflected light is received by the image sensor 206 and displayed on the raindrop detection image area 214. On the other hand, at a place where raindrops are attached on the outer wall surface of the windshield 105, the illumination light is transmitted through the outer wall surface of the windshield 105, and the transmitted light is not received by the image sensor 206. Therefore, in the raindrop detection image area 214 of the captured image data, the non-attached portion where the raindrop does not adhere to the outer wall surface of the windshield 105 becomes a high brightness image portion (high pixel value), while the raindrop adheres. The attached part becomes a low-brightness image part (low pixel value). From these differences, it is possible to grasp not only the presence of raindrops but also the amount of raindrops.

FIG. 13 is an explanatory diagram of raindrop detection processing according to the first embodiment.
In the raindrop detection process of the first embodiment, the wiper 107 is driven when it is determined that the number of raindrops has increased using information on the raindrop detection image area 214 of the captured image data acquired from the imaging unit 101. Specifically, the raindrop detection image area 214 is divided into eight sections (x = 1 to 8) in the horizontal direction of the image as shown in FIG. 13, for example, and the total luminance value y (x, ta) in each raindrop detection section x. Is calculated. Note that “ta” is the imaging timing of the lighting imaging frame. Then, the raindrop detection condition is judged based on the total luminance value y (x, ta) of each raindrop detection section x, and if the raindrop detection condition is satisfied, the raindrop is attached from the situation where no raindrop is attached (the raindrop increases). And the wiper 107 is driven. Conversely, the raindrop non-detection condition is determined based on the total luminance value y (x, ta) of each raindrop detection section x, and when the raindrop non-detection condition is satisfied, the driving of the wiper 107 is stopped. The conditions for starting and stopping the driving of the wiper 107 are not limited to this, and can be set as appropriate. For example, the threshold value does not have to be a fixed value, and may be changed as appropriate according to a change in the situation around the host vehicle on which the imaging unit 200 is mounted. Moreover, the threshold value of the start condition and the stop condition may be the same value or different values.

FIG. 14 is an explanatory diagram showing the relationship between the imaging frame and raindrop detection in the first embodiment.
In general, an imaging frame (sensing frame) for detecting a white line (partition line) on the road surface in the imaging area and an identification target such as another vehicle, and an imaging frame (raindrop detection) for detecting raindrops Frame) is often a different imaging frame. However, in this case, since there is a time gap between the sensing frames before and after the raindrop detection frame, the object to be identified is between the imaging time of the previous sensing frame and the imaging time of the subsequent sensing frame. There is a time during which an image for identifying an object cannot be acquired, causing problems such as a reduction in recognition accuracy of the identification object. Therefore, in the first embodiment, as shown in FIG. 14, an image having an image (vehicle detection image region) for detecting the identification target and a raindrop detection image region for detecting the raindrop 203 is displayed. , And can be obtained with a single imaging frame frame2,4,6,8,10,12. In this case, a dedicated imaging frame (raindrop detection frame) for detecting raindrops is not necessary, and the above-described problems do not occur.

  Here, in the first embodiment, in order to obtain a suitable vehicle detection image area (for example, an image area with high contrast), an automatic exposure correction function that changes the exposure amount according to the situation (brightness, etc.) of the imaging area. The automatic exposure control (AEC) is implemented. Specifically, the exposure time of the next imaging frame is adjusted by the exposure amount control unit 102C of the image analysis unit 102 in accordance with the luminance value of the center of the captured image (pixels in the vehicle detection image area 213) in the previous imaging frame. Automatic exposure control (AEC) for changing (exposure amount) is performed.

  When such automatic exposure control is performed, even if the raindrop adhesion state on the windshield is exactly the same, there is a possibility that the brightness and contrast of the raindrop detection image area may be changed by changing the exposure time. In this case, as a result of obtaining the raindrop detection image region 214 having different brightness, contrast, etc., the raindrop amount cannot be detected appropriately, and problems such as malfunction of the wiper 107 may occur. Therefore, in the first embodiment, the exposure period is set to be changed by automatic exposure control outside the light irradiation period from the light source unit 202. In this case, even if the exposure period is changed, the light source light receiving time by the image sensor 206 is constant during the light irradiation period, so that it is possible to appropriately detect the amount of raindrops and suppress problems such as malfunction of the wiper 107. it can.

In the first embodiment, as described above, when raindrops adhere to the windshield, the amount of light received from the light source unit 202 by the image sensor 206 decreases, and the brightness value of the raindrop detection image area decreases. To detect raindrops on the windshield. Specifically, the difference between the total luminance value y (x, ta) in each raindrop detection section x and the immediately preceding total luminance value y (x, ta-1) (hereinafter referred to as “lighting difference”) e _y . Let (x, ta) be a raindrop fluctuation component, and it can be understood from this raindrop fluctuation component that raindrops have adhered.

In the first embodiment, the lighting difference e _y (x, ta) = y (x, ta) −y (x, ta−1) is the two consecutive lighting imaging frames frame2, 4, 6, 8, It shows the amount of change in the total luminance value (the amount of change during illumination) within the time required to image 10, 12 (ta = 1, 2, 3, 4, 5, 6). Factors that cause the lighting difference e _y (x, ta) to fluctuate include factors (raindrop fluctuation components) caused by the attachment of raindrops or the amount of attachment of raindrops due to the removal of raindrops by the wiper 107, as well as the image sensor 206. A factor (disturbance light fluctuation component) due to a change in the amount of incident disturbance light and a factor (non-disturbance light fluctuation component) other than the disturbance light fluctuation component can be included. If these disturbance light fluctuation components and non-disturbance light fluctuation components are included in the lighting difference e _y (x, ta), if this lighting difference is treated as a raindrop fluctuation component, a false detection of raindrops will occur. May cause problems such as malfunction of the wiper 107.

Here, as the non-disturbing light fluctuation component, the BLC fluctuation component due to the black level correction function of the image sensor 206, the optical characteristics of optical members such as lenses, mirrors, and prisms, and the light emission amount of the light source are mainly temperature. The temperature fluctuation component that fluctuates due to changes and changes the luminance value of the raindrop detection image area, the optical characteristics of these optical members and the light emission amount of the light source fluctuate due to deterioration over time, and the luminance value of the raindrop detection image area fluctuates And the like. However, the temperature fluctuation component and the temporal deterioration fluctuation component are very gentle fluctuations, and the period during which the fluctuation component causes a significant change in the lighting difference e _y (x, ta) This is longer than the change observation period in which the lighting difference e _y (x, ta) is observed (in the first embodiment, the time required to take two consecutive lighting imaging frames). Therefore, the temperature fluctuation component and the temporal deterioration fluctuation component can be excluded from the non-disturbing light fluctuation component that fluctuates the lighting difference e _y (x, ta).
Then, it is considered that the factors (variation components) that fluctuate the lighting difference e _y (x, ta) are only the disturbance light fluctuation component and the non-disturbance light fluctuation component BLC fluctuation component excluding the raindrop fluctuation component. Can do.

Therefore, in the first embodiment, the disturbance light fluctuation component and the BLC fluctuation are included in the lighting difference e _y (x, ta) indicating the change amount of the total luminance value in each raindrop detection section x (the change amount during illumination). It is determined whether or not a component is included, and the lighting difference e _y (x, ta) when no disturbance light fluctuation component or BLC fluctuation component is included is handled as a raindrop fluctuation component. As a result, raindrops can be appropriately detected without being affected by disturbance light fluctuation components and non-disturbance light fluctuation components, and problems such as malfunction of the wiper 107 can be avoided.

  Here, the disturbance light fluctuation component can be obtained, for example, as follows. That is, in the first embodiment, as illustrated in FIG. 14, the light source unit 202 is turned on and off alternately for each imaging frame. Specifically, in the odd-numbered imaging frames frames 1, 3, 5, 7, 9, and 11, the captured image data (image data when the light is turned off) captured with the light source unit 202 turned off is captured, and the even-numbered imaging frames are captured. In frames 2, 4, 6, 8, 10, and 12, captured image data (lighted image data) captured in a state where the light source unit 202 is lit is captured.

At this time, the image data at the time of extinction of the image area for raindrop detection obtained by the imaging frames frame 1, 3, 5, 7, 9, 11 (tb = 1, 2, 3, 4, 5, 6) at the time of extinction is the light source This is image data obtained by capturing only disturbance light other than light source light from the unit 202. Note that “tb” is the imaging timing of the imaging frame when the light is off. Therefore, the sum of the luminance values d (x, tb) in each raindrop detection section x in the immediately preceding imaging frame at the time of turn-off is obtained in each raindrop detection section x of the image area for raindrop detection obtained from the imaging frame at the time of lighting. It can be grasped as a disturbance light component included in the total luminance value y (x, ta). Therefore, in the first embodiment, the difference between the luminance value sum d (x, tb) in each raindrop detection section x and the immediately preceding luminance value sum d (x, tb−1) in the imaging frame when the light is extinguished (hereinafter, “ off time difference _{"hereinafter.) e d (x, tb} ) is referred to as disturbance light fluctuation component.

FIG. 15 is a flowchart showing a flow of raindrop detection processing in the first embodiment.
The light source control unit 102B of the image analysis unit 102 controls the light emission timing of the light source unit 202 in conjunction with the acquisition of the image signal from the signal processing unit 208. First, the imaging frame frames1,3 (tb = 1, 1) 2) and the imaging frame frames 2 and 4 (ta = 1, 2) at the time of lighting are imaged (S1). Next, the detection processing unit 102A of the image analysis unit 102 calculates the total luminance value y (x, ta = 1), y (x) in each raindrop detection section x of the raindrop detection image area in the lighting imaging frames frames 2 and 4. , Ta = 2) is calculated (S2). Then, the lighting difference e _y (x, ta = 2) = y (x, ta = 2) −y (x, ta = 1) is calculated for each raindrop detection section x (S3).

In calculating the lighting difference e _y (x, ta = 2), the detection processing unit 102A of the image analysis unit 102 first acquires the captured image data of the lighting imaging frame frame2 (ta = 1) from the imaging unit 101. . When the detection processing unit 102A calculates the total luminance value y (x, ta = 1) of the lighting imaging frame frame2 from the acquired captured image data, the detection processing unit 102A temporarily stores it in the storage unit 102D or delays its output. Or After that, when the detection processing unit 102A of the image analysis unit 102 acquires the captured image data of the next lighting imaging frame frame4 from the imaging unit 101 and calculates the total luminance value y (x, ta = 2), it is temporarily stored. Alternatively, the lighting difference e _y (x, ta = 2) is calculated using the previous delayed total luminance value y (x, ta = 1).

Next, the detection processing unit 102A determines whether or not the lighting difference e _y (x, ta = 2) of each raindrop detection section x calculated in this way is less than the first difference threshold Ey. The number of raindrop detection sections x whose lighting difference e _y (x, ta = 2) is less than the first difference threshold Ey is obtained (S4). The first difference threshold value Ey at this time is an appropriate value that can detect the amount of change in the lighting difference e _y (x, ta = 2) when the amount of raindrop attachment varies due to raindrop attachment or raindrop removal by the wiper 107. The value is set as appropriate. In the first embodiment, when the raindrops are attached, the total luminance value is decreased. Therefore, the lighting difference e _y (x, ta) = y (x, ta) −y (x, ta− when the raindrops are attached). Since 1) takes a negative value, the first difference threshold Ey is appropriately set as a negative value.

Here, the amount of change (the amount of change during illumination) of the total luminance value y (x, ta) in each raindrop detection section x is calculated from the total luminance value y (x, ta) of the current lighting imaging frame. Although the lighting difference e _y (x, ta) obtained by subtracting the total luminance value y (x, ta−1) of the lighting imaging frame immediately before that is used, it is not limited to this.

For example, the total luminance value y (x, ta-1) of the immediately previous lighting imaging frame may be subtracted from the total luminance value y (x, ta) of the current lighting imaging frame. In this case, in the case where the total luminance value becomes small when raindrops are attached as in the first embodiment, the lighting difference e _y (x, ta) = y (x, ta−1) −y when the raindrops are attached. Since (x, ta) takes a positive value, the first difference threshold Ey is appropriately set to a positive value. At this time, in processing step S4, the number of raindrop detection sections x in which the lighting difference e _y (x, ta) of each raindrop detection section x is equal to or greater than the first difference threshold value Ey is obtained.

Further, for example, it may be an absolute value of a difference between the total luminance value y (x, ta-1) of the immediately previous lighting imaging frame and the total luminance value y (x, ta) of the current lighting imaging frame. . In this case, in the case where the total luminance value becomes small when raindrops adhere as in the first embodiment, the lighting difference e _y (x, ta) = | y (x, ta) −y ( Since x, ta-1) | takes a positive value, the first difference threshold value Ey is appropriately set to a positive value. At this time, in processing step S4, the number of raindrop detection sections x in which the lighting difference e _y (x, ta) of each raindrop detection section x is equal to or greater than the first difference threshold value Ey is obtained.

  Note that the change observation period for observing the change amount (change amount during illumination) of the total luminance value y (x, ta) in each raindrop detection section x can also be set as appropriate, so that it is limited to the difference between two consecutive imaging frames. Instead, a difference between two or more distant imaging frames may be used.

  Further, as the amount of change (total amount of change during illumination) of the total luminance value y (x, ta) in each raindrop detection section x, for example, an average value or median value of a plurality of total luminance values that are close in time is used. Thus, a value smoothed in the time direction may be used. By smoothing in the time direction in this way, the pulse-like noise that occurs in a short time can be mitigated, and the risk of accidental changes in lighting due to temporal fluctuations in the signal is reduced. You can do it.

When the detection processing unit 102A of the image analysis unit 102 determines the number of raindrop detection sections x whose lighting difference e _y (x, ta = 2) is less than the first difference threshold value Ey, the determined number is the first determination. It is determined whether or not the threshold value F1 is exceeded (S5). The first determination threshold value F1 can also be set as appropriate. If the determined number of divisions is equal to or less than the first determination threshold value F1 (No in S5), the raindrop non-detection processing result is output from the image analysis unit 102 to the wiper control unit 106, assuming that no raindrop has been detected. (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

On the other hand, when the determined number of divisions exceeds the first determination threshold value F1 (Yes in S5), the lighting difference e _y (x, ta = 2) varies due to an increase in the amount of raindrops attached due to raindrops. It is highly possible. However, the lighting time difference _{e y (x, ta = 2} ) variation of its lit difference _{e y (x, ta = 2} ) 2 single lit imaging frame calculated change observed period (successive frame2,4 There is a possibility that it is caused by a change in the amount of disturbance light within a period of time required for imaging the image or a black level correction function (BLC). Therefore, in the first embodiment, the disturbance light fluctuation component and the BLC fluctuation component are included in the lighting difference e _y (x, ta = 2) determined to be less than the first difference threshold Ey. A process for determining whether or not the

Specifically, the detection processing unit 102A of the image analysis unit 102 firstly adds the luminance value sum d (x, tb = 1) in each raindrop detection section x of the raindrop detection image area in the unlit imaging frames frames1, 3. , D (x, tb = 2) is calculated (S6). Then, the extinction difference e _d (x, tb = 2) = d (x, tb = 2) −d (x, tb = 1) for each raindrop detection section x is calculated (S7). The calculation method of the turn-off difference e _d (x, tb = 2) is the same as the calculation method of the turn-on difference e _y (x, ta = 2).

Next, the detection processing unit 102A determines whether or not the difference e _d (x, tb = 2) at the time of extinction of each raindrop detection section x calculated in this way is less than the second difference threshold Ed. off time difference _{e d (x, tb = 2} ) Find the number of the rain detection segment x is less than the second difference threshold Ed (S8). Here, the disturbance light fluctuation component that can be mistaken as the raindrop fluctuation component is a case where the extinction difference e _d (x, tb) = d (x, tb) −d (x, tb−1) takes a negative value. The second difference threshold Ed is appropriately set as a negative value. In addition, about the determination whether it is less than the 2nd difference threshold Ed, the raindrop detection which judged that the lighting difference _ey (x, ta = 2) was less than the 1st difference threshold Ey in the process step S4 mentioned above. You may make it perform only to the division | segmentation x.

Further, here, the amount of change (the amount of change during non-illumination) of the luminance value sum d (x, tb) in each raindrop detection section x is calculated from the luminance value sum d (x, tb) of the imaging frame when the light is turned off this time. Although the light-off time difference e _d (x, tb) obtained by subtracting the luminance value sum d (x, tb−1) of the light-off imaging frame immediately before that is used, it is not limited to this.

For example, the sum of the luminance values d (x, tb) of the imaging frame when the light is turned off may be subtracted from the sum of luminance values d (x, tb-1) of the imaging frame when the light is turned off immediately before. In this case, when the amount of disturbance light that can be mistaken as a raindrop fluctuation component is reduced in the case where the total luminance value is reduced when raindrops are attached as in the first embodiment, the extinction difference e _d (x, tb) = d (x , Tb−1) −d (x, tb) has a positive value, and therefore the second difference threshold Ed is appropriately set to a positive value. At this time, in the process step S8, so that the light-off difference e _{d (x,} tb) of each raindrop detection segment x is determined the number of rain detection segment x is the second difference threshold Ed more.

Further, for example, it may be an absolute value of a difference between the sum of luminance values d (x, tb-1) of the immediately previous imaging frame when the light is turned off and the total luminance value d (x, tb) of the current imaging frame when the light is turned off. . In this case, when the amount of disturbance light that can be mistaken as a raindrop fluctuation component is reduced in the case where the total luminance value is reduced when raindrops are attached as in the first embodiment, the extinction difference e _d (x, tb) = | d ( x, tb) −d (x, tb−1) | takes a positive value, and therefore the second difference threshold Ed is appropriately set to a positive value. At this time, in the process step S8, so that the light-off difference e _{d (x,} tb) of each raindrop detection segment x is determined the number of rain detection segment x is the second difference threshold Ed more.

  Note that the period for calculating the amount of change (total amount of change during non-illumination) of the luminance value sum d (x, tb) within each raindrop detection section x can also be set as appropriate, and thus is not limited to the difference between two consecutive imaging frames. Differences between two or more distant imaging frames may be used.

Further, the period for calculating the turn-off difference e _d (x, tb) does not need to coincide with the change observation period for calculating the turn-on difference e _y (x, ta), but this change observation period. It is desirable to set the period close to. For example, the lighting time difference _{e y (x, ta = 2} ) as off difference component corresponding to, for _{example, e d (x, tb =} 3) -e d (x, tb = 2) _and, e d (x, tb = 3) -e _d (x, tb = 1) may be used.

Detection processing unit 102A of the image analysis unit 102, once determined the number of light-off difference _{e d (x, tb = 2} ) is the rain detection segment x is less than the second difference threshold value Ed, the number of sections calculated the second determination It is determined whether or not the threshold value F2 is exceeded, and the determination result (the determination result Htb at the time of extinction) is temporarily stored in the storage unit 102D of the image analysis unit 102 (S9). The second determination threshold value F2 can also be set as appropriate.

Thereafter, the detection processing unit 102A of the image analysis unit 102 reads the past extinction determination result Htb-2 from the storage unit 102D (S10), and obtains the extinction determination result Htb-2 in the past processing step S8. When the number of sections (that is, the difference e _d when extinguished (x, tb−2) <the number of sections satisfying the second difference threshold Ed) is less than or equal to the second determination threshold F2 (No in S11) ), The lighting difference e _y (x, ta = 2) that is less than the first difference threshold Ey described above is not caused by a change in the amount of disturbance light, and is not caused by the black level correction function. It is determined that this is because the amount of raindrops increased due to the attachment of raindrops. Therefore, the raindrop detection processing result is output from the image analysis unit 102 to the wiper control unit 106 as raindrops are detected (S13). As a result, the wiper control unit 106 performs control such as starting to drive the wiper 107.

On the other hand, when the past extinction determination result Htb-2 read from the storage unit 102D indicates that the number of classifications obtained in the past processing step S8 exceeds the second determination threshold F2 (in S11) Yes), the lighting difference e _y (x, ta = 2) that is less than the first difference threshold Ey described above may be due to a change in the amount of disturbance light or a black level correction function. high. Therefore, assuming that no raindrop has been detected, the processing result of raindrop non-detection is output from the image analysis unit 102 to the wiper control unit 106 (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

  Here, in the present embodiment, the determination result whether or not the number of divisions obtained in the current processing step S8 exceeds the second determination threshold value F2 is not used, and the past processing step S8 (in this case, the processing two times before) The determination result of whether or not the number of divisions obtained in step S8) exceeds the second determination threshold value F2 is used. This is to properly grasp the BLC fluctuation component due to the black level correction function. This will be described in detail below.

FIG. 16 is a graph showing an example in which the output signal (luminance value of the raindrop detection image area) output from the image sensor 206 at the time of lighting varies due to the black level correction function.
When strong disturbance light such as retrograde is input to the image sensor 206, the black level correction function is exhibited. As a result, the output signal output from the image sensor 206 gradually changes toward the target value after level correction by the black level correction function, even if the amount of light actually received by the image sensor 206 is constant. In the example shown in FIG. 16, the level correction is started at time t1, the output signal when the image sensor 206 is turned on gradually decreases from Von1, and reaches the target value Von2 at time t3 when the level correction is completed. In this example, since level correction is continued until time t4, the output signal during lighting is constant at Von2 from time t3 to time t4. Thereafter, when the level correction is completed at time t4, the output signal when the image sensor 206 is turned on gradually rises from Von2, and returns to the target value Von1 at time t6 when the completion of the level correction is completed.

In this example, since the total luminance value y (x, ta) at the time of lighting gradually decreases from the time t1 to the time t3, for example, the first difference even though no raindrops are attached. The result of raindrop detection that a raindrop detection condition that the number of sections of the lighting difference e _y (x, ta) that is less than the threshold value Ey exceeds the first determination threshold value F1 satisfies the raindrop detection condition and a raindrop is detected is erroneously sent to the wiper control unit. There is a risk of output. On the contrary, the raindrop is detected because the raindrop detection condition does not satisfy the raindrop detection condition that the number of sections of the lighting difference e _y (x, ta) that is less than the first difference threshold Ey exceeds the first determination threshold F1 even though the raindrop has adhered. There is a possibility that the result of raindrop non-detection not being output is erroneously output to the wiper control unit.

FIG. 17 is a graph illustrating an example in which the output signal (the luminance value of the raindrop detection image area) output from the image sensor 206 when the light is turned off varies due to the black level correction function.
The output signal of the image sensor 206 at the time of turning off has a slight output value Voff1 due to the incidence of disturbance light or the like. In the example shown in FIG. 17, level correction by the black level correction function is performed at the same timing as in the example of FIG. Therefore, when the level correction is started at time t1, the output signal of the image sensor 206 at the time of extinguishing gradually decreases from Voff1. However, since the output signal of the image sensor 206 when the light is turned off is originally low, it bottoms out at the time t2 before the time t3 when the level correction of the output signal of the image sensor 206 when the light is turned on is completed. It becomes zero and becomes constant at zero after that. On the other hand, the output signal of the image sensor 206 at the time of extinguishing does not rise immediately even when the level correction is completed at time t4, and starts to rise from zero at time t5 after time t4. Then, it returns to the target value Voff1 at time t6 when the end of level correction is completed.

If the fluctuation of the output signal when the light is turned off due to the level correction by the black level correction function shows the same fluctuation as the output signal when the light is turned on, that is, the output signal when the light is turned off continues to decrease until time t3 when the level correction is completed. In addition, if the rise starts at time t4 when the level correction ends, fluctuations due to the level correction also occur in the output signal at the time of lighting immediately before or immediately after the light extinction from the fluctuation of the output signal at the time of the light extinction. I can understand that. Therefore, from the determination result of whether or not the number of categories obtained in the current processing step S8 exceeds the second determination threshold F2, the determination result of the current processing step S5 (that is, the current lighting difference e _y (x, It is possible to appropriately grasp whether or not the determination result of whether or not the raindrop detection condition that the number of sections of ta) exceeds the first determination threshold F1 includes a level correction error.

However, as shown in FIG. 17, the fluctuation of the output signal when the light is turned off due to the level correction by the black level correction function does not show the same fluctuation as the output signal when the light is turned on. That is, the output signal at the time of extinction becomes zero at the time t2 before the time t3 when the level correction is completed, and thereafter takes a constant value as zero. Therefore, from time t2 to time t3, no raindrops are attached, but the total luminance value y (x, ta) at the time of lighting is gradually decreased by level correction, so that it is less than the first difference threshold Ey. While the raindrop detection condition that the number of sections of the lighting difference e _y (x, ta) that exceeds the first determination threshold F1 is satisfied (Yes in S5), the luminance value sum d (x, tb) when the lamp is turned off It remains constant at zero even during level correction. Therefore, the turn-off determination result Htb that the number of sections of the turn-off difference e _d (x, tb) that is less than the second difference threshold Ed is equal to or less than the second determination threshold F2 is output, and the determination result of the processing step S5 is As a result of being determined not to be affected by disturbance light or level correction, a raindrop detection result indicating that a raindrop has been detected is erroneously output to the wiper control unit.

On the other hand, in the present embodiment, whether or not the determination result of the processing step S5 is affected by ambient light or level correction by the black level correction function is determined based on the previous (two times before) processing steps S6 to S6. The turn-off determination result Htb-2 determined in S8 is read (S10), and the turn-off determination result Htb-2 is used (S11). As a result, the determination result of processing step S5 between time t2 and time t3 is the luminance value sum d (x, tb-2) at the time of extinction between time t1 and time t2, that is, by level correction. The luminance value sum d (x, tb-2) at the time of extinguishing that gradually decreases can be used. As a result, the brightness sum d (x, tb-2) at the time of off is the number of segments off when the difference _{e d (x, tb-2} ) is less than the second difference threshold Ed calculated from the second determination threshold value The turn-off determination result Htb-2 that exceeds F2 is output. As a result, it is determined that the determination result of processing step S5 is affected by disturbance light or level correction, and the result of raindrop non-detection that no raindrop is detected is correctly output to the wiper control unit.

  As described above, according to the first embodiment, the raindrop detection that the raindrop is detected even though the raindrop amount has not actually increased due to the influence of the level correction by the black level correction function (even though it is not raining). It is possible to avoid a situation in which the result of the above is erroneously output to the wiper control unit and the wiper operates erroneously.

  In the first embodiment, the influence of the level correction on the determination result (S5) based on the total luminance value at the time of lighting this time is determined using the past determination result Htb-2 at the time of turning off. The influence of the level correction on the determination result based on the total brightness value at the time of lighting may be determined using the current determination result at turning off Htb. In this case, it is possible to avoid a situation in which a raindrop non-detection result that no raindrop is detected is erroneously output even though the raindrop amount has not actually decreased.

  Specifically, as shown in FIG. 16, from time t4 to time t5, even when the amount of raindrops does not decrease, the total luminance value y (x, ta) at the time of lighting gradually increases by level correction. On the other hand, as shown in FIG. 17, the output signal at the time of extinction remains constant at zero until time t5 after time t4, and starts to rise from zero at time t5. Therefore, the influence of level correction on the total luminance value y (x, ta) during lighting from time t4 to time t5 is the sum of luminance values d (x, ta during lighting) from time t4 to time t5. It cannot be determined using tb). At this time, if the influence of the level correction on the determination result based on the total luminance value at the time of lighting in the past is determined using the determination result Htb at the time of extinction at this time, the lighting from time t4 to time t5 is performed. The influence of level correction on the total luminance value y (x, ta-2) at the time can be determined using the total luminance value d (x, tb) when the light is extinguished between time t5 and time t6. It becomes. Since the luminance value sum d (x, tb) at the time of extinction from time t5 to time t6 gradually increases due to the level correction, the total luminance value y (x at lighting) from time t4 to time t5. , Ta-2) can be appropriately determined.

  In the present embodiment, the determination result based on the total luminance value at the time of lighting is used in the current processing, but the determination result in the past processing may be used. In this case, both the lighting determination result Hta based on the total luminance value during lighting and the lighting determination result Htb based on the sum of luminance values when extinguished are temporarily stored in the storage unit 102D, and in subsequent processing May be read out and used for the turn-on determination result Hta-1 and the turn-off determination result Htb-3 based on the sum of luminance values at turn-off.

  In the present embodiment, in the period immediately after time t6, whether or not the determination result of processing step S5 is affected by ambient light or level correction is determined when the light is turned off between time t5 and time t6. The sum of the brightness values, that is, the sum of the brightness values at the time of extinguishing which is gradually decreased by the level correction is used. For this reason, the determination result of the processing step S5 is not affected by the level correction, but is determined to be affected by the ambient light or the level correction, and the raindrop amount actually increases. However, the result of no raindrop detection that no raindrop has been detected is output to the wiper control unit. However, even in this case, the result of the raindrop detection that the raindrop has been detected is output to the wiper control unit by making a judgment at the time of turn-off using the sum of luminance values at the time of turn-off after time t6 after a while. become. That is, the wiper operates only a little after the raindrop amount starts to increase, and there are few substantial adverse effects.

  Further, in the present embodiment, an example of suppressing erroneous detection or detection omission due to level correction by the black level correction function of the image sensor 206 has been described, but due to other causes such as an automatic exposure correction function of the image sensor 206, If the total luminance value at the time of lighting is constant but the total luminance value at the time of lighting fluctuates, and this may cause erroneous detection or detection omission, such erroneous detection or omission can be similarly suppressed. It is. Therefore, the same applies to level correction by brightness correction, contrast correction, and the like performed to effectively use the dynamic range of an image.

The above is the raindrop detection process performed after completion of imaging of the lighting imaging frame frame4 (ta = 2). When this raindrop detection processing is completed, the imaging of the lighting imaging frame frame6 (ta = 3) is performed next. The same raindrop detection process is repeatedly performed after completion. Note that the lighting difference at this time is e _y (x, ta = 3) = y (x, ta = 3) −y (x, ta = 2), and the lighting difference is e _d (x, tb = 3) = d (x, tb = 3) −d (x, tb = 2). In this manner, the raindrop detection process is repeatedly performed while the function of detecting the attachment of raindrops is enabled. However, the execution timing of the raindrop detection process can be arbitrarily set, and the raindrop detection process is not performed every time after completion of the imaging of the lighting imaging frame frame6 (ta = 3), for example, the lighting imaging frame frame6 ( The raindrop detection process may be performed every time the imaging of ta = 3) is performed twice or more.

Further, in the first embodiment, as a raindrop detection condition, the number of raindrop detection sections x in which the lighting difference e _y (x, ta) is less than the first difference threshold Ey exceeds the first determination threshold F1. Although conditions are included, other conditions may be adopted as raindrop detection conditions instead of or in addition to this condition. For example, a condition may be adopted in which an average value or a variance value of the lighting difference e _y (x, ta) of the all raindrop detection section x is derived and the derived value exceeds or falls below a predetermined threshold value. This condition is effective in that the resistance to locally generated noise can be increased.

Further, as the amount of change in the total luminance value y (x, ta) in each raindrop detection section x (the amount of change during illumination), the current total luminance value y (x, ta) in each raindrop detection section x and the immediately preceding A condition using a parameter different from the lighting difference e _y (x, ta) that is a difference from the total luminance value y (x, ta-1) may be adopted.
For example, for each pixel in each raindrop detection section x, a pixel-specific difference value between the current pixel value and the previous pixel value is derived, and the number of pixels for which the pixel-specific difference value exceeds or falls below a predetermined threshold is calculated as follows: It may be used as a parameter instead of the lighting difference e _y (x, ta). In this case, it is effective in that raindrops can be appropriately detected even when raindrops locally adhere within each raindrop detection section x.

  In addition, for example, a condition may be employed in which an average value or a variance value of pixel-specific difference values in each raindrop detection section x is derived, and the derived value exceeds or falls below a predetermined threshold. This condition is effective in that it is possible to enhance resistance to noise that occurs locally in each raindrop detection section x. However, the difference value for each pixel may obtain a result in which the sign is reversed locally. If the raindrop detection condition is simply determined from the average value of the difference value for each pixel, the raindrop fluctuation component is evaluated to be unduly small. There is a fear. Therefore, for example, a condition that the sum of absolute values of pixel-specific difference values in each raindrop detection section x exceeds a predetermined threshold value may be adopted.

  Since raindrops adhere with spatial variations, the pixel-specific difference values in each raindrop detection section x also have spatial variations. Therefore, the raindrop detection condition can also be determined from the variance value of the difference value for each pixel in each raindrop detection section x. In addition, when the light source light is refracted or reflected by raindrops as in the second embodiment to be described later, the light source light is locally focused on the image sensor 206, and the pixel-specific difference in each raindrop detection section x. The value may increase locally. This may be reflected in the raindrop detection conditions.

Further, the first embodiment is an example in which it is determined whether or not raindrops are detected, but the amount of raindrops attached per unit time is estimated based on the lighting difference e _y (x, ta), and estimated. The amount of raindrops may be output as a result of raindrop detection. In this case, for example, the wiper control unit 106 can execute control for selecting the operation speed (operation mode such as Low, High, Interval, etc.) of the wiper 107 according to the amount of raindrops acquired from the image analysis unit 102. Become.

[Modification 1]
Next, a modified example of the first embodiment (hereinafter referred to as “modified example 1”) will be described.
In the first embodiment described above, the lighting difference e _y (x, ta) that is the amount of change (the amount of change during illumination) of the total luminance value y (x, ta) of each raindrop detection category x during lighting is the raindrop detection condition. However, in the first modification, such a lighting difference e _y (x, ta) is not calculated, and each raindrop detection classification x at the time of lighting is not calculated. It is determined whether or not the total luminance value y (x, ta) satisfies the raindrop detection condition.

FIG. 18 is a flowchart showing a flow of raindrop detection processing in the first modification.
When the detection processing unit 102A of the image analysis unit 102 calculates the total luminance value y (x, ta = 1) in each raindrop detection section x of the raindrop detection image area (S1, S2), each calculated raindrop detection section It is determined whether or not the total luminance value y (x, ta = 1) of x is less than the third difference threshold Ey ′, and the total luminance value y (x, ta = 1) is less than the third difference threshold Ey ′. The number of raindrop detection sections x is calculated (S31). As described above, considering that the value of the total luminance value y (x, ta) decreases as the raindrop adhesion amount increases, the third difference threshold Ey ′ can detect that raindrops adhere to a predetermined amount or more. An appropriate value is set as appropriate.

  When the detection processing unit 102A of the image analysis unit 102 obtains the number of raindrop detection sections x whose total luminance value y (x, ta = 1) is less than the third difference threshold value Ey ′, the number of obtained sections is determined as the third determination. It is determined whether or not the threshold value F1 ′ is exceeded (S32). This third determination threshold value F1 'can also be set as appropriate. When the determined number of divisions is equal to or smaller than the third determination threshold value F1 ′ (No in S32), the raindrop non-detection processing result is output from the image analysis unit 102 to the wiper control unit 106, assuming that no raindrop has been detected. (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

  On the other hand, when the determined number of divisions exceeds the third determination threshold value F1 ′ (Yes in S32), the total luminance value y (x, ta = 1) becomes the third difference threshold value Ey because a predetermined amount or more of raindrops have adhered. It is likely that a value less than 'is taken. However, the value of the total luminance value y (x, ta = 1) may be due to ambient light or the black level correction function (BLC). Therefore, also in the first modification, the disturbance light component and the BLC component are included in the total luminance value y (x, ta = 1) determined to be less than the third difference threshold Ey ′, as in the first embodiment. A process for determining whether or not it is included is performed.

Specifically, the detection processing unit 102A of the image analysis unit 102 firstly adds the luminance value sums d (x, tb = 0) and d (x, tb = 1) in each raindrop detection section x of the raindrop detection image area. ) Is calculated (S6), and a light-off time difference e _d (x, tb = 1) = d (x, tb = 1) −d (x, tb = 0) for each raindrop detection section x is calculated (S7). ). The off time difference _{e d (x, tb = 1} ) of the raindrop detection segment x calculated is determined whether each less than a second difference threshold value Ed, off when the difference _e d (x, tb = 1 ) Is determined below the second difference threshold Ed to determine the number of raindrop detection sections x (S8).

Detection processing unit 102A of the image analysis unit 102, once determined the number of light-off difference _{e d (x, tb = 1} ) is the rain detection segment x is less than the second difference threshold value Ed, the number of sections calculated the second determination It is determined whether or not the threshold value F2 is exceeded, and the determination result (the determination result Htb at the time of extinction) is temporarily stored in the storage unit 102D of the image analysis unit 102 (S9). Thereafter, the detection processing unit 102A of the image analysis unit 102 reads the past extinction determination result Htb-2 from the storage unit 102D (S10), and obtains the extinction determination result Htb-2 in the past processing step S8. When the number of sections (that is, the difference e _d when extinguished (x, tb−2) <the number of sections satisfying the second difference threshold Ed) is less than or equal to the second determination threshold F2 (No in S11) ), The total luminance value y (x, ta = 1) which is less than the third difference threshold Ey ′ described above is not affected by ambient light or the black level correction function, and raindrops adhere to a predetermined amount or more. Judgment is due. Therefore, the raindrop detection processing result is output from the image analysis unit 102 to the wiper control unit 106 as raindrops are detected (S13). As a result, the wiper control unit 106 performs control such as starting to drive the wiper 107.

  On the other hand, when the past extinction determination result Htb-2 read from the storage unit 102D indicates that the number of classifications obtained in the past processing step S8 exceeds the second determination threshold F2 (in S11) Yes), the total luminance value y (x, ta = 1) that is less than the third difference threshold Ey ′ described above is highly likely to be caused by ambient light or the black level correction function. Therefore, assuming that no raindrop has been detected, the processing result of raindrop non-detection is output from the image analysis unit 102 to the wiper control unit 106 (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

  In the first modification, the determination result whether or not the number of divisions obtained in the current processing step S8 exceeds the second determination threshold value F2 is not used, and the past processing step S8 (in this case, the processing two times before) The reason for using the determination result as to whether or not the number of divisions obtained in step S8) exceeds the second determination threshold value F2 is the same as in the first embodiment.

According to the first modification, it is not necessary to calculate the amount of change (the amount of change during illumination) of the total luminance value y (x, ta) in each raindrop detection section x, so that the processing step can be simplified. There is. However, as described above, the components that change the total luminance value y (x, ta) of each raindrop detection section x at the time of lighting include the optical characteristics of optical members such as lenses, mirrors, and prisms, and the light emission amount of the light source. The temperature fluctuation component that fluctuates the brightness value of the raindrop detection image area due to temperature changes, the optical characteristics of these optical members, the light emission amount of the light source, etc. fluctuate due to deterioration over time, and the brightness value of the raindrop detection image area There is also a time-dependent deterioration fluctuation component that fluctuates. If the lighting difference e _y (x, ta) is calculated and the raindrop detection condition is determined using the lighting difference e _y (x, ta) as in Embodiment 1 described above, the influence of these temperature fluctuation components and temporal deterioration fluctuation components can be excluded. However, in the present modification 1, since it is affected by these temperature fluctuation components and temporal deterioration fluctuation components, in the system in which the influences of these temperature fluctuation components and temporal deterioration fluctuation components are large, the first embodiment described above. It is possible to detect raindrops with higher accuracy.

[Embodiment 2]
Next, another embodiment (hereinafter, this embodiment is referred to as “embodiment 2”) in which the object detection device according to the present invention is applied to an attached matter detection device used in an in-vehicle device control system as in the first embodiment described above. Will be explained.
In the first embodiment described above, the reflection deflecting prism 230 is used, and the illumination light is transmitted through the outer surface of the windshield at the location where raindrops are attached, and the illumination light is reflected from the outer surface of the windscreen at locations where raindrops are not attached. The configuration is such that light is received by 206. On the other hand, in the second embodiment, the illumination light is transmitted through the outer surface of the windshield at the location where the raindrop is not attached, and the illumination light transmitted through the outer surface of the windshield is reflected at the location where the raindrop is attached. It is the structure which receives light. Specifically, the configuration is as shown in FIG.

  In the example of FIG. 19, the illumination light emitted from the light source unit 202 is directly incident on the inner wall surface of the windshield 105, but may be configured via an optical path changing member such as a mirror. In this case, similarly to the first embodiment described above, the light source unit 202 as the light source device can be mounted on the sensor substrate 207.

  A light ray A in FIG. 19 is a light ray emitted from the light source unit 202 and passing through the windshield 105. When the raindrop 203 does not adhere to the outer wall surface of the windshield 105, the light emitted from the light source unit 202 toward the windshield 105 passes through the windshield 105 like the light ray A and remains on the vehicle 100 as it is. Leak outside.

  A light beam B in FIG. 19 is a light beam that is emitted from the light source unit 202 and is regularly reflected by the inner wall surface of the windshield 105 and is incident on the imaging unit 200. Part of the light traveling from the light source unit 202 toward the windshield 105 is regularly reflected by the inner wall surface of the windshield 105. The polarization component of the specularly reflected light (ray B) is generally dominated by the S polarization component (horizontal polarization component S) that oscillates in a direction perpendicular to the incident plane (direction perpendicular to the paper surface of FIG. 19). Is known to be. The specularly reflected light (light ray B) irradiated from the light source unit 202 and specularly reflected by the inner wall surface of the windshield 105 does not vary depending on the presence or absence of the raindrop 203 adhering to the outer wall surface of the windshield 105. In addition, it becomes disturbance light that lowers the detection accuracy of raindrop detection. In the second embodiment, a polarizing filter layer that cuts the light beam B (horizontal polarization component S) is provided in the optical filter 205, and it is possible to suppress the raindrop detection accuracy from being deteriorated by the light beam B.

  A light ray C in FIG. 19 is a light ray that is emitted from the light source unit 202 through the inner wall surface of the windshield 105 and then reflected by raindrops attached to the outer wall surface of the windshield 105 and incident on the imaging unit 200. is there. A part of the light traveling from the light source unit 202 toward the windshield 105 passes through the inner wall surface of the windshield 105, but the transmitted light is more in the vertical polarization component P than in the horizontal polarization component S. When raindrops 203 are attached on the outer wall surface of the windshield 105, the light transmitted through the inner wall surface of the windshield 105 does not leak out like the light ray A, but is reflected multiple times inside the raindrop. Then, the light passes through the windshield 105 again toward the imaging unit 200 and enters the imaging unit 200. At this time, since the infrared light transmission filter region 212 of the front-stage filter 210 in the optical filter 205 of the imaging unit 200 is configured to transmit the emission wavelength (infrared light) of the light source unit 202, the light ray C is red. It passes through the outside light transmission filter region 212. In addition, since the polarizing filter layer that cuts the light beam B (horizontal polarization component S) described above is formed in the longitudinal direction of the metal wire having a wire grid structure so as to transmit the vertical polarization component P, the light beam C is polarized. The filter layer is also transmitted. Therefore, the light ray C reaches the image sensor 206, and raindrops are detected based on the amount of light received.

  A light ray D in FIG. 19 is a light ray that passes through the windshield 105 from the outside of the windshield 105 and enters the imaging unit 200. Although this ray D can also be disturbance light at the time of raindrop detection, in the second embodiment, most of the ray D is cut by the infrared light transmission filter region 212 of the front-stage filter 210 in the optical filter 205. Therefore, it is possible to suppress the raindrop detection accuracy from being lowered by the light ray D.

  A light ray E in FIG. 19 is a light ray that passes through the windshield 105 from the outside of the windshield 105 and enters the imaging unit 200. The light beam E is cut in the infrared light band by the infrared light cut filter region 211 of the pre-stage filter 210 in the optical filter 205, and only the light in the visible region is imaged. This captured image is used for detection of a head lamp of an oncoming vehicle, a tail lamp of a preceding vehicle, and a white line.

  In the second embodiment, since light source light refracted or reflected by raindrops is incident on the image sensor 206, raindrops adhere to the outer wall surface of the windshield 105 in the raindrop detection image region 214 of the captured image data. The non-attached portion that is not attached becomes a low-luminance image portion (low pixel value), while the attached portion where raindrops are attached becomes a high-luminance image portion (high pixel value). However, by setting various threshold values in consideration of such differences, the same processing as in the first embodiment described above can be realized.

FIG. 20 is a flowchart showing a flow of raindrop detection processing according to the second embodiment.
In the following description, the description of the same parts as those in the first embodiment will be omitted, and the description will focus on parts different from those in the first embodiment.
First, the detection processing unit 102A of the image analysis unit 102 calculates a lighting difference e _y (x, ta = 2) for each raindrop detection section x (S1 to S3). Then, it is determined whether or not the calculated lighting difference e _y (x, ta = 2) of each raindrop detection section x is less than the first difference threshold Ey, and the lighting difference e _y (x, ta = 2). ) Is determined to be the number of raindrop detection sections x that are less than the first difference threshold Ey (S4).

However, in the case where the total luminance value increases when raindrops adhere as in the second embodiment, the lighting difference e _y (x, ta) = y (x, ta) −y (x, Since ta-1) takes a positive value, the first difference threshold Ey is appropriately set to a positive value. At this time, in processing step S4, the number of raindrop detection sections x in which the lighting difference e _y (x, ta) of each raindrop detection section x is equal to or greater than the first difference threshold value Ey is obtained.

Further, for example, as the amount of change (total amount of change during illumination) of the total luminance value y (x, ta) in each raindrop detection section x, for example, the total luminance value y (x, ta−1) of the immediately previous lighting imaging frame. ) Is subtracted from the total luminance value y (x, ta) of the lighting imaging frame at this time, in the case where the total luminance value becomes large when the raindrop is attached as in the second embodiment, the raindrop is attached. The lighting difference e _y (x, ta) = y (x, ta−1) −y (x, ta) takes a negative value, so that the first difference threshold Ey is appropriately set to a negative value. . At this time, in the processing step S4, the number of raindrop detection sections x whose lighting difference e _y (x, ta) of each raindrop detection section x is less than the first difference threshold value Ey is obtained.

Further, for example, as the amount of change (total amount of change during illumination) of the total luminance value y (x, ta) in each raindrop detection section x, the total luminance value y (x, ta-1) of the immediately previous lighting imaging frame When using the absolute value of the difference from the total luminance value y (x, ta) of the imaging frame at the time of lighting this time, even when raindrops are attached as in the second embodiment, raindrops are attached. Since the lighting difference e _y (x, ta) = | y (x, ta) −y (x, ta−1) | takes a positive value, the first difference threshold Ey is a positive value as appropriate. Is set. At this time, in processing step S4, the number of raindrop detection sections x in which the lighting difference e _y (x, ta) of each raindrop detection section x is equal to or greater than the first difference threshold value Ey is obtained.

When the detection processing unit 102A of the image analysis unit 102 determines the number of raindrop detection sections x whose lighting difference e _y (x, ta = 2) is less than the first difference threshold value Ey, the determined number is the first determination. It is determined whether or not the threshold value F1 is exceeded, and the determination result (lighting determination result Hta) is temporarily stored in the storage unit 102D of the image analysis unit 102 (S21).

Thereafter, the detection processing unit 102A of the image analysis unit 102 reads the past lighting determination result Hta-2 from the storage unit 102D (S22), and obtains the lighting determination result Hta-2 in the past processing step S4. When the number of sections (that is, the number of sections where the lighting difference e _y (x, ta−2) is less than the first difference threshold Ey) is equal to or less than the third determination threshold F1 ′ (S23) No), the raindrop non-detection processing result is output from the image analysis unit 102 to the wiper control unit 106, assuming that no raindrop is detected (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

On the other hand, when the past lighting determination result Hta-2 read from the storage unit 102D indicates that the number of categories obtained in the past processing step S4 exceeds the third determination threshold F1 ′ (S23). Yes), there is a high possibility that the lighting difference e _y (x, ta−2) has fluctuated due to an increase in the amount of raindrops attached due to the attachment of raindrops. However, the fluctuation of the lighting difference e _y (x, ta−2) may be caused by ambient light or level correction of the black level correction function (BLC). Therefore, in the second embodiment, next, a process for determining whether or not the disturbance light fluctuation component and the BLC fluctuation component are included in the lighting difference e _y (x, ta−2). carry out.

Specifically, the detection processing unit 102A of the image analysis unit 102 firstly adds the luminance value sum d (x, tb = 1) in each raindrop detection section x of the raindrop detection image area in the unlit imaging frames frames1, 3. , D (x, tb = 2) is calculated (S6). Then, the extinction difference e _d (x, tb = 2) = d (x, tb = 2) −d (x, tb = 1) for each raindrop detection section x is calculated (S7). The detection processing unit 102A is turned off when the difference _{e d (x, tb = 2} ) of each raindrop detection segment x calculated is determined whether a second difference threshold Ed more thereof, extinguished when the difference e _d The number of raindrop detection sections x where (x, tb = 2) is less than the second difference threshold Ed is determined (S8).

In the case where the total luminance value increases when raindrops adhere as in the second embodiment, the disturbance light fluctuation component and the BLC fluctuation component, which can be mistaken as raindrop fluctuation components, are cases where the amount of light increases. In this case, as the amount of change in luminance value sum d (x, tb) in each raindrop detection section x (non-illumination change amount), the light-off time difference e _d (x, tb) = d (x, tb) −d When (x, tb−1) is used, the turn-off difference e _d (x, tb) takes a positive value, so the second difference threshold Ed is appropriately set to a positive value. Therefore, in the process step S8, so that the light-off difference e _{d (x,} tb) of each raindrop detection segment x is determined the number of rain detection segment x is the second difference threshold Ed more.

Further, for example, as the amount of change in the luminance value sum d (x, tb) in each raindrop detection section x (the amount of change during non-illumination), the luminance value sum d (x, tb-1) of the immediately preceding extinguished imaging frame. When the sum of the luminance values d (x, tb) of the imaging frame when the light is off is subtracted from this time, it is misidentified as a raindrop fluctuation component in the case where the total luminance value increases when raindrops adhere as in the second embodiment. When the amount of disturbance light to be obtained or the amount of light is increased by level correction, the extinction difference e _d (x, tb) = d (x, tb−1) −d (x, tb) takes a negative value, so the second difference threshold Ed Is appropriately set to a negative value. At this time, in the process step S8, thereby determining the number of light-off difference e _{d (x,} tb) is the rain detection segment x are each less than a second difference threshold Ed of each raindrop detection segment x.

Further, for example, as the amount of change in the luminance value sum d (x, tb) in each raindrop detection section x (the amount of change during non-illumination), the luminance value sum d (x, tb-1) of the immediately preceding extinguished imaging frame. When the absolute value of the difference between the brightness value sum d (x, tb) of the imaging frame when the light is turned off this time is used, as a raindrop fluctuation component in the case where the total brightness value increases when raindrops adhere as in the second embodiment. Even when ambient light that can be misidentified or when the amount of light increases due to level correction, the extinction difference e _d (x, tb) = | d (x, tb) −d (x, tb−1) | takes a positive value. The second difference threshold Ed is appropriately set as a positive value. At this time, in the process step S8, so that the light-off difference e _{d (x,} tb) of each raindrop detection segment x is determined the number of rain detection segment x is the second difference threshold Ed more.

Thereafter, in the second embodiment, it is determined whether the determined number of divisions exceeds the second determination threshold value F2 (S24). When the determined number of divisions is equal to or less than the second determination threshold value F2 (No in S24), the lighting difference e _y (x, ta-2) that is equal to or more than the first difference threshold value Ey described above is the amount of disturbance light. It is determined that it is not caused by the change, is not caused by the black level correction function, but is caused by the increase in the amount of raindrops attached due to the raindrops being attached. Therefore, the raindrop detection processing result is output from the image analysis unit 102 to the wiper control unit 106 as raindrops are detected (S13). As a result, the wiper control unit 106 performs control such as starting to drive the wiper 107.

On the other hand, when the determined number of divisions exceeds the second determination threshold F2 (Yes in S24), the above-described lighting difference e _y (x, ta-2) is caused by a change in the amount of disturbance light or black There is a high possibility that it is caused by the level correction function. Therefore, assuming that no raindrop has been detected, the processing result of raindrop non-detection is output from the image analysis unit 102 to the wiper control unit 106 (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

  Here, in this embodiment, without using the determination result of whether or not the number of divisions obtained in the current processing step S4 exceeds the first determination threshold value F1, the past processing step S4 (in this case, the processing two times before) The determination result of whether or not the number of divisions obtained in step S4) exceeds the first determination threshold value F1 is used. This is to properly grasp the BLC fluctuation component due to the black level correction function. This will be described in detail below.

As described in the first embodiment, as shown in FIGS. 16 and 17, the fluctuation of the output signal at the time of turn-off during the level correction by the black level correction function does not show the same fluctuation as the output signal at the time of lighting. That is, as shown in FIG. 17, the output signal at the time of extinguishing remains constant at zero until time t5 after time t4 when the level correction ends, and starts increasing from zero at time t5. . Therefore, during the period from time t4 to time t5, the raindrop amount does not change, but the total luminance value y (x, ta) at the time of lighting is gradually increased by level correction. It is erroneously determined that the raindrop detection condition that the number of sections of the lighting difference e _y (x, ta) exceeds the first determination threshold value F1 is satisfied. On the other hand, since the luminance value sum d (x, tb) at the time of extinction is constant at zero from time t4 to time t5, the extinction difference e _d (less than the second difference threshold Ed). It is determined that the number of sections of x, tb) is equal to or less than the second determination threshold value F2. As a result, the determination result erroneously determined that the raindrop detection condition is satisfied based on the total luminance value y (x, ta) at the time of lighting is determined not to be affected by disturbance light or level correction. The raindrop detection result that the raindrop is detected is erroneously output to the wiper control unit.

On the other hand, in the second embodiment, the lighting determination result Hta-2 indicating whether or not the number of sections calculated in the past processing step S4 exceeds the first determination threshold F1 is used. Whether or not Hta-2 is affected by disturbance light or level correction is determined by whether or not the number of segments calculated in the processing step S8 after two times (this time) exceeds the second determination threshold value F2. Is done. As a result, for the lighting determination result Hta-2 from time t4 to time t5, the luminance value sum d (x, tb) at the time of extinction from time t5 to time t6, that is, gradually by level correction. The luminance value sum d (x, tb) at the time of extinguishing that rises to can be used. As a result, the brightness sum d (x, tb) at the Off Off time difference e _{d (x,} tb) is calculated from the the number of segments is smaller than the second difference threshold Ed exceeds the second determination threshold value F2 It will be judged. Therefore, it is determined that the lighting determination result Hta-2 from time t4 to time t5 is affected by disturbance light or level correction, and the result of raindrop non-detection that no raindrop is detected is the correct wiper. Output to the control unit.

[Modification 2]
Next, a modified example of the second embodiment (hereinafter referred to as “modified example 2”) will be described.
In the second embodiment described above, the lighting difference e _y (x, ta) that is the amount of change (the amount of change during illumination) of the total luminance value y (x, ta) of each raindrop detection category x during lighting is the raindrop detection condition. However, in the second modification, such a lighting difference e _y (x, ta) is not calculated, and each raindrop detection classification x at the time of lighting is calculated. It is determined whether or not the total luminance value y (x, ta) satisfies the raindrop detection condition.

FIG. 21 is a flowchart showing the flow of raindrop detection processing in the second modification.
When the detection processing unit 102A of the image analysis unit 102 calculates the total luminance value y (x, ta = 1) in each raindrop detection section x of the raindrop detection image area (S1, S2), each calculated raindrop detection section It is determined whether or not the total luminance value y (x, ta = 1) of x is equal to or greater than the fourth difference threshold Ey ″, and the total luminance value y (x, ta = 1) is the fourth difference threshold Ey ′. 'The number of raindrop detection sections x that are greater than or equal to is obtained (S41). As described above, in consideration of the fact that the total luminance value y (x, ta) increases as the amount of raindrops increases, the fourth difference threshold Ey ″ detects that raindrops have adhered to a predetermined amount or more. Appropriate values are set as appropriate.

  When the detection processing unit 102A of the image analysis unit 102 determines the number of raindrop detection sections x whose total luminance value y (x, ta = 1) is equal to or greater than the fourth difference threshold value Ey ″, the determined number of sections is the third. It is determined whether or not the determination threshold value F1 ′ is exceeded, and the determination result (lighting determination result Hta ′) is temporarily stored in the storage unit 102D of the image analysis unit 102 (S42).

  Thereafter, the detection processing unit 102A of the image analysis unit 102 reads the past lighting determination result Hta-2 ′ from the storage unit 102D (S43), and the past processing step S41 in the lighting determination result Hta-2 ′. When the number of divisions obtained in (i.e., the number of divisions in which the total luminance value y (x, ta = 1) is equal to or greater than the fourth difference threshold Ey ″) is equal to or less than the third determination threshold F1 ′. (No in S44), the raindrop non-detection processing result is output from the image analysis unit 102 to the wiper control unit 106, assuming that no raindrop was detected (S12). Accordingly, the wiper control unit 106 performs control such as stopping the driving of the wiper 107.

  On the other hand, when the past lighting determination result Hta-2 ′ read from the storage unit 102D indicates that the number of categories obtained in the past processing step S41 exceeds the third determination threshold F1 ′ ( It is highly possible that the total luminance value y (x, ta = 1) has a value equal to or greater than the fourth difference threshold value Ey ″ due to the attachment of raindrops over a predetermined amount. However, the value of the total luminance value y (x, ta = 1) may be due to ambient light or the black level correction function (BLC). Therefore, also in the second modified example, the disturbance light component and the BLC component are added to the total luminance value y (x, ta = 1) determined to be equal to or greater than the fourth difference threshold value Ey ″ as in the second embodiment. To determine whether or not is included.

[Modification 3]
Next, another modified example of the first and second embodiments described above (hereinafter, this modified example is referred to as “modified example 3”) will be described.
In the first and second embodiments described above (including various modifications), the lighting difference e _y (x, ta) and the lighting total luminance value y (x, ta) are obtained even if there is no influence of level correction. However, there is a possibility that the result of false detection or detection omission of raindrops may be shown due to the influence of ambient light. Specifically, the above-described first embodiment will be described as an example. For example, although the amount of raindrops is not actually increased, the lighting difference e _y (x, In some cases, the number of sections for which ta) is less than the first difference threshold Ey exceeds the first determination threshold F1. Even in this case, the past turn-off difference e _d (x, tb-2) is also less than the second difference threshold Ed due to the influence of the amount of disturbance light, and the past turn-off determination result Htb-2 indicates that raindrops are not detected. If it shows a result, the situation where the result of a raindrop detection is output accidentally can be prevented. However, since there is a time difference between the turn-on difference e _y (x, ta) and the turn-off difference ed (x, tb−2), the amount of disturbance light may vary depending on the time difference. In this case, when the number of sections whose lighting difference e _y (x, ta) is less than the first difference threshold value Ey exceeds the first determination threshold value F1 due to the influence of the change in the amount of disturbance light, when the light is turned off in the past The difference e _d (x, tb-2) does not become less than the second difference threshold Ed, and the past turn-off determination result Htb-2 indicates the result of raindrop detection, and the result of raindrop detection is erroneously output. Will be.
Therefore, in the third modification, in order to stably prevent the erroneous detection of raindrops and the detection omission due to the influence of such disturbance light, the following detection processing is performed.

FIG. 22 is a flowchart showing the flow of raindrop detection processing in the third modification.
First, the detection processing unit 102A of the image analysis unit 102 first sums the luminance values d (x, tb = 1) and d (x, tb = 2) in each raindrop detection section x in the image area for raindrop detection in the imaging frame when the light is extinguished. ) is calculated (S1, S6), off time difference _e d (x for each raindrop detection segment x, calculates the tb = 2) (S7). Then, the detection processing unit 102A determines whether or not the extinction difference e _d (x, tb = 2) of each raindrop detection section x calculated in this way is less than the second difference threshold Ed, respectively. The number of raindrop detection sections x whose time difference e _d (x, tb = 2) is less than the second difference threshold Ed is obtained (S8).

  Next, the detection processing unit 102A of the image analysis unit 102 determines whether or not the obtained number of divisions exceeds the second determination threshold F2 (S51), and the obtained number of divisions exceeds the second determination threshold F2. If it is (No in S51), the result of raindrop non-detection is output to the wiper control unit 106 until a predetermined time period elapses (S52, S53, S12). Therefore, even when the determined number of divisions is equal to or less than the second determination threshold F2 (Yes in S51), until a certain period elapses after it is determined that the determined number of divisions exceeds the second determination threshold F2. The result of no raindrop detection is output to the wiper control unit 106 (S53, S12).

On the other hand, when it is determined that the determined number of divisions is equal to or smaller than the second determination threshold value F2 after a certain period of time has passed (No in S51, No in S53), the detection processing unit 102A then turns on. A total luminance value y (x, ta = 1) and y (x, ta = 2) in each raindrop detection section x of the image area for raindrop detection in the imaging frame is calculated (S2), and each raindrop detection section x is calculated. The lighting difference e _y (x, ta = 2) is calculated (S3). Thereafter, the detection processing unit 102A is lit difference _{e y (x, ta = 2} ) of each raindrop detection segment x calculated is determined whether each less than the first difference threshold value Ey, lit difference e _y The number of raindrop detection sections x in which (x, ta = 2) is less than the first difference threshold Ey is obtained (S4).

When the detection processing unit 102A of the image analysis unit 102 determines the number of raindrop detection sections x whose lighting difference e _y (x, ta = 2) is less than the first difference threshold value Ey, the determined number is the first determination. It is determined whether or not the threshold value F1 is exceeded, and the determination result (lighting determination result Hta) is temporarily stored in the storage unit 102D of the image analysis unit 102 (S54).

Thereafter, the detection processing unit 102A of the image analysis unit 102 reads the past lighting determination result Hta-2 from the storage unit 102D (S55), and the lighting determination result Hta-2 is obtained in the past processing step S4. The number of sections (that is, the number of sections where the lighting difference e _y (x, ta−2) is less than the first difference threshold Ey) is equal to or less than the first determination threshold F1 (in S56) No), assuming that no raindrop was detected, the processing result of raindrop non-detection is output from the image analysis unit 102 to the wiper control unit 106 (S12).

On the other hand, when the past lighting determination result Hta-2 read from the storage unit 102D indicates that the number of categories obtained in the past processing step S4 exceeds the first determination threshold F1 (in S56) Yes), the lighting difference e _y (x, ta−2) is not caused by a change in the amount of disturbance light, and is not caused by the black level correction function. Judged to be due to the increase in quantity. Therefore, the raindrop detection processing result is output from the image analysis unit 102 to the wiper control unit 106 as raindrops are detected (S13).

According to the third modification, when it is determined in this process that the number of sections whose turn-off difference e _d (x, tb) is less than the second difference threshold Ed exceeds the second determination threshold F2 (S51). Yes), the result of raindrop non-detection is always output to the wiper control unit 106 for a certain period thereafter (S52, S53, S12). As a result, the determination result (lighting time difference e _y (x, ta) is the first difference between the lighting time difference e _y (x, ta) from the past (two times before) processing to the processing after the current processing. Regardless of whether the number of sections that is less than the difference threshold value Ey exceeds the first determination threshold value F1, the result of raindrop non-detection is always output to the wiper control unit 106. As a result, when the number of sections whose lighting difference e _y (x, ta) is less than the first difference threshold value Ey exceeds the first determination threshold value F1 due to the change in the amount of disturbance light, the time difference affects Thus, it is possible to avoid a situation in which the raindrop detection result is erroneously output without the corresponding turn-off difference e _d (x, tb−2) being less than the second difference threshold Ed.

In the third modification, when it is determined in this process that the number of sections whose turn-off difference e _d (x, tb) is less than the second difference threshold Ed exceeds the second determination threshold F2 (S51). Yes), this is an example in which the determination result for the lighting difference e _y (x, ta) in the processes before and after that becomes invalid. However, the present invention is not limited to this, and the lighting difference e _y (x, Only the determination result for ta) may be invalidated, or only the determination result for the lighting difference e _y (x, ta) in subsequent processing may be invalidated.

  In the first and second embodiments described above (including various modifications), the information stored in the storage unit 102D is the information on the determination results Htb, Hta ′, and Hta, but the parameters used to derive these determination results ( Information on the total luminance value at the time of lighting, the difference at the time of lighting, the sum of the luminance values at the time of extinction, the difference at the time of extinguishing, etc.) are stored, and the determination result obtained by reading the past parameters and performing the determination process may be used Good.

  In addition, in the past process, the determination result on the total brightness value and the lighting difference in the past process and the determination result on the sum of the brightness values and the difference in the off time in the past process are used. Thus, the result finally output to the wiper control unit is temporarily derived and stored in the storage unit 102D, and the determination result of the luminance value sum at the time of extinction and the difference at the time of extinction in the current process is used, The result output to the wiper control unit at the time of the past process stored in the storage unit 102D may be updated and output to the wiper control unit.

  In Embodiments 1 and 2 described above (including various modifications), the light receiving means is an image sensor, and the total luminance value (light receiving amount during illumination) obtained from two-dimensional data (captured image data) or during lighting is obtained. The difference (change amount during illumination), the sum of brightness values during light extinction (light reception amount during non-illumination), or the difference during light extinction (variation amount during non-illumination) are used as parameters for determining raindrop detection conditions. Absent. For example, a sensor in which a plurality of light receiving elements (photoelectric conversion elements) are arranged in a line may be used, and these parameters may be calculated from one-dimensional data and used to determine raindrop detection conditions. Further, these parameters may be calculated from an output value (amount of received light) obtained from one light receiving element (photoelectric conversion element) and used for determination of raindrop detection conditions.

  Further, for each output value (light reception amount) obtained from one light receiving element, parameters such as a light reception amount during illumination or a change during illumination, a light reception amount during non-illumination or a change during non-illumination may be calculated, These parameters may be calculated using a value obtained by combining output values (light reception amounts) obtained from a plurality of light receiving elements. As specific examples, these parameters are calculated using values obtained by integrating the two-dimensional data obtained by the light receiving means within a certain range, or values obtained by integrating the two-dimensional data obtained by the light receiving means in the vertical direction or the horizontal direction are used. These parameters may be used for calculation, or may be calculated using a value obtained by integrating all the one-dimensional data obtained by the light receiving means. In this way, when calculating these parameters using the value obtained by combining the output values (light reception amounts) obtained from a plurality of light receiving elements, the influence of noise generated in each light receiving element is reduced or the number of data is reduced. Thus, the calculation amount and the storage capacity can be reduced.

In the above-described first and second embodiments (including various modifications), the raindrop detection image area is imaged together with the vehicle detection image area, but only the raindrop detection image area is imaged ( It may be a device dedicated to raindrop detection.
Further, in the above-described first and second embodiments (including various modifications), the example in which raindrops attached on the windshield (windshield) of the vehicle are detected as detection objects has been described. However, the object detection according to the present invention is described. The apparatus is not limited to this, and can be widely used. For example, raindrops adhering to a light transmissive member such as a lens or cover of a surveillance camera or a window of a building may be detected as a detection target. Note that the material of the light transmissive member is not limited to glass, and various transparent or translucent materials such as plastic can be used.
In addition, the detection target is not limited to raindrops, and is not limited to a deposit that adheres to a light-transmitting member such as a windshield.
The processing result of the detection process is not limited to the case of controlling the operation of the wiper 107, but the control of the removing means that removes raindrops by blowing off by wind or drying by heat, or the control of some device that is not the removing means. You may use it.

Further, in the above-described first and second embodiments (including various modified examples, but modified examples), the order of the determination process using the lighting difference and the total luminance value during lighting and the determination process using the unlit difference May be reversed. For example, in the above-described first embodiment, after performing the determination process whether the number of sections whose lighting difference e _y (x, ta) is less than the first difference threshold value Ey exceeds the first determination threshold value F1 first. In addition, a determination process is performed to determine whether or not the number of sections whose past turn-off difference e _d (x, tb−2) is less than the second difference threshold Ed exceeds the second determination threshold F2. The order of processing may be reversed.

What was demonstrated above is an example, and there exists an effect peculiar for every following aspect.
(Aspect A)
Within the illumination range of the light source light based on the amount of light received during illumination received by the light receiving means such as the image sensor 206 when the light source light such as the light source unit 202 repeatedly irradiates and non-irradiates the light source light. In an object detection device for detecting a detection target such as raindrops, the non-illuminated light reception amount such as the sum of luminance values d (x, tb) received by the light receiving means when the light source light is not illuminated by the light source, and the non-illumination The amount of light received during illumination, such as the total luminance value y (x, ta-2) in the previous period, and the time of non-illumination corresponding to the amount of light received during non-illumination. The presence state of the detection object in the illumination range is determined by using at least one of the received light amounts during illumination such as the total luminance value y (x, ta + 2) in the later period than the illumination immediately after the illumination. Predetermined to detect And having a detection processing means such as a detection processing section 102A for performing the detection process to determine whether or not the condition.
In this aspect, in the detection process for determining whether or not a predetermined condition for detecting the presence state of the detection target within the illumination range of the light source light is satisfied, the non-illuminated light reception amount and the illumination light reception amount are used. . Since the amount of light received during illumination varies according to the state of presence of the detection target within the illumination range, the state of presence of the detection target within the illumination range can be detected from the amount of light received during illumination. The presence state of the detection target within the illumination range can be controlled in the same manner as various controls (such as wiper drive control) using the detection processing result of the conventional object detection device.
The main factors that fluctuate the amount of light received during illumination are the factors due to changes in the presence of the detection object within the illumination range (increase or decrease in the detection object), as well as error factors due to ambient light incident on the light receiving means, There are error factors other than ambient light (such as an automatic correction function provided in the light receiving means). Whether or not erroneous detection or detection omission may occur due to these error factors can be determined from the amount of light received during non-illumination received by the light receiving means when light source light is not irradiated. The amount of light received during non-illumination does not vary even if the presence of the detection object changes, so when the amount of light received during non-illumination varies, the amount of light received during illumination also varies due to error factors such as ambient light It is because it is considered. Conventionally, the non-illuminated light reception amount for grasping error factors such as disturbance light is obtained at a time as close as possible to the illumination light reception amount. This is because the more distant the two are, the more likely the error factor status will change during the time difference, making it difficult to properly grasp the error factor of the received light amount from the non-illuminated light amount. Because it was thought to be. Therefore, the amount of light received during illumination is normally used immediately before or after the non-illumination corresponding to the amount of light received during non-illumination for grasping the error factor of the amount of light received during illumination.
However, if the illumination amount of the received light amount during illumination is immediately before or immediately after the non-illumination amount of the received light amount during non-illumination for grasping the error factor of the received light amount during illumination in this way, It has been found that the error factor due to the correction function may not be properly grasped, resulting in erroneous detection of detection objects and detection omissions.
More specifically, when an automatic correction function such as an automatic exposure correction function or a black level correction function provided in the light receiving unit is exhibited, even if the actual light reception amount is constant, the received light amount ( The output value) gradually changes toward the target value after correction by the automatic correction function. When the received light amount (output value) output from the light receiving means gradually decreases toward the target value after correction by the automatic correction function, the light reception amount during illumination that originally had a high output value is during the time until the target value is reached. It continues to decline toward the target value. However, the amount of light received during non-illumination originally has a low output value, so it may bottom out in the middle of the time required to reach the target value by the automatic correction function, and may continue to be constant thereafter. . Normally, a certain amount of time is required until the amount of light received (output value) output from the light receiving means reaches the target value by the automatic correction function, and the length of the time is the repetition cycle of light source light irradiation and non-irradiation Longer than. Therefore, when the amount of light received during non-illumination reaches a constant value during the time it takes to reach the target value by the automatic correction function, the amount of light received during illumination immediately before or after the non-illuminated light reception amount is A situation where it continues to decline can occur. In this situation, that is, the amount of light received at the time of illumination has decreased but the amount of light received at the time of non-illumination has not decreased (is constant). The situation is the same as when there was a change. Therefore, if the received light amount during illumination is immediately before or after the received light amount during non-illumination for grasping the error factor of the received light amount during illumination, the received light amount during illumination varies due to the automatic correction function. This may not be properly grasped from the amount of light received during non-illumination.
Therefore, in this aspect, as the light reception amount at the time of illumination, not just the light reception amount at the time of illumination but immediately before or after the light reception amount at the time of non-illumination for grasping the error factor of the light reception amount at the time of illumination. The amount of light received during illumination in the later period is used immediately after the amount of light received during illumination or the amount of light received during non-illumination. In this case, if the amount of light received during non-illumination bottoms out or saturates in the middle of the time until the target value is reached by the automatic correction function, and then a constant value is maintained after that, it remains constant. The cause of the error in the received light amount immediately before or immediately after the non-illuminated received light amount that has become the value is the received light amount before the illumination or the non-illuminated light after the non-illuminated received light amount that has reached the constant value. This is determined by the amount of received light. Therefore, as the amount of light received during illumination, the amount of light received during illumination or immediately after the amount of light received during non-illumination in an appropriate period prior to immediately before the amount of light received during non-illumination for grasping the error factor of the amount of light received during illumination By setting the received light amount during illumination in an appropriate period later, the error correction factor of the received light amount immediately before or immediately after the non-illuminated received light amount that has reached the constant value varies depending on the automatic correction function. Judgment can be made using the amount of light received during non-illumination. As a result, the detection target that may occur when using the received light amount immediately before or immediately after the non-illuminated received light amount to grasp the error factor of the illuminated received light amount as the illuminated received light amount or Detection omission can be suppressed.

(Aspect B)
In the aspect A, the detection processing unit is configured to calculate a non-illumination change amount such as a turn-off difference e _d (x, tb) indicating a temporal change in the non-illumination light reception amount, and an illumination light reception amount in the previous period. Illumination time variation e _y (x, ta-2) or the like indicating a time change, and lighting time difference e _y (x, ta + 2) or the like indicating time variation of the illumination light reception amount in the subsequent period. A detection process for determining whether or not the predetermined condition is satisfied is performed using at least one of the illumination variations.
According to this, the optical characteristics of optical members such as lenses, mirrors, and prisms, the light emission amount of the light source, etc. fluctuate due to temperature changes, and the temperature fluctuations that change the brightness value of the image area for raindrop detection, It is possible to appropriately detect the detection target by excluding the influence of the temporal deterioration variation that changes the luminance value of the image area for raindrop detection by changing the optical characteristics and the light emission amount of the light source due to the temporal deterioration. .

(Aspect C)
In the aspect A or B, a storage unit such as a storage unit 102D that stores at least one received light amount during the non-illumination and at least one received light amount during the illumination, and the detection processing unit includes: A detection process for determining whether or not the predetermined condition is satisfied is performed using the at least one received light amount stored in the storage unit.
According to this, as the amount of light received at the time of illumination, not from the amount of light received at the time of illumination immediately before or immediately after the amount of light received at the time of illumination for grasping the error factor of the amount of light received at the time of illumination. In addition, it is possible to easily realize the process using the illumination light reception amount in the subsequent period rather than immediately after the illumination light reception amount or the non-illumination light reception amount in the previous period.

(Aspect D)
In the aspect B, there is a storage unit such as a storage unit 102D that stores at least one of the non-illumination change amount and the at least one illumination change amount, and the detection processing unit includes the storage unit A detection process for determining whether or not the predetermined condition is satisfied is executed using the at least one change amount stored in the means.
According to this, as the amount of change at the time of illumination, not from the amount of change at the time of illumination immediately before or immediately after the amount of change at the time of illumination for grasping the error factor of the amount of change at the time of illumination, but from immediately before the amount of change at the time of non-illumination In addition, the process using the illumination change amount in the subsequent period can be easily realized rather than immediately after the illumination change amount or the non-illumination change amount in the previous period.

(Aspect E)
In the aspect A or B, the predetermined condition is a first condition for determining the presence state of the detection object in the illumination range using the light reception amount during illumination without using the light reception amount during non-illumination. And a second condition for determining whether or not the determination result of the first condition is made the processing result of the detection process using the non-illuminated received light amount, and the at least one illuminated received light amount Among the determination results of the first condition using lighting (determination results Hta, Hta ′, etc. during lighting) and the determination result of the second condition using the non-illuminated received light amount (determination results Htb, etc. during extinguishing) Storage means such as a storage unit 102D for storing at least one of the determination results, wherein the detection processing means uses the at least one determination result stored in the storage means to satisfy the predetermined condition. Detection process to determine whether or not It is characterized by performing.
According to this, as the amount of light received at the time of illumination, not from the amount of light received at the time of illumination immediately before or immediately after the amount of light received at the time of illumination for grasping the error factor of the amount of light received at the time of illumination. In addition, it is possible to easily realize the process using the illumination light reception amount in the subsequent period rather than immediately after the illumination light reception amount or the non-illumination light reception amount in the previous period.

(Aspect F)
In any one of the aspects A to E, the detection object is an attachment such as raindrops attached to a light-transmitting member such as the windshield 105 irradiated with light from the light source. And
According to this, the deposit | attachment adhering on a light transmissive member can be detected appropriately.

(Aspect G)
In any one of the aspects A to F, the detection processing unit includes a predetermined time including a non-illumination time corresponding to the non-illumination light reception amount when the non-illumination light reception amount satisfies a predetermined non-execution condition. Instead of the result of the detection process using the amount of light received during illumination within the range, a result determined without using the amount of light received during illumination is used as the result of the detection process.
According to this, as described in Modification 3 above, the influence of the time difference between the non-illuminated received light amount and the illuminated received light amount used for the detection process (change in disturbance light during the time difference, etc.) is suppressed. Is possible.

(Aspect H)
Based on the object detection apparatus according to any one of the aspects A to G and the result of the detection process in the object detection apparatus, the operation of an object removal unit such as a wiper 107 that removes the detection target is controlled. An object removal operation control system comprising control means such as a wiper control unit 106.
According to this, it becomes possible to appropriately control the operation of the object removing means.

(Aspect I)
In an object detection method for detecting a detection target within an illumination range of light source light based on a received light amount during illumination received by a light receiving means during illumination of the light source light by a light source that periodically repeats irradiation and non-irradiation of the light source light A non-illuminated received light amount received by the light receiving means when the light source light is not illuminated by the light source, and an illuminated received light amount in a period before the non-illuminated previous light corresponding to the non-illuminated received light amount, And the presence of the detection object in the illumination range using at least one of the received light amount during illumination in a later period than the illumination immediately after non-illumination corresponding to the received light amount during non-illumination. It is characterized by determining whether or not a predetermined condition for detecting the situation is satisfied.
According to this, an error in the detection target that may occur when using the received light amount immediately before or immediately after the non-illuminated received light amount for grasping the error factor of the illuminated received light amount as the illuminated received light amount. Detection or detection omission can be suppressed.

(Aspect J)
An object detection program for causing a computer of the object detection apparatus according to any one of the aspects A to G to function, wherein the computer functions as the detection processing unit.
According to this, an error in the detection target that may occur when using the received light amount immediately before or immediately after the non-illuminated received light amount for grasping the error factor of the illuminated received light amount as the illuminated received light amount. Detection or detection omission can be suppressed.
This program can be distributed or obtained in a state of being recorded on a recording medium such as a CD-ROM. It is also possible to distribute and obtain signals by placing this program and distributing or receiving signals transmitted by a predetermined transmission device via transmission media such as public telephone lines, dedicated lines, and other communication networks. Is possible. At the time of distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, it is not necessary for all data constituting the computer program to exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave including the computer program. Further, the transmission method for transmitting a computer program from a predetermined transmission device includes a case where data constituting the program is transmitted continuously and a case where it is transmitted intermittently.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 102A Detection processing part 102B Light source control part 102C Exposure amount control part 102D Storage part 105 Windshield 106 Wiper control unit 107 Wiper 108 Vehicle travel control unit 200 Imaging part 202 Light source part 203 Raindrop 204 Imaging Lens 205 Optical filter 206 Image sensor 207 Sensor substrate 208 Signal processing unit 230 Reflection deflection prism

JP2013-117514A

Claims (10)

In an object detection device for detecting a detection object within an illumination range of light source light based on a light reception amount received by a light receiving means during illumination of the light source light by a light source that periodically repeats irradiation and non-irradiation of the light source light ,
A non-illuminated received light amount received by the light receiving means when the light source light is not illuminated by the light source, and an illuminated received light amount in a period before the non-illuminated immediately preceding illumination corresponding to the non-illuminated received light amount; and The presence state of the detection object in the illumination range using at least one of the received light amount during illumination in a later period than the illumination immediately after non-illumination corresponding to the received light amount during non-illumination An object detection apparatus comprising detection processing means for executing a detection process for determining whether or not a predetermined condition for detecting an image is satisfied. The object detection apparatus according to claim 1,
The detection processing means includes a non-illumination change amount indicating a time change of the non-illumination received light amount, an illumination change amount indicating a time change of the illumination received light amount in the previous period, and an illumination time in the subsequent period. Object detection characterized in that a detection process for determining whether or not the predetermined condition is satisfied is performed using at least one of the illumination-time variation amounts indicating the temporal variation of the received light amount. apparatus. In the object detection device according to claim 1 or 2,
Storing means for storing at least one received light amount of the non-illuminated received light amount and the at least one illuminated received light amount;
The object detection apparatus, wherein the detection processing unit executes a detection process for determining whether or not the predetermined condition is satisfied using the at least one received light amount stored in the storage unit. The object detection device according to claim 2,
Storage means for storing a change amount of at least one of the non-illumination change amount and the at least one illumination change amount;
The object detection apparatus, wherein the detection processing unit executes a detection process for determining whether or not the predetermined condition is satisfied using the at least one change amount stored in the storage unit. In the object detection device according to claim 1 or 2,
The predetermined condition includes a first condition for determining a presence state of a detection target in the illumination range using the light reception amount during illumination without using the light reception amount during non-illumination, and the light reception during non-lighting. A second condition for determining whether the determination result of the first condition is a processing result of the detection process using a quantity,
Storage means for storing the determination result of the first condition using the at least one light reception amount during illumination and the determination result of the second condition using the non-illumination light reception amount Have
The object detection apparatus, wherein the detection processing means executes a detection process for determining whether or not the predetermined condition is satisfied using the at least one determination result stored in the storage means. In the object detection device according to any one of claims 1 to 5,
The object detection apparatus according to claim 1, wherein the detection target is an attachment that adheres to a light transmissive member irradiated with light from the light source. The object detection device according to any one of claims 1 to 6,
When the non-illuminated light reception amount satisfies a predetermined non-execution condition, the detection processing means uses the illumination light reception amount within a predetermined time range including non-illumination time corresponding to the non-illumination light reception amount. An object detection apparatus characterized by using, as a result of the detection process, a result determined without using the amount of light received during illumination instead of the result of the process. The object detection device according to any one of claims 1 to 7,
An object removal operation control system comprising: a control unit that controls an operation of an object removal unit that removes the detection target object based on a result of the detection process in the object detection device. In an object detection method for detecting a detection target within an illumination range of light source light based on a received light amount during illumination received by a light receiving means during illumination of the light source light by a light source that periodically repeats irradiation and non-irradiation of the light source light ,
A non-illuminated received light amount received by the light receiving means when the light source light is not illuminated by the light source, and an illuminated received light amount in a period before the non-illuminated immediately preceding illumination corresponding to the non-illuminated received light amount; and The presence state of the detection object in the illumination range using at least one of the received light amount during illumination in a later period than the illumination immediately after non-illumination corresponding to the received light amount during non-illumination An object detection method characterized by determining whether or not a predetermined condition for detecting an image is satisfied. An object detection program for causing a computer of the object detection apparatus according to any one of claims 1 to 7 to function,
An object detection program that causes the computer to function as the detection processing means.

Images