JPH10272959A - Arousal level estimation device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide a wakefulness estimating device capable of easily and efficiently estimating a wakefulness of a driver by using a driver's blink and behavioral characteristics. SOLUTION: A blink is detected from a driver's face image, a long blink for a predetermined time or more is detected based on the blink time, and a long blink is determined from the total number of blinks and the number of long blinks within a predetermined evaluation period. Is calculated. Further, a behavior indicating a behavioral characteristic of the driver, for example, a standard deviation [EMH-SD] of the horizontal movement of the eyeball is detected. Then, a prediction formula Y = A + B [Lrate] + C [EMH-SD] (A,
(B and C are prediction coefficients), and a drowsiness predicted value [Y] is calculated, and the drowsiness predicted value is discriminated by a predetermined threshold to evaluate the arousal level.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is suitable for alerting a driver's awakening and eventually a decrease in arousal by simply paying attention to the driver's blink and issuing a warning to arouse driving attention. The present invention relates to a wakefulness estimation device.

[0002]

[Related Background Art] Recently, various systems for estimating a driver's arousal level based on various information and issuing a warning when a decrease in the arousal level is detected, for example, to arouse driving attention have been developed. Have been. One of the methods to estimate this kind of arousal
Finally, there is a method in which a driver's blink is used as an evaluation index. For example, Japanese Patent Application Laid-Open No. 61-175129 discloses a method of counting the number of blinks per unit time and determining a decrease in arousal level. I have. However, when the number of blinks per unit time is used as an index for arousal level evaluation, errors due to individual differences in blinks are likely to occur, and the accuracy of estimation cannot be improved.

Therefore, the present applicant has previously filed Japanese Patent Application No. 8-913.
No. 24, and a paper [52.
A method of estimating a decrease in arousal level by focusing on the driver's blink time was proposed as announced as a method for evaluating arousal level during driving (9632415). A method of estimating the degree of awakening focusing on the blink time is based on a frequency distribution of the blink time, and a threshold (blink time) for determining a long blink from a standard blink time distribution width and a center time of the distribution width. ) Is set, and the ratio of the total number of blinks within a predetermined period to the number of occurrences of long blinks exceeding the above-described threshold is obtained, and a decrease in arousal level is determined by evaluating this ratio.

According to such a method, there is an advantage that it is possible to absorb individual differences such as a blink time and a blink frequency and to accurately evaluate the arousal level.

[0005]

As an index for estimating the arousal level, not only the above-mentioned blinking but also, for example, a change in brain waves can be used. However, in consideration of estimating the degree of awakening of a driver mounted on an automobile, measurement of an electroencephalogram which requires sticking of an electrode to the driver's head or the like is not practical. Therefore, the degree of drowsiness (arousal level)
Information on the behavior of the driver, such as the frequency of the driver's head tilting forward or backward, the frequency of the line of sight movement, and the frequency of various driving operations such as the steering wheel operation and the turn signal operation. It is considered that it is desirable to detect the characteristic) without contacting the driver.

[0006] However, by using information on the above-mentioned behavior of the driver other than the blinking time, or by using the information on the behavior of the driver together with the information on the blinking, how to make the awakening degree of the driver easier can be improved. In addition, there are still some problems in estimating with high accuracy. In particular, there is a problem in estimating the arousal level using the information (behavioral characteristics) on the behavior of the driver without being affected by individual differences.

The present invention has been made in view of such circumstances, and its object is to wake up the driver using not only information on the driver's blink but also other behavioral characteristics of the driver. It is an object of the present invention to provide a waking degree estimating device that can efficiently estimate the degree of simplicity with high accuracy.

[0008]

SUMMARY OF THE INVENTION In order to achieve the above object, a waking degree estimating apparatus according to the present invention detects a driver's blink from a driver's face image captured by, for example, a camera, and detects the detected blink. Along with detecting a long blink for a predetermined time or more based on the time of the blink, and including a long blink occurrence ratio calculating means for obtaining a long blink occurrence ratio from the total number of blinks and the number of long blinks within a predetermined evaluation period A behavior detection means for detecting a behavior indicating a behavioral characteristic of the driver, wherein the awakening degree determination means uses a sleepiness prediction model based on the occurrence ratio of the long blink and the behavior information of the driver. It is characterized in that the driver's arousal level is evaluated in accordance with a prediction formula obtained by regression analysis.

In particular, when detecting the long blink,
For example, a reference time [To] of a blink specific to the driver is obtained from a blink time of the driver at the time of awakening, and a blink exceeding a time [Ts] obtained by increasing the reference time by a predetermined ratio [r%] is long. Blinks are detected, and the occurrence ratio [Lrate] of long blinks to the total number of blinks within a predetermined time is determined. On the other hand, as the information on the behavior indicating the behavioral characteristics of the driver, for example, a standard deviation [EMH-SD] of the eye movement indicating the driver's line of sight is calculated.

In the arousal level determining means, for example, the relationship between the arousal level and the occurrence ratio of long blinks [Lrate] and the standard deviation of eye movement [EMH-SD] obtained by regression analysis using a drowsiness prediction model Y = A + B [Lrate] + C [EMH-SD] (A,
(B and C are prediction coefficients), and a sleepiness prediction value [Y] is calculated, and the sleepiness prediction value is discriminated by a predetermined threshold to evaluate the arousal level.

That is, the degree of arousal is estimated in accordance with a prediction formula shown in a multiple regression model (sleepiness prediction model) obtained by integrating information on blinks and information on other behavioral characteristics such as standard deviation of eye movements. By doing so, the degree of awakening of the driver and, consequently, a decrease in the degree of awakening are detected simply and with high accuracy.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of an awakening degree estimating apparatus according to the present invention will be described with reference to the drawings. FIG.
FIG. 1 conceptually shows the configuration of an embodiment device mounted on the vehicle 1. In FIG. 2, reference numeral 2 denotes a TV camera for imaging the face of the driver D, particularly the eye area. In the figure, reference numeral 3 denotes a display (multiplex information display device) for displaying various information as images and presenting it to the driver D, and reference numeral 4 denotes a speaker for outputting a voice message, an alarm sound and the like. The TV camera 2, the display 3, and the speaker 4 are incorporated in, for example, an instrument panel in front of a driver's seat.

The awakening degree estimating apparatus according to this embodiment has a T
A blink of the driver is detected from a face image of the driver captured by the V camera 2 to estimate the degree of awakening of the driver. When the degree of awakening decreases, a message is displayed via the display 3 and a speaker 4 is displayed. It plays the role of alerting the driver and stimulating driving attention. This device is realized by, for example, an electronic control unit (ECU) mainly composed of a microprocessor, and is schematically configured as shown in FIG.

That is, this arousal level estimating apparatus is, for example, shown in FIG.
As shown in the functional block configuration of FIG.
The recognition process is performed at 0, and a partial image of the eye region is extracted at a predetermined cycle, for example. The blink detection unit 11 detects blinking by detecting, particularly, the opening and closing of the eyelids from the temporal change of the image. The blink time calculation unit 12 includes the blink detection unit 11
Each time a blink is detected, the blink time (blink time) [t] indicated by the closed eye time from the start to the end of the closed eye is measured. Such blink time detection processing is repeatedly executed over a predetermined period under the management of a timer, for example.

The blink reference time calculation unit 14 has a strong sense of "starting (starting)" as in the initial stage of driving, for example, and recognizes that the driver D is sufficiently awake. When it can be considered, based on the blink time [t] obtained over a predetermined period by the blink time calculation unit 12, a reference time [To] of the blink at the time of awakening unique to the driver D is obtained. I have. The reference time [To] is calculated as the average value of the blink time [t] in the predetermined period, for example, 5 minutes at the start of operation.

The long blink time setting section 15 sets a blink evaluation time [Ts] for detecting a long blink from the blink of the driver D, particularly in association with a decrease in arousal level. The reference time [T obtained by the time calculation unit 14
o] by a predetermined ratio [r (%)], the evaluation time [Ts] is determined as Ts = To + To · r / 100. The ratio is given as, for example, 5%, 10%, 15%, or 20%, and is actually set to, for example, 10% based on a simulation result or the like described later. That is, in the long blink time setting unit 15, when the arousal level decreases,
Generally, in consideration of the fact that a long blink is likely to occur, a blink evaluation time [Ts] for detecting a long blink is used in order to use the long blink as a determination index of a decrease in arousal level.
Is set based on the reference time [To] of the blink specific to the driver D.

Although the blink evaluation time [Ts] is set here based on the blink reference time [To], as proposed by the present applicant in Japanese Patent Application No. Hei 8-91324,
Based on the frequency distribution of the blink time [t], a standard distribution time width of the blink and a center time of the distribution width are obtained, and the blink evaluation time is calculated according to the blink frequency distribution information (time width and center time). It is of course possible to set [Ts].

The long blink detection section 16 calculates the blink B of the driver D from the blink time [t] of the driver D sequentially detected by the blink time calculation section 12 in accordance with the blink evaluation time [Ts] set as described above. Blinks exceeding the evaluation time [Ts] are detected as long blinks. The long blink occurrence ratio calculation unit 17 calculates, for example, the total number of blinks [Ntotal] evaluated by the long blink detection unit 16 and the long blink detection unit 1.
The number of long blinks [Nlo
ng]. Then, the occurrence ratio calculation unit 17 calculates the occurrence ratio of long blinks [Lra] from the total number of blinks [Ntotal] in a predetermined evaluation period set in advance and the number of long blinks [Nlong] in the blinks.
te (%)] as Lrate = 100 · Nlong / Ntotal.

The long blink occurrence ratio calculating section 17 counts the number of long blinks [Nlong] and the other standard number of blinks [Nnormal], and calculates Lrate = 100 · Nlong / (Nlong). + Nnormal) to determine the occurrence ratio [Lrate] of long blinks.

On the other hand, the eyeball movement detecting section 21 detects, in particular, the horizontal movement of the eyeball from the partial image showing the eye area of the driver D obtained by the image processing section 10. The movement of the eyeball in the horizontal direction indicates the behavioral characteristics (behavior) of the driver D such as, for example, checking the information of the adjacent lane from the attention state of the driving lane or checking the information via the rearview mirror or the side mirror (gaze of the mirror). It is shown. Thus, the standard deviation calculation unit 22 obtains the standard deviation [EMH-SD] of the movement of the eyeball from the information on the horizontal movement of the eyeball (the position of the pupil in the eye). The standard deviation of the horizontal movement of the eyeball is calculated as an average value of the standard deviation per unit time obtained in, for example, a 60-second step in five minutes.

The arousal level prediction calculating section 23 calculates the arousal level and the occurrence ratio of the long blink [Lrate] obtained by multiple regression analysis using a drowsiness prediction model as described later.
(%)] And a standard deviation of eye movement [EMH-SD] Y = A + B · Lrate + C · EMH-SD (A,
(B and C are prediction coefficients) to calculate a sleepiness prediction value [Y]. For example, the prediction coefficients A, B, and C are defined as five levels of arousal level as follows, and the rate of increase in setting the above-described blink evaluation time [Ts] based on various simulation results [ r%] is set to 10% as described later, for example, A = 2.812, B = 0.032, and C = −0.695. That is, the above-mentioned prediction formula is obtained as follows: Y = 2.812 + 0.032 · Lrate−0.695 ·
Given as EMH-SD, the ratio of occurrence of long blinks [Lrate (%)] during the blink of driver D is calculated as described above, and the standard deviation [EMH-SD] of the eye movement at that time is calculated. The drowsiness predicted value [Y] is obtained by calculating the prediction formula.

By the way, a prediction equation y = a + b · Lrate (a, b represents a relationship between the awakening degree obtained by the multiple regression analysis using the drowsiness prediction model and the occurrence ratio [Lrate (%)] of the long blink. When the drowsiness prediction value [y] is calculated based on the prediction coefficient, the prediction coefficients a and b are given as, for example, a = 1.238 and b = 0.046 under the same evaluation condition. That is, the prediction formula in this case is given as y = 1.238 + 0.046 · Lrate.

The five levels of the arousal level are, for example, level 1 ... seemingly drowsy (the movement of the line of sight is fast and frequent; blinking is stable and movement is active) level 2 ... slightly sleepy (the line of sight) Slow movement. Lips open.) Level 3 ... sleepy (blinks are slow and frequent. There is movement of the mouth.) Level 4 ... quite sleepy (conscious blinks, blinks and gaze are slow. .) Level 5: Set to be very sleepy (closes eyelids, head tilts back and forth). Therefore, when the drowsiness predicted values [Y] and [y] exceed [3], it is estimated (determined) that the arousal level is low and the user is likely to sleep as shown below.

The awakening degree prediction calculation unit 23 calculates the drowsiness prediction value [Y] obtained based on the long blink occurrence ratio [Lrate] and the standard deviation [EMH-SD] of the horizontal movement of the eyeball. The arousal level is determined by the arousal level determination unit 24 to determine the arousal level reduction. When the arousal level decrease determination unit 24 detects a decrease in the arousal level, a warning is issued using the display 3 and the speaker 4 described above, and the driver D is alerted to driving attention. Will be

Note that the processing for determining a decrease in arousal level shown in the above-described functional blocks is actually executed under a microprocessor according to a control routine shown in FIG. That is, at the beginning of the operation start, for example, the driver D
The blinking time [t] is measured (steps S1, S2, S
3) The average of the blink time is calculated as a blink reference time [To] unique to the driver D (step S4).
Thereafter, one unit of the measurement target period is set to 5 minutes, and the subsequent blinking time [t] is measured (steps S5, S6, S5).
S7).

Thereafter, the reference time [To] is set to 1 according to the blink time [t] detected every 5 minutes as described above.
An occurrence rate [Lrate (= Long10)] of a long blink detected under the blink evaluation time [Ts] set to be 0% longer is calculated (step S8). At the same time, the horizontal movement of the eyeball indicating the movement of the line of sight is detected, and the standard deviation [EMH-SD] of the horizontal movement is obtained based on the movement information (step S).
9).

Then, the occurrence ratio of long blinks [Lrate (= Lo
ng10)] and the standard deviation of eye movement [EMH-SD] are calculated, then a drowsiness predicted value [Y] is calculated according to the above-described prediction formula (step S10), and is obtained by, for example, this calculation. The sleepiness prediction value (degree of arousal) is displayed on the display 3 (step S11). The display of the predicted drowsiness value (degree of arousal) is performed by, for example, displaying the degree of arousal in a bar graph or changing the display color of the information in accordance with the five levels set as described above.

Then, the sleepiness predicted value [Y] obtained as described above is evaluated (step S12). If the level (predicted value) exceeds [3], for example, the driver D is awakened. Then, an alarm is issued to urge driving attention (step S13). If the level of the arousal level is equal to or less than [3], the processing from step S5 described above is repeatedly executed, whereby the estimating process of the arousal level based on the blink information for the next five minutes is executed again.

Here, the occurrence ratio of the long blink described above [Lra
te (= Long10)] and the standard deviation of the horizontal movement of the eyeball [EMH
-SD] and the evaluation of the drowsiness predicted value [Y] based on this will be described in more detail. As the evaluation index of the arousal level, not only the blink described above but also, for example, an electroencephalogram, an electrocardiogram,
It is conceivable to use a temporal change of a physiological index such as breathing or a temporal change of a performance index such as a steering operation characteristic indicated by a steering wheel angle. FIG. 4 is a simulation experiment result showing the relationship between each of these indices and the drowsiness of the driver D, which was objectively evaluated by a third party at that time.

In FIG. 4, Cz (α / β) and Pz (α / β)
Is the ratio of the spectral power values of the α-wave and β-wave of the electroencephalogram obtained at the top Cz region and the Pz region of the driver D. Blink-No is the number of blinks in 5 seconds, for example, obtained from the eye movement of the driver D.
Is the average blink time, HR is the heart rate per minute of driver D, and Resp is the respiratory rate. Speed is the running speed of the vehicle, Steer is the average value of the steering wheel angle, and Steer-SD is the deviation of the steering wheel angle. Sleepiness is the driver's D sleepiness evaluated by a third party, and MWS is the driver's D subjective sleepiness evaluation.

As shown in the simulation experiment results illustrated in FIG. 4, the driver D sleepiness (Sleepine
ss) increases as the operation time elapses. Moreover, driver D's drowsiness depends on the number of blinks (Blink-No) and blink time (Blin-No).
k-Dur). In other words, drowsiness increases due to the monotony and familiarity of the driving operation over time, and furthermore, fatigue, and the number of blinks tends to decrease and the blink time tends to increase as the drowsiness increases. Furthermore, the Cz (α / β) and Pz (α / β) of the electroencephalogram also tend to increase with the lapse of driving time (increase in sleepiness), and conversely, HR
And Resp show a decreasing trend.

On the other hand, a plurality of drivers D (subject: Subj.1,
12), the driver D
Sleepiness expression, eg driver D evaluated by a third party
When the behavioral characteristics (behavior) of the driver D that occurs when the expression of the drowsiness is high are examined from the sleepiness of the driver D and the subjective evaluation MWS of the drowsiness by the driver D itself, for example, as shown in FIG. Behavioral features that cause significant changes. That is, when the drowsiness increases, the plurality of drivers D share
Not only does a noticeable change in the blinking occur, but the movement of the driver D itself tends to be slow. It was also found that the frequency of driving operations such as speed adjustment, mirror check, blinker operation, etc., decreased as drowsiness increased.

In particular, focusing on the behavioral features of the mirror confirmation and the blinker operation at the time of lane change in the simulation experiment results described above, as shown in FIG. It has been found that there is a general tendency that the frequency of movement of the indicated line of sight decreases and the frequency of driving operations such as blinker operation decreases.

Therefore, in the present invention, in order to evaluate the arousal level (drowsiness fluctuation) without contact with the driver D, the present invention pays particular attention to the blinking of the driver and the horizontal movement of the eyeball indicating the line of sight. We studied to improve accuracy and reduce individual differences. Specifically, attention was first focused on long blinks that increase with a decrease in arousal level, and the relationship between the occurrence ratio of long blinks and the arousal level during a predetermined period was examined. In particular, as a pre-process, a blink evaluation time [Ts] for determining a long blink is set as an average blink time peculiar to the driver D at the time of awakening as a reference time [To], and the reference time [To].
5%, 10%, 15%, 2
Each blink evaluation time [Ts] when each is set to 0%
Below, the occurrence ratio [Lrate] of long blinks of the plurality of drivers D for a predetermined time (for example, 5 minutes) is set to [Long5], [Lo
ng10], [Long15], and [Long20], respectively, and examined their relationship with arousal level.

That is, the occurrence ratio of the long blink [Long5],
For [Long10], [Long15], and [Long20], the sample center time is defined as a unit of delay (Lag) given as a period of, for example, 60 seconds, and the Lag is calculated on a moving average basis.
A relational analysis with drowsiness (degree of arousal) was performed while shifting the value by one unit, and a relational coefficient R under the condition where the relational coefficient was maximized was evaluated. As a result of this evaluation, the relation coefficient R in the occurrence ratio [Long20] becomes smaller than the occurrence ratios [Long5], [Long10], and [Long15], and the long blink evaluation time [Ts] becomes 20% or more. I found that it is better not to set it long. In the analysis of variance related to the Lag value, the ratio of occurrence of long blinks [Long5],
It was confirmed that there was no significant difference between [Long10] and [Long15]. However, each of these indices is data having a physiologically different meaning.

This evaluation is performed by multiple regression analysis using the sleepiness prediction model described above, and a prediction equation y = a + b · Lrate showing the relationship between the degree of arousal and the occurrence ratio of long blinks [Lrate (%)]. Akaike's information amount represented by AIC = 2n logeσ + 2p (where n is the number of data and p is the number of regression coefficients) According to the reference AIC, the prediction was performed by obtaining a prediction model that minimizes the information amount reference AIC.

In the analysis of variance, each index (the ratio of occurrence of long blinks) was determined as an average for 300 seconds, and the expression of drowsiness was determined as an average for 60 seconds. Therefore, the result of the analysis of variance indicates that the time characteristic of sleepiness prediction is, for example, 120 seconds to 180 seconds as an average of the index from 0 seconds to 300 seconds.
This means that the average of drowsiness at the time point of seconds is predicted, and the predicted delay is 120 seconds. Therefore, when the Lag value indicated by the cycle of 60 seconds is given by [−1], actually when the index is obtained after 300 seconds, 2
It means that you predict sleepiness at the time of 40 seconds,
The predicted delay is shown to be 60 seconds. However, since it is less than half of the average period of actual sleepiness of 150 seconds,
Practically, the change in drowsiness can be predicted ahead of 90 seconds on average.

On the other hand, as a behavioral feature which is another index different from blinking, attention is paid to information on the movement of the line of sight such as the gaze of the mirror, that is, the information on the horizontal movement of the eyeball. Estimation of arousal level using ratios was also studied. Specifically, the rate of occurrence of long blinks [Lrate],
In particular, [Long10] and the standard deviation of the horizontal movement of the eyeball [EM
H-SD], a predictive model of the relationship with the arousal level (drowsiness) for the two types of indices Y = A is obtained from sample data from a plurality of drivers D obtained by simulation. + B · Lrate + C · EMH-SD An approximation was made by the linear expression, and the average of the prediction coefficients A, B, and C when the relationship became the highest and the relationship coefficient R at that time was obtained. Then, according to Akaike's information criterion AIC, a prediction model that minimizes the information criterion AIC was evaluated. As a result, when the prediction coefficients A, B, and C are given as A = 2.812, B = 0.032, and C = −0.695, that is, the prediction equation is expressed as Y = 2.812 + 0.032. Lrate-0.695
When given as EMH-SD, the occurrence ratio of long blinks described above [Long
10], it was confirmed that a higher relationship with the arousal level was obtained than when only using [10].

Specifically, the occurrence ratio of long blinks [Long10]
In the prediction of arousal level based on the prediction formula using only
The average of the multiple relation coefficients was [0.71]. On the other hand, as described above, the occurrence ratio of long blinks [Long10]
When the arousal level is predicted under the prediction formula obtained by adding the standard deviation [EMH-SD] of the horizontal movement of the eyeball, the average of the multiple relation coefficient is [0.75], and the accuracy of the prediction model is improved. That was confirmed. Akaike's information criterion AI
It was confirmed that the value of C was also improved from [-134.3] to [-138.7].

FIGS. 7 and 8 show measured values of drowsiness obtained from drowsiness expressions of a plurality of drivers D (subjects: Subj. 1 to 12) and the occurrence ratio [Lo] of the long blink described above.
ng10], and a drowsiness calculated according to the prediction formula based on the long blink occurrence ratio [Long10] and the standard deviation of the horizontal movement of the eyeball [EMH-SD]. It is shown as a predicted value [Y]. In FIGS. 7 and 8, the characteristics indicated by thick lines indicate predicted drowsiness values [y] and [Y], and the characteristics indicated by thin lines indicate actual changes in drowsiness.

As shown in FIGS. 7 and 8, respectively, the drowsiness predicted value [y],
[Y] is the drowsiness (arousal level) that precedes the actual drowsiness
Can be said to be well predicted, and moreover, the drowsiness is predicted by absorbing the individuality of the driver D almost satisfactorily. Also, as shown in FIG. 8, it is shown that calculating the sleepiness prediction value in accordance with the prediction model using the standard deviation [EMH-SD] of the horizontal movement of the eyeball of the driver D can improve the estimation accuracy. .

Accordingly, the occurrence ratio [L
According to this device, which predicts a decrease in arousal level (drowsiness) using not only the rate] but also other behavioral characteristics, specifically, the standard deviation of horizontal movement of the eyeball [EMH-SD]. It is possible to determine a decrease in the awakening degree of the driver D with high prediction accuracy and effectively absorbing various personalities.

In particular, as described above, the occurrence ratio of long blinks [L
rate] and the standard deviation [EMH-SD] of the horizontal movement of the eyeball indicating the behavioral feature, and also easily calculates the drowsiness predicted value [Y] with high accuracy according to a prediction formula represented by a linear expression. it can. In addition, the processing load is light, and the system configuration can be simplified. Further, since it is only necessary to perform an operation in accordance with the prediction equation represented by the linear equation, the processing speed can be increased, and the time delay of the detection can be reduced.

Further, the prediction accuracy is improved while absorbing the individual differences, and further, the processing is simplified and the processing speed is increased.
Further, the system configuration can be simplified and the cost can be reduced. Therefore, there is a great advantage in practical use as an awakening degree estimation device mounted on each vehicle. Note that the present invention is not limited to the above embodiment. For example, the increase rate [r%] with respect to the reference time [To] for determining the long blink evaluation time [Ts] may be set based on more simulation results and the like. That is, the blink evaluation time [Ts] may be set so as to obtain a higher relationship. In addition to the horizontal movement of the eyeball, it is of course possible to construct a prediction model by introducing information indicating other behavioral characteristics, such as the above-described forward tilt and backward tilt of the driver D's head. It is also possible to operate the vehicle's brake mechanism to decelerate, or to activate an automatic driving mode based on recognition of white lines on the road or control of the distance between other vehicles using the estimated result of the arousal level decrease. . In addition, the present invention can be variously modified and implemented without departing from the gist thereof.

[0045]

As described above, according to the present invention, a blink of a driver is detected, and a blink longer than a predetermined time is detected based on the detected time of the blink. Along with a long blink occurrence ratio calculating means for obtaining a long blink occurrence ratio from the total number of blinks and the long blink times, a behavior detecting means for detecting a behavior indicating the behavioral characteristics of the driver is provided. The arousal level determination means evaluates the arousal level of the driver according to a prediction formula obtained by a regression analysis using a sleepiness prediction model, based on the occurrence ratio of long blinks and the behavior information of the driver.

In particular, a long blink is detected from the blink time of the driver in accordance with the reference time [To] of the blink specific to the driver.
The occurrence ratio [Lrate] of a long blink with respect to the total number of blinks within a predetermined time is obtained, and information on the behavior indicating the behavioral characteristics of the driver is, for example, a standard deviation [EMH- SD], and a prediction formula Y = A + B [Lrate] + C [EMH-SD] obtained by a regression analysis using a drowsiness prediction model, for example.
(A, B, and C are prediction coefficients), and a sleepiness prediction value [Y] is calculated to evaluate the arousal level.

Accordingly, it is possible to simply and highly accurately detect the driver's arousal level and, consequently, the decrease in the arousal level, and to effectively absorb individual differences and estimate the arousal level with high precision. it can. In particular, a drowsiness prediction value is calculated based on a prediction formula indicating a relationship between a degree of arousal obtained by a regression analysis using a drowsiness prediction model and a rate of occurrence of long blinks, and the drowsiness prediction value is discriminated by a predetermined threshold, and the awakening degree is determined. Therefore, practically significant effects can be obtained, for example, the system configuration can be simplified and the processing speed can be increased while sufficiently increasing the estimation accuracy.

[Brief description of the drawings]

FIG. 1 is a view conceptually showing a configuration of a vehicle equipped with a waking degree estimation device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a functional block configuration of a waking degree estimation device shown in FIG. 1;

FIG. 3 is a diagram showing a flow of a wakefulness estimation process in the wakefulness estimation device shown in FIG. 2;

FIG. 4 is a diagram showing a simulation result of a relationship between a plurality of indices of arousal degree having different physical meanings and objectively evaluated drowsiness.

[FIG. 5] Based on simulation results of a plurality of drivers (subjects: Subj. 1 to 12), a plurality of arousal indices (behavioral characteristics) having different physical meanings and objectively evaluated. The figure which shows the case where a strong relationship is recognized with drowsiness.

FIG. 6 is a diagram showing changes in behavioral characteristics including lane change, mirror confirmation, and blinker operation with the elapse of driving time by simulation.

FIG. 7 is a diagram showing a comparison between a measured drowsiness value of a plurality of drivers D (subjects: Subj. 1 to 12) and a drowsiness prediction value obtained from the occurrence ratio of long blinks.

FIG. 8 shows a comparison between a measured drowsiness value of a plurality of drivers D (subjects: Subj. 1 to 12) and a drowsiness prediction value obtained from a long blink and horizontal movement of an eyeball. FIG.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Vehicle 2 TV camera 3 Display 4 Speaker 10 Image processing unit 11 Blink detection unit 12 Blink time calculation unit 14 Blink reference time calculation unit 15 Long blink time setting unit 16 Long blink detection unit 17 Long blink occurrence ratio calculation unit 21 Eye movement Detection unit 22 Standard deviation calculation unit 23 Arousal level prediction calculation unit 24 Arousal level determination unit

Claims (3)

[Claims] 1. A blink detecting means for detecting a blink of a driver, a means for detecting a blink longer than a predetermined time based on a time of the detected blink, a total number of blinks and a long blink within a predetermined evaluation period. An occurrence ratio calculating means for calculating an occurrence ratio of a long blink from the number of times, a behavior detecting means for detecting the behavior of the driver, and a drowsiness prediction model based on the occurrence ratio of the long blink and the behavior information of the driver. A vigilance estimating device comprising: a vigilance determination means for evaluating the vigilance of the driver according to a prediction formula obtained by a regression analysis using the arousal level. 2. The method according to claim 1, wherein the behavior detecting means obtains a standard deviation [EMH-SD] of eye movement indicating a driver's line of sight movement, and the arousal degree determining means calculates awakening degree [Y] and a long blink. [Lrate] and the standard deviation of eye movement [EMH]
-SD], which is a prediction formula obtained from regression analysis of a drowsiness prediction model showing a relationship with Y-A + B [Lrate] + C [EMH-SD] (A,
The sleepiness prediction value [Y] is calculated based on (B, C are prediction coefficients), and the sleepiness prediction value is discriminated by a predetermined threshold value to evaluate the waking degree, and the waking degree estimation according to claim 1, wherein apparatus. 3. The long blink detecting means obtains a reference time [To] of a blink specific to the driver from a blink time of the driver at the time of awakening, and determines the reference time to a predetermined ratio [r].
The alertness estimating apparatus according to claim 1, wherein a blink exceeding a time [Ts] increased by [%] is detected as a long blink.