WO2024087205A1 - 司机状态评估方法、装置、电子设备及存储介质 - Google Patents

司机状态评估方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2024087205A1
WO2024087205A1 PCT/CN2022/128380 CN2022128380W WO2024087205A1 WO 2024087205 A1 WO2024087205 A1 WO 2024087205A1 CN 2022128380 W CN2022128380 W CN 2022128380W WO 2024087205 A1 WO2024087205 A1 WO 2024087205A1
Authority
WO
WIPO (PCT)
Prior art keywords
driver
information
analysis
fusion
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/128380
Other languages
English (en)
French (fr)
Inventor
韩永刚
顾子贤
王玉玲
何堃
覃海传
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Streamax Technology Co Ltd
Original Assignee
Streamax Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Streamax Technology Co Ltd filed Critical Streamax Technology Co Ltd
Priority to CN202280062260.4A priority Critical patent/CN118119543A/zh
Priority to PCT/CN2022/128380 priority patent/WO2024087205A1/zh
Priority to EP22963186.6A priority patent/EP4495833A4/en
Publication of WO2024087205A1 publication Critical patent/WO2024087205A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W40/09—Driving style or behaviour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/90—Single sensor for two or more measurements
    • B60W2420/905—Single sensor for two or more measurements the sensor being an xyz axis sensor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/10—Longitudinal speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/10—Longitudinal speed
    • B60W2520/105—Longitudinal acceleration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/12—Lateral speed
    • B60W2520/125—Lateral acceleration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/043—Identity of occupants
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/221—Physiology, e.g. weight, heartbeat, health or special needs
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/40—Dynamic objects, e.g. animals, windblown objects
    • B60W2554/406—Traffic density
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/80—Spatial relation or speed relative to objects
    • B60W2554/802—Longitudinal distance
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/10—Historical data
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/45—External transmission of data to or from the vehicle
    • B60W2556/50—External transmission of data to or from the vehicle of positioning data, e.g. GPS [Global Positioning System] data

Definitions

  • the present application relates to the field of vehicle driving technology, and in particular to a driver status assessment method, device, electronic device and storage medium.
  • the driver's driving state has a very serious impact on safe driving. Therefore, the driver should be kept in a good driving state as much as possible.
  • the existing driver status assessment methods usually judge whether the driver is tired by the length of time the eyes are closed, judge whether the driver is distracted by the angle of the face, and judge whether the driver is in a dangerous state by single behavioral events such as sudden acceleration, sudden deceleration, and sharp turns.
  • many drivers do not necessarily close their eyes when they are tired.
  • the driver wears sunglasses or the strong light reflection of the glasses makes the eye state invisible.
  • the strong light irradiation makes the driver squint and look like closing his eyes.
  • an embodiment of the present application provides a data processing system and a driver status assessment method and a data processing method thereof, aiming to at least solve the problem of low assessment accuracy in the prior art.
  • a first aspect of an embodiment of the present application provides a driver status assessment method, comprising: performing a behavior analysis on the driver based on the driver's current driving behavior data and mental state data, obtaining the driver's behavior events, and generating time series event information based on the behavior events; performing a driving habit analysis on the driver based on the driver's normal driving behavior data in a current time period, and generating the driver's driving habit information in the current time period; calling a driver portrait library based on the driver's identity information to obtain the driver portrait information of the driver; using a preset information fusion analysis model to perform information fusion analysis on the time series event information, the driving habit information, and the driver portrait information, and evaluating the driver's status based on the analysis results to obtain the driver's status assessment result.
  • a preset information fusion analysis model is used to perform information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and the driver's status is evaluated based on the analysis results to obtain the driver's status evaluation result, including: using a shallow fusion network in the information fusion analysis model to perform early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate an early fusion analysis vector; using a deep fusion network in the information fusion analysis model to perform late fusion analysis on the early fusion analysis vector and the driver portrait information to generate a late fusion analysis vector; evaluating the driver's status according to the late fusion analysis vector to obtain the driver's status evaluation result, wherein the driver's status evaluation result includes one or more of fatigue level information, distraction information and driving risk level information.
  • a shallow fusion network in the information fusion analysis model is used to perform early fusion analysis on the time series event information, the driving habit information, and the driver portrait information to generate an early fusion analysis vector, comprising: inputting the time series event information as an input vector into the shallow fusion network, and using the behavioral event analysis channel of the shallow fusion network to perform multi-head self-attention analysis processing, residual connection processing, and normalization processing on the time series event information to generate a first analysis vector; inputting the driving habit information as an input vector into the shallow fusion network, and using the shallow The driving habit analysis channel of the shallow fusion network performs multi-head self-attention analysis, residual connection and standardization on the driving habit information to generate a second analysis vector; the driver portrait information is input into the shallow fusion network as an input vector, and the driver portrait analysis channel of the shallow fusion network is used to perform multi-head self-attention analysis, residual connection, standardization, collision detection and logistic regression on the driver portrait
  • the deep fusion network in the information fusion analysis model is used to perform late fusion analysis on the early fusion analysis vector and the driver portrait information to generate a late fusion analysis vector, comprising: inputting the early fusion analysis vector as an input vector into the deep fusion network, and using the fusion result analysis channel in the deep fusion network to perform multi-head self-attention analysis processing, residual connection processing, standardization processing and connection processing on the early fusion analysis vector to generate a fourth analysis vector; inputting the driver portrait information as an input vector into the deep fusion network, and using the driver portrait analysis channel in the deep fusion network to perform multi-head self-attention analysis processing, residual connection processing, standardization processing, collision detection processing and logistic regression processing on the driver portrait information to generate a fifth analysis vector; performing dot product processing on the fourth analysis vector and the fifth analysis vector to obtain a late fusion analysis vector.
  • a behavior analysis is performed on the driver based on the driver's current driving behavior data and mental state data, the driver's behavior events are obtained, and time series event information is generated based on the behavior events, wherein the time series event information includes the following information items: event category information, event start time information, event end time information, event start speed information, event end speed information, relative collision minimum time information, minimum collision distance information, collision cumulative time information, collision duration information, traffic flow information, vehicle position information, and vehicle driving time information.
  • the fifth possible implementation method of the first aspect after the steps of performing behavior analysis on the driver based on the driver's current driving behavior data and mental state data, obtaining the driver's behavior events, and generating time series event information based on the behavior events, it also includes: obtaining real-time weather status information under the current driving scenario, and adding the real-time weather status information as an information item to the time series event information.
  • the sixth possible implementation method of the first aspect after the steps of performing behavior analysis on the driver based on the driver's current driving behavior data and mental state data, obtaining the driver's behavior events, and generating time series event information based on the behavior events, it also includes: obtaining real-time road information in the current driving scenario, and adding the real-time road information as an information item to the time series event information.
  • the second aspect of an embodiment of the present application provides a driver status assessment device, including: a first generation module, used to perform a behavior analysis on the driver based on the driver's current driving behavior data and mental state data, obtain the driver's behavior events, and generate time series event information based on the behavior events; a second generation module, used to perform a driving habit analysis on the driver based on the driver's normal driving behavior data in the current time period, and generate the driver's driving habit information in the current time period; an acquisition module, used to call a driver portrait library based on the driver's identity information to obtain the driver portrait information of the driver; an assessment module, used to perform an information fusion analysis on the time series event information, the driving habit information and the driver portrait information using a preset information fusion analysis model, and evaluate the driver's status based on the analysis result to obtain the driver's status assessment result.
  • a driver status assessment device including: a first generation module, used to perform a behavior analysis on the driver based on the driver's current driving behavior data and mental
  • a third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.
  • a fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.
  • the embodiments of the present application have the following beneficial effects: by obtaining the time series event information corresponding to the driver's current behavior event, the driver's driving habit information in the current time period, and the driver's driver portrait information, and then using a preset information fusion analysis model to perform information fusion analysis on the time series event information, driving habit information, and driver portrait information to obtain an analysis result, and then evaluating the driver's status based on the analysis result to obtain the driver's status evaluation result, combining a series of time series event features and driver portraits to judge the driver's status, thereby improving the accuracy of driver fatigue status recognition, distraction status recognition, and dangerous driving recognition.
  • FIG1 is a basic method flow chart of a driver status assessment method provided by an embodiment of the present application.
  • FIG2 is a flow chart of a method for performing information fusion analysis in a driver status assessment method provided in an embodiment of the present application
  • FIG3 is a schematic diagram of a structure of an information fusion analysis model in a driver status assessment method provided in an embodiment of the present application
  • FIG4 is a flow chart of a method for performing early fusion analysis in a driver status assessment method provided in an embodiment of the present application
  • FIG5 is a schematic diagram of a structure of a shallow fusion network in a driver status assessment method provided in an embodiment of the present application
  • FIG6 is a flow chart of a method for performing late fusion analysis in the driver status assessment method provided in an embodiment of the present application
  • FIG7 is a schematic diagram of a structure of a deep fusion network in a driver status assessment method provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of the structure of a driver status assessment device provided in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an electronic device provided in an embodiment of the present application.
  • FIG1 is a basic method flow chart of a driver status assessment method provided by an embodiment of the present application. The details are as follows:
  • S11 Analyze the driver's behavior based on his current driving scene data, driving behavior data, and mental state data, obtain the driver's behavior events, and generate time series event information based on the behavior events.
  • the driver's current driving scene data, driving behavior data and mental state data can be obtained through data collection by cameras and sensors installed on the vehicle.
  • the vehicle's built-in advanced driving assistance system ADAS, Advanced Driving Assistance System
  • ADAS Advanced Driving Assistance System
  • the vehicle's built-in indoor digital system can use the DIS camera to capture the driver's face to obtain a facial image, and calculate the driver's facial position, facial key points, driver's expression, driver's eye opening and closing state and other mental state data based on the facial information displayed in the facial image.
  • DIS DIS Digital Indoor System
  • the vehicle's motion state can be detected by sensors such as six-axis sensors and GPS sensors installed on the vehicle to obtain driving behavior data such as sudden acceleration, sudden deceleration, sharp turns, and vehicle speed.
  • driving behavior data such as sudden acceleration, sudden deceleration, sharp turns, and vehicle speed.
  • the driver status evaluation system when the driver's behavior is analyzed based on the driving scene data, driving behavior data and mental state data, the obtained data can be respectively input into the relevant event confirmation algorithm for event confirmation processing, and the event confirmation algorithm confirms the driver's behavior event in the current state. After confirming the behavior event, the corresponding time series event information is generated according to the confirmed behavior event. It can be understood that the driver's behavior events can be divided into several types.
  • 69 types of behavior events are included, which include lane keeping events, front collision events, eye closing events, down-glancing events, yawning events, acceleration events, deceleration events, turning events, rapid acceleration events, rapid deceleration events, and sharp turning events.
  • a corresponding event confirmation algorithm can be configured for each behavior event type in the driver status evaluation system.
  • the input data of each event confirmation algorithm is different.
  • the corresponding input data can be obtained from these driving scene data, driving behavior data and mental state data according to the input requirements of each event confirmation algorithm, so that each event confirmation algorithm can perform behavior analysis according to its input data to confirm whether the driver has executed the behavior event corresponding to the event confirmation algorithm in the current state. If the event confirmation algorithm confirms that the driver has executed the corresponding behavior event, the corresponding time series event information is generated according to the corresponding behavior event.
  • the time series event information includes the following information items: event category information, event start time information, event end time information, event start speed information, event end speed information, relative collision minimum time information, minimum collision distance information, collision cumulative time information, collision duration information, traffic flow information, vehicle position information, and vehicle driving time information.
  • a weather information acquisition module may be configured in the driver status evaluation system, and the real-time weather status information in the current driving scene, such as rainy days, snowy days, foggy days, sunny days, etc., may be acquired through the weather information acquisition module.
  • the real-time weather status information is fused with the time series event information, and the real-time weather status information is added as an information item to the time series event information. In this way, the weather status is added as an influencing factor when evaluating the driver's status, thereby improving the accuracy of the evaluation result.
  • a map module may be configured in the driver status evaluation system to obtain road information in the current driving scenario, such as traffic lights, intersections, highways, corridors, provincial roads, etc. After obtaining the road information, the road information is fused with the time series event information, and the road information is added as an information item to the time series event information. In this way, the road condition is added as an influencing factor when evaluating the driver's status, thereby improving the accuracy of the evaluation results.
  • S12 Analyze the driver's driving habits according to the driver's normal driving behavior data in the current time period, and generate the driver's driving habit information in the current time period.
  • the driving behavior data generated within a time range closest to the current moment can be used as the normal driving behavior data of the driver in the current time period.
  • the normal driving behavior data is generated regularly according to the set time interval, for example, once per second, and the normal driving behavior data obtained is time series data.
  • the normal driving behavior data includes vehicle speed data, traffic data, position data, six-axis data, etc., wherein the traffic data includes the number of vehicles in front and the distance of the vehicle in front from the vehicle.
  • the driver's driving habits can be analyzed according to the normal driving behavior data, the driver's real-time driving rules can be counted, and the driving habit information of the driver in the current time period can be generated.
  • the driving habit information of the driver in the current time period includes the driver's following habit in the current time period, the driver's lane keeping habit in the current event segment, and the driver's speed keeping habit in the current time period.
  • a driver portrait can be constructed based on the driver's historical driving data and stored in a driver portrait library.
  • the driver portrait library includes a large number of driver portraits, and a corresponding relationship between the driver portrait and the driver's identity information is established.
  • the driver portrait information of the driver can be obtained by calling the driver portrait library according to the driver's identity information.
  • the driver portrait information includes information such as the effective time period of the driver's eyes closing, the time period of closing and false alarm, the type of false alarm of closing eyes, etc.
  • S14 Using a preset information fusion analysis model to perform information fusion analysis on the time series event information, the driving habit information and the driver portrait information, and evaluating the driver's status based on the analysis results to obtain the driver's status evaluation result.
  • the preset information fusion analysis model is obtained by training the neural network through sample data.
  • the information fusion analysis model is trained to perform information fusion analysis based on time series event information, driving habit information and driver portrait information, evaluate the driver's status according to the analysis results, and obtain the driver's status evaluation result.
  • the information fusion analysis model can use the event stream as input based on the time series event information, and confirm the event through the event stream. When it is confirmed that some of the driver's behavioral events are abnormal, the driver's status can be accurately evaluated, which can greatly reduce the false alarm rate.
  • the information fusion analysis model combines driving habit information and driver portrait information for information fusion analysis, and can use the driver portrait for multi-layer constraints, perceive the driver's status of thousands of people, effectively solve the false alarm caused by the personalized differences of different drivers, and improve the accuracy of the perception of thousands of people.
  • the information fusion analysis model can evaluate the driver's status according to the analysis results, it can effectively improve the accuracy and timeliness of the driver's status evaluation by classifying the driver's status and grading the driver's fatigue and driving risk.
  • the driver status assessment method can obtain the time series event information corresponding to the driver's current behavior event, the driver's driving habit information in the current time period, and the driver's driver portrait information, and then use a preset information fusion analysis model to perform information fusion analysis on the time series event information, driving habit information and driver portrait information to obtain an analysis result, and then evaluate the driver's status based on the analysis result to obtain the driver's status assessment result, which combines a series of time series event features and driver portraits to judge the driver's status, thereby improving the accuracy of driver fatigue status recognition, distraction status recognition and dangerous driving recognition.
  • FIG. 2 is a flow chart of a method for performing information fusion analysis in the driver status assessment method provided in an embodiment of the present application, as follows:
  • S23 Evaluate the driver's state according to the late fusion analysis vector to obtain a state evaluation result of the driver, wherein the driver's state evaluation result includes one or more of fatigue level information, distraction information and driving risk level information.
  • FIG. 3 is a structural diagram of an information fusion analysis model in a driver state assessment method provided in an embodiment of the present application.
  • the information fusion analysis model includes a shallow fusion network and a deep fusion network.
  • the shallow fusion network can be built based on the behavior encoder framework of the transformer model.
  • the information fusion analysis of the time series event information, the driving habit information and the driver portrait information is performed using a preset information fusion analysis model, the time series event information, the driving habit information and the driver portrait information can be input into the shallow fusion network together, and the shallow fusion network analyzes the early fusion analysis vector based on the time series event information, the driving habit information and the driver portrait information.
  • the early fusion analysis vector obtained by the shallow fusion network analysis is input into the deep fusion network
  • the driver portrait information is also input into the deep fusion network while the early fusion analysis vector is input into the deep fusion network
  • the late fusion analysis vector is analyzed by the deep fusion network based on the early fusion analysis vector and the driver portrait information.
  • the driver's state can be classified in the information fusion analysis model, and each state category has a corresponding feature vector.
  • the vector similarity comparison can be performed based on the late fusion analysis vector and the feature vectors corresponding to various state categories, so as to determine the driver's state according to the vector similarity, including determining the driver's fatigue level, determining whether the driver is analyzed, and determining the driving risk level.
  • the driver's state is evaluated and the driver's state evaluation result is obtained.
  • the driver's state evaluation result includes one or more of the fatigue level information, distraction information and driving risk level information.
  • the information fusion analysis model when training the information fusion analysis model, it is possible to judge whether the driver's state is fatigue in the shallow fusion network and the deep fusion network respectively, so as to obtain two pre-trained networks through training, and then train the information fusion analysis model by combining the two pre-trained networks, which can effectively reduce the amount of sample data required for training the information fusion analysis model.
  • FIG. 4 is a flow chart of a method for performing early fusion analysis in the driver status evaluation method provided in an embodiment of the present application, as detailed as follows:
  • FIG. 5 is a schematic diagram of the structure of a shallow fusion network in the driver state assessment method provided in the embodiment of the present application.
  • the shallow fusion network contains three analysis channels, namely, a behavior event analysis channel, a driving habit analysis channel, and a driver portrait analysis channel.
  • a multi-head self-attention analysis layer Multi-Head Self Attention
  • An&Norm residual connection and normalization processing layer
  • the time series event information can be input as an input vector into the shallow fusion network, and the time series event information can be processed by multi-head self-attention analysis, residual connection processing and normalization processing by using the behavior event analysis channel of the shallow fusion network to generate a first analysis vector.
  • the driving habit analysis channel of the shallow fusion network can be used to perform multi-head self-attention analysis, residual connection processing and normalization processing on the driving habit information, and a second analysis vector can be generated.
  • multiple multi-head self-attention analysis, residual connection and normalization processing can be performed on the time series event information and driving habit information according to the actual analysis requirements to generate the first analysis vector and the second analysis vector.
  • the driver portrait analysis channel is provided with a multi-head self-attention analysis layer (Multi-Head Self Attention), a residual connection and normalization processing layer (Add&Norm), a collision detection layer (FCL) and a logistic regression processing layer (softmax).
  • Multi-Head Self Attention Multi-Head Self Attention
  • Add&Norm residual connection and normalization processing layer
  • FCL collision detection layer
  • softmax logistic regression processing layer
  • the driver portrait information is input as an input vector into the shallow fusion network, and the driver portrait information is subjected to multi-head self-attention analysis, residual connection, normalization, collision detection and logistic regression processing by using the driver portrait analysis channel of the shallow fusion network to generate the third analysis vector.
  • the first analysis vector and the second analysis vector can be connected by the connection function (concat) to obtain the connection vector. Then, the connection vector and the third analysis vector are subjected to dot product processing (Dot-product) to obtain the early fusion analysis vector.
  • the driver portrait analysis channel can perform multiple self-attention analysis, residual connection processing and normalization processing on the driver portrait information according to actual analysis requirements.
  • the shallow fusion network performs early fusion analysis on the time series event information, driving habit information and driver portrait information, which can constrain the local features of the network, so that the output analysis vector focuses more on key behavior events and filters out weakly correlated behavior events.
  • FIG. 6 is a flow chart of a method for performing late fusion analysis in the driver status evaluation method provided in an embodiment of the present application, as detailed as follows:
  • S63 Perform a dot product process on the fourth analysis vector and the fifth analysis vector to obtain a late fusion analysis vector.
  • FIG. 7 is a structural schematic diagram of a deep fusion network in the driver state assessment method provided by the embodiment of the present application.
  • the deep fusion network contains two analysis channels, namely an early fusion result analysis channel and a driver portrait analysis channel.
  • the early fusion result analysis channel is provided with a multi-head self-attention analysis layer (Multi-Head Self Attention), a residual connection and normalization processing layer (Add&Norm), and a connection processing layer (concat).
  • Multi-Head Self Attention multi-head self-attention analysis layer
  • Add&Norm residual connection and normalization processing layer
  • concat connection processing layer
  • the early fusion analysis vector output by the shallow fusion network is input into the deep fusion network as the input vector of the deep fusion network, and the fusion result analysis channel in the deep fusion network is used to perform multi-head self-attention analysis processing, residual connection processing, normalization processing and connection processing on the early fusion analysis vector, so as to generate a fourth analysis vector.
  • the driver portrait analysis channel is provided with a multi-head self-attention analysis layer (Multi-Head Self Attention), a residual connection and normalization processing layer (Add&Norm), a collision detection layer (FCL) and a logistic regression processing layer (softmax).
  • the driver portrait information is once again input as an input vector into the deep fusion network.
  • the driver portrait information is subjected to multi-head self-attention analysis, residual connection, standardization, collision detection and logistic regression processing by using the driver portrait analysis channel in the deep fusion network to generate the fifth analysis vector.
  • the late fusion analysis vector can be obtained by performing dot product processing (Dot-product) on the fourth analysis vector and the fifth analysis vector.
  • Dot-product dot product processing
  • late fusion through the deep fusion network can optimize the weight distribution of each parameter of the network on a global basis.
  • the early fusion analysis of the shallow fusion network combined with the late fusion analysis of the deep fusion network can effectively solve the problem of false alarms caused by individual differences in drivers, and improve the accuracy of the perception of drivers.
  • FIG8 is a schematic diagram of the structure of a driver status evaluation device provided in an embodiment of the present application. The details are as follows:
  • the driver status evaluation device includes: a first generation module 81, a second generation module 82, an acquisition module 83 and an evaluation module 84.
  • the first generation module 81 is used to analyze the driver's behavior according to the driver's current driving scene data, driving behavior data and mental state data, obtain the driver's behavior events, and generate time series event information according to the behavior events.
  • the second generation module 82 is used to analyze the driver's driving habits according to the driver's normal driving behavior data in the current time period, and generate the driver's driving habit information in the current time period.
  • the acquisition module 83 is used to call the driver portrait library according to the driver's identity information to obtain the driver portrait information of the driver.
  • the evaluation module 84 is used to use a preset information fusion analysis model to perform information fusion analysis on the time series event information, the driving habit information and the driver portrait information, and evaluate the driver's state according to the analysis results to obtain the driver's state evaluation result.
  • the driver status evaluation device corresponds one-to-one to the above-mentioned driver status evaluation method, and will not be described in detail here.
  • Figure 9 is a schematic diagram of an electronic device provided in an embodiment of the present application.
  • the electronic device can be a vehicle-mounted device or a cloud device, and the communication unit is used to send and receive data and signaling.
  • the electronic device 9 of this embodiment includes: a processor 91, a memory 92, and a computer program 93 stored in the memory 92 and executable on the processor 91, such as a driver status assessment program or a data processing program.
  • the processor 91 executes the computer program 92, the steps in the above-mentioned driver status assessment method embodiments or data processing methods are implemented.
  • the processor 91 executes the computer program 93, the functions of each module/unit in the above-mentioned device embodiments are implemented.
  • the computer program 93 may be divided into one or more modules/units, which are stored in the memory 92 and executed by the processor 91 to complete the present application.
  • the one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 93 in the electronic device 9.
  • the computer program 93 may be divided into a first generation module, a second generation module, an acquisition module, and an evaluation module, and the specific functions of each module are as described above.
  • the electronic device may include, but is not limited to, a processor 91 and a memory 92.
  • FIG9 is merely an example of the electronic device 9 and does not limit the electronic device 9.
  • the electronic device may include more or fewer components than shown in the figure, or may combine certain components, or different components.
  • the electronic device may also include an input/output device, a network access device, a bus, etc.
  • the processor 91 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or any conventional processor, etc.
  • the memory 92 may be an internal storage unit of the electronic device 9, such as a hard disk or memory of the electronic device 9.
  • the memory 92 may also be an external storage device of the electronic device 9, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 9.
  • the memory 92 may also include both an internal storage unit of the electronic device 9 and an external storage device.
  • the memory 92 is used to store the computer program and other programs and data required by the electronic device.
  • the memory 92 may also be used to temporarily store data that has been output or is to be output.
  • the technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration.
  • the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
  • the functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.
  • the disclosed devices/terminal equipment and methods can be implemented in other ways.
  • the device/terminal equipment embodiments described above are only schematic.
  • the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
  • the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program.
  • the computer program can be stored in a computer-readable storage medium.
  • the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form.
  • the computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electric carrier signal telecommunication signal and software distribution medium.
  • the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.
  • computer-readable media do not include electric carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

本申请提供一种司机状态评估方法、装置、电子设备及存储介质，其方法包括：根据司机当前的驾驶行为数据和精神状态数据对司机进行行为分析，获取司机的行为事件，并根据行为事件生成时序事件信息；根据司机当前时间段的常态驾驶行为数据对司机进行驾驶习惯分析，生成司机当前时间段的驾驶习惯信息；根据司机的身份信息调用司机画像库，获取司机的司机画像信息；采用预设的信息融合分析模型对时序事件信息、驾驶习惯信息和司机画像信息进行信息融合分析，并根据分析结果评估司机的状态，获得司机的状态评估结果。该方法结合一系列时序事件特征和司机画像综合来判断司机的状态，提高了司机疲劳状态识别、分心状态识别和危险驾驶识别的准确率。

Description

司机状态评估方法、装置、电子设备及存储介质 技术领域
本申请涉及车辆驾驶技术领域，尤其涉及一种司机状态评估方法、装置、电子设备及存储介质。
 背景技术
随着车辆的不断普及，交通事故也随之增多。驾驶员的驾驶状态对安全行车的影响非常严重，因此，应尽可能的使驾驶员处于良好的驾驶状态。司机在驾驶过程中，若司机因疲劳过度、注意力分散等问题导致驾驶状态较差时，可能导致判断能力下降，极易发生道路交通事故。目前，现有的司机状态评估方法通常通过闭眼时长判断是否疲劳、通过人脸朝向角度判断是否分心，通过急加速、急减速、急转弯等单一行为事件判断是否处于危险状态。然而，很多司机疲劳时不一定会闭眼，司机戴墨镜或者眼镜强光反射导致眼睛状态不可见，强光照射导致司机眯眼看着像闭眼，这些场景都很难准确地判定司机是否疲劳，无法对疲劳进行有效分级。人脸朝向角度容易受制于安装角度、安装距离的影响，摄像头安装距离较远导致瞳孔不可见、受眼镜影响、司机眼晴大小影响，导致视线检测不准确，在真实应用场景中误判、漏判的情况较多。急加速、急减速、急转弯等单一行为事件，虽然有一定概率是由危险驾驶导致的。但是并非和危险驾驶不是充分必要关系。有些驾驶场景下即使出现急加速、急减速、急转弯等行为，也只是司机的驾驶习惯导致，与危险无关。因此，现有的司机状态评估方法存在评估准确性低的问题。
 技术问题
有鉴于此，本申请实施例提供了一种数据处理系统及其司机状态评估方法和数据处理方法，旨在至少解决现有技术中存在评估准确性低的问题。
 技术解决方案
本申请实施例的第一方面提供了一种司机状态评估方法，包括：根据司机当前的驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息；根据司机当前时间段的常态驾驶行为数据对所述司机进行驾驶习惯分析，生成所述司机当前时间段的驾驶习惯信息；根据所述司机的身份信息调用司机画像库，获取所述司机的司机画像信息；采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果。
结合第一方面，在第一方面的第一种可能实现方式中，采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果的步骤，包括：采用所述信息融合分析模型中的浅层融合网络对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行早融合分析，生成早融合分析向量；采用所述信息融合分析模型中的深度融合网络对所述早融合分析向量和所述司机画像信息进行晚融合分析，生成晚融合分析向量；根据所述晚融合分析向量评估所述司机的状态，获得所述司机的状态评估结果，其中，所述司机的状态评估结果包括疲劳级别信息、分心信息和驾驶风险级别信息中的一项或多项信息。
结合第一方面的第一种可能实现方式，在第一方面的第二种可能实现方式中，采用所述信息融合分析模型中的浅层融合网络对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行早融合分析，生成早融合分析向量的步骤，包括：将所述时序事件信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的行为事件分析通道对所述时序事件信息进行多头自注意力分析处理、残差连接处理和标准化处理，生成第一分析向量；将所述驾驶习惯信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的驾驶习惯分析通道对所述驾驶习惯信息进行多头自注意力分析处理、残差连接处理和标准化处理，生成第二分析向量；将所述司机画像信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的司机画像分析通道对所述司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，生成第三分析向量；将所述第一分析向量和所述第二分析向量进行连接处理，获得连接向量；将所述连接向量与所述第三分析向量进行点积处理，获得早融合分析向量。
结合第一方面的第一或第二种可能实现方式，在第一方面的第三种可能实现方式中，采用所述信息融合分析模型中的深度融合网络对所述早融合分析向量和所述司机画像信息进行晚融合分析，生成晚融合分析向量的步骤，包括：将所述早融合分析向量作为输入向量输入到所述深度融合网络，采用所述深度融合网络中的融合结果分析通道对所述早融合分析向量进行多头自注意力分析处理、残差连接处理、标准化处理和连接处理，生成第四分析向量；将所述司机画像信息作为输入向量输入至所述深度融合网络，采用所述深度融合网络中的司机画像分析通道对所述司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，生成第五分析向量；将所述第四分析向量与所述第五分析向量进行点积处理，获得晚融合分析向量。
结合第一方面，在第一方面的第四种可能实现方式中，根据司机当前的驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息的步骤中，所述时序事件信息包括以下信息项：事件类别信息、事件起始时间信息、事件结束时间信息、事件起始速度信息、事件结束速度信息、相对碰撞最小时间信息、最小碰撞距离信息、碰撞累积时间信息、碰撞持续时间信息、车流信息、车辆位置信息、车辆行驶时长信息。
结合第一方面的第四种可能实现方式，在第一方面的第五种可能实现方式中，根据司机当前的驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息的步骤之后，还包括：获取当前驾驶场景下的实时天气状态信息，将所述实时天气状态信息作为信息项添加至所述时序事件信息中。
结合第一方面的第四或第五种可能实现方式，在第一方面的第六种可能实现方式中，根据司机当前的驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息的步骤之后，还包括：获取当前驾驶场景下的实时道路信息，将所述实时道路信息作为信息项添加至所述时序事件信息中。
本申请实施例的第二方面提供了一种司机状态评估装置，包括：第一生成模块，用于根据司机当前的驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息；第二生成模块，用于根据司机当前时间段的常态驾驶行为数据对所述司机进行驾驶习惯分析，生成所述司机当前时间段的驾驶习惯信息；获取模块，用于根据所述司机的身份信息调用司机画像库，获取所述司机的司机画像信息；评估模块，用于采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果。
本申请实施例的第三方面提供了一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如第一方面任一项所述方法的步骤。
本申请实施例的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如第一方面任一项所述方法的步骤。
 有益效果
本申请实施例与现有技术相比存在的有益效果是：通过获取司机当前行为事件所对应的时序事件信息、司机当前时间段的驾驶习惯信息以及司机的司机画像信息，然后，采用预设的信息融合分析模型对时序事件信息、驾驶习惯信息和司机画像信息进行信息融合分析，得到分析结果，进而根据分析结果来评估司机的状态，获得司机的状态评估结果，结合了一系列的时序事件特征和司机画像来判断司机的状态，提高了司机疲劳状态识别、分心状态识别和危险驾驶识别的准确率。
 附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种司机状态评估方法的基本方法流程图；
图2为本申请实施例提供的司机状态评估方法中进行信息融合分析的一种方法流程图；
图3为本申请实施例提供的司机状态评估方法中信息融合分析模型的一种结构示意图；
图4为本申请实施例提供的司机状态评估方法中进行早融合分析的一种方法流程图；
图5为本申请实施例提供的司机状态评估方法中浅层融合网络的一种结构示意图；
图6为本申请实施例提供的司机状态评估方法中进行晚融合分析的一种方法流程图；
图7为本申请实施例提供的司机状态评估方法中深度融合网络的一种结构示意图；
图8为本申请实施例提供的一种司机状态评估装置的结构示意图；
图9为本申请实施例提供的一种电子设备的示意图。
 本发明的实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
为了说明本申请所述的技术方案，下面通过具体实施例来进行说明。
本申请的一些实施例中，请参阅图1，图1为本申请实施例提供的一种司机状态评估方法的基本方法流程图。详述如下：
S11：根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息。
本实施例中，司机当前的驾驶场景数据、驾驶行为数据和精神状态数据可以通过车辆上安装的摄像头和传感器进行数据采集获得。示例性的，可以通过车辆内置的高级驾驶辅助系统（ADAS，Advanced Driving Assistance System）采用ADAS摄像头对车辆周围环境进行图像拍摄，获取环境图像，根据环境图像展示中周围环境信息计算获得车辆所在的车道位置、周围车辆位置及周围行人位置等驾驶场景数据。可以通过车辆内置的室内数字系统（DIS，DIS Digital Indoor System）采用DIS摄像头对司机人脸进行图像拍摄，获取人脸图像，根据人脸图像中展示人脸信息计算获得司机的人脸位置、人脸关键点、司机表情、司机眼睛开合状态等精神状态数据。可以通过车辆上安装的六轴传感器、GPS传感器等传感器进行车辆运动状态检测，获得急加速、急减速、急转弯、车速等驾驶行为数据。在司机状态评估系统中，根据驾驶场景数据、驾驶行为数据和精神状态数据对司机进行行为分析时，可以通过将这些获得的数据分别输入到相关的事件确认算法中进行事件确认处理，由事件确认算法来确认当前状态下司机的行为事件，在确认行为事件后，根据已确认的行为事件生成对应的时序事件信息。可以理解的是，司机的行为事件可以划分为若干种，在本实施例中包含有69种行为事件类型，示例性的，包含有车道保持事件，前碰事件，闭眼事件、下瞟事件、打哈欠事件、加速事件、减速事件、转弯事件、急加速事件、急减速事件、急转弯事件等类型。在本实施例中，可以在司机状态评估系统中为每种行为事件类型配置一种对应的事件确认算法。在本实施例中，每种事件确认算法的输入数据都不相同，在获得驾驶场景数据、驾驶行为数据和精神状态数据后，可以由各事件确认算法各自的输入需求从这些获得驾驶场景数据、驾驶行为数据和精神状态数据中获取对应的输入数据，以使各事件确认算法各自根据其输入数据进行行为分析，确认司机在当前状态下是否有执行事件确认算法对应的行为事件。若事件确认算法确认司机有执行其对应的行为事件，则根据其对应的行为事件生成对应的时序事件信息。
本申请的一些实施例中，时序事件信息包括有以下信息项：事件类别信息、事件起始时间信息、事件结束时间信息、事件起始速度信息、事件结束速度信息、相对碰撞最小时间信息、最小碰撞距离信息、碰撞累积时间信息、碰撞持续时间信息、车流信息、车辆位置信息、车辆行驶时长信息。
本申请的一些实施例中，还可以在司机状态评估系统中配置一天气信息获取模块，通过该天气信息获取模块获取当前驾驶场景下的实时天气状态信息，例如雨天、雪天、雾霾天、晴天等。在获得实时天气状态信息后，将该实时天气状态信息与时序事件信息进行数据融合，将实时天气状态信息作为一个信息项添加至时序事件信息中。以此实现在评估司机状态时加入天气状态这一项影响因素，提高评估结果的准确性。
本申请的一些实施例中，还可以在司机状态评估系统中配置一地图模块，通过该地图模块获取当前驾驶场景下的道路信息，例如红绿灯、交叉路口、高速公路、过道、省道等信息。在获得道路信息后，将该道路信息与时序事件信息进行数据融合，将道路信息作为一个信息项添加至时序事件信息中。以此实现在评估司机状态时加入路况这一项影响因素，提高评估结果的准确性。
S12：根据司机当前时间段的常态驾驶行为数据对所述司机进行驾驶习惯分析，生成所述司机当前时间段的驾驶习惯信息。
本实施例中，可以将距离当前时刻最近的一段时间范围内产生的驾驶行为数据作为司机当前时间段的常态驾驶行为数据。示例性的，假设当前时刻为12点15分，预设时间范围为5分钟，则此时认为12点10分至12点15分之间这段时间为司机当前时间段。在本实施例中，常态驾驶行为数据按照设定的时间间隔定时产生，例如1秒产生一次，获取得到常态驾驶行为数据为时间序列数据。在本实施例中，常态驾驶行为数据包括车速数据、车流数据、位置数据、六轴数据等，其中，车流数据包括前方车辆的数量和前方车辆距离本车的距离。在本实施例中，获得司机当前时间段的常态驾驶行为数据后，可以根据该常态驾驶行为数据对司机进行驾驶习惯分析，统计司机的实时驾驶规律，生成该司机当前时间段的驾驶习惯信息。其中，司机当前时间段的驾驶习惯信息包括司机当前时间段跟车习惯，司机当前事件段车道保持习惯，司机当前时间段车速保持习惯等信息。
S13：根据所述司机的身份信息调用司机画像库，获取所述司机的司机画像信息。
本实施例中，可以根据司机的历史驾驶数据构建司机画像并将司机画像存储在司机画像库。在司机画像库中包括有大量的司机画像，并建立有司机画像与司机身份信息之间的对应关系。在本实施例中，可以通过根据司机的身份信息调用司机画像库，获取司机的司机画像信息。在本实施例中，司机画像信息包括司机闭眼有效时间段，闭跟误报时间段，闭眼误报类型等等信息。
S14：采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果。
本实施例中，预设的信息融合分析模型为通过样本数据进行神经网络训练获得。信息融合分析模型被训练用于根据时序事件信息、驾驶习惯信息和司机画像信息进行信息融合分析，根据分析结果评估司机的状态，获得司机的状态评估结果。在本实施例中，信息融合分析模型可以根据时序事件信息，利用事件流作为输入，通过事件流进行事件确认，当确认司机有部分行为事件出现异常即可准确评估司机的状态，可以大幅度减少了漏报率。信息融合分析模型通过结合驾驶习惯信息和司机画像信息进行信息融合分析，可以利用司机画像进行多层约束，感知千人千面的司机状态，有效解决不同司机个性化差异导致的误报，提升了千人千面感知的准确率。信息融合分析模型可以根据分析结果评估司机的状态时，可以通过将司机状态进行分类，并将司机疲劳分级和驾驶风险分级，有效地提升司机状态评估的准确性和及时性。
以上可以看出，本申请实施例提供的司机状态评估方法可以通过获取司机当前行为事件所对应的时序事件信息、司机当前时间段的驾驶习惯信息以及司机的司机画像信息，然后，采用预设的信息融合分析模型对时序事件信息、驾驶习惯信息和司机画像信息进行信息融合分析，得到分析结果，进而根据分析结果来评估司机的状态，获得司机的状态评估结果，结合了一系列的时序事件特征和司机画像来判断司机的状态，提高了司机疲劳状态识别、分心状态识别和危险驾驶识别的准确率。
本申请的一些实施例中，请参阅图2，图2为本申请实施例提供的司机状态评估方法中进行信息融合分析的一种方法流程图，详细如下：
S21：采用所述信息融合分析模型中的浅层融合网络对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行早融合分析，生成早融合分析向量；
S22：采用所述信息融合分析模型中的深度融合网络对所述早融合分析向量和所述司机画像信息进行晚融合分析，生成晚融合分析向量；
S23：根据所述晚融合分析向量评估所述司机的状态，获得所述司机的状态评估结果，其中，所述司机的状态评估结果包括疲劳级别信息、分心信息和驾驶风险级别信息中的一项或多项信息。
本实施例中，请一并参阅图3，图3为本申请实施例提供的司机状态评估方法中信息融合分析模型的一种结构示意图。如图3所示，信息融合分析模型包括浅层融合网络和深层融合网络。示例性的，浅层融合网络可以基于transformer模型的行为encoder框架搭建而成。在本实施例中，采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析时，具体可以先将时序事件信息、驾驶习惯信息和司机画像信息一并输入到浅层融合网络中，由浅层融合网络根据该时序事件信息、驾驶习惯信息和司机画像信息分析得出早融合分析向量。然后将浅层融合网络分析得出的早融合分析向量输入到深层融合网络，并且在将早融合分析向量输入到深层融合网络的同时将司机画像信息也输入到深层融合网络中，由深度融合网络根据该早融合分析向量和司机画像信息分析得出晚融合分析向量。示例性的，在本实施例种，信息融合分析模型中可以将司机状态进行分类，每种状态类别具有对应的特征向量，在深度融合网络分析得出晚融合分析向量后，可以根据该晚融合分析向量与各种状态类别对应的特征向量进行向量相似度比对，从而根据向量相似度来确定司机的状态，其中包括确定司机的疲劳等级、确定司机是否分析以及确定驾驶风险等级。从而评估得出司机的状态，获得司机的状态评估结果。其中，司机的状态评估结果包括疲劳级别信息、分心信息和驾驶风险级别信息中的一项或多项信息。在本实施例中，基于信息融合分析模型的结构，在训练信息融合分析模型时，可以分别在浅层融合网络和深度融合网络判断司机状态是否为疲劳，以此训练获得两个预训练网络，进而在联合两个预训练网络来训练信息融合分析模型，可以有效减少训练信息融合分析模型所需要的样本数据量。
本申请的一些实施例中，请参阅图4，图4为本申请实施例提供的司机状态评估方法中进行早融合分析的一种方法流程图，详细如下：
S41：将所述时序事件信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的行为事件分析通道对所述时序事件信息进行多头自注意力分析处理、残差连接处理和标准化处理，生成第一分析向量；
S42：将所述驾驶习惯信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的驾驶习惯分析通道对所述驾驶习惯信息进行多头自注意力分析处理、残差连接处理和标准化处理，生成第二分析向量；
S43：将所述司机画像信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的司机画像分析通道对所述司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，生成第三分析向量；
S44：将所述第一分析向量和所述第二分析向量进行连接处理，获得连接向量；
S45：将所述连接向量与所述第三分析向量进行点积处理，获得早融合分析向量。
本实施例中，请一并参阅图5，图5为本申请实施例提供的司机状态评估方法中浅层融合网络的一种结构示意图。如图5所示，浅层融合网络中包含有三个分析通道，分别为行为事件分析通道、驾驶习惯分析通道和司机画像分析通道。其中，行为事件分析通道和驾驶习惯分析通道中设置有多头自注意力分析层（Multi-Head Self Attention）和残差连接及标准化处理层（Add&Norm）。在本实施例中，可以将时序事件信息作为输入向量输入至浅层融合网络，通过采用浅层融合网络的行为事件分析通道对时序事件信息进行多头自注意力分析处理、残差连接处理和标准化处理，即可生成第一分析向量。通过将驾驶习惯信息作为输入向量输入至浅层融合网络，采用浅层融合网络的驾驶习惯分析通道对驾驶习惯信息进行多头自注意力分析处理、残差连接处理和标准化处理，即可生成第二分析向量。在本实施例中，可以根据实际分析需求对时序事件信息和驾驶习惯信息进行多次的多头自注意力分析处理、残差连接处理和标准化处理来生成第一分析向量和第二分析向量。在本实施例中，司机画像分析通道中设置有多头自注意力分析层（Multi-Head Self Attention）、残差连接及标准化处理层（Add&Norm）、碰撞检测层（FCL）和逻辑回归处理层（softmax）。在本实施例中，将司机画像信息作为输入向量输入至浅层融合网络，通过采用浅层融合网络的司机画像分析通道对司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，即可生成第三分析向量。在本实施例中，当浅层融合网络中的三个分析通道分别得出第一分析向量、第二分析向量和第三分析向量后，可以通过连接函数（concat）将第一分析向量和所述第二分析向量进行连接处理，获得连接向量。进而，再将连接向量和第三分析向量进行点积处理（Dot-product），即可获得早融合分析向量。在本实施例中，司机画像分析通道在生成第三分析向量过程中，可以根据实际分析需求对司机画像信息进行多次的多头自注意力分析处理、残差连接处理和标准化处理。在本实施例中，通过浅层融合网络对时序事件信息、驾驶习惯信息和司机画像信息进行早融合分析，可以约束网络的局部特征，使得输出的分析向量更关注于关键行为事件，过滤弱相关性行为事件。
本申请的一些实施例中，请参阅图6，图6为本申请实施例提供的司机状态评估方法中进行晚融合分析的一种方法流程图，详细如下：
S61：将所述早融合分析向量作为输入向量输入到所述深度融合网络，采用所述深度融合网络中的融合结果分析通道对所述早融合分析向量进行多头自注意力分析处理、残差连接处理、标准化处理和连接处理，生成第四分析向量；
S62：将所述司机画像信息作为输入向量输入至所述深度融合网络，采用所述深度融合网络中的司机画像分析通道对所述司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，生成第五分析向量；
S63：将所述第四分析向量与所述第五分析向量进行点积处理，获得晚融合分析向量。
本实施例中，图7为本申请实施例提供的司机状态评估方法中深度融合网络的一种结构示意图。如图7所示，深度融合网络中包含有两个分析通道，分别为早融合结果分析通道和司机画像分析通道。其中，早融合结果分析通道中设置有多头自注意力分析层（Multi-Head Self Attention）和残差连接及标准化处理层（Add&Norm）和连接处理层（concat）。在本实施例中，将浅层融合网络输出的早融合分析向量作为深度融合网络的输入向量输入至该深度融合网络中，采用深度融合网络中的融合结果分析通道对早融合分析向量进行多头自注意力分析处理、残差连接处理、标准化处理和连接处理，即可生成第四分析向量。司机画像分析通道中设置有多头自注意力分析层（Multi-Head Self Attention）、残差连接及标准化处理层（Add&Norm）、碰撞检测层（FCL）和逻辑回归处理层（softmax）。在本实施例中，将司机画像信息再一次作为输入向量输入至深度融合网络，通过采用深度融合网络中的司机画像分析通道对司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，即可生成第五分析向量。深度融合网络生成第四分析向量和第五分析向量后，通过将第四分析向量和第五分析向量进行点积处理（Dot-product），即可获得晚融合分析向量。在本实施例中，通过深度融合网络进行晚融合可以在全局基础上优化网络各个参数的权重分布。通过浅层融合网络的早融合分析结合深度融合网络的晚融合分析，可有效解决由于司机个性化差异导致误报的问题，提升了司机千人千面感知的准确率。
可以理解的是，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本申请的一些实施例中，请参阅图8，图8为本申请实施例提供的一种司机状态评估装置的结构示意图。详述如下：
司机状态评估装置包括：第一生成模块81、第二生成模块82、获取模块83以及评估模块84。其中，所述第一生成模块81用于根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息。所述第二生成模块82用于根据司机当前时间段的常态驾驶行为数据对所述司机进行驾驶习惯分析，生成所述司机当前时间段的驾驶习惯信息。所述获取模块83用于根据所述司机的身份信息调用司机画像库，获取所述司机的司机画像信息。所述评估模块84用于采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果。
所述司机状态评估装置，与上述的司机状态评估方法一一对应，此处不再赘述。
本申请的一些实施例中，请参阅图9，图9为本申请实施例提供的一种电子设备的示意图，该电子设备可以为车载设备端，也可以为云端，通讯单元用于收发数据和信令。如图9所示，该实施例的电子设备9包括：处理器91、存储器92以及存储在所述存储器92中并可在所述处理器91上运行的计算机程序93，例如司机状态评估程序或数据处理程序。所述处理器91执行所述计算机程序92时实现上述各个司机状态评估方法实施例或数据处理方法中的步骤。或者，所述处理器91执行所述计算机程序93时实现上述各装置实施例中各模块/单元的功能。
示例性的，所述计算机程序93可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器92中，并由所述处理器91执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序93在所述电子设备9中的执行过程。例如，所述计算机程序93可以被分割成第一生成模块、第二生成模块、获取模块和评估模块，各模块具体功能如上所述。
所述电子设备可包括，但不仅限于，处理器91、存储器92。本领域技术人员可以理解，图9仅仅是电子设备9的示例，并不构成对电子设备9的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述电子设备还可以包括输入输出设备、网络接入设备、总线等。
所称处理器91可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器 (Digital Signal Processor，DSP)、专用集成电路 (Application Specific Integrated Circuit，ASIC)、现成可编程门阵列 (Field-Programmable Gate Array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器92可以是所述电子设备9的内部存储单元，例如电子设备9的硬盘或内存。所述存储器92也可以是所述电子设备9的外部存储设备，例如所述电子设备9上配备的插接式硬盘，智能存储卡（Smart Media Card, SMC），安全数字（Secure Digital, SD）卡，闪存卡（Flash Card）等。进一步地，所述存储器92还可以既包括所述电子设备9的内部存储单元也包括外部存储设备。所述存储器92用于存储所述计算机程序以及所述电子设备所需的其他程序和数据。所述存储器92还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/终端设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/终端设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括是电载波信号和电信信号。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (10)

  1. 一种司机状态评估方法，其特征在于，包括：
    根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息；
    根据司机当前时间段的常态驾驶行为数据对所述司机进行驾驶习惯分析，生成所述司机当前时间段的驾驶习惯信息；
    根据所述司机的身份信息调用司机画像库，获取所述司机的司机画像信息；
    采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果。
  2. 根据权利要求1所述的司机状态评估方法，其特征在于，采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果的步骤，包括：
    采用所述信息融合分析模型中的浅层融合网络对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行早融合分析，生成早融合分析向量；
    采用所述信息融合分析模型中的深度融合网络对所述早融合分析向量和所述司机画像信息进行晚融合分析，生成晚融合分析向量；
    根据所述晚融合分析向量评估所述司机的状态，获得所述司机的状态评估结果，其中，所述司机的状态评估结果包括疲劳级别信息、分心信息和驾驶风险级别信息中的一项或多项信息。
  3. 根据权利要求2所述的司机状态评估方法，其特征在于，采用所述信息融合分析模型中的浅层融合网络对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行早融合分析，生成早融合分析向量的步骤，包括：
    将所述时序事件信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的行为事件分析通道对所述时序事件信息进行多头自注意力分析处理、残差连接处理和标准化处理，生成第一分析向量；
    将所述驾驶习惯信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的驾驶习惯分析通道对所述驾驶习惯信息进行多头自注意力分析处理、残差连接处理和标准化处理，生成第二分析向量；
    将所述司机画像信息作为输入向量输入至所述浅层融合网络，采用所述浅层融合网络的司机画像分析通道对所述司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，生成第三分析向量；
    将所述第一分析向量和所述第二分析向量进行连接处理，获得连接向量；
    将所述连接向量与所述第三分析向量进行点积处理，获得早融合分析向量。
  4. 根据权利要求2或3所述的司机状态评估方法，其特征在于，采用所述信息融合分析模型中的深度融合网络对所述早融合分析向量和所述司机画像信息进行晚融合分析，生成晚融合分析向量的步骤，包括：
    将所述早融合分析向量作为输入向量输入到所述深度融合网络，采用所述深度融合网络中的融合结果分析通道对所述早融合分析向量进行多头自注意力分析处理、残差连接处理、标准化处理和连接处理，生成第四分析向量；
    将所述司机画像信息作为输入向量输入至所述深度融合网络，采用所述深度融合网络中的司机画像分析通道对所述司机画像信息进行多头自注意力分析处理、残差连接处理、标准化处理、碰撞检测处理和逻辑回归处理，生成第五分析向量；
    将所述第四分析向量与所述第五分析向量进行点积处理，获得晚融合分析向量。
  5. 根据权利要求1所述的司机状态评估方法，其特征在于，根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息的步骤中，所述时序事件信息包括以下信息项：
    事件类别信息、事件起始时间信息、事件结束时间信息、事件起始速度信息、事件结束速度信息、相对碰撞最小时间信息、最小碰撞距离信息、碰撞累积时间信息、碰撞持续时间信息、车流信息、车辆位置信息、车辆行驶时长信息。
  6. 根据权利要求5所述的司机状态评估方法，其特征在于，根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息的步骤之后，还包括：
    获取当前驾驶场景下的实时天气状态信息，将所述实时天气状态信息作为信息项添加至所述时序事件信息中。
  7. 根据权利要求5或6所述的司机状态评估方法，其特征在于，根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息的步骤之后，还包括：
    获取当前驾驶场景下的道路信息，将所述道路信息作为信息项添加至所述时序事件信息中。
  8. 一种司机状态评估装置，其特征在于，包括：
    第一生成模块，用于根据司机当前的驾驶场景数据、驾驶行为数据和精神状态数据对所述司机进行行为分析，获取所述司机的行为事件，并根据所述行为事件生成时序事件信息；
    第二生成模块，用于根据司机当前时间段的常态驾驶行为数据对所述司机进行驾驶习惯分析，生成所述司机当前时间段的驾驶习惯信息；
    获取模块，用于根据所述司机的身份信息调用司机画像库，获取所述司机的司机画像信息；
    评估模块，用于采用预设的信息融合分析模型对所述时序事件信息、所述驾驶习惯信息和所述司机画像信息进行信息融合分析，并根据分析结果评估所述司机的状态，获得所述司机的状态评估结果。
  9. 一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述方法的步骤。
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述方法的步骤。
PCT/CN2022/128380 2022-10-28 2022-10-28 司机状态评估方法、装置、电子设备及存储介质 Ceased WO2024087205A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202280062260.4A CN118119543A (zh) 2022-10-28 2022-10-28 司机状态评估方法、装置、电子设备及存储介质
PCT/CN2022/128380 WO2024087205A1 (zh) 2022-10-28 2022-10-28 司机状态评估方法、装置、电子设备及存储介质
EP22963186.6A EP4495833A4 (en) 2022-10-28 2022-10-28 DRIVER CONDITION EVALUATION METHOD AND APPARATUS, ELECTRONIC DEVICE AND STORAGE MEDIUM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/128380 WO2024087205A1 (zh) 2022-10-28 2022-10-28 司机状态评估方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024087205A1 true WO2024087205A1 (zh) 2024-05-02

Family

ID=90829749

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/128380 Ceased WO2024087205A1 (zh) 2022-10-28 2022-10-28 司机状态评估方法、装置、电子设备及存储介质

Country Status (3)

Country Link
EP (1) EP4495833A4 (zh)
CN (1) CN118119543A (zh)
WO (1) WO2024087205A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118802354A (zh) * 2024-07-26 2024-10-18 中关村科学城城市大脑股份有限公司 基于网络行为画像的网络行为检测方法、装置和电子设备
CN118797526A (zh) * 2024-09-10 2024-10-18 中电科新型智慧城市研究院有限公司 一种事件告警方法、装置、终端设备及存储介质
CN118916834A (zh) * 2024-07-18 2024-11-08 北京中科睿途科技有限公司 用于司机行为预测的多模态数据融合方法及装置
CN120288054A (zh) * 2025-03-28 2025-07-11 安康市道路运输服务中心 基于时空图神经网络的驾驶员危险行为干预系统及方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108407816A (zh) * 2018-01-19 2018-08-17 杭州砺玛物联网科技有限公司 一种汽车驾驶员驾驶评估方法及系统
CN108423006A (zh) * 2018-02-02 2018-08-21 辽宁友邦网络科技有限公司 一种辅助驾驶预警方法及系统
CN110909718A (zh) * 2019-12-11 2020-03-24 深圳市锐明技术股份有限公司 驾驶状态识别方法、装置及车辆

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3266668B1 (en) * 2016-07-06 2025-02-19 Continental Autonomous Mobility Germany GmbH Device for determining driving warning information
EP3416147B1 (en) * 2017-06-13 2020-01-15 Volvo Car Corporation Method for providing drowsiness alerts in vehicles
US10960895B1 (en) * 2017-09-27 2021-03-30 State Farm Mutual Automobile Insurance Company Automatically tracking driving activity
US10745019B2 (en) * 2017-12-18 2020-08-18 International Business Machines Corporation Automatic and personalized control of driver assistance components
WO2020160331A1 (en) * 2019-01-30 2020-08-06 Cobalt Industries Inc. Systems and methods for verifying and monitoring driver physical attention

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108407816A (zh) * 2018-01-19 2018-08-17 杭州砺玛物联网科技有限公司 一种汽车驾驶员驾驶评估方法及系统
CN108423006A (zh) * 2018-02-02 2018-08-21 辽宁友邦网络科技有限公司 一种辅助驾驶预警方法及系统
CN110909718A (zh) * 2019-12-11 2020-03-24 深圳市锐明技术股份有限公司 驾驶状态识别方法、装置及车辆

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4495833A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118916834A (zh) * 2024-07-18 2024-11-08 北京中科睿途科技有限公司 用于司机行为预测的多模态数据融合方法及装置
CN118802354A (zh) * 2024-07-26 2024-10-18 中关村科学城城市大脑股份有限公司 基于网络行为画像的网络行为检测方法、装置和电子设备
CN118797526A (zh) * 2024-09-10 2024-10-18 中电科新型智慧城市研究院有限公司 一种事件告警方法、装置、终端设备及存储介质
CN120288054A (zh) * 2025-03-28 2025-07-11 安康市道路运输服务中心 基于时空图神经网络的驾驶员危险行为干预系统及方法
CN120288054B (zh) * 2025-03-28 2025-09-12 安康市道路运输服务中心 基于时空图神经网络的驾驶员危险行为干预系统及方法

Also Published As

Publication number Publication date
EP4495833A1 (en) 2025-01-22
CN118119543A (zh) 2024-05-31
EP4495833A4 (en) 2025-07-02

Similar Documents

Publication Publication Date Title
WO2024087205A1 (zh) 司机状态评估方法、装置、电子设备及存储介质
US11783601B2 (en) Driver fatigue detection method and system based on combining a pseudo-3D convolutional neural network and an attention mechanism
CN108351968B (zh) 一种针对犯罪活动的告警方法、装置、存储介质及服务器
CN112105537B (zh) 驾驶风险计算装置和方法
Saiprasert et al. Driver behaviour profiling using smartphone sensory data in a V2I environment
CN110895662A (zh) 车辆超载报警方法、装置、电子设备及存储介质
CN110047272B (zh) 一种基于大数据的智能交通行人行为监控报警系统
WO2024087204A1 (zh) 设备端的驾驶风险行为干预方法、装置、设备及存储介质
CN113676702A (zh) 基于视频流的目标追踪监测方法、系统、装置及存储介质
CN114299473A (zh) 一种基于多源信息融合的驾驶员行为识别方法
CN118072551A (zh) 基于时空轨迹的安全预警方法、装置、系统及电子设备
US20230377456A1 (en) Mobile real time 360-degree traffic data and video recording and tracking system and method based on artifical intelligence (ai)
JP2020042785A (ja) 無人車内の乗客状態の識別方法、装置、機器及び記憶媒体
JP2026048690A (ja) システム等
CN112200148A (zh) 一种去中心化的交通诚信评价系统
WO2024243804A1 (zh) 司机的驾驶风险分析方法、系统、电子设备及存储介质
CN115909651A (zh) 车内人身安全保护方法、装置、设备及存储介质
WO2025000127A1 (zh) 司机状态检测方法、装置、电子设备及存储介质
CN114758315B (zh) 车辆信号灯的识别方法、识别模型的训练方法及相关设备
CN111241918A (zh) 一种基于人脸识别的车用防跟踪方法及系统
JP7140895B1 (ja) 事故分析装置、事故分析方法及びプログラム
Kashevnik et al. Driver intelligent support system in internet of transportation things: Smartphone-based approach
CN118043248B (zh) 司机的驾驶风险分析方法、系统、电子设备及存储介质
Thakrar et al. Enhancing Driver Safety through Real-Time Feedback on Driving Behavior: A Deep Learning Approach
KR102892590B1 (ko) 딥 러닝 기반의 지능형 영상 분석 시스템

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 202280062260.4

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22963186

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18849298

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022963186

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022963186

Country of ref document: EP

Effective date: 20241014

NENP Non-entry into the national phase

Ref country code: DE