WO2020192498A1 - 手握方向盘状态的检测方法及装置 - Google Patents

手握方向盘状态的检测方法及装置 Download PDF

Info

Publication number
WO2020192498A1
WO2020192498A1 PCT/CN2020/079742 CN2020079742W WO2020192498A1 WO 2020192498 A1 WO2020192498 A1 WO 2020192498A1 CN 2020079742 W CN2020079742 W CN 2020079742W WO 2020192498 A1 WO2020192498 A1 WO 2020192498A1
Authority
WO
WIPO (PCT)
Prior art keywords
steering wheel
state
hand
area information
driver
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/079742
Other languages
English (en)
French (fr)
Inventor
唐晨
张志伟
张骁迪
丁春辉
王进
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ArcSoft Corp Ltd
Original Assignee
ArcSoft Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ArcSoft Corp Ltd filed Critical ArcSoft Corp Ltd
Priority to EP20779920.6A priority Critical patent/EP3951645A4/en
Priority to US17/054,179 priority patent/US11423673B2/en
Priority to JP2021557708A priority patent/JP7253639B2/ja
Priority to KR1020217035279A priority patent/KR102830024B1/ko
Publication of WO2020192498A1 publication Critical patent/WO2020192498A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30268—Vehicle interior

Definitions

  • This application relates to the field of image detection technology, and in particular to a method and device for detecting the state of a steering wheel in a hand.
  • the embodiments of the present application provide a method and device for detecting the state of a steering wheel held by a hand, so as to at least solve the technical problem that the driver cannot detect the mastering state of the steering wheel in the process of driving a vehicle in the related art.
  • a method for detecting the state of holding a steering wheel including: detecting a video stream collected from a target vehicle to obtain a detection result; and obtaining ROI region information carried in the detection result One is the state of association with ROI area information two in a two-dimensional plane, where the ROI area information one is the ROI area information of the steering wheel of the target vehicle, and the ROI area information two is the hand of the driver of the target vehicle ROI area information; determine the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state.
  • the video stream includes at least one of the following: a depth video stream, an infrared IR video stream, and an RGB video stream.
  • detecting the video stream collected from the target vehicle, and obtaining the detection result includes: detecting the image in the video stream to obtain the steering wheel position of the target vehicle and the driver's hand position, To get the test result.
  • acquiring the associated state of the ROI region information one and the ROI region information two carried in the detection result in a two-dimensional plane includes: using a pre-trained classification model to compare the steering wheel position and the driver's hand position
  • the positional relationship of is divided into different categories, where the categories include: the hand position is on the steering wheel position, and the hand position is not on the steering wheel position.
  • acquiring the associated state of the ROI area information one and the ROI area information two carried in the detection result in a two-dimensional plane includes: comparing the position of the steering wheel with the position of the driver's hand in the two-dimensional plane The coincidence rate and the first threshold value are used to determine the aggregation state of the ROI area information 1 and the ROI area information 2 in a two-dimensional plane, wherein the aggregation state is used to indicate that the ROI area information 1 is in the Whether there is an intersection between the two-dimensional plane and the second ROI area information.
  • the method further includes: acquiring depth information of the ROI region information one and the ROI region information two carried in the detection result, and determining the difference between the ROI region information one and the ROI region information two The depth state, wherein the depth states of the first ROI area information and the second ROI area information are used to indicate whether the depth ranges of the first ROI area information and the second ROI area information are consistent.
  • the method further includes: judging whether the hand position and the steering wheel position can grow into a connected region according to the depth information of the ROI region information 1 and the ROI region information 2 by using a region growth method to determine The current state between the driver's hand and the steering wheel of the target vehicle.
  • acquiring the depth information of ROI area information 1 and ROI area information 2 carried in the detection result, and determining the depth state of ROI area information 1 and ROI area information 2 includes: modeling the depth of the steering wheel position to obtain The depth model of the steering wheel position; obtain the actual depth of the hand position and the theoretical depth of the hand position in the depth model; determine the steering wheel according to the actual depth and the theoretical depth of the hand position Position and depth state of the hand position.
  • detecting the images in the video stream to obtain the steering wheel position of the target vehicle includes: determining the initial position of the steering wheel of the target vehicle, and determining whether the image in the video stream is within the initial position Through detection, the steering wheel position of the target vehicle is obtained.
  • detecting the images in the video stream to obtain the driver's hand position includes: obtaining the driver's hand position corresponding to the image in the video stream through a positioning model, wherein, the positioning model is obtained by training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: an image in the video stream and the driver's hand corresponding to the image in the video stream. Department location.
  • detecting the image in the video stream to obtain the driver's hand position further includes: detecting the image in the video stream to locate the driver's arm position; The arm position corrects the hand position obtained based on the positioning model to obtain the hand position.
  • the driver's arm position is represented by three-dimensional information of a predetermined number of points along the arm corresponding to the arm detected from the image in the video stream.
  • determining the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state includes: when the hand position is on the steering wheel position or the aggregate state is used to indicate When the ROI area information one intersects the ROI area information two in the two-dimensional plane, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is a real grip state; When the position of the part is not on the steering wheel position or the set state is used to indicate that the ROI area information one does not have an intersection with the ROI area information two in the two-dimensional plane, it is determined that the driver’s hand is The current state of the steering wheel of the target vehicle is the disengaged state.
  • determining the current state between the driver's hand and the steering wheel of the target vehicle according to the association state and the depth state includes: in the aggregate state, indicating that the ROI area information is in the In the case that there is no intersection between the two-dimensional plane and the ROI region information, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the disengaged state; the collective state represents the ROI region information 1.
  • the depth state indicates that the depth range of the ROI area information one and the ROI area information two are inconsistent
  • determine the driving The current state of the driver’s hand and the steering wheel of the target vehicle is in a virtual grip state; in the case where the collective state indicates that the ROI area information one intersects the ROI area information two in the two-dimensional plane,
  • the depth state indicates that when the depth range of the first ROI area information is consistent with the depth range of the second ROI area information, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is a real grip state.
  • determining the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state and the depth state includes: when the hand position is on the steering wheel position, and The depth state indicates that when the depth range of the ROI area information 1 and the ROI area information 2 is consistent, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is a virtual grip state; When the position of the steering wheel is at the steering wheel position, and the depth state indicates that the depth range of the ROI area information one and the ROI area information two is consistent, it is determined that the driver’s hand is in relation to the steering wheel of the target vehicle.
  • the current state is a real grip state; when the hand position is not at the steering wheel position, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is a disengaged state.
  • the current state between the driver's hand and the steering wheel of the target vehicle includes one of the following: two hands separated from the steering wheel, one hand separated from the steering wheel, two-hand virtual grip state, one-hand virtual grip state, and two hands real grip status.
  • the method for detecting the state of the steering wheel in the hand further includes: detecting the number of hands; When the number of the hands is two, the associated state of the steering wheel position and the hand position of each hand of the driver in the two-dimensional plane is acquired.
  • a detection device for the state of a hand-held steering wheel including: a detection unit, configured to detect a video stream collected from a target vehicle to obtain a detection result; To obtain the associated state of the ROI area information one and the ROI area information two carried in the detection result in a two-dimensional plane, wherein the ROI area information one is the ROI area information of the steering wheel of the target vehicle, and the ROI area The second information is the ROI area information of the driver's hand of the target vehicle; the determining unit is configured to determine the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state.
  • the video stream includes at least one of the following: a depth video stream, an infrared IR video stream, and an RGB video stream.
  • the detection unit is configured to detect images in the video stream to obtain the steering wheel position of the target vehicle and the driver's hand position to obtain the detection result.
  • the acquiring unit includes: an associated state acquiring module, configured to use a pre-trained classification model to classify the positional relationship between the steering wheel position and the driver's hand position into different categories, wherein the The categories include: the hand position is on the steering wheel position, and the hand position is not on the steering wheel position.
  • the acquiring unit includes: an associated state acquiring module, configured to compare the coincidence rate of the steering wheel position and the driver's hand position in the two-dimensional plane with a first threshold to determine the ROI
  • the acquiring unit further includes: a depth state acquiring module, configured to acquire depth information of ROI region information one and ROI region information two carried in the detection result, and to determine ROI region information one and ROI region information two
  • the depth state of the ROI area information one and the ROI area information two is used to indicate whether the depth ranges of the ROI area information one and the ROI area information two are consistent.
  • the depth state acquisition module is further configured to determine whether the hand position and the steering wheel position can grow into a connected region by using a region growing method according to the depth information of the ROI region information one and the ROI region information two To determine the current state between the driver's hand and the steering wheel of the target vehicle.
  • the depth state acquisition module includes: a modeling sub-module for modeling the depth of the steering wheel position to obtain a depth model of the steering wheel position; and an acquisition sub-module for acquiring the hand The actual depth of the position and the theoretical depth of the hand position in the depth model; a determining sub-module is used to determine the steering wheel position and the hand position according to the actual depth and the theoretical depth of the hand position The depth state.
  • the detection unit includes: a steering wheel position detection module, configured to determine the initial position of the steering wheel of the target vehicle, and detect in the initial position according to the image in the video stream to obtain the Steering wheel position.
  • a steering wheel position detection module configured to determine the initial position of the steering wheel of the target vehicle, and detect in the initial position according to the image in the video stream to obtain the Steering wheel position.
  • the detection unit includes: a hand position detection module, configured to obtain the driver's hand position corresponding to the image in the video stream through a positioning model, wherein the positioning model is used
  • a hand position detection module configured to obtain the driver's hand position corresponding to the image in the video stream through a positioning model, wherein the positioning model is used
  • Multiple sets of training data are obtained through training, and each set of training data in the multiple sets of training data includes: an image in a video stream and a driver's hand position corresponding to the image in the video stream.
  • the detection unit further includes: an arm position detection module, which is used to detect images in the video stream and locate the driver’s arm position; and a hand position correction module, which is used for The arm position corrects the hand position obtained based on the positioning model to obtain the hand position.
  • an arm position detection module which is used to detect images in the video stream and locate the driver’s arm position
  • a hand position correction module which is used for The arm position corrects the hand position obtained based on the positioning model to obtain the hand position.
  • the determining unit includes: a first determining subunit, configured to indicate that the ROI region information is on the two-dimensional plane when the hand position is on the steering wheel position or the aggregate state When there is an intersection with the second ROI region information, it is determined that the current state of the driver’s hand and the steering wheel of the target vehicle is the real grip state; the second determining subunit is used when the hand position is not The position of the steering wheel or the state of the set is used to indicate that the ROI area information one does not have an intersection with the ROI area information two in the two-dimensional plane, determining the relationship between the driver’s hand and the target vehicle The current state of the steering wheel is the disengaged state.
  • the determining unit further includes: a third determining subunit, configured to indicate, in the set state, that the ROI area information one does not have an intersection with the ROI area information two in the two-dimensional plane Next, it is determined that the current state of the driver’s hand and the steering wheel of the target vehicle is the disengaged state; the fourth determining subunit is used to indicate that the ROI area information is in the two-dimensional plane in the aggregate state When there is an intersection with the second ROI area information, and the depth state indicates that the depth range of the first ROI area information is not consistent with the second depth range of the ROI area information, it is determined that the driver’s hand and the target vehicle
  • the current state of the steering wheel of is a virtual grip state;
  • the fifth determining subunit is used to indicate that the ROI area information one has an intersection with the ROI area information two in the two-dimensional plane when the aggregate state indicates, And the depth state indicates that when the depth range of the first ROI area information is consistent with the depth range of the second ROI area information, it is determined that the current state of the driver's
  • the determining unit further includes: a sixth determining subunit, configured to: when the hand position is on the steering wheel position, and the depth state represents the ROI area information one and the ROI area When the depth range of the second information is inconsistent, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the virtual grip state; the seventh determining subunit is used for when the hand position is on the steering wheel position When the depth state indicates that the depth range of the ROI area information 1 and the ROI area information 2 is consistent, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the real grip state; The eight determination subunit is used to determine that the current state of the driver's hand and the steering wheel of the target vehicle is the disengaged state when the hand position is not at the steering wheel position.
  • a sixth determining subunit configured to: when the hand position is on the steering wheel position, and the depth state represents the ROI area information one and the ROI area When the depth range of the second information is inconsistent, it is determined that the current state of the driver
  • the current state between the driver's hand and the steering wheel of the target vehicle includes one of the following: two hands separated from the steering wheel, one hand separated from the steering wheel, two-hand virtual grip state, one-hand virtual grip state, and two hands real grip status.
  • the acquiring unit detects the number of hands before acquiring the associated state of the ROI area information one and the ROI area information two carried in the detection result in the two-dimensional plane; When the number is two, the associated state of the position of the steering wheel and the position of each hand of the driver in the two-dimensional plane is acquired.
  • a storage medium includes a stored program, wherein the program executes the method for detecting the state of holding a steering wheel in any one of the foregoing.
  • a processor is also provided, the processor is used to run a program, wherein the method for detecting the state of holding a steering wheel according to any one of the above is executed when the program is running.
  • the video stream collected from the target vehicle is detected to obtain the detection result; and the associated state of the ROI region information 1 and the ROI region information 2 carried in the detection result in the two-dimensional plane is obtained, where: ROI area information one is the ROI area information of the steering wheel of the target vehicle, ROI area information two is the ROI area information of the driver's hand of the target vehicle; and the current state between the driver's hand and the steering wheel of the target vehicle is determined according to the associated state
  • the method detects the state of the driver’s hand holding the steering wheel, and achieves the purpose of determining the state of the driver’s hand and the steering wheel of the target vehicle based on the ROI area information of the driver’s hand and the ROI area information of the steering wheel obtained from image analysis.
  • the technical effect of improving the accuracy of detecting the state of the driver's hand and the steering wheel is achieved, and the technical problem of the related technology that cannot be detected in the driver's grasping state of the steering wheel while driving the vehicle is solved.
  • FIG. 1 is a flowchart of a method for detecting the state of holding a steering wheel according to an embodiment of the present application
  • FIG. 2 is a flowchart of an optional detection method for the state of holding a steering wheel according to an embodiment of the present application
  • Fig. 3 is a preferred flowchart of a method for detecting a state of holding a steering wheel according to an embodiment of the present application
  • Fig. 4 is a schematic diagram of a detection device for a state of holding a steering wheel according to an embodiment of the present application.
  • ADAS Advanced Driver Assistance System
  • the vehicle can be controlled by the autonomous steering system, and it is acceptable to leave the steering wheel with the hand.
  • ADAS can only focus on assistance, requiring the driver to concentrate on driving the vehicle. At this time, it is not possible to have both hands off the steering wheel. Therefore, the ADAS system needs to add the driver's steering wheel release detection to provide appropriate assistance in different situations.
  • the traditional steering wheel hand-off detection uses hardware for detection. Specifically, using hardware to detect is to directly use a capacitive sensor to detect the driver's grip on the steering wheel.
  • the advantage is that the sensor is in direct contact with the driver, and the results obtained are accurate and sensitive.
  • the cost of the hand-off detection system is often higher.
  • a method embodiment of a method for detecting the state of a hand-held steering wheel is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be implemented in a computer system such as a set of computer-executable instructions. Execution, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.
  • Fig. 1 is a flow chart of a method for detecting the state of holding a steering wheel according to an embodiment of the present application. As shown in Fig. 1, the method for detecting the state of holding a steering wheel includes the following steps:
  • Step S102 Detect the video stream collected from the target vehicle to obtain the detection result.
  • the aforementioned video stream may include at least one of the following: a deep video stream, an infrared IR video stream, and an RGB video stream.
  • IR is the abbreviation of Infrared Radiation, which is a wireless communication method that can transmit wireless data. Since the light in the car (for example, the driver's cab) usually changes with the driving environment, during the day when the weather is fine, the light in the car (for example, the driver's cab) is brighter, at night or in a cloudy day and in the tunnel, the car ( For example, the light in the cab is relatively dark, while the infrared camera is less affected by changes in illumination and has the ability to work around the clock. Therefore, you can choose an infrared camera (including near-infrared cameras, etc.) to obtain infrared IR video streams with better quality than ordinary cameras. Thereby improving the accuracy of the detection results.
  • an infrared camera including near-infrared cameras, etc.
  • the installation location of the capture device for capturing video streams is not specifically limited, as long as the foregoing video streams can be captured.
  • the acquisition perspective can be a downward perspective, a first-person perspective, and a third-person perspective on the top of the target vehicle.
  • the aforementioned collection device may be a road monitoring device, for example, a monitoring device at a road checkpoint.
  • detecting the video stream collected from the target vehicle to obtain the detection result may include: detecting the image in the video stream to obtain the steering wheel position of the target vehicle and the driver's hand position to obtain the detection result.
  • detecting the image in the video stream to obtain the steering wheel position of the target vehicle may include: determining the initial position of the steering wheel of the target vehicle, and detecting the initial position according to the image in the video stream to obtain the target The position of the steering wheel of the vehicle.
  • detecting the image in the video stream to obtain the driver’s hand position may include: obtaining the driver’s hand position corresponding to the image in the video stream through a positioning model, where the positioning model is a multi-use model.
  • a set of training data is obtained through training, and each set of training data in the multiple sets of training data includes: the image in the video stream and the driver's hand position corresponding to the image in the video stream.
  • Step S104 Obtain the correlation status of ROI area information 1 and ROI area information 2 carried in the detection result on a two-dimensional plane, where ROI area information 1 is the ROI area information of the steering wheel of the target vehicle, and ROI area information 2 is the target vehicle ROI area information of the driver’s hand.
  • the above detection result also carries the ROI area information of the steering wheel of the target vehicle and the ROI area information of the driver's hand of the target vehicle.
  • a method for obtaining the associated state of the ROI area information 1 and the ROI area information 2 carried in the detection result on a two-dimensional plane may be: using a pre-trained classification model to compare the steering wheel position with the driver’s hand
  • the positional relationship of the position of the hand is divided into different categories, among which the categories include: the position of the hand is on the position of the steering wheel, and the position of the hand is not on the position of the steering wheel. Specifically, if the hand position is not on the steering wheel position, it may include the hand position below or adjacent to the steering wheel.
  • obtaining the correlation status of the ROI region information one and the ROI region information two carried in the detection result on a two-dimensional plane may include: comparing the position of the steering wheel with the position of the driver's hand on the two-dimensional plane The coincidence rate and the first threshold are used to determine the collection status of ROI region information 1 in a two-dimensional plane and ROI region information 2, where the collection status is used to indicate whether ROI region information 1 is in a two-dimensional plane and ROI region information 2 is There is an intersection.
  • the coincidence rate of the steering wheel position and the driver's hand position in the two-dimensional plane can be compared with a first threshold, and the comparison result is that the steering wheel position and the driver's hand position in the two-dimensional plane have a greater coincidence rate than the first threshold.
  • a threshold value determine that the hand position is at the steering wheel position; in the case where the comparison result is that the coincidence rate of the steering wheel position and the driver's hand position in the two-dimensional plane is not greater than the first threshold, it is determined that the hand position is not at the steering wheel position To determine the positional relationship between the driver’s hand and the steering wheel of the target vehicle.
  • Step S106 Determine the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state.
  • determining the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state may include: when the hand position is at the steering wheel position, determining that the current state of the driver's hand and the steering wheel of the target vehicle is a real grip State; when the hand position is not on the steering wheel position, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the disengaged state.
  • the video stream collected from the target vehicle can be detected to obtain the detection result; then the ROI region information carried in the detection result 1 and the ROI region information 2 are associated with the two-dimensional plane, where the ROI region information One is the ROI area information of the steering wheel of the target vehicle, and the second is the ROI area information of the driver's hand of the target vehicle; and the current state between the driver's hand and the steering wheel of the target vehicle is determined according to the associated state.
  • the video stream of the target vehicle can be detected to obtain the detection result, and then the correlation state of the ROI area information one and the ROI area information two on the two-dimensional plane can be determined, and the driver's hand and the target can be determined based on the correlation state
  • the current state between the steering wheel of the vehicle realizes the ROI area information of the driver's hand and the ROI area information of the steering wheel obtained according to the image analysis.
  • the purpose of determining the state of the driver's hand and the steering wheel of the target vehicle is achieved.
  • this method can actually only determine whether the driver's hand and the steering wheel are visually overlapping, so as to determine whether the current state of the driver's hand and the steering wheel of the target vehicle is the real grip state or the disengaged state. If the driver holds the steering wheel in a virtual manner, that is, the hand makes an action of holding the steering wheel directly above the steering wheel, and does not actually touch the steering wheel, it is impossible to correctly judge that the driver has released the steering wheel.
  • the depth information of the driver’s hand and the target vehicle’s steering wheel in the depth video stream is used to determine whether the depth ranges of the two are consistent. Whether the current state of the driver's hand and the steering wheel of the target vehicle is a real grip state, a virtual grip state or a disengaged state.
  • a method embodiment of a method for detecting the state of a hand-held steering wheel is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be implemented in a computer system such as a set of computer-executable instructions. Execution, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.
  • Fig. 2 is a flowchart of an optional method for detecting the state of holding a steering wheel according to an embodiment of the present application. As shown in Fig. 2, the method for detecting the state of holding a steering wheel includes the following steps:
  • Step S202 Detect the video stream collected from the target vehicle to obtain the detection result.
  • the video stream includes at least one of the following: a depth video stream, an infrared IR video stream, and an RGB video stream. Since only RGB video stream or infrared IR video stream or a combination of RGB video stream and infrared IR video stream are used alone, the depth position of the driver's hand and the steering wheel cannot be judged, and the virtual grip cannot be detected. In addition, when driving at night, only using the RGB video stream may affect the judgment of the driver's hands and the state of the steering wheel. Therefore, in the embodiments of the present application, in order to make the detection result more comprehensive and adaptable, the video stream may include multiple types, but each type includes a deep video stream.
  • the video stream collection device is not specifically limited as long as it can collect in-depth video streams; the installation location of the collection device is also not specifically limited, as long as the above-mentioned video stream can be collected.
  • the acquisition perspective can be a downward perspective, a first-person perspective, and a third-person perspective on the top of the target vehicle.
  • the aforementioned collection device may be a road monitoring device, for example, a monitoring device at a road checkpoint.
  • step S202 detecting the video stream collected from the target vehicle, and obtaining the detection result may include: detecting the image in the video stream to obtain the steering wheel position of the target vehicle and the driver's hand position to obtain the detection result .
  • Step S204 Obtain the associated state of ROI area information 1 and ROI area information 2 carried in the detection result in a two-dimensional plane, where ROI area information 1 is the ROI area information of the steering wheel of the target vehicle, and ROI area information 2 is the target vehicle's ROI area information of the driver's hand.
  • the ROI area here is the region of interest (Region of Interest, ROI for short).
  • the processed image is outlined in the form of boxes, circles, ellipses, and irregular polygons.
  • the area is called the area of interest.
  • the ROI area can be used as a key area of image analysis during image analysis.
  • the ROI area information here is the coordinate position in the two-dimensional plane.
  • acquiring the association status of the ROI region information one and the ROI region information two carried in the detection result in a two-dimensional plane may include: using a pre-trained classification model to compare the position of the steering wheel and the position of the driver's hand
  • the position relationship of is divided into different categories. Among them, the categories include: the position of the hand is on the position of the steering wheel, and the position of the hand is not on the position of the steering wheel.
  • obtaining the associated state of the ROI area information one and the ROI area information two carried in the detection result in the two-dimensional plane may include: comparing the position of the steering wheel with the position of the driver's hand in the two-dimensional plane.
  • the coincidence rate and the first threshold are used to determine the aggregation state of the ROI area information 1 in the two-dimensional plane and the ROI area information 2, where the aggregation state is used to indicate whether the ROI area information 1 is in the two-dimensional plane and the ROI area information 2 exists Intersection.
  • the coincidence rate of the steering wheel position and the driver's hand position in the two-dimensional plane can be compared with a first threshold, and the comparison result is that the steering wheel position and the driver's hand position in the two-dimensional plane have a greater coincidence rate than the first threshold.
  • a threshold value determine that the hand position is at the steering wheel position; in the case where the comparison result is that the coincidence rate of the steering wheel position and the driver's hand position in the two-dimensional plane is not greater than the first threshold, it is determined that the hand position is not at the steering wheel position To determine the positional relationship between the driver’s hand and the steering wheel of the target vehicle.
  • Step S206 Obtain the depth information of the ROI area information 1 and the ROI area information 2 carried in the detection result, and determine the depth state of the ROI area information 1 and the ROI area information 2.
  • the depth state of the first ROI area information and the second ROI area information includes whether the depth ranges of the first ROI area information and the second ROI area information are consistent.
  • acquiring the depth information of ROI area information 1 and ROI area information 2 carried in the detection result, and determining the depth state of ROI area information 1 and ROI area information 2 may include: positioning the steering wheel (ROI area information 1) Model the depth of the steering wheel to obtain the depth model of the steering wheel position (ROI area information one); obtain the actual depth of the hand position (ROI area information two) and the theoretical depth of the hand position (ROI area information two) in the depth model; According to the actual depth and theoretical depth of the hand position (ROI region information 2), the depth state of the steering wheel position (ROI region information 1) and the hand position (ROI region information 2) are determined.
  • the theoretical depth of the hand position (ROI region information 2) can be obtained by inputting the actual depth of the hand position (ROI region information 2) into the depth model of the steering wheel position (ROI region information 1) based on the output of the depth model Yes, the depth model of the steering wheel position is obtained through machine learning training using multiple sets of training data.
  • determining the depth state of the steering wheel position and the hand position according to the actual depth and the theoretical depth of the hand position may include: comparing the actual depth of the hand position and the theoretical depth of the hand position in the depth model with the first depth. The two thresholds are compared to determine the depth state of the steering wheel position and the hand position.
  • the hand position and the steering wheel position are inconsistent, that is, the depth range of the ROI area information 1 and the ROI area information 2 Inconsistent; when the actual depth of the hand position and the theoretical depth of the hand position in the depth model are not greater than the second threshold, it is determined that the hand position is consistent with the steering wheel position, that is, the depth range of ROI area information 1 and ROI area information 2 Consistent.
  • Step S208 Determine the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state and the depth state.
  • determining the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state and the depth state may include: when the hand position is on the steering wheel position, and the depth state represents the hand position When it is inconsistent with the steering wheel position, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the virtual grip state; when the hand position is at the steering wheel position, and the depth state indicates that the hand position is consistent with the steering wheel position, the driver is determined The current state of the hand of the driver and the steering wheel of the target vehicle is the real grip state; when the hand position is not at the position of the steering wheel, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the disengaged state.
  • determining the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state and the depth state may also include: representing the ROI region information in the aggregate state-in the two-dimensional plane and the ROI If there is no intersection between the second region information, the current state of the driver's hand and the steering wheel of the target vehicle is determined to be the off state; the set state means that the ROI region information 1 has an intersection with the ROI region information 2 in the two-dimensional plane , And the depth state indicates that the hand position is inconsistent with the steering wheel position, the current state of the driver’s hand and the steering wheel of the target vehicle is determined to be the virtual grip state; the aggregate state indicates the ROI area information one in the two-dimensional plane and the ROI area information two When there is an intersection, and the depth state indicates that the hand position is consistent with the steering wheel position, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the real grip state.
  • the video stream collected from the target vehicle can be detected to obtain the detection result; then the ROI region information carried in the detection result 1 and the ROI region information 2 are associated with the two-dimensional plane, where the ROI region information One is the ROI area information of the steering wheel of the target vehicle, and the second is the ROI area information of the driver's hand of the target vehicle; obtain the depth information of the ROI area information one and the ROI area information two carried in the detection result to determine the ROI area
  • the depth state of the information one and the ROI area information two; the current state between the driver's hand and the steering wheel of the target vehicle is determined according to the correlation state and the depth state.
  • the video stream of the target vehicle can be detected to obtain the detection result, and then the correlation state and depth state of the ROI area information one and the ROI area information two in the two-dimensional plane are determined, and the driving is determined based on the correlation state and the depth state
  • the current state between the driver’s hand and the steering wheel of the target vehicle realizes the ROI area information of the driver’s hand and the ROI area information of the steering wheel obtained by image analysis.
  • the state of the driver’s hand and the steering wheel of the target vehicle is determined.
  • the purpose is not only to determine whether the current state of the driver’s hand and the steering wheel of the target vehicle is the real grip state or the disengaged state, but also to further determine the virtual grip state, so as to improve the detection of the driver’s hand and steering wheel state.
  • the technical effect of accuracy further solves the technical problem in related technologies that cannot detect the driver's mastery state of the steering wheel during driving of the vehicle.
  • the method of region growth determines according to the depth information of ROI area information 1 and ROI area information 2 Whether the hand position and the steering wheel position can grow into a connected area, if the judgment result is that the area growth map of the hand position and the area growth map of the steering wheel position cannot be merged into a connected area, it means that the depth range of the hand position and the steering wheel position is inconsistent , It is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the virtual grip state.
  • the image in the video stream may be detected to locate the driver's arm position; the hand position obtained based on the positioning model is corrected according to the arm position to obtain the hand position; or According to the arm position and the steering wheel position, it is further judged whether the current state of the driver's hand and the steering wheel of the target vehicle is a virtual grip state; among them, the driver's arm position is detected along the line of the arm corresponding to the arm according to the image in the video stream The three-dimensional information representation of a predetermined number of points.
  • the method for detecting the state of holding a steering wheel may further include: detecting the number of hands; When the number of hands detected is two, the correlation state of the position of the steering wheel and the hand position of each hand of the driver in the two-dimensional plane is obtained.
  • the current state between the driver's hand and the steering wheel of the target vehicle may include one of the following: hands off the steering wheel, one hand off the steering wheel, two hands virtual grip state, one hand virtual grip state, and two hands real grip state , As shown in Figure 3.
  • FIG. 3 is a preferred flowchart of a method for detecting the state of a steering wheel according to an embodiment of the present application.
  • the depth video stream, the combination of the depth video stream and the infrared IR video stream, and the depth video stream Any one of the combination with the RGB video stream, the combination of the depth video stream and the infrared IR video stream, and the RGB video stream is input to the detection module, that is, the input video stream can be only the depth video stream, the depth video stream and Combination of infrared IR video stream, combination of depth video stream and RGB video stream, combination of depth video stream and infrared IR video stream, and RGB video stream.
  • the detection module detects the received video stream, and the detection of the video stream by the detection module is divided into the following two aspects.
  • detecting the image in the video stream to obtain the steering wheel position of the target vehicle may include: determining the initial position of the steering wheel of the target vehicle, and detecting in the initial position according to the image in the video stream, to obtain the position of the steering wheel of the target vehicle. . That is, to detect the continuous images carried in the video stream input to the detection module, where the detection of the steering wheel can be based on prior knowledge to give an approximate range, that is, to obtain the initial position, and perform detection within the approximate range , Get the position of the steering wheel of the target vehicle. As shown in Figure 3, after the steering wheel is detected, it is determined whether the steering wheel is detected, and the position of the steering wheel is output if the determination result is yes; otherwise, it returns to re-detect.
  • detecting the image in the video stream to obtain the driver’s hand position may include: obtaining the driver’s hand position corresponding to the image in the video stream through a positioning model, where the positioning model is a multi-use model.
  • a set of training data is obtained through training, and each set of training data in the multiple sets of training data includes: the image in the video stream and the driver's hand position corresponding to the image in the video stream.
  • the ROI area information 1 of the steering wheel and the ROI area information 2 of the driver's hand of the target vehicle are output to the judgment module as shown in FIG. 3, and the judgment module is started.
  • the judgment module can be used to obtain the associated state of the ROI region information 1 and the ROI region information 2 carried in the detection result in a two-dimensional plane, where the ROI region information 1 is the ROI region information of the steering wheel of the target vehicle, and the ROI region
  • the second information is the ROI area information of the driver's hand of the target vehicle; and is used to determine the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state.
  • the method for detecting the state of the steering wheel can use a camera to detect the target vehicle to obtain the video stream of the target vehicle, and the video stream (for example, the combination of the depth video stream, the depth video stream and the infrared IR video stream) ,
  • the combination of the depth video stream and the RGB video stream, the combination of the depth video stream and the infrared IR video stream and the RGB video stream) real-time analysis and detection, through the detection results of the position of the steering wheel and the hand in the two-dimensional plane Associate the state and the depth state, and in some applications, determine whether the driver is holding the steering wheel with both hands, leaving the steering wheel with one hand, or leaving the steering wheel with both hands based on the positional relationship between human limbs, hands, and steering wheel.
  • the virtual grip can also be accurately determined. At the same time, it can also be used at night to detect the situation of the driver's steering wheel being released or gripped. In the driving process of the driver, monitoring and alarming the driver's behavior of leaving the steering wheel with one or both hands or holding the steering wheel, and giving the corresponding correction feedback mechanism, is of great significance to reducing traffic accidents.
  • a request instruction is generated, where the request instruction is used to control the target vehicle to activate the automatic driving function to prevent the target vehicle from deviating and causing a traffic accident.
  • a device for detecting the state of holding a steering wheel is also provided. It should be noted that the device for detecting the state of holding a steering wheel in an embodiment of the present application can be used to perform the state of holding a steering wheel provided by the embodiment of the present application.
  • the detection method. The detection device for the state of holding a steering wheel provided by an embodiment of the present application will be introduced below.
  • FIG. 4 is a schematic diagram of a detection device for a state of holding a steering wheel according to an embodiment of the present application.
  • the detection device for a state of holding a steering wheel includes a detection unit 41, an acquisition unit 43 and a determination unit 45.
  • the detection device for the state of holding the steering wheel will be described in detail below.
  • the detection unit 41 is used to detect the video stream collected from the target vehicle to obtain the detection result.
  • the obtaining unit 43 is configured to obtain the associated state of the ROI area information 1 and the ROI area information 2 carried in the detection result in a two-dimensional plane, where the ROI area information 1 is the ROI area information of the steering wheel of the target vehicle, and the ROI area information 2 is ROI area information of the driver's hand of the target vehicle.
  • the determining unit 45 is used to determine the current state between the driver's hand and the steering wheel of the target vehicle according to the associated state.
  • the detection unit 41 can be used to detect the video stream collected from the target vehicle to obtain the detection result; and the acquisition unit 43 can be used to obtain the ROI area information one and the ROI area information two carried in the detection result in two dimensions.
  • the video stream of the target vehicle can be detected to obtain the detection result, and then the correlation state of the ROI area information one and the ROI area information two in the two-dimensional plane is determined, and the driver's hand and the target vehicle are determined based on the correlation state.
  • the current state between the steering wheel of the driver is realized based on the image analysis of the ROI area information of the driver’s hand and the ROI area information of the steering wheel.
  • the purpose of determining the state of the driver’s hand and the steering wheel of the target vehicle is achieved, thereby improving the
  • the technical effect of the accuracy of the detection of the driver's hand and the state of the steering wheel further solves the technical problem of the related technology that cannot detect the driver's grasping state of the steering wheel during the driving of the vehicle.
  • the video stream includes at least one of the following: a depth video stream, an infrared IR video stream, and an RGB video stream.
  • the detection unit 41 is configured to detect images in the video stream to obtain the steering wheel position of the target vehicle and the driver's hand position to obtain the detection result.
  • the acquiring unit 43 includes: an associated state acquiring module, configured to use a pre-trained classification model to classify the positional relationship between the steering wheel position and the driver's hand position into different categories, where the categories include : The hand position is on the steering wheel position, but the hand position is not on the steering wheel position.
  • the acquiring unit 43 includes: an associated state acquiring module, which is used to compare the coincidence rate of the steering wheel position and the driver's hand position in the two-dimensional plane with the first threshold to determine the ROI area information.
  • the set state of the two-dimensional plane and the ROI area information two where the set state is used to indicate whether there is an intersection between the ROI area information one and the ROI area information two in the two-dimensional plane.
  • the acquiring unit 43 further includes: a depth state acquiring module, configured to acquire depth information of ROI region information 1 and ROI region information 2 carried in the detection result, and determine ROI region information 1 and ROI region The depth state of the second information, where the depth states of the first ROI area information and the second ROI area information are used to indicate whether the depth ranges of the first ROI area information and the second ROI area information are consistent.
  • a depth state acquiring module configured to acquire depth information of ROI region information 1 and ROI region information 2 carried in the detection result, and determine ROI region information 1 and ROI region The depth state of the second information, where the depth states of the first ROI area information and the second ROI area information are used to indicate whether the depth ranges of the first ROI area information and the second ROI area information are consistent.
  • the depth state acquisition module is also used to determine whether the hand position and the steering wheel position can grow into a connected region according to the depth information of the ROI region information 1 and the ROI region information 2 by using the region growth method .
  • the depth state acquisition module includes: a modeling sub-module, which is used to model the depth of the steering wheel position to obtain a depth model of the steering wheel position; and the acquisition sub-module is used to obtain the actual hand position.
  • the depth and the theoretical depth of the hand position in the depth model; the determination sub-module is used to determine the steering wheel position and the depth state of the hand position according to the actual depth and the theoretical depth of the hand position, wherein the theoretical depth
  • the depth is calculated by inputting the actual depth of the hand position into the depth model of the steering wheel position, and the depth model is obtained through machine learning training using multiple sets of training data.
  • the detection unit 41 includes a steering wheel position detection module for determining the initial position of the steering wheel of the target vehicle, and detecting the initial position according to the image in the video stream to obtain the steering wheel position of the target vehicle.
  • the detection unit 41 includes: a hand position detection module, which is used to obtain the driver’s hand position corresponding to the image in the video stream through a positioning model, wherein the positioning model is a multi-group
  • the training data is obtained by training, and each of the multiple sets of training data includes: the image in the video stream and the driver's hand position corresponding to the image in the video stream.
  • the detection unit 41 may further include: an arm position detection module, which is used to detect images in the video stream and locate the driver’s arm position; and a hand position correction module, which is used to correct The position corrects the hand position based on the positioning model to obtain the hand position.
  • an arm position detection module which is used to detect images in the video stream and locate the driver’s arm position
  • a hand position correction module which is used to correct The position corrects the hand position based on the positioning model to obtain the hand position.
  • the determining unit 45 includes: a first determining subunit, which is used to indicate the ROI area information when the hand position is on the steering wheel position or the collective state-in the two-dimensional plane and When there is an intersection of the second ROI area information, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the real grip state; the second determination subunit is used to indicate when the hand position is not on the steering wheel position or the collective state When there is no intersection between the ROI area information 1 and the ROI area information 2 in the two-dimensional plane, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the off state.
  • the determining unit 45 further includes: a third determining subunit, which is used to determine driving when the aggregate state indicates that the ROI area information 1 does not have an intersection with the ROI area information 2 in the two-dimensional plane.
  • the current state of the driver’s hand and the steering wheel of the target vehicle is the disengaged state;
  • the fourth determining sub-unit is used to indicate that the ROI area information one intersects the ROI area information two in the two-dimensional plane in the aggregate state, and the The depth state indicates that when the depth range of the ROI area information 1 and the ROI area information 2 is inconsistent, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the virtual grip state;
  • the fifth determining subunit is used to set The state indicates that the ROI area information one has an intersection with the ROI area information two in a two-dimensional plane, and the depth state indicates that the depth range of the ROI area information one and the ROI area information two are consistent, determine the driver The current state of your hand and the steering wheel of the target vehicle is a real grip state.
  • the determining unit 45 further includes: a sixth determining subunit, which is used when the hand position is at the steering wheel position, and the depth state indicates that the ROI area information one and the ROI area When the depth range of the second information is inconsistent, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the virtual grip state; the seventh determining subunit is used when the hand position is on the steering wheel position, and the ROI area information When the first is consistent with the depth range of the second ROI area information, it is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the real grip state; the eighth determining subunit is used for when the hand position is not on the steering wheel position, It is determined that the current state of the driver's hand and the steering wheel of the target vehicle is the disengaged state.
  • the current state between the driver's hand and the steering wheel of the target vehicle includes one of the following: two hands are separated from the steering wheel, one hand is separated from the steering wheel, two-hand virtual grip state, one-hand virtual grip state, two hands real state Grip state.
  • the acquiring unit is also used to detect the number of hands before acquiring the ROI area information one and the ROI area information two carried in the detection result in a two-dimensional plane; the detection result is When the number of the hands is two, the associated state of the steering wheel position and the hand position of each hand of the driver in the two-dimensional plane is acquired.
  • the detection device for the state of the steering wheel described above includes a processor and a memory.
  • the detection unit 41, the acquisition unit 43, and the determination unit 45 are all stored as program units in the memory, and the processor executes the program units stored in the memory to implement The corresponding function.
  • the above-mentioned processor contains a kernel, and the kernel calls the corresponding program unit from the memory.
  • One or more kernels can be set, and the state of the driver's hand and the steering wheel of the target vehicle can be determined according to the associated state by adjusting the kernel parameters.
  • the above-mentioned memory may include non-permanent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least A memory chip.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash memory
  • a storage medium includes a stored program, wherein the program executes any one of the aforementioned methods for detecting the state of holding a steering wheel.
  • a processor which is used to run a program, wherein, when the program is running, any one of the above-mentioned methods for detecting the state of the steering wheel is executed.
  • a device in the embodiment of the present application, includes a processor, a memory, and a program stored in the memory and running on the processor.
  • the processor executes the program, the following steps are implemented: ROI area information 1 and ROI area information 2 in the two-dimensional plane associated state carried in the detection result, where ROI area information 1 is the ROI area information of the steering wheel of the target vehicle, ROI
  • the second area information is the ROI area information of the driver's hand of the target vehicle; the current state between the driver's hand and the steering wheel of the target vehicle is determined according to the associated state.
  • the embodiment of the present application also provides a computer program product, which when executed on a data processing device, is suitable for executing a program that initializes the following method steps: detecting a video stream collected from a target vehicle to obtain a detection result ; Obtain the associated state of ROI area information 1 and ROI area information 2 carried in the detection result in a two-dimensional plane, where ROI area information 1 is the ROI area information of the steering wheel of the target vehicle, and ROI area information 2 is the driver of the target vehicle ROI area information of the hand of the driver; determine the current state between the driver’s hand and the steering wheel of the target vehicle according to the associated state.
  • the disclosed technical content can be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units may be a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one detection unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
  • Steering Control In Accordance With Driving Conditions (AREA)

Abstract

本申请公开了一种手握方向盘状态的检测方法及装置。其中，该方法包括：对从目标车辆中采集的视频流进行检测，得到检测结果；获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。本申请解决了相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的技术问题。

Description

手握方向盘状态的检测方法及装置
本申请要求于2019年3月28日提交至中国国家知识产权局、申请号为201910245398.8、发明名称为“手握方向盘状态的检测方法及装置”的专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本申请涉及图像检测技术领域，具体而言，涉及一种手握方向盘状态的检测方法及装置。
背景技术
目前，汽车成为一种很普遍的交通工具。私家车、运输用车的数量快速增长，给人们生活带来便利。同时驾驶人员能力、经验的参差不齐，安全意识不够到位，超负荷疲劳驾驶等，也带来很多交通问题。根据事故统计，有很大部分交通事故的发生，是由驾驶人员在车辆行驶过程中分心，打电话、发送信息、吃东西等引发的。在做这些干扰开车的行为时，驾驶人员的单手或者双手脱离了方向盘，失去对方向盘的完全掌控，手在与不在方向盘上，其注意力分散情况有很大差异。当危险发生时，驾驶人员便很难马上做出避险反应，酿成事故。而目前，还无法实现对于司机手脱离方向盘的行为进行监控和报警。
针对上述相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的问题，目前尚未提出有效的解决方案。
发明内容
本申请实施例提供了一种手握方向盘状态的检测方法及装置，以至少解决相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的技术问题。
根据本申请实施例的一个方面，提供了一种手握方向盘状态的检测方法，包括：对从目标车辆中采集的视频流进行检测，得到检测结果；获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，所述ROI区域信息一为所述目标车辆的方向盘的ROI区域信息，所述ROI区域信息二为所述目标车辆的驾驶员的手的ROI区域信息；根据所述关联状态确定所述驾驶员的手与所述目标车辆 的方向盘之间的当前状态。
可选地，所述视频流包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。
可选地，对从目标车辆中采集的视频流进行检测，得到检测结果包括：对所述视频流中的图像进行检测，得到所述目标车辆的方向盘位置和所述驾驶员的手部位置，以得到所述检测结果。
可选地，获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态包括：使用预先训练的分类模型将所述方向盘位置和所述驾驶员的手部位置的位置关系分为不同的类别，其中，所述类别包括：所述手部位置在所述方向盘位置上，所述手部位置不在所述方向盘位置上。
可选地，获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态包括：比较所述方向盘位置与所述驾驶员的手部位置在所述二维平面的重合率和第一阈值，以确定所述ROI区域信息一在二维平面中与所述ROI区域信息二的集合状态，其中，所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二是否存在交集。
可选地，所述方法还包括：获取所述检测结果中携带的所述ROI区域信息一与所述ROI区域信息二的深度信息，确定所述ROI区域信息一与所述ROI区域信息二的深度状态，其中，所述ROI区域信息一与所述ROI区域信息二的深度状态用于表示所述ROI区域信息一与所述ROI区域信息二的深度范围是否一致。
可选地，所述方法还包括：根据所述ROI区域信息一与所述ROI区域信息二的深度信息，利用区域生长的方法判断手部位置与方向盘位置能否生长成为一个连通区域，以确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
可选地，获取检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，确定ROI区域信息一与ROI区域信息二的深度状态包括：对所述方向盘位置的深度进行建模，得到所述方向盘位置的深度模型；获取所述手部位置的实际深度和所述手部位置在所述深度模型中的理论深度；根据所述手部位置的实际深度和理论深度，确定所述方向盘位置和所述手部位置的深度状态。
可选地，对所述视频流中的图像进行检测，得到所述目标车辆的方向盘位置包括：确定所述目标车辆的方向盘的初始位置，根据所述视频流中的图像在所述初始位置内检测，得到所述目标车辆的方向盘位置。
可选地，对所述视频流中的图像进行检测，得到所述驾驶员的手部位置包括：通过定位模型，获取与所述视频流中的图像对应的所述驾驶员的手部位置，其中，所述定位模型为使用多组训练数据训练得到的，所述多组训练数据中的每组训练数据均包括：视频流中的图像和所述视频流中的图像对应的驾驶员的手部位置。
可选地，对所述视频流中的图像进行检测，得到所述驾驶员的手部位置还包括：对所述视频流中的图像进行检测，定位得到所述驾驶员的手臂位置；根据所述手臂位置对基于所述定位模型得到的手部位置进行修正，得到所述手部位置。
可选地，所述驾驶员的手臂位置由根据所述视频流中的图像检测得到的手臂对应的手臂沿线上的预定数量的点的三维信息表示。
可选地，根据所述关联状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括：当所述手部位置在所述方向盘位置上或所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；当所述手部位置不在所述方向盘位置上或所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
可选地，根据所述关联状态和所述深度状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括：在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集的情况下，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态；在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态。
可选地，根据所述关联状态和所述深度状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括：当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握 状态；当所述手部位置不在所述方向盘位置上时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
可选地，所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括以下之一：双手脱离方向盘、单手脱离方向盘、双手虚握状态、单手虚握状态、双手实握状态。
可选地，在获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态之前，该手握方向盘状态的检测方法还包括：检测到手的数量；在检测到所述手的数量为两个时，获取所述方向盘位置与所述驾驶员的每只手的手部位置在所述二维平面的关联状态。
根据本申请实施例的另外一个方面，还提供了一种手握方向盘状态的检测装置，包括：检测单元，用于对从目标车辆中采集的视频流进行检测，得到检测结果；获取单元，用于获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，所述ROI区域信息一为所述目标车辆的方向盘的ROI区域信息，所述ROI区域信息二为所述目标车辆的驾驶员的手的ROI区域信息；确定单元，用于根据所述关联状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
可选地，所述视频流包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。
可选地，所述检测单元，用于对所述视频流中的图像进行检测，得到所述目标车辆的方向盘位置和所述驾驶员的手部位置，以得到所述检测结果。
可选地，所述获取单元包括：关联状态获取模块，用于使用预先训练的分类模型将所述方向盘位置和所述驾驶员的手部位置的位置关系分为不同的类别，其中，所述类别包括：所述手部位置在所述方向盘位置上，所述手部位置不在所述方向盘位置上。
可选地，所述获取单元包括：关联状态获取模块，用于比较所述方向盘位置与所述驾驶员的手部位置在所述二维平面的重合率和第一阈值，以确定所述ROI区域信息一在二维平面中与所述ROI区域信息二的集合状态，其中，所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二是否存在交集。
可选地，所述获取单元还包括：深度状态获取模块，用于获取所述检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，并确定ROI区域信息一与ROI区域信息二的深度状态，其中，所述ROI区域信息一与所述ROI区域信息二的深度状态用于表示所述ROI区域信息一与所述ROI区域信息二的深度范围是否一致。
可选地，所述深度状态获取模块还用于根据所述ROI区域信息一与所述ROI区域 信息二的深度信息，用区域生长的方法判断手部位置与方向盘位置能否生长成为一个连通区域，以确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
可选地，所述深度状态获取模块包括：建模子模块，用于对所述方向盘位置的深度进行建模，得到所述方向盘位置的深度模型；获取子模块，用于获取所述手部位置的实际深度和所述手部位置在所述深度模型中的理论深度；确定子模块，用于根据所述手部位置的实际深度和理论深度，确定所述方向盘位置和所述手部位置的深度状态。
可选地，所述检测单元包括：方向盘位置检测模块，用于确定所述目标车辆的方向盘的初始位置，根据所述视频流中的图像在所述初始位置内检测，得到所述目标车辆的方向盘位置。
可选地，所述检测单元包括：手部位置检测模块，用于通过定位模型，获取与所述视频流中的图像对应的所述驾驶员的手部位置，其中，所述定位模型为使用多组训练数据训练得到的，所述多组训练数据中的每组训练数据均包括：视频流中的图像和所述视频流中的图像对应的驾驶员的手部位置。
可选地，所述检测单元还包括：手臂位置检测模块，用于对所述视频流中的图像进行检测，定位得到所述驾驶员的手臂位置；手部位置修正模块，用于根据所述手臂位置对基于所述定位模型得到的手部位置进行修正，得到所述手部位置。
可选地，所述确定单元包括：第一确定子单元，用于当所述手部位置在所述方向盘位置上或所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；第二确定子单元，用于当所述手部位置不在所述方向盘位置上或所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
可选地，所述确定单元还包括：第三确定子单元，用于在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集的情况下，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态；第四确定子单元，用于在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；第五确定子单元，用于在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目 标车辆的方向盘的当前状态为实握状态。
可选地，所述确定单元还包括：第六确定子单元，用于当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；第七确定子单元，用于当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；第八确定子单元，用于当所述手部位置不在所述方向盘位置上时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
可选地，所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括以下之一：双手脱离方向盘、单手脱离方向盘、双手虚握状态、单手虚握状态、双手实握状态。
可选地，所述获取单元在获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态之前，检测手的数量；在所述检测结果为所述手的数量为两个时，获取所述方向盘位置与所述驾驶员的每只手的手部位置在所述二维平面的关联状态。
根据本申请实施例的另外一个方面，还提供了一种存储介质，所述存储介质包括存储的程序，其中，所述程序执行上述中任意一项所述的手握方向盘状态的检测方法。
根据本申请实施例的另外一个方面，还提供了一种处理器，所述处理器用于运行程序，其中，所述程序运行时执行上述中任意一项所述的手握方向盘状态的检测方法。
在本申请实施例中，采用对从目标车辆中采集的视频流进行检测，得到检测结果；并获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；以及根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态的方式对驾驶员手握方向盘状态进行检测，实现了根据图像分析得到的驾驶员的手的ROI区域信息一与方向盘的ROI区域信息二确定驾驶员的手与目标车辆的方向盘的状态的目的，达到了提高在对驾驶员的手与方向盘的状态进行检测的精确度的技术效果，进而解决了相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的技术问题。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申 请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1是根据本申请实施例的手握方向盘状态的检测方法的流程图；
图2是根据本申请实施例的可选的手握方向盘状态的检测方法的流程图；
图3是根据本申请实施例的手握方向盘状态的检测方法的优选的流程图；
图4是根据本申请实施例的手握方向盘状态的检测装置的示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
随着先进驾驶辅助系统(Advanced Driver Assistance System，简称ADAS)的发展，驾驶人员在很多时候都可以依赖其进行操作，但是如果过于信赖ADAS系统，就比较容易出现滥用该系统的风险，例如，在低速和倒车或停车的情况下，可以通过自主的转向系统来控制车辆，手脱离方向盘是可以接受的。但是在高速路段，ADAS只能以辅助为主，需要驾驶人员集中注意力在驾驶车辆上，此时就不可出现双手脱离方向盘的情况。因此，ADAS系统需要加入驾驶员方向盘脱手检测，在不同情境下给出合适的辅助。
传统的方向盘脱手检测利用硬件进行检测。具体地，利用硬件进行检测是直接使用电容传感器来检测驾驶员对方向盘的握紧程度。其优点在于，传感器与驾驶员直接接触，得到的结果准确、反应灵敏，然而往往会导致脱手检测系统的成本较高。
实施例1
根据本申请实施例，提供了一种手握方向盘状态的检测方法的方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图1是根据本申请实施例的手握方向盘状态的检测方法的流程图，如图1所示，该手握方向盘状态的检测方法包括如下步骤：
步骤S102，对从目标车辆中采集的视频流进行检测，得到检测结果。
其中，上述视频流可以包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。其中，IR是红外线Infrared Radiation的简称，是一种无线通讯方式，可以进行无线数据的传输。由于车内(例如，驾驶室)的光线通常会跟随行车环境而变化，在天气晴好的白天，车内(例如，驾驶室)的光线较为明亮，在夜晚或者阴天以及隧道内，车内(例如，驾驶室)的光线则较暗，而红外摄像头受光照变化影响小，具有全天候工作的能力，因此可以选择红外摄像头(包括近红外摄像头等)获取质量优于普通摄像头的红外线IR视频流，从而提高检测结果的准确度。
另外，在本申请实施例中对于用于采集视频流的采集设备的安装位置也不做具体限定，只要可以采集到上述视频流即可。对于采集视角可以在目标车辆的顶端向下的视角、第一人视角、第三人视角。上述采集设备可以为道路监控设备，例如，道路关卡处的监控设备等。
另外，对从目标车辆中采集的视频流进行检测，得到检测结果可以包括：对视频流中的图像进行检测，得到目标车辆的方向盘位置和驾驶员的手部位置，以得到检测结果。
作为一种可选的实施例，对视频流中的图像进行检测，得到目标车辆的方向盘位置可以包括：确定目标车辆的方向盘的初始位置，根据视频流中的图像在初始位置内检测，得到目标车辆的方向盘位置。
另外一个方面，对视频流中的图像进行检测，得到驾驶员的手部位置可以包括：通过定位模型，获取与视频流中的图像对应的驾驶员的手部位置，其中，定位模型为使用多组训练数据训练得到的，多组训练数据中的每组训练数据均包括：视频流中的图像和视频流中的图像对应的驾驶员的手部位置。
步骤S104，获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面 上的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息。
即，上述检测结果中还携带有目标车辆的方向盘的ROI区域信息和目标车辆的驾驶员的手的ROI区域信息。
在一个实施例中，获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面上的关联状态的一种方法可以为：使用预先训练的分类模型将方向盘位置和驾驶员的手部位置的位置关系分为不同类别，其中，类别包括：手部位置在方向盘位置上，手部位置不在方向盘位置上。具体地，手部位置不在方向盘位置上可以包括手部位置在所述方向盘下方或相邻的位置。
在另外一个可选的实施例中，获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面上的关联状态可以包括：比较方向盘位置与驾驶员的手部位置在二维平面的重合率和第一阈值，以确定ROI区域信息一在二维平面中与ROI区域信息二的集合状态，其中，集合状态用于表示ROI区域信息一在二维平面中与ROI区域信息二是否存在交集。
例如，可以将方向盘位置与驾驶员的手部位置在二维平面的重合率与第一阈值进行比较，在比较结果为方向盘位置与驾驶员的手部位置在二维平面的重合率大于第一阈值的情况下，确定手部位置在方向盘位置上；在比较结果为方向盘位置与驾驶员的手部位置在二维平面的重合率不大于第一阈值的情况下，确定手部位置不在方向盘位置上，从而确定驾驶员的手与目标车辆的方向盘之间的位置关系。
步骤S106，根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。
具体地，根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态可以包括：当手部位置在方向盘位置上时，确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态；当手部位置不在方向盘位置上时，确定驾驶员的手与目标车辆的方向盘的当前状态为脱离状态。
通过上述步骤，可以对从目标车辆中采集的视频流进行检测，得到检测结果；然后获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；并根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。在该实施例中，可以对目标车辆的视频流进行检测得到检测结果，再确定ROI区域信息一与ROI区域信息二在二维平面上的关联状态，基于此关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态，实现了根据图像分析得到的驾 驶员的手的ROI区域信息一与方向盘的ROI区域信息二，确定驾驶员的手与目标车辆的方向盘的状态的目的，达到了提高在对驾驶员的手与方向盘的状态进行检测的精确度的技术效果，进而解决了相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的技术问题。但这种方式实际上只能判断出驾驶员手部与方向盘在视觉上是否重叠，以确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态还是脱离状态。如果驾驶员虚握方向盘，即，手在方向盘正上方做出握方向盘的动作，实际上并未接触到方向盘，就无法正确判断驾驶员已经有方向盘脱手的行为。为了进一步提高对驾驶员的手握方向盘状态的检测精确度，本申请实施例中通过使用深度视频流中驾驶员的手与目标车辆的方向盘的深度信息判断两者的深度范围是否一致，以确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态、虚握状态还是脱离状态。下面结合以下实施例进行详细说明。
实施例2
根据本申请实施例，提供了一种手握方向盘状态的检测方法的方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图2是根据本申请实施例的可选的手握方向盘状态的检测方法的流程图，如图2所示，该手握方向盘状态的检测方法包括如下步骤：
步骤S202，对从目标车辆中采集的视频流进行检测，得到检测结果。
可选地，视频流包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。由于仅仅是单一使用RGB视频流或红外线IR视频流或RGB视频流与红外线IR视频流的组合，无法判断驾驶员的手与方向盘的深度位置，检测不出虚握的情况。另外，在夜间行车时，只使用RGB视频流则可能影响判断驾驶员的手与方向盘的状态。因此，在本申请实施例中，为了使得检测结果更加全面、适应性更好，视频流中可以包括多种类型，但是每种类型中均包括深度视频流。
另外，在本申请实施例中对视频流的采集设备不做具体限定，只要可以采集深度视频流即可；对于采集设备的安装位置也不做具体限定，只要可以采集到上述视频流即可。对于采集视角可以在目标车辆的顶端向下的视角、第一人视角、第三人视角。上述采集设备可以为道路监控设备，例如，道路关卡处的监控设备等。
在步骤S202中，对从目标车辆中采集的视频流进行检测，得到检测结果可以包括：对视频流中的图像进行检测，得到目标车辆的方向盘位置和驾驶员的手部位置，以得 到检测结果。
步骤S204，获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息。
其中，这里的ROI区域为感兴趣区域(Region of Interest，简称ROI)，在机器视觉、图像处理中，从被处理的图像以方框、圆、椭圆、不规则多边形等方式勾勒出需要处理的区域，称为感兴趣区域。该ROI区域在图像分析时可以作为图像分析的重点区域。这里的ROI区域信息是二维平面内的坐标位置。
作为一种可选的实施例，获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态可以包括：使用预先训练的分类模型将方向盘位置和驾驶员的手部位置的位置关系分为不同的类别，其中，类别包括：手部位置在方向盘位置上，手部位置不在方向盘位置上。
作为另一种可选的实施例，获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态可以包括：比较方向盘位置与驾驶员的手部位置在二维平面的重合率和第一阈值，以确定ROI区域信息一在二维平面中与ROI区域信息二的集合状态，其中，集合状态用于表示ROI区域信息一在二维平面中与ROI区域信息二是否存在交集。
例如，可以将方向盘位置与驾驶员的手部位置在二维平面的重合率与第一阈值进行比较，在比较结果为方向盘位置与驾驶员的手部位置在二维平面的重合率大于第一阈值的情况下，确定手部位置在方向盘位置上；在比较结果为方向盘位置与驾驶员的手部位置在二维平面的重合率不大于第一阈值的情况下，确定手部位置不在方向盘位置上，从而确定驾驶员的手与目标车辆的方向盘之间的位置关系。
步骤S206，获取检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，确定ROI区域信息一与ROI区域信息二的深度状态。
可选地，ROI区域信息一与ROI区域信息二的深度状态包括：ROI区域信息一与ROI区域信息二的深度范围是否一致。
在一个实施例中，获取检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，确定ROI区域信息一与ROI区域信息二的深度状态可以包括：对方向盘位置(ROI区域信息一)的深度进行建模，得到方向盘位置(ROI区域信息一)的深度模型；获取手部位置(ROI区域信息二)的实际深度和手部位置(ROI区域信息二)在深度模型中的理论深度；根据手部位置(ROI区域信息二)的实际深度和理论深度，确定方 向盘位置(ROI区域信息一)和手部位置(ROI区域信息二)的深度状态。其中，获取手部位置(ROI区域信息二)的理论深度可以是将手部位置(ROI区域信息二)的实际深度输入方向盘位置(ROI区域信息一)的深度模型后，基于深度模型的输出得到的，方向盘位置的深度模型是使用多组训练数据通过机器学习训练得到的。
在一个实施例中，根据手部位置的实际深度和理论深度，确定方向盘位置和手部位置的深度状态可以包括：将手部位置的实际深度与手部位置在深度模型中的理论深度与第二阈值进行比较，以确定方向盘位置和手部位置的深度状态。具体地，当手部位置的实际深度与手部位置在深度模型中的理论深度大于第二阈值时，确定手部位置与方向盘位置不一致，即，ROI区域信息一与ROI区域信息二的深度范围不一致；当手部位置的实际深度与手部位置在深度模型中的理论深度不大于第二阈值时，确定手部位置与方向盘位置一致，即，ROI区域信息一与ROI区域信息二的深度范围一致。
步骤S208，根据关联状态和深度状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。
作为一种可选的实施例，根据关联状态和深度状态确定驾驶员的手与目标车辆的方向盘之间的当前状态可以包括：当手部位置在方向盘位置上时，并且深度状态表示手部位置与方向盘位置不一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为虚握状态；当手部位置在方向盘位置上时，并且深度状态表示手部位置与方向盘位置一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态；当手部位置不在方向盘位置上时，确定驾驶员的手与目标车辆的方向盘的当前状态为脱离状态。
作为另一种可选的实施例，根据关联状态和深度状态确定驾驶员的手与目标车辆的方向盘之间的当前状态还可以包括：在集合状态表示ROI区域信息一在二维平面中与ROI区域信息二不存在交集的情况下，确定驾驶员的手与目标车辆的方向盘的当前状态为脱离状态；在集合状态表示ROI区域信息一在二维平面中与ROI区域信息二存在交集的情况下，并且深度状态表示手部位置与方向盘位置不一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为虚握状态；在集合状态表示ROI区域信息一在二维平面中与ROI区域信息二存在交集的情况下，并且深度状态表示手部位置与方向盘位置一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态。
通过上述步骤，可以对从目标车辆中采集的视频流进行检测，得到检测结果；然后获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；获取检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，确定ROI区域信息一与ROI区域信息二的深度状态；根据关联状 态和深度状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。在该实施例中，可以对目标车辆的视频流进行检测得到检测结果，再确定ROI区域信息一与ROI区域信息二在二维平面的关联状态和深度状态，基于此关联状态和深度状态确定驾驶员的手与目标车辆的方向盘之间的当前状态，实现了根据图像分析得到的驾驶员的手的ROI区域信息一与方向盘的ROI区域信息二确定驾驶员的手与目标车辆的方向盘的状态的目的，不仅可以确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态还是脱离状态，还可以进一步判断出虚握状态，达到了提高在对驾驶员的手与方向盘的状态进行检测的精确度的技术效果，进而解决了相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的技术问题。
另外，为了提高对驾驶员的手与方向盘的状态进行检测的精确度，作为一种可选的实施例，还可以根据ROI区域信息一与ROI区域信息二的深度信息，用区域生长的方法判断手部位置与方向盘位置能否生长成为一个连通区域，若判断结果是手部位置的区域生长图与方向盘位置的区域生长图无法融合为一个连通区域，说明手部位置与方向盘位置的深度范围不一致，确定驾驶员的手与目标车辆的方向盘的当前状态为虚握状态。
作为另一种可选的实施例，还可以对视频流中的图像进行检测，定位得到驾驶员的手臂位置；根据手臂位置对基于定位模型得到的手部位置进行修正，得到手部位置；或者根据手臂位置和方向盘位置进一步判断确定驾驶员的手与目标车辆的方向盘的当前状态是否为虚握状态；其中，驾驶员的手臂位置由根据视频流中的图像检测得到的手臂对应的手臂沿线上的预定数量的点的三维信息表示。
为了细化检测结果，在获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态之前，该手握方向盘状态的检测方法还可以包括：检测手的数量；在检测到手的数量为两个时，获取方向盘位置与驾驶员的每只手的手部位置在二维平面的关联状态。
通过上述实施例可知，驾驶员的手与目标车辆的方向盘之间的当前状态可以包括以下之一：双手脱离方向盘、单手脱离方向盘、双手虚握状态、单手虚握状态、双手实握状态，如图3中所示。
例如，图3是根据本申请实施例的手握方向盘状态的检测方法的优选的流程图，如图3所示，可以将深度视频流、深度视频流与红外线IR视频流的组合、深度视频流与RGB视频流的组合、深度视频流与红外线IR视频流以及RGB视频流的组合中的任意一种视频流输入到检测模块中，即，输入视频流可以是只有深度视频流、深度视频流和红外线IR视频流的组合、深度视频流与RGB视频流的组合、深度视频流与红外线 IR视频流以及RGB视频流的组合。检测模块对接收到的视频流进行检测，其中，检测模块对视频流的检测分为以下两个方面。
一个方面，对视频流中的图像进行检测，得到目标车辆的方向盘位置可以包括：确定目标车辆的方向盘的初始位置，根据视频流中的图像在初始位置内检测，得到目标车辆的方向盘的位置一。即，对输入到检测模块的视频流中携带的连续图像进行检测，其中，对方向盘的检测可以为先根据先验知识，给出大致范围，即，得到初始位置，在该大致范围内进行检测，得到目标车辆的方向盘的位置一。如图3所示，在对方向盘进行检测之后，判断是否检测到方向盘，在判断结果为是的情况下输出方向盘的位置一；反之，返回进行重新检测。
另外一个方面，对视频流中的图像进行检测，得到驾驶员的手部位置可以包括：通过定位模型，获取与视频流中的图像对应的驾驶员的手部位置，其中，定位模型为使用多组训练数据训练得到的，多组训练数据中的每组训练数据均包括：视频流中的图像和视频流中的图像对应的驾驶员的手部位置。
其中，在检测到方向盘的情况下，将方向盘的ROI区域信息一和目标车辆的驾驶员的手的ROI区域信息二输出到如图3中所示的判断模块中，并启动判断模块。
具体地，该判断模块可以用于获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；并用于根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。
最后，输出驾驶员的手与目标车辆的方向盘之间的当前状态。
通过本申请实施例提供的手握方向盘状态的检测方法可以利用摄像头对目标车辆进行检测，得到目标车辆的视频流，对视频流(例如，深度视频流、深度视频流与红外线IR视频流的组合、深度视频流与RGB视频流的组合、深度视频流与红外线IR视频流以及RGB视频流的组合中的一种)进行实时分析检测，通过对检测结果中方向盘与手的位置在二维平面的关联状态以及深度状态，并在一些应用中结合人肢体与手、方向盘的位置关系判断驾驶员是双手握方向盘、单手脱离方向盘、双手脱离方向盘，对于虚握的情况也可以准确判断。同时也可以在夜间使用，检测驾驶员的方向盘脱手、虚握的情况。在驾驶员的驾驶过程中，对驾驶员单手或双手脱离、虚握方向盘的行为进行监控和报警，并给出相应的修正反馈机制，对减少交通事故发生有着重要的意义。
优选的，在检测结果为双手脱离、虚握方向盘时，会生成请求指令，其中，该请求指令用于控制目标车辆启动自动驾驶功能，以防止目标车辆出现跑偏引起交通事故。
实施例3
根据本申请实施例还提供了一种手握方向盘状态的检测装置，需要说明的是，本申请实施例的手握方向盘状态的检测装置可以用于执行本申请实施例所提供的手握方向盘状态的检测方法。以下对本申请实施例提供的手握方向盘状态的检测装置进行介绍。
图4是根据本申请实施例的手握方向盘状态的检测装置的示意图，如图4所示，该手握方向盘状态的检测装置包括：检测单元41，获取单元43以及确定单元45。下面对该手握方向盘状态的检测装置进行详细说明。
检测单元41，用于对从目标车辆中采集的视频流进行检测，得到检测结果。
获取单元43，用于获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息。
确定单元45，用于根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。
在该实施例中，可以利用检测单元41对从目标车辆中采集的视频流进行检测，得到检测结果；并利用获取单元43获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；以及确定单元45根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。在该实施例中，可以对目标车辆的视频流进行检测得到检测结果，再确定ROI区域信息一与ROI区域信息二在二维平面的关联状态，基于此关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态，实现了根据图像分析得到的驾驶员的手的ROI区域信息一与方向盘的ROI区域信息二确定驾驶员的手与目标车辆的方向盘的状态的目的，达到了提高对驾驶员的手与方向盘的状态进行检测的精确度的技术效果，进而解决了相关技术中无法实现对驾驶员在驾驶车辆过程中对方向盘的掌握状态进行检测的技术问题。
作为一种可选的实施例，视频流包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。
作为一种可选的实施例，检测单元41，用于对视频流中的图像进行检测，得到目标车辆的方向盘位置和驾驶员的手部位置，以得到检测结果。
作为一种可选的实施例，获取单元43包括：关联状态获取模块，用于使用预先训 练的分类模型将方向盘位置和驾驶员的手部位置的位置关系分为不同的类别，其中，类别包括：手部位置在方向盘位置上，手部位置不在方向盘位置上。
作为一种可选的实施例，获取单元43包括：关联状态获取模块，用于比较方向盘位置与驾驶员的手部位置在二维平面的重合率和第一阈值，以确定ROI区域信息一在二维平面中与ROI区域信息二的集合状态，其中，集合状态用于表示ROI区域信息一在二维平面中与ROI区域信息二是否存在交集。
作为一种可选的实施例，获取单元43还包括：深度状态获取模块，用于获取检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，并确定ROI区域信息一与ROI区域信息二的深度状态，其中ROI区域信息一与ROI区域信息二的深度状态用于表示ROI区域信息一与ROI区域信息二的深度范围是否一致。
作为一种可选的实施例，深度状态获取模块，还用于根据ROI区域信息一与ROI区域信息二的深度信息，用区域生长的方法判断手部位置与方向盘位置能否生长成为一个连通区域。
作为一种可选的实施例，深度状态获取模块包括：建模子模块，用于对方向盘位置的深度进行建模，得到方向盘位置的深度模型；获取子模块，用于获取手部位置的实际深度和手部位置在深度模型中的理论深度；确定子模块，用于根据所述手部位置的实际深度和理论深度，确定所述方向盘位置和所述手部位置的深度状态，其中，理论深度是将手部位置的实际深度输入方向盘位置的深度模型计算后输出的，深度模型是使用多组训练数据通过机器学习训练得到的。
作为一种可选的实施例，检测单元41包括：方向盘位置检测模块，用于确定目标车辆的方向盘的初始位置，根据视频流中的图像在初始位置内检测，得到目标车辆的方向盘位置。
作为一种可选的实施例，检测单元41包括：手部位置检测模块，用于通过定位模型，获取与视频流中的图像对应的驾驶员的手部位置，其中，定位模型为使用多组训练数据训练得到的，多组训练数据中的每组训练数据均包括：视频流中的图像和视频流中的图像对应的驾驶员的手部位置。
作为一种可选的实施例，检测单元41还可以包括：手臂位置检测模块，用于对视频流中的图像进行检测，定位得到驾驶员的手臂位置；手部位置修正模块，用于根据手臂位置对基于定位模型得到的手部位置进行修正，得到手部位置。
作为一种可选的实施例，确定单元45包括：第一确定子单元，用于当手部位置在方向盘位置上或集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI 区域信息二存在交集时，确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态；第二确定子单元，用于当手部位置不在方向盘位置上或集合状态用于表示ROI区域信息一在二维平面中与ROI区域信息二不存在交集时，确定驾驶员的手与目标车辆的方向盘的当前状态为脱离状态。
作为一种可选的实施例，确定单元45还包括：第三确定子单元，用于在集合状态表示ROI区域信息一在二维平面中与ROI区域信息二不存在交集的情况下，确定驾驶员的手与目标车辆的方向盘的当前状态为脱离状态；第四确定子单元，用于在集合状态表示ROI区域信息一在二维平面中与ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为虚握状态；第五确定子单元，用于在集合状态表示ROI区域信息一在二维平面中与ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态。
作为一种可选的实施例，确定单元45还包括：第六确定子单元，用于当手部位置在方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为虚握状态；第七确定子单元，用于当手部位置在方向盘位置上时，并且所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定驾驶员的手与目标车辆的方向盘的当前状态为实握状态；第八确定子单元，用于当手部位置不在方向盘位置上时，确定驾驶员的手与目标车辆的方向盘的当前状态为脱离状态。
作为一种可选的实施例，驾驶员的手与目标车辆的方向盘之间的当前状态包括以下之一：双手脱离方向盘、单手脱离方向盘、双手虚握状态、单手虚握状态、双手实握状态。
作为一种可选的实施例，获取单元在获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态之前，还用于检测手的数量；在所述检测结果为所述手的数量为两个时，获取所述方向盘位置与所述驾驶员的每只手的手部位置在所述二维平面的关联状态。
上述手握方向盘状态的检测装置包括处理器和存储器，上述检测单元41，获取单元43以及确定单元45等均作为程序单元存储在存储器中，由处理器执行存储在存储器中的上述程序单元来实现相应的功能。
上述处理器中包含内核，由内核去存储器中调取相应的程序单元。内核可以设置 一个或以上，通过调整内核参数根据关联状态确定驾驶员的手与目标车辆的方向盘的状态。
上述存储器可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)，存储器包括至少一个存储芯片。
根据本申请实施例的另外一个方面，还提供了一种存储介质，存储介质包括存储的程序，其中，程序执行上述中任意一项的手握方向盘状态的检测方法。
根据本申请实施例的另外一个方面，还提供了一种处理器，处理器用于运行程序，其中，程序运行时执行上述中任意一项的手握方向盘状态的检测方法。
在本申请实施例中还提供了一种设备，该设备包括处理器、存储器及存储在存储器上并可在处理器上运行的程序，处理器执行程序时实现以下步骤：对从目标车辆中采集的视频流进行检测，得到检测结果；获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。
在本申请实施例中还提供了一种计算机程序产品，当在数据处理设备上执行时，适于执行初始化有如下方法步骤的程序：对从目标车辆中采集的视频流进行检测，得到检测结果；获取检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，ROI区域信息一为目标车辆的方向盘的ROI区域信息，ROI区域信息二为目标车辆的驾驶员的手的ROI区域信息；根据关联状态确定驾驶员的手与目标车辆的方向盘之间的当前状态。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
在本申请的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，可以为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个检测单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅是本申请的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (36)

  1. 一种手握方向盘状态的检测方法，包括：
    对从目标车辆中采集的视频流进行检测，得到检测结果；
    获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，所述ROI区域信息一为所述目标车辆的方向盘的ROI区域信息，所述ROI区域信息二为所述目标车辆的驾驶员的手的ROI区域信息；
    根据所述关联状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
  2. 根据权利要求1所述的方法，其中，所述视频流包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。
  3. 根据权利要求1所述的方法，其中，对从目标车辆中采集的视频流进行检测，得到检测结果包括：对所述视频流中的图像进行检测，得到所述目标车辆的方向盘位置和所述驾驶员的手部位置，以得到所述检测结果。
  4. 根据权利要求1所述的方法，其中，获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态包括：使用预先训练的分类模型将所述方向盘位置和所述驾驶员的手部位置的位置关系分为不同的类别，其中，所述类别包括：所述手部位置在所述方向盘位置上，所述手部位置不在所述方向盘位置上。
  5. 根据权利要求1所述的方法，其中，获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态包括：比较所述方向盘位置与所述驾驶员的手部位置在所述二维平面的重合率和第一阈值，以确定所述ROI区域信息一在二维平面中与所述ROI区域信息二的集合状态，其中，所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二是否存在交集。
  6. 根据权利要求4或5所述的方法，其中，所述方法还包括：获取所述检测结果中携带的所述ROI区域信息一与所述ROI区域信息二的深度信息，并确定所述ROI区域信息一与所述ROI区域信息二的深度状态，其中，所述ROI区域信息一与所述ROI区域信息二的深度状态用于表示所述ROI区域信息一与所述ROI区域信息二的深度范围是否一致。
  7. 根据权利要求6所述的方法，其中，所述方法还包括：根据所述ROI区域信息一 与所述ROI区域信息二的深度信息，利用区域生长的方法判断手部位置与方向盘位置能否生长成为一个连通区域，以确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
  8. 根据权利要求6所述的方法，其中，获取检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，确定所述ROI区域信息一与所述ROI区域信息二的深度状态包括：
    对所述方向盘位置的深度进行建模，得到所述方向盘位置的深度模型；
    获取所述手部位置的实际深度和所述手部位置在所述深度模型中的理论深度；
    根据所述手部位置的实际深度和理论深度，确定所述方向盘位置和所述手部位置的深度状态。
  9. 根据权利要求3所述的方法，其中，对所述视频流中的图像进行检测，得到所述目标车辆的方向盘位置包括：
    确定所述目标车辆的方向盘的初始位置，根据所述视频流中的图像在所述初始位置内检测，得到所述目标车辆的方向盘位置。
  10. 根据权利要求3所述的方法，其中，对所述视频流中的图像进行检测，得到所述驾驶员的手部位置包括：
    通过定位模型，获取与所述视频流中的图像对应的所述驾驶员的手部位置，其中，所述定位模型为使用多组训练数据训练得到的，所述多组训练数据中的每组训练数据均包括：视频流中的图像和所述视频流中的图像对应的驾驶员的手部位置。
  11. 根据权利要求10所述的方法，其中，对所述视频流中的图像进行检测，得到所述驾驶员的手部位置还包括：
    对所述视频流中的图像进行检测，定位得到所述驾驶员的手臂位置；
    根据所述手臂位置对基于所述定位模型得到的手部位置进行修正，得到所述手部位置。
  12. 根据权利要求11所述的方法，其中，所述驾驶员的手臂位置由根据所述视频流中的图像检测得到的手臂对应的手臂沿线上的预定数量的点的三维信息表示。
  13. 根据权利要求4或5所述的方法，其中，根据所述关联状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括：
    当所述手部位置在所述方向盘位置上或所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；
    当所述手部位置不在所述方向盘位置上或所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
  14. 根据权利要求6所述的方法，其中，所述方法包括：根据所述关联状态和所述深度状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
  15. 根据权利要求14所述的方法，其中，根据所述关联状态和所述深度状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括：
    在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集的情况下，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态；
    在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；
    在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态。
  16. 根据权利要求14所述的方法，其中，根据所述关联状态和所述深度状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括：
    当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；
    当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；
    当所述手部位置不在所述方向盘位置上时，确定所述驾驶员的手与所述目标 车辆的方向盘的当前状态为脱离状态。
  17. 根据权利要求1所述的方法，其中，所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括以下之一：双手脱离方向盘、单手脱离方向盘、双手虚握状态、单手虚握状态、双手实握状态。
  18. 根据权利要求1所述的方法，其中，在获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态之前，还包括：
    检测手的数量；
    在检测到所述手的数量为两个时，获取所述方向盘位置与所述驾驶员的每只手的手部位置在所述二维平面的关联状态。
  19. 一种手握方向盘状态的检测装置，包括：
    检测单元，设置为对从目标车辆中采集的视频流进行检测，得到检测结果；
    获取单元，设置为获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态，其中，所述ROI区域信息一为所述目标车辆的方向盘的ROI区域信息，所述ROI区域信息二为所述目标车辆的驾驶员的手的ROI区域信息；
    确定单元，设置为根据所述关联状态确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
  20. 根据权利要求19所述的装置，其中，所述视频流包括以下至少之一：深度视频流，红外线IR视频流，RGB视频流。
  21. 根据权利要求19所述的装置，其中，所述检测单元，用于对所述视频流中的图像进行检测，得到所述目标车辆的方向盘位置和所述驾驶员的手部位置，以得到所述检测结果。
  22. 根据权利要求19所述的装置，其中，所述获取单元包括：关联状态获取模块，用于使用预先训练的分类模型将所述方向盘位置和所述驾驶员的手部位置的位置关系分为不同的类别，其中，所述类别包括：所述手部位置在所述方向盘位置上，所述手部位置不在所述方向盘位置上。
  23. 根据权利要求19所述的装置，其中，所述获取单元包括：关联状态获取模块，用于比较所述方向盘位置与所述驾驶员的手部位置在所述二维平面的重合率和第一阈值，以确定所述ROI区域信息一在二维平面中与所述ROI区域信息二的集合状 态，其中，所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二是否存在交集。
  24. 根据权利要求22或23所述的装置，其中，所述获取单元还包括：深度状态获取模块，用于获取所述检测结果中携带的ROI区域信息一与ROI区域信息二的深度信息，并确定ROI区域信息一与ROI区域信息二的深度状态，其中，所述ROI区域信息一与所述ROI区域信息二的深度状态用于表示所述ROI区域信息一与所述ROI区域信息二的深度范围是否一致。
  25. 根据权利要求24所述的装置，其中，所述深度状态获取模块，还用于根据所述ROI区域信息一与所述ROI区域信息二的深度信息，利用区域生长的方法判断手部位置与方向盘位置能否生长成为一个连通区域，以确定所述驾驶员的手与所述目标车辆的方向盘之间的当前状态。
  26. 根据权利要求24所述的装置，其中，所述深度状态获取模块包括：
    建模子模块，设置为对所述方向盘位置的深度进行建模，得到所述方向盘位置的深度模型；
    获取子模块，设置为获取所述手部位置的实际深度和所述手部位置在所述深度模型中的理论深度；
    确定子模块，设置为根据所述手部位置的实际深度和理论深度，确定所述方向盘位置和所述手部位置的深度状态。
  27. 根据权利要求21所述的装置，其中，所述检测单元包括：
    方向盘位置检测模块，设置为确定所述目标车辆的方向盘的初始位置，根据所述视频流中的图像在所述初始位置内检测，得到所述目标车辆的方向盘位置。
  28. 根据权利要求21所述的装置，其中，所述检测单元包括：
    手部位置检测模块，设置为通过定位模型，获取与所述视频流中的图像对应的所述驾驶员的手部位置，其中，所述定位模型为使用多组训练数据训练得到的，所述多组训练数据中的每组训练数据均包括：视频流中的图像和所述视频流中的图像对应的驾驶员的手部位置。
  29. 根据权利要求21所述的装置，其中，所述检测单元还包括：
    手臂位置检测模块，设置为对所述视频流中的图像进行检测，定位得到所述驾驶员的手臂位置；
    手部位置修正模块，设置为根据所述手臂位置对基于所述定位模型得到的手部位置进行修正，得到所述手部位置。
  30. 根据权利要求22或23所述的装置，其中，所述确定单元包括：
    第一确定子单元，设置为当所述手部位置在所述方向盘位置上或所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；
    第二确定子单元，设置为当所述手部位置不在所述方向盘位置上或所述集合状态用于表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
  31. 根据权利要求24所述的装置，其中，所述确定单元还包括：
    第三确定子单元，设置为在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二不存在交集的情况下，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态；
    第四确定子单元，设置为在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；
    第五确定子单元，设置为在所述集合状态表示所述ROI区域信息一在所述二维平面中与所述ROI区域信息二存在交集的情况下，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态。
  32. 根据权利要求24所述的装置，其中，所述确定单元还包括：
    第六确定子单元，设置为当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围不一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为虚握状态；
    第七确定子单元，设置为当所述手部位置在所述方向盘位置上时，并且所述深度状态表示所述ROI区域信息一与所述ROI区域信息二的深度范围一致时，确定所述驾驶员的手与所述目标车辆的方向盘的当前状态为实握状态；
    第八确定子单元，设置为当所述手部位置不在所述方向盘位置上时，确定所 述驾驶员的手与所述目标车辆的方向盘的当前状态为脱离状态。
  33. 根据权利要求19所述的装置，其中，所述驾驶员的手与所述目标车辆的方向盘之间的当前状态包括以下之一：双手脱离方向盘、单手脱离方向盘、双手虚握状态、单手虚握状态、双手实握状态。
  34. 根据权利要求19所述的装置，其中，所述获取单元，还用于在获取所述检测结果中携带的ROI区域信息一与ROI区域信息二在二维平面的关联状态之前，检测手的数量；在所述检测结果为所述手的数量为两个时，获取所述方向盘位置与所述驾驶员的每只手的手部位置在所述二维平面的关联状态。
  35. 一种存储介质，所述存储介质包括存储的程序，其中，所述程序执行权利要求1至18中任意一项所述的手握方向盘状态的检测方法。
  36. 一种处理器，所述处理器用于运行程序，其中，所述程序运行时执行权利要求1至18中任意一项所述的手握方向盘状态的检测方法。
PCT/CN2020/079742 2019-03-28 2020-03-17 手握方向盘状态的检测方法及装置 Ceased WO2020192498A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP20779920.6A EP3951645A4 (en) 2019-03-28 2020-03-17 METHOD AND APPARATUS FOR DETECTING STATE OF HOLDING OF A STEERING WHEEL BY HANDS
US17/054,179 US11423673B2 (en) 2019-03-28 2020-03-17 Method and device for detecting state of holding steering wheel
JP2021557708A JP7253639B2 (ja) 2019-03-28 2020-03-17 手によるハンドルの把握状態の検出方法及び装置
KR1020217035279A KR102830024B1 (ko) 2019-03-28 2020-03-17 핸드에 의한 핸들 홀딩 상태의 검출 방법 및 장치

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910245398.8 2019-03-28
CN201910245398.8A CN111753589B (zh) 2019-03-28 2019-03-28 手握方向盘状态的检测方法及装置

Publications (1)

Publication Number Publication Date
WO2020192498A1 true WO2020192498A1 (zh) 2020-10-01

Family

ID=72611359

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/079742 Ceased WO2020192498A1 (zh) 2019-03-28 2020-03-17 手握方向盘状态的检测方法及装置

Country Status (6)

Country Link
US (1) US11423673B2 (zh)
EP (1) EP3951645A4 (zh)
JP (1) JP7253639B2 (zh)
KR (1) KR102830024B1 (zh)
CN (1) CN111753589B (zh)
WO (1) WO2020192498A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112818802A (zh) * 2021-01-26 2021-05-18 四川天翼网络服务有限公司 一种银行柜台人员举手识别方法及系统
CN114495073A (zh) * 2022-01-29 2022-05-13 上海商汤临港智能科技有限公司 方向盘脱手检测方法及装置、电子设备和存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7272329B2 (ja) * 2020-07-21 2023-05-12 トヨタ自動車株式会社 車両の制御装置
CN112818839A (zh) * 2021-01-29 2021-05-18 北京市商汤科技开发有限公司 驾驶员违章行为识别方法、装置、设备及介质
CN112926544A (zh) * 2021-04-12 2021-06-08 上海眼控科技股份有限公司 一种驾驶状态确定方法、装置、设备和存储介质
CN114155514A (zh) * 2021-12-10 2022-03-08 安徽江淮汽车集团股份有限公司 基于视觉的驾驶员脱手检测方法及系统
CN114312849B (zh) * 2022-01-27 2024-03-26 中国第一汽车股份有限公司 一种车辆运行状态确定方法、装置、车辆和存储介质
US11770612B1 (en) 2022-05-24 2023-09-26 Motorola Mobility Llc Electronic devices and corresponding methods for performing image stabilization processes as a function of touch input type
US12067231B2 (en) * 2022-05-24 2024-08-20 Motorola Mobility Llc Electronic devices and corresponding methods for capturing image quantities as a function of touch input type
CN115393791A (zh) * 2022-08-11 2022-11-25 浙江大华技术股份有限公司 一种异物检测方法、设备和计算机可读存储介质
CN117876933B (zh) * 2024-01-15 2025-11-28 浙江吉利控股集团有限公司 底盘漏装检测系统、方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140245858A1 (en) * 2013-03-01 2014-09-04 Wesley C. JOHNSON Adaptor for turn signal lever
CN104276080A (zh) * 2014-10-16 2015-01-14 北京航空航天大学 客车驾驶员手离方向盘检测预警系统及预警方法
CN109034111A (zh) * 2018-08-17 2018-12-18 北京航空航天大学 一种基于深度学习的驾驶员手离方向盘检测方法及系统

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5136473B2 (ja) 2009-03-12 2013-02-06 株式会社デンソー 乗員姿勢推定装置
CN102263937B (zh) * 2011-07-26 2013-07-24 华南理工大学 基于视频检测的驾驶员驾驶行为监控装置及监控方法
JP6398994B2 (ja) * 2013-02-13 2018-10-03 ティーケー ホールディングス インク．Ｔｋ Ｈｏｌｄｉｎｇｓ Ｉｎｃ． ハンドルの手検出システム
JP6413647B2 (ja) * 2014-10-31 2018-10-31 三菱自動車工業株式会社 操作入力装置
US9443320B1 (en) 2015-05-18 2016-09-13 Xerox Corporation Multi-object tracking with generic object proposals
US10043084B2 (en) * 2016-05-27 2018-08-07 Toyota Jidosha Kabushiki Kaisha Hierarchical context-aware extremity detection
JP6820533B2 (ja) 2017-02-16 2021-01-27 パナソニックＩｐマネジメント株式会社 推定装置、学習装置、推定方法、及び推定プログラム
JP6722355B2 (ja) * 2017-03-07 2020-07-15 コンチネンタル オートモーティヴ ゲゼルシャフト ミット ベシュレンクテル ハフツングＣｏｎｔｉｎｅｎｔａｌ Ａｕｔｏｍｏｔｉｖｅ ＧｍｂＨ ステアリングホイールの手動操作を検出するための装置及び方法
JP2019040465A (ja) 2017-08-25 2019-03-14 トヨタ自動車株式会社 行動認識装置，学習装置，並びに方法およびプログラム
US20190246036A1 (en) * 2018-02-02 2019-08-08 Futurewei Technologies, Inc. Gesture- and gaze-based visual data acquisition system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140245858A1 (en) * 2013-03-01 2014-09-04 Wesley C. JOHNSON Adaptor for turn signal lever
CN104276080A (zh) * 2014-10-16 2015-01-14 北京航空航天大学 客车驾驶员手离方向盘检测预警系统及预警方法
CN109034111A (zh) * 2018-08-17 2018-12-18 北京航空航天大学 一种基于深度学习的驾驶员手离方向盘检测方法及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3951645A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112818802A (zh) * 2021-01-26 2021-05-18 四川天翼网络服务有限公司 一种银行柜台人员举手识别方法及系统
CN112818802B (zh) * 2021-01-26 2022-07-05 四川天翼网络服务有限公司 一种银行柜台人员举手识别方法及系统
CN114495073A (zh) * 2022-01-29 2022-05-13 上海商汤临港智能科技有限公司 方向盘脱手检测方法及装置、电子设备和存储介质

Also Published As

Publication number Publication date
JP7253639B2 (ja) 2023-04-06
CN111753589B (zh) 2022-05-03
CN111753589A (zh) 2020-10-09
EP3951645A1 (en) 2022-02-09
JP2022528847A (ja) 2022-06-16
US20220004788A1 (en) 2022-01-06
KR102830024B1 (ko) 2025-07-03
US11423673B2 (en) 2022-08-23
KR20210140771A (ko) 2021-11-23
EP3951645A4 (en) 2022-06-01

Similar Documents

Publication Publication Date Title
US11423673B2 (en) Method and device for detecting state of holding steering wheel
CN104276080B (zh) 客车驾驶员手离方向盘检测预警系统及预警方法
CN110858295B (zh) 一种交警手势识别方法、装置、整车控制器及存储介质
JP7005933B2 (ja) 運転者監視装置、及び運転者監視方法
CN109383525A (zh) 驾驶员状态掌握装置、驾驶员状态掌握系统及方法
CN105388021A (zh) Adas虚拟开发与测试系统
CN106934876A (zh) 一种车辆异常驾驶事件的识别方法及系统
CN110765807A (zh) 驾驶行为分析、处理方法、装置、设备和存储介质
CN105069976A (zh) 一种疲劳检测和行驶记录综合系统及疲劳检测方法
CN104085396A (zh) 一种全景车道偏离预警方法及系统
CN111028384A (zh) 自动驾驶车辆的故障智能分类方法和系统
CN107097794A (zh) 道路车道线的侦测系统及其方法
CN106184232A (zh) 一种基于驾驶员视角的车道偏离预警控制方法
CN109664889B (zh) 一种车辆控制方法、装置和系统以及存储介质
CN113421402A (zh) 基于红外相机的乘客体温及疲劳驾驶行为检测系统及方法
CN113119982A (zh) 操作状态识别与处理方法、装置、设备、介质及程序产品
CN112818726A (zh) 车辆违章预警方法、装置、系统及存储介质
CN111967384A (zh) 车辆信息处理方法、装置、设备及计算机可读存储介质
CN108256487B (zh) 一种基于反向双目的驾驶状态检测装置和方法
CN106494409A (zh) 基于ar增强现实和大数据的车辆控制的驾驶辅助系统
CN116255997B (zh) 基于视觉导航单元的实时车道检测动态校验系统
CN108304745A (zh) 一种驾驶员驾驶行为检测方法、装置
CN113221734A (zh) 影像识别方法及装置
CN112329555A (zh) 一种针对驾驶人员手部动作姿态的智能混合感知系统
CN117173856A (zh) 分神驾驶报警方法、装置、车辆、介质、设备及程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20779920

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021557708

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20217035279

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020779920

Country of ref document: EP

Effective date: 20211028