WO2019179417A1 - 数据融合方法以及相关设备 - Google Patents

数据融合方法以及相关设备 Download PDF

Info

Publication number
WO2019179417A1
WO2019179417A1 PCT/CN2019/078646 CN2019078646W WO2019179417A1 WO 2019179417 A1 WO2019179417 A1 WO 2019179417A1 CN 2019078646 W CN2019078646 W CN 2019078646W WO 2019179417 A1 WO2019179417 A1 WO 2019179417A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
roadside
result
target object
sensing device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/078646
Other languages
English (en)
French (fr)
Inventor
于欢
杨肖
宋永刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP19771774.7A priority Critical patent/EP3754448B1/en
Priority to JP2020550669A priority patent/JP7386173B2/ja
Publication of WO2019179417A1 publication Critical patent/WO2019179417A1/zh
Priority to US17/021,911 priority patent/US11987250B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
    • G05D1/024—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors in combination with a laser
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/04—Traffic conditions
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G01S13/865—Combination of radar systems with lidar systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G01S13/867—Combination of radar systems with cameras
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/003—Transmission of data between radar, sonar or lidar systems and remote stations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/40—Means for monitoring or calibrating
    • G01S7/4004—Means for monitoring or calibrating of parts of a radar system
    • G01S7/4026—Antenna boresight
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0225—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving docking at a fixed facility, e.g. base station or loading bay
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0242—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using non-visible light signals, e.g. IR or UV signals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0255—Control of position or course in two dimensions specially adapted to land vehicles using acoustic signals, e.g. ultra-sonic singals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0259—Control of position or course in two dimensions specially adapted to land vehicles using magnetic or electromagnetic means
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0278—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using satellite positioning signals, e.g. GPS
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/028—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using a RF signal
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0285—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using signals transmitted via a public communication network, e.g. GSM network
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/646—Following a predefined trajectory, e.g. a line marked on the floor or a flight path
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/251—Fusion techniques of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9316—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles combined with communication equipment with other vehicles or with base stations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/932—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles using own vehicle data, e.g. ground speed, steering wheel direction
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9323—Alternative operation using light waves
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present invention relates to the field of automatic driving, and in particular, to a data fusion method and related equipment.
  • the perception of the road environment is the primary task of achieving autonomous driving. After the self-driving vehicle senses the environment of the road, it can avoid other vehicles or pedestrians on the road to achieve safe driving. In order to realize the perception of the road environment, the prior art self-driving vehicle detects the other vehicles or pedestrians in the road through the vehicle sensing device mounted on the vehicle, thereby sensing the environment of the road. However, in the prior art, the sensing range of the vehicle sensing device is relatively small, and it is difficult to meet the requirements for realizing automatic driving.
  • the embodiment of the present application provides a data fusion method and related device, which can superimpose the sensing range of the roadside sensing device and the sensing range of the vehicle sensing device, thereby effectively expanding the sensing range.
  • a data fusion method which can be applied to a vehicle device side or a roadside device side, including:
  • vehicle sensing data wherein the vehicle sensing data is obtained by detecting, by the vehicle sensing device, a road environment within a sensing range;
  • roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the vehicle sensing data and the roadside sensing data are fused by a fusion formula to obtain a first fusion result.
  • the fusion formula is expressed as:
  • result r is a roadside result set
  • the roadside result set is used to represent the roadside sensing data
  • result v is a vehicle result set
  • vehicle result set is used to represent the vehicle sensing data
  • y is The first fusion result is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • w r is the confidence factor of the roadside sensing device
  • w r (w r1 , w r2 , ..., w rM ), result r (roadside 1 , roadside 2 , ..., roadside M )
  • M is The roadside sensing device senses the number of target objects in the range
  • w ri is a confidence factor corresponding to the target object i within the sensing range of the roadside sensing device
  • the roadside i is the target object within the sensing range of the roadside sensing device.
  • the confidence factor is determined jointly based on the sensing device parameters, the perceived distance of the target object, and the perceived angle of the target object.
  • the confidence factor w can be obtained according to the following formula:
  • S k is the sensing device parameter
  • R i is the sensing distance of the target object
  • ⁇ j is the sensing angle of the target object
  • g is a calibration parameter table obtained by the sensing device.
  • the confidence factor can be obtained by comprehensively considering the confidence of the plurality of sensors.
  • the confidence of the plurality of sensors can be comprehensively considered by weighting or averaging.
  • the vehicle result set includes at least one vehicle result unit, the at least one vehicle result unit has a one-to-one correspondence with at least one target object, and each of the at least one vehicle result unit has a vehicle result
  • the unit is used to describe the characteristics of the corresponding target object from a multi-dimensional perspective.
  • any one of the at least one vehicle result unit is represented as vehicle j (p vj , v vj , s vj , c vj ), wherein p vj is represented as detection by the vehicle sensing device
  • p vj is represented as detection by the vehicle sensing device
  • v vj is the speed of the target object j detected by the vehicle sensing device
  • s vj is the size of the target object j detected by the vehicle sensing device
  • c vj Indicated as the color of the target object j detected by the vehicle sensing device
  • N is the number of target objects within the sensing range of the vehicle sensing device
  • j is a natural number, 0 ⁇ j ⁇ N.
  • the roadside result set includes at least one roadside result unit, and the at least one roadside result unit has a one-to-one correspondence with at least one target object, and the at least one roadside result unit is in the unit.
  • Each roadside result unit is used to describe the characteristics of the corresponding target object from a multidimensional perspective.
  • any one of the at least one roadside result unit is represented as roadside i (p vi , v vi , s vi , c vi ), wherein p vi is represented as the roadside sensing
  • p vi is represented as the roadside sensing
  • v vi is the speed of the target object i detected by the roadside sensing device
  • s vi is the target object detected by the roadside sensing device.
  • the size of c vi is the color of the target object i detected by the roadside sensing device
  • M is the number of target objects within the sensing range of the roadside sensing device
  • i is a natural number, 0 ⁇ i ⁇ M .
  • the method before the merging the vehicle sensing data and the roadside sensing data by the fusion formula to obtain a fusion result, the method further includes:
  • the merging the vehicle sensing data and the roadside sensing data to obtain a first fusion result includes:
  • the matching relationship between the roadside result unit of the roadside result set and the vehicle result unit of the vehicle result set is found by the deviation network.
  • the vehicle result unit, i, j are natural numbers.
  • the deviation network Deviation is represented by a reverse propagation BP neural network.
  • the method further includes:
  • the deviation network is adjusted based on the evaluation result.
  • the acquiring vehicle sensing data when applied to the roadside device side, includes: receiving vehicle sensing data of at least one vehicle device;
  • the method further includes:
  • the target vehicle device is configured to fuse vehicle perception data of the target vehicle device with the first fusion result to obtain a second fusion result
  • the target vehicle equipment belongs to the at least one vehicle device.
  • a data fusion method is provided, which is applied to a vehicle device side, and includes the following steps:
  • the roadside device receives a first fusion result sent by the roadside device, where the first fusion result is that the roadside device over-fusion formula fuses the vehicle sensing data and the roadside sensing data sent by at least one vehicle device Obtaining, the roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the vehicle sensing data and the first fusion result are fused to obtain a second fusion result.
  • the fusion formula is expressed as:
  • result r is a roadside result set
  • the roadside result set is used to represent the roadside sensing data
  • result v is a vehicle result set
  • vehicle result set is used to represent the vehicle sensing data
  • y is The first fusion result is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • w r is the confidence factor of the roadside sensing device
  • w r (w r1 , w r2 , ..., w rM ), result r (roadside 1 , roadside 2 , ..., roadside M )
  • M is The roadside sensing device senses the number of target objects in the range
  • w ri is a confidence factor corresponding to the target object i within the sensing range of the roadside sensing device
  • the roadside i is the target object within the sensing range of the roadside sensing device.
  • the confidence factor is determined jointly based on the sensing device parameters, the perceived distance of the target object, and the perceived angle of the target object.
  • the confidence factor w can be obtained according to the following formula:
  • S k is the sensing device parameter
  • R i is the sensing distance of the target object
  • ⁇ j is the sensing angle of the target object
  • g is a calibration parameter table obtained by the sensing device.
  • the confidence factor can be obtained by comprehensively considering the confidence of the plurality of sensors.
  • the confidence of the plurality of sensors can be comprehensively considered by weighting or averaging.
  • the vehicle result set includes at least one vehicle result unit, the at least one vehicle result unit has a one-to-one correspondence with at least one target object, and each of the at least one vehicle result unit has a vehicle result
  • the unit is used to describe the characteristics of the corresponding target object from a multi-dimensional perspective.
  • any one of the at least one vehicle result unit is represented as vehicle j (p vj , v vj , s vj , c vj ), wherein p vj is represented as detection by the vehicle sensing device
  • p vj is represented as detection by the vehicle sensing device
  • v vj is the speed of the target object j detected by the vehicle sensing device
  • s vj is the size of the target object j detected by the vehicle sensing device
  • c vj Indicated as the color of the target object j detected by the vehicle sensing device
  • N is the number of target objects within the sensing range of the vehicle sensing device
  • j is a natural number, 0 ⁇ j ⁇ N.
  • the roadside result set includes at least one roadside result unit, and the at least one roadside result unit has a one-to-one correspondence with at least one target object, and the at least one roadside result unit is in the unit.
  • Each roadside result unit is used to describe the characteristics of the corresponding target object from a multidimensional perspective.
  • any one of the at least one roadside result unit is represented as roadside i (p vi , v vi , s vi , c vi ), wherein p vi is represented as the roadside sensing
  • p vi is represented as the roadside sensing
  • v vi is the speed of the target object i detected by the roadside sensing device
  • s vi is the target object detected by the roadside sensing device.
  • the size of c vi is the color of the target object i detected by the roadside sensing device
  • M is the number of target objects within the sensing range of the roadside sensing device
  • i is a natural number, 0 ⁇ i ⁇ M .
  • the method before the merging the vehicle sensing data and the roadside sensing data by the fusion formula to obtain a fusion result, the method further includes:
  • the merging the vehicle sensing data and the roadside sensing data to obtain a first fusion result includes:
  • the matching relationship between the roadside result unit of the roadside result set and the vehicle result unit of the vehicle result set is found by the deviation network.
  • the vehicle result unit, i, j are natural numbers.
  • the deviation network Deviation is represented by a reverse propagation BP neural network.
  • the method further includes:
  • the deviation network is adjusted based on the evaluation result.
  • a fusion device comprising means for performing the method of the first aspect.
  • a fusion device comprising means for performing the method of the second aspect.
  • a fifth aspect provides a fusion device, a memory, and a processor and a communication module coupled to the memory, wherein: the communication module is configured to send or receive externally sent data, and the memory is configured to store program code.
  • the processor is configured to invoke the program code stored in the memory to perform the method described in any one of the first aspect or the second aspect.
  • a sixth aspect a computer readable storage medium, comprising instructions, when the instructions are run on a fusion device, causing the fusion device to perform any one of the first aspect or the second aspect Methods.
  • a seventh aspect a computer program product comprising instructions for causing a computer to perform the method of any one of the first aspect or the second aspect, when it is run on a computer.
  • the roadside sensing data detected by the roadside sensing device and the vehicle sensing data detected by the vehicle sensing device are combined to realize the sensing range of the roadside sensing device and the vehicle sensing device.
  • the sensing range is superimposed to effectively extend the sensing range.
  • FIG. 1 is a schematic diagram of an application scenario involved in an embodiment of the present application
  • FIG. 2 is a schematic view showing an installation space angle of a sensing device according to an embodiment of the present application
  • FIG. 3 is a schematic diagram of mounting coordinates of a sensing device according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a perceptual coordinate system involved in an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a reverse propagation neural network according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an inter-frame loopback involved in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a multi-frame association involved in an embodiment of the present application.
  • FIG. 8 is a schematic flowchart diagram of a first data fusion method according to an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of a second data fusion method according to an embodiment of the present application.
  • FIG. 10 is a schematic flowchart diagram of a second data fusion method according to an embodiment of the present application.
  • FIG. 14 are schematic structural diagrams of four fusion devices provided by an embodiment of the present application.
  • FIG. 1 is a schematic diagram of an application scenario involved in an embodiment of the present application.
  • at least one roadside device is installed on the roadside of the road, and at least one of the vehicles traveling in the middle of the road is equipped with the vehicle equipment. among them:
  • the roadside device is used to detect the road environment from the roadside angle to obtain roadside sensing data.
  • the roadside device may be configured with a roadside sensing device, which may include at least one roadside sensor, such as a microwave radar, a millimeter wave radar, etc., capable of recognizing a target object within a sensing range (eg, a vehicle) And the location, speed and size of the pedestrian) and other roadside perception data.
  • the roadside sensing device may further include a roadside sensor such as a camera. In addition to recognizing the roadside sensing data such as the position, velocity and size of the target object within the sensing range, the camera can also recognize the target object within the sensing range.
  • the color of the road for example, the color of the vehicle and the color of the clothing on the pedestrian), etc., the roadside sensing data.
  • the roadside sensing device may use any one of the roadside sensors alone, or may use any of the roadside sensors simultaneously.
  • the roadside sensing data may be described in the form of a roadside result set, and the roadside result set may include a plurality of roadside result units, and each roadside result unit corresponds to one target object.
  • the roadside result unit can be represented as roadside(p r , v r , s r , c r ), where p r represents the position of the target object detected by the roadside sensing device, v r Indicated as the speed of the target object detected by the roadside sensing device, s r is the size of the target object detected by the roadside sensing device, and r r is the target object detected by the roadside sensing device.
  • the roadside result set can be expressed as result r (roadside 1 , roadside 2 , ..., roadside M ), where M is the number of target objects within the range of the roadside sensing device. More specifically, taking the form of a matrix as an example, the roadside result set can be expressed as:
  • the vehicle equipment is used to detect the road environment from the perspective of the vehicle to obtain vehicle perception data.
  • the vehicle device may be configured with a vehicle sensing device, which may include at least one vehicle sensor, such as a combined inertial navigation, microwave radar, millimeter wave radar, camera, and the like.
  • vehicle sensors can detect different vehicle perception data.
  • the vehicle sensing device can recognize the roadside sensing data of the position, velocity, and the like of the target object by combining the inertial navigation.
  • the vehicle sensing device can recognize roadside sensing data such as the position, velocity and size of the target object within the sensing range through the microwave radar and the millimeter wave radar.
  • the vehicle sensing device is capable of recognizing roadside sensing data such as the position, speed, size, and color of the target object within the sensing range by the camera. It will be understood that the above specific examples are merely illustrative of vehicle sensors and should not be specifically limited.
  • the vehicle sensing device may use any one of the vehicle sensors alone, or may use any of the vehicle sensors simultaneously.
  • the vehicle perception data may be described in the form of a vehicle result set, the vehicle result set may include a plurality of vehicle result units, each vehicle result unit corresponding to one target object, and the vehicle result unit describes the characteristics of the target object from a multi-dimensional angle, for example, Position, speed, size and color, and more.
  • the vehicle result unit can be represented as vehicle(p v , v v , s v , c v ), where p v represents the position of the target object detected by the vehicle sensing device, and v v is expressed as The speed of the target object detected by the vehicle sensing device, s v is the size of the target object detected by the vehicle sensing device, and c v is the color of the target object detected by the vehicle sensing device, and the vehicle result is The set can be expressed as result v (vehicle 1 ,vehicle 2 ,...,vehicle N ), where N is the number of target objects within the sensing range of the vehicle sensing device. More specifically, taking the form of a matrix as an example, the vehicle result set can be expressed as:
  • the roadside device and the vehicle device can be connected by wireless, thereby implementing data communication.
  • the roadside device and/or the vehicle device may use the fusion formula to detect the roadside sensing data detected by the roadside sensing device within the sensing range and the detected by the vehicle sensing device within the sensing range.
  • the vehicle senses data for data fusion to obtain a first fusion result.
  • f is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • the function f can be expressed as:
  • w r is the confidence factor of the roadside sensing device
  • w v is the confidence factor of the vehicle sensing device
  • the confidence factor of the roadside sensing device can be divided more finely so that different elements in the roadside result unit respectively correspond to different confidence factors.
  • the confidence factor of the vehicle sensing device can also be divided more finely so that different elements in the vehicle result unit respectively correspond to different confidence factors.
  • the confidence factor may be jointly determined according to the sensing device parameters, the sensing distance of the target object, and the sensing angle of the target object.
  • the sensing device parameters are related to the initial accuracy of the sensing device itself, the installation space angle, and the mounting coordinates.
  • the perceived distance of the target object is the distance between the target object and the sensing device in the sensing coordinate system.
  • the perceived angle of the target object is the angle formed by the target object and the sensing device in the sensing coordinate system.
  • the sensing device includes a plurality of sensors
  • the confidence factor can be obtained by comprehensively considering the confidence of the plurality of sensors by weighting or averaging.
  • the confidence factor can be obtained according to the following formula:
  • S k is the sensing device parameter
  • R i is the sensing distance
  • ⁇ j is the sensing angle
  • g is the calibration parameter table obtained by the sensing device.
  • the calibration parameter table can be obtained by inversely using data of a large number of known target objects during the calibration of the sensor device. It is not difficult to understand that if the accuracy of the sensing device parameter is higher, the value of the confidence factor is larger. If the accuracy of the sensing device parameter is lower, the value of the confidence factor is smaller; if the sensing distance is smaller, the confidence factor is The larger the value, the larger the perceived distance is, the smaller the value of the confidence factor is. If the perceived angle is smaller, the value of the confidence factor is larger. If the perceived angle is larger, the value of the confidence factor is smaller.
  • the pitch angle of the ground, roll is the roll angle of the sensing device relative to the ground of the road.
  • the yaw angle yaw can be defined as the angle obtained by the sensing device rotating around the y-axis.
  • the pitch angle pitch can be defined as the sensing device.
  • the angle at which the x-axis is rotated, the roll angle roll can be defined as the angle at which the sensing device rotates about the z-axis.
  • the sensing distance of the target object and the sensing angle of the target object can be obtained as follows: as shown in FIG. 4, the sensing range of the sensing device is divided into different distances by using the sensing device as a center. , the fan-shaped area at different angles, thus constructing a perceptual coordinate system.
  • the sensing device determines the sensing distance R i of the target object and the sensing angle ⁇ j of the target object according to the sector region in which the target object falls into the sensing coordinate system.
  • the roadside device and/or the vehicle device need to match the roadside sensing data and the vehicle sensing data to obtain a matching result, so that the roadside device and / or the vehicle device can data fusion of the roadside sensing data and the vehicle sensing data according to the matching result.
  • the meaning of matching the roadside sensing data and the vehicle sensing data is exemplified below: It is assumed that the roadside sensing data expressed in the form of the roadside result set is: result r (roadside 1 , roadside 2 , ..., roadside M ), The vehicle sensing data represented by the vehicle result set form is: result v (vehicle 1 , vehicle 2 , ..., vehicle N ), M is the number of target objects within the sensing range of the roadside sensing device, and N is within the sensing range of the vehicle sensing device. The number of target objects, M>N.
  • the roadside 1 is a roadside result unit for detecting the target object 1 by the roadside sensing device
  • the roadside 2 is a roadside result unit for detecting the target object 1 by the roadside sensing device, ..., roadside M
  • Vehicle 1 is a vehicle sensor means for detecting the target object 1 obtained vehicle unit result
  • vehicle 2 is a vehicle sensing apparatus of the target object 1 unit detects the vehicle results obtained, ..., vehicle N as vehicle sensing apparatus
  • the target object N performs the detected vehicle result unit.
  • roadside 1 and vehicle 1 are the result units obtained by detecting the target object 1, and there is a matching relationship between them; roadside 2 and vehicle 2 are the result units obtained by detecting the target object 2, There is a matching relationship between the two; ...; roadside N and vehicle N are the result units obtained by detecting the target object N, and there is a matching relationship between the two. Therefore, matching the roadside sensing data with the vehicle sensing data is to find out the matching relationship between the roadside result unit in the roadside sensing data and the vehicle result unit in the vehicle sensing data.
  • the roadside device and/or the vehicle device may find out the matching relationship between the roadside result unit in the roadside sensing data and the vehicle result unit in the vehicle sensing data through the deviation network. Specifically, if the roadside result unit and the vehicle result unit are used as inputs to the deviation network, the deviation network will output a matching result between the roadside result unit and the vehicle result unit. If the matching result of the deviation network output is that the two match, the matching relationship between the roadside result unit and the vehicle result unit may be considered. If the matching result output by the deviation network is that the two do not match, the roadside result may be considered. There is no match between the unit and the vehicle result unit.
  • roadside 1 and vehicle 1 are used as input to the deviation network, and the deviation of the output of the deviation network is matched, it can be determined that there is a matching relationship between roadside 1 and vehicle 1 ; if roadside 1 is to be And vehicle 2 as the input of the deviation network, the deviation of the output of the deviation network is that the two do not match, then it can be determined that there is no matching relationship between roadside 1 and vehicle 2 .
  • ⁇ p ij is the position deviation value
  • ⁇ p ij fabs(p ri -p vj )
  • p ri is the position of the target object i detected by the roadside sensing device
  • p vj is the target object detected by the vehicle sensor The position of j, fabs for the absolute value function
  • P p is the position deviation factor, a confidence factor corresponding to the position of the target object i detected by the roadside sensing device, a confidence factor corresponding to the position of the target object j detected by the vehicle sensing device;
  • P v is the speed deviation factor, a confidence factor corresponding to the speed of the target object i detected by the roadside sensing device, a confidence factor corresponding to the speed of the target object j detected by the vehicle sensing device;
  • P s is the size deviation factor, a confidence factor corresponding to the size of the target object i detected by the roadside sensing device, a confidence factor corresponding to the size of the target object j detected by the vehicle sensing device;
  • P c is the speed deviation factor, a confidence factor corresponding to the color of the target object i detected by the roadside sensing device, a confidence factor corresponding to the color of the target object j detected by the vehicle sensing device;
  • the activation function may be a Leaky Rectified Linear Unit (LReLU), a Parameterized Rectified Linear Unit (PReLU), and a randomized leaked corrected linear unit (Randomized Leaky Rectified).
  • the deviation network is described by taking the BP neural network as an example.
  • the deviation network may also be a Long Short-Term Memory (LSTM), a residual network (Residential Networking, ResNet), Recurrent Neural Networks (RNN), etc., are not specifically limited herein.
  • LSTM Long Short-Term Memory
  • ResNet Residential Networking
  • RNN Recurrent Neural Networks
  • the above content only uses the single-side roadside sensing data and the vehicle sensing data to achieve matching, and the confidence of the matching result is not high.
  • it is considered to achieve matching by using two or even multiple frames of road side sensing data and vehicle sensing data to improve the confidence of the matching result.
  • the roadside device and/or the vehicle device may also evaluate the confidence of the matching result by means of interframe loopback and/or multiframe association to obtain an evaluation result, and adjust the deviation network according to the evaluation result.
  • the inter-frame loopback is mainly calculated according to the matching result obtained by the cross-matching between the roadside sensing data of the adjacent frame and the vehicle sensing data.
  • the inter-frame loopback is mainly obtained according to the intra-frame matching result of the adjacent frame and the inter-frame matching result.
  • the intra-frame matching result is a matching result obtained by matching result units obtained by detecting different target devices in the same frame to the same target object.
  • the inter-frame matching result is a matching result obtained by matching the result units obtained by detecting the same target object in the adjacent frame by the same sensing device.
  • T 1 is the first matching result
  • T 2 is the second matching result
  • T 3 is the third matching result
  • T 4 is the fourth matching result.
  • the first matching result is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame
  • the matching result of the detected vehicle result unit of the detected target object j is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame
  • the matching result of the detected vehicle result unit of the detected target object j is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object
  • the second matching result is an inter-frame matching result of the vehicle sensing device, that is, the corresponding vehicle result unit of the target object j detected by the vehicle sensing device in the ith frame and the vehicle sensing device are detected in the i+1 frame.
  • the matching result of the corresponding vehicle result unit of the target object j measured.
  • the third matching result is the intra-frame matching result of the (i+1)th frame, that is, the corresponding vehicle result unit and the drive test sensing device of the target object j detected by the vehicle sensing device in the (i+1)th frame are at the i-th
  • the matching result of the corresponding roadside result unit of the target object j detected by the +1 frame are at the i-th.
  • the fourth matching result is the inter-frame matching result of the roadside sensing device, that is, the corresponding roadside result unit and the roadside sensing device of the target object j detected by the roadside sensing device in the i+1th frame are The matching result of the corresponding roadside result unit of the target object j detected by the i-th frame.
  • the multi-frame association is mainly obtained according to the inter-frame loopback of multiple consecutive frames.
  • the sensing range of the roadside sensing device is superimpose the sensing range of the roadside sensing device and the sensing range of the vehicle sensing device, thereby effectively expanding the sensing range of the roadside sensing device and/or the vehicle sensing device.
  • the number of target objects detected by the roadside sensing device within the sensing range is three (target object 1, target object 2, target object 3), and the sensing range of the vehicle sensing device is detected.
  • the number of target objects to arrive is two (target object 3, target object 4)
  • the sensing range of the fusion result includes four targets (objects) Object 1, target object 2, target object 3, target object 4).
  • the above content of the article focuses on how to realize the data fusion of the roadside sensing data detected by the roadside sensing device within the sensing range and the vehicle sensing data detected by the vehicle sensing device within the sensing range.
  • the following describes how the roadside sensing device and/or the vehicle sensing device can realize the extended sensing range by using the above data fusion scheme from the perspective of data fusion and related equipment.
  • FIG. 8 is a schematic flowchart diagram of a first data fusion method provided by an embodiment of the present application. As shown in FIG. 8, the data fusion method in the embodiment of the present application specifically includes the following steps:
  • the vehicle device acquires vehicle sensing data, wherein the vehicle sensing data is that the vehicle sensing device detects the road environment within the sensing range by using the vehicle sensor to obtain vehicle sensing data.
  • the vehicle sensing device may be configured with a vehicle sensing device, and the vehicle sensing device includes at least one vehicle sensor, such as a combined inertial navigation, microwave radar, millimeter wave radar, a camera, etc., which can recognize Vehicle perception data of the target object within the perceived range.
  • vehicle sensing data may include the position, speed, size, color, and the like of the target object.
  • vehicle sensing device may use any one of the vehicle sensors alone, or may use any of the vehicle sensors simultaneously.
  • the roadside device acquires the roadside sensing data, wherein the roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range by the roadside sensor.
  • the roadside device may be configured with a roadside sensing device, and the roadside sensing device includes at least one roadside sensor, such as a microwave radar, a millimeter wave radar, etc., capable of recognizing the sensing range.
  • the roadside sensing data may include the position, speed, size, color, and the like of the target object.
  • the roadside sensing device may use any one of the roadside sensors alone, or may use any of the roadside sensors simultaneously.
  • the roadside device sends the roadside sensing data to the vehicle device. Accordingly, the vehicle device receives the roadside sensing data transmitted by the roadside device.
  • the vehicle device matches the roadside sensing data and the vehicle sensing data to obtain a matching result.
  • the vehicle device may find out the matching relationship between the roadside result unit in the roadside sensing data and the vehicle result unit in the vehicle sensing data through the deviation network. Specifically, if the roadside result unit and the vehicle result unit are used as inputs to the deviation network, the deviation network will output a matching result between the roadside result unit and the vehicle result unit.
  • S105 The vehicle device evaluates the confidence of the matching result by means of inter-frame loopback and/or multi-frame association, and adjusts the deviation network according to the evaluation result.
  • the first matching result is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame
  • the matching result of the detected vehicle result unit of the detected target object j is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame.
  • the second matching result is an inter-frame matching result of the vehicle sensing device, that is, the corresponding vehicle result unit of the target object j detected by the vehicle sensing device in the ith frame and the vehicle sensing device are detected in the i+1 frame.
  • the matching result of the corresponding vehicle result unit of the target object j measured.
  • the third matching result is the intra-frame matching result of the (i+1)th frame, that is, the corresponding vehicle result unit and the drive test sensing device of the target object j detected by the vehicle sensing device in the (i+1)th frame are at the i-th
  • the matching result of the corresponding roadside result unit of the target object j detected by the +1 frame are at the i-th.
  • the fourth matching result is the inter-frame matching result of the roadside sensing device, that is, the corresponding roadside result unit and the roadside sensing device of the target object j detected by the roadside sensing device in the i+1th frame are The matching result of the corresponding roadside result unit of the target object j detected by the i-th frame.
  • the multi-frame association is mainly obtained according to the inter-frame loopback of multiple consecutive frames.
  • the vehicle device fuses the vehicle sensing data and the roadside sensing data by a fusion formula to obtain a first fusion result.
  • the vehicle device can perform data fusion by using the fusion formula to the roadside sensing data detected by the roadside sensing device within the sensing range and the vehicle sensing data detected by the vehicle sensing device within the sensing range. Obtain the first fusion result.
  • f is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • the function f can be expressed as:
  • w r is the confidence factor of the roadside sensing device
  • w v is the confidence factor of the vehicle sensing device
  • the embodiment of the present application does not describe the fusion of the roadside sensing data and the vehicle sensing data in detail.
  • FIG. 9 is a schematic flowchart diagram of a second data fusion method provided by an embodiment of the present application. As shown in FIG. 9, the data fusion method in the embodiment of the present application specifically includes the following steps:
  • the roadside device acquires the roadside sensing data, wherein the roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range by the roadside sensor.
  • the roadside device may be configured with a roadside sensing device, and the roadside sensing device includes at least one roadside sensor, such as a microwave radar, a millimeter wave radar, etc., capable of recognizing the sensing range.
  • the roadside sensing data may include the position, speed, size, color, and the like of the target object.
  • the roadside sensing device may use any one of the roadside sensors alone, or may use any of the roadside sensors simultaneously.
  • the vehicle device acquires vehicle sensing data, wherein the vehicle sensing data is that the vehicle sensing device detects the road environment within the sensing range by the vehicle sensor to obtain vehicle sensing data.
  • the vehicle device may be configured with a vehicle sensing device, and the vehicle sensing device includes at least one vehicle sensor, such as a combined inertial navigation, microwave radar, millimeter wave radar, a camera, etc., capable of recognizing the sensing range.
  • the vehicle sensing data may include the position, speed, size, color, and the like of the target object.
  • vehicle sensing device may use any one of the vehicle sensors alone, or may use any of the vehicle sensors simultaneously.
  • the vehicle device sends the vehicle perception data to the roadside device. Accordingly, the roadside device receives vehicle sensing data transmitted by the vehicle device.
  • the roadside device matches the roadside sensing data and the vehicle sensing data to obtain a matching result.
  • the roadside sensing device can find out the matching relationship between the roadside result unit in the roadside sensing data and the vehicle result unit in the vehicle sensing data through the deviation network. Specifically, if the roadside result unit and the vehicle result unit are used as inputs to the deviation network, the deviation network will output a matching result between the roadside result unit and the vehicle result unit.
  • S205 The roadside device evaluates the confidence of the matching result by means of interframe loopback and/or multiframe association to obtain an evaluation result, and adjusts the deviation network according to the evaluation result.
  • the first matching result is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame
  • the matching result of the detected vehicle result unit of the detected target object j is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame.
  • the second matching result is an inter-frame matching result of the vehicle sensing device, that is, the corresponding vehicle result unit of the target object j detected by the vehicle sensing device in the ith frame and the vehicle sensing device are detected in the i+1 frame.
  • the matching result of the corresponding vehicle result unit of the target object j measured.
  • the third matching result is the intra-frame matching result of the (i+1)th frame, that is, the corresponding vehicle result unit and the drive test sensing device of the target object j detected by the vehicle sensing device in the (i+1)th frame are at the i-th
  • the matching result of the corresponding roadside result unit of the target object j detected by the +1 frame are at the i-th.
  • the fourth matching result is the inter-frame matching result of the roadside sensing device, that is, the corresponding roadside result unit and the roadside sensing device of the target object j detected by the roadside sensing device in the i+1th frame are The matching result of the corresponding roadside result unit of the target object j detected by the i-th frame.
  • the multi-frame association is mainly obtained according to the inter-frame loopback of multiple consecutive frames.
  • the roadside device combines the vehicle sensing data and the roadside sensing data by a fusion formula to obtain a first fusion result.
  • the roadside sensing device can use the fusion formula to detect the roadside sensing data detected by the roadside sensing device within the sensing range and the vehicle sensing data detected by the vehicle sensing device within the sensing range. Data fusion is performed to obtain the first fusion result.
  • f is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • the function f can be expressed as:
  • w r is the confidence factor of the roadside sensing device
  • w v is the confidence factor of the vehicle sensing device
  • the embodiment of the present application does not describe the fusion of the roadside sensing data and the vehicle sensing data in detail.
  • FIG. 10 a schematic flowchart of a third data fusion method provided by an embodiment of the present application is shown.
  • the data fusion method in the embodiment of the present application specifically includes the following steps:
  • the roadside device acquires the roadside sensing data, wherein the roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range by the roadside sensor.
  • the roadside device may be configured with a roadside sensing device, and the roadside sensing device includes at least one roadside sensor, such as a microwave radar, a millimeter wave radar, etc., capable of recognizing the sensing range.
  • the roadside sensing data may include the position, speed, size, color, and the like of the target object.
  • the roadside sensing device may use any one of the roadside sensors alone, or may use any of the roadside sensors simultaneously.
  • the at least one vehicle device acquires vehicle sensing data, wherein the vehicle sensing data is that the vehicle sensing device detects the road environment within the sensing range by using the vehicle sensor to obtain vehicle sensing data.
  • the vehicle device may be configured with a vehicle sensing device including at least one vehicle sensor, such as a combined inertial navigation, microwave radar, millimeter wave radar, a camera, etc., capable of recognizing the sensing range.
  • the vehicle sensing data may include the position, speed, size, color, and the like of the target object.
  • vehicle sensing device may use any one of the vehicle sensors alone, or may use any of the vehicle sensors simultaneously.
  • the at least one vehicle device sends the vehicle perception data to the roadside device. Accordingly, the roadside device receives vehicle sensing data transmitted by at least one vehicle device.
  • the roadside device matches the roadside sensing data with the vehicle sensing data sent by the at least one vehicle device to obtain a matching result.
  • the roadside device may find out the matching relationship between the roadside result unit in the roadside sensing data and the vehicle result unit in the vehicle sensing data through the deviation network.
  • the roadside result unit and the vehicle result unit are used as inputs to the deviation network, and the deviation network will output a matching result between the roadside result unit and the vehicle result unit.
  • S305 The roadside device evaluates the confidence of the matching result by means of interframe loopback and/or multiframe association to obtain an evaluation result, and adjusts the deviation network according to the evaluation result.
  • the first matching result is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame
  • the matching result of the detected vehicle result unit of the detected target object j is the intra-frame matching result of the ith frame, that is, the corresponding roadside result unit of the target object j detected by the roadside sensing device in the ith frame and the vehicle sensing device at the ith frame.
  • the second matching result is an inter-frame matching result of the vehicle sensing device, that is, the corresponding vehicle result unit of the target object j detected by the vehicle sensing device in the ith frame and the vehicle sensing device are detected in the i+1 frame.
  • the matching result of the corresponding vehicle result unit of the target object j measured.
  • the third matching result is the intra-frame matching result of the (i+1)th frame, that is, the corresponding vehicle result unit and the drive test sensing device of the target object j detected by the vehicle sensing device in the (i+1)th frame are at the i-th
  • the matching result of the corresponding roadside result unit of the target object j detected by the +1 frame are at the i-th.
  • the fourth matching result is the inter-frame matching result of the roadside sensing device, that is, the corresponding roadside result unit and the roadside sensing device of the target object j detected by the roadside sensing device in the i+1th frame are The matching result of the corresponding roadside result unit of the target object j detected by the i-th frame.
  • the multi-frame association is mainly obtained according to the inter-frame loopback of multiple consecutive frames.
  • the roadside device fuses the vehicle sensing data sent by the at least one vehicle device with the roadside sensing data by using a fusion formula to obtain a first fusion result.
  • the roadside device may perform data of the roadside sensing data detected by the roadside sensing device within the sensing range and the vehicle sensing data detected by the vehicle sensing device within the sensing range by using a fusion formula. Fusion to obtain the first fusion result.
  • f is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • the function f can be expressed as:
  • w r is the confidence factor of the roadside sensing device
  • w v is the confidence factor of the vehicle sensing device
  • the first fusion result may be obtained by combining the vehicle sensing data sent by one or more vehicle sensing devices with the roadside sensing data of the roadside sensing device by using a fusion formula. Specifically limited.
  • the embodiment of the present application does not describe the fusion of the roadside sensing data and the vehicle sensing data in detail.
  • the roadside device transmits the first fusion result to the target vehicle device.
  • the target vehicle device receives the first fusion result sent by the roadside device.
  • the target vehicle device belongs to the at least one vehicle device.
  • the target vehicle device fuses the first fusion result with the vehicle perception data of the target vehicle device to obtain a second fusion result.
  • the target vehicle device may perform data fusion between the first fusion result and the vehicle sensing data of the target vehicle device by using a fusion formula to obtain a second fusion result.
  • the process of fusing the first fusion result and the vehicle sensing data is similar to the process of fusing the roadside sensing data and the vehicle sensing data, and the description is not further described herein.
  • the roadside device may fuse the vehicle sensing data sent by the plurality of vehicle sensing devices with the roadside sensing data of the vehicle to obtain a first fusion result with a larger sensing range (the sensing range here is the roadside transmission). And superimposing the sensing range of the sensing device and the sensing range of the plurality of vehicle sensing devices, and then transmitting the first fusion result to the target vehicle device for the target vehicle device to fuse the first fusion result with the vehicle sensing data. Expand the sensing range of the vehicle sensing device.
  • the embodiment of the present application provides a fusion device, which can be applied to a vehicle device or a roadside device.
  • the merging device may be a chip, a programmable component, a circuit component, a device (ie, a merging device is a vehicle device or a roadside device) or a system, etc., and is not specifically limited herein.
  • the fusion device 100 includes a sensor system 104 , a control system 106 , a peripheral device 108 , a power source 110 , and a computing device 112 .
  • Computing device 112 can include a processor 113 and a memory 114. Computing device 112 may be part of a controller or controller of fusion device 100.
  • the memory 114 can include instructions 115 that the processor 113 can run, and can also store map data 116.
  • the components of the fusion device 100 can be configured to operate in a manner interconnected with each other and/or with other components coupled to the various systems.
  • power source 110 can provide power to all components of fusion device 100.
  • Computing device 111 can be configured to receive data from sensor system 104, control system 106, and peripherals 108 and control them.
  • fusion device 100 may include more, fewer, or different systems, and each system may include more, fewer, or different components. Moreover, the systems and components shown may be combined or divided in any number of ways.
  • the sensor system 104 can include a number of sensors for sensing a road environment within a range of perception of the fusion device 100.
  • the sensors of the sensor system include a GPS 126, an IMU (Inertial Measurement Unit) 128, a Radio Detection and Radar Ranging (RADAR) unit 130, a Laser Ranging (LIDAR) unit 132, a camera 134, and Actuator 136 to modify the position and/or orientation of the sensor.
  • IMU Inertial Measurement Unit
  • RADAR Radio Detection and Radar Ranging
  • LIDAR Laser Ranging
  • Actuator 136 to modify the position and/or orientation of the sensor.
  • the GPS module 126 can be any sensor for estimating the geographic location of the vehicle.
  • the GPS module 126 may include a transceiver that estimates the position of the fusion device 100 relative to the earth based on satellite positioning data.
  • computing device 111 can be used in conjunction with map data 116 to use GPS module 126 to estimate the location of a lane boundary on a road on which fusion device 100 can travel.
  • the GPS module 126 can take other forms as well.
  • the IMU 128 may be for sensing changes in position and orientation of the vehicle based on inertial acceleration and any combination thereof.
  • the combination of sensors can include, for example, an accelerometer and a gyroscope. Other combinations of sensors are also possible.
  • the RADAR unit 130 can be regarded as an object detecting system for detecting characteristics of a target object using radio waves such as the distance, height, direction or speed of the object.
  • the RADAR unit 130 can be configured to transmit radio waves or microwave pulses that can bounce off any object in the course of the wave.
  • the object may return a portion of the energy of the wave to a receiver (eg, a dish or antenna), which may also be part of the RADAR unit 130.
  • the RADAR unit 130 can also be configured to perform digital signal processing on the received signal (bounce from the object) and can be configured to identify the target object.
  • LIDAR Light Detection and Ranging
  • the LIDAR unit 132 includes a sensor that senses or detects a target object in the road environment within the range using the light sensing or detecting fusion device 100.
  • LIDAR is an optical remote sensing technique that can measure the distance to a target object or other properties of a target object by illuminating the target with light.
  • LIDAR unit 132 can include a laser source and/or a laser scanner configured to emit laser pulses, and a detector for receiving reflections of the laser pulses.
  • the LIDAR unit 132 can include a laser range finder that is reflected by a rotating mirror and scans the laser around the digitized scene in one or two dimensions to acquire distance measurements at specified angular intervals.
  • LIDAR unit 132 may include components such as light (eg, laser) sources, scanners and optical systems, photodetectors, and receiver electronics, as well as position and navigation systems.
  • the LIDAR unit 132 can be configured to image an object using ultraviolet (UV), visible, or infrared light, and can be used with a wide range of target objects, including non-metallic objects.
  • a narrow laser beam can be used to map physical features of an object with high resolution.
  • wavelengths in the range of from about 10 microns (infrared) to about 250 nanometers (UV) can be used.
  • Light is typically reflected via backscattering.
  • Different types of scattering are used for different LIDAR applications such as Rayleigh scattering, Mie scattering and Raman scattering, and fluorescence.
  • LIDAR can thus be referred to as Rayleigh laser RADAR, Mie LIDAR, Raman LIDAR, and sodium/iron/potassium fluorescent LIDAR.
  • Appropriate combinations of wavelengths may allow remote mapping of objects, for example by looking for wavelength dependent changes in the intensity of the reflected signal.
  • Three-dimensional (3D) imaging can be achieved using both a scanned LIDAR system and a non-scanning LIDAR system.
  • "3D gated viewing laser radar” is an example of a non-scanning laser ranging system that uses a pulsed laser and a fast gating camera.
  • the imaging LIDAR can also use a high-speed detector array that is typically built on a single chip using CMOS (Complementary Metal Oxide Semiconductor) and CCD (Charge Coupled Device) manufacturing techniques. And modulating the sensitive detector array to perform.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge Coupled Device
  • each pixel can be locally processed by high speed demodulation or gating such that the array can be processed to represent an image from the camera.
  • thousands of pixels can be acquired simultaneously to create a 3D point cloud representing the object or scene detected by the LIDAR unit 132.
  • a point cloud can include a set of vertices in a 3D coordinate system. These vertices may be defined, for example, by X, Y, Z coordinates, and may represent the outer surface of the target object.
  • the LIDAR unit 132 can be configured to create a point cloud by measuring a large number of points on the surface of the target object, and can output the point cloud as a data file. As a result of the 3D scanning process of the object through the LIDAR unit 132, the point cloud can be used to identify and visualize the target object.
  • the point cloud can be rendered directly to visualize the target object.
  • a point cloud may be converted to a polygonal or triangular mesh model by a process that may be referred to as surface reconstruction.
  • Example techniques for converting a point cloud to a 3D surface may include a Delaunay triangulation, an alpha shape, and a rotating sphere. These techniques include building a network of triangles on existing vertices of a point cloud.
  • Other example techniques may include converting a point cloud to a volumetric distance field, and reconstructing such an implicit surface as defined by a moving cube algorithm.
  • Camera 134 may be any camera (eg, a still camera, video camera, etc.) for acquiring images of the road environment in which the vehicle is located. To this end, the camera can be configured to detect visible light, or can be configured to detect light from other portions of the spectrum, such as infrared or ultraviolet light. Other types of cameras are also possible. Camera 134 may be a two-dimensional detector or may have a three-dimensional spatial extent. In some examples, camera 134 may be, for example, a distance detector configured to generate a two-dimensional image indicative of the distance from camera 134 to several points in the environment. To this end, the camera 134 can use one or more distance detection techniques.
  • a distance detector configured to generate a two-dimensional image indicative of the distance from camera 134 to several points in the environment. To this end, the camera 134 can use one or more distance detection techniques.
  • camera 134 can be configured to use structured light technology in which fusion device 100 illuminates an object in the environment with a predetermined light pattern, such as a grid or checkerboard pattern, and uses camera 134 to detect a predetermined light pattern from the object. reflection. Based on the distortion in the reflected light pattern, the fusion device 100 can be configured to detect the distance of a point on the object.
  • the predetermined light pattern may include infrared light or light of other wavelengths.
  • Actuator 136 can be configured, for example, to modify the position and/or orientation of the sensor.
  • Sensor system 104 may additionally or alternatively include components in addition to those shown.
  • Control system 106 can be configured to control the operation of fusion device 100 and its components. To this end, control system 106 can include sensor fusion algorithm 144, computer vision system 146, navigation or route control system 148, and obstacle avoidance system 150.
  • Sensor fusion algorithm 144 may include, for example, an algorithm (or a computer program product that stores the algorithm) that computing device 111 may operate. Sensor fusion algorithm 144 can be configured to accept data from sensor 104 as an input. The data may include, for example, data representing information sensed at the sensors of sensor system 104. Sensor fusion algorithm 144 may include, for example, a Kalman filter, a Bayesian network, or another algorithm. The sensor fusion algorithm 144 may also be configured to provide various ratings based on data from the sensor system 104, including, for example, an assessment of individual objects and/or features in the environment in which the vehicle is located, an assessment of a particular situation, and/or based on An assessment of the likely impact of a particular situation. Other evaluations are also possible.
  • Computer vision system 146 may be any system configured to process and analyze images captured by camera 134 to identify objects and/or features in the environment in which fusion device 100 is located, such as lane information, Traffic signals and obstacles. To this end, computer vision system 146 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, or other computer vision techniques. In some examples, computer vision system 146 may additionally be configured as a mapping environment, following an object, estimating the speed of an object, and the like.
  • SFM Structure from Motion
  • the navigation and route control system 148 can be any system configured to determine the driving route of the vehicle.
  • the navigation and route control system 148 can additionally be configured to dynamically update the driving route while the vehicle is in operation.
  • navigation and route control system 148 can be configured to combine data from sensor fusion algorithm 144, GPS module 126, and one or more predetermined maps to determine a driving route for the vehicle.
  • the obstacle avoidance system 150 can be any system configured to identify, evaluate, and avoid or otherwise cross obstacles in the environment in which the vehicle is located.
  • Control system 106 may additionally or alternatively include components in addition to those shown.
  • Peripheral device 108 can be configured to allow fusion device 100 to interact with external sensors, other vehicles, and/or users.
  • peripheral device 108 can include, for example, wireless communication system 152, touch screen 154, microphone 156, and/or speaker 158.
  • Wireless communication system 152 can be any system configured to be wirelessly coupled to one or more other vehicles, sensors, or other entities, either directly or via a communication network.
  • the wireless communication system 152 can include an antenna and chipset for communicating with other vehicles, sensors, or other entities, either directly or through an air interface.
  • the chipset or the entire wireless communication system 152 can be arranged to communicate in accordance with one or more other types of wireless communications (e.g., protocols) such as those described in Bluetooth, IEEE 802.11 (including any IEEE 802.11 revision).
  • Wireless communication system 152 can take other forms as well.
  • cellular technology such as GSM, CDMA, UMTS (Universal Mobile Telecommunications System), EV-DO, WiMAX or LTE (Long Term Evolution)), Zigbee, DSRC (Dedicated Short Range Communications) , dedicated short-range communication) and RFID (Radio Frequency Identification) communication, and so on.
  • Wireless communication system 152 can take other forms as well.
  • the touch screen 154 can be used by a user to input commands to the fusion device 100.
  • the touch screen 154 can be configured to sense at least one of a position and a movement of a user's finger via a capacitive sensing, a resistive sensing, or a surface acoustic wave process or the like.
  • the touch screen 154 may be capable of sensing finger movement in a direction parallel to the touch screen surface or in the same plane as the touch screen surface, in a direction perpendicular to the touch screen surface, or in both directions, and may also be capable of sensing application to The level of pressure on the surface of the touch screen.
  • Touch screen 154 may be formed from one or more translucent or transparent insulating layers and one or more translucent or transparent conductive layers. Touch screen 154 can take other forms as well.
  • the microphone 156 can be configured to receive audio (eg, a voice command or other audio input) from a user of the fusion device 100.
  • the speaker 158 can be configured to output audio to a user of the fusion device 100.
  • Peripheral device 108 may additionally or alternatively include components in addition to those shown.
  • the power source 110 can be configured to provide power to some or all of the components of the fusion device 100.
  • the power source 110 can include, for example, a rechargeable lithium ion or lead acid battery.
  • one or more battery packs can be configured to provide power.
  • Other power materials and configurations are also possible.
  • power source 110 and energy source 120 can be implemented together, as in some all-electric vehicles.
  • Processor 113 included in computing device 111 may include one or more general purpose processors and/or one or more special purpose processors (eg, image processors, digital signal processors, etc.). Insofar as the processor 113 includes more than one processor, such processors can work individually or in combination. Computing device 111 may implement the function of controlling vehicle 100 based on input received through user interface 112.
  • processors 113 may include one or more general purpose processors and/or one or more special purpose processors (eg, image processors, digital signal processors, etc.). Insofar as the processor 113 includes more than one processor, such processors can work individually or in combination.
  • Computing device 111 may implement the function of controlling vehicle 100 based on input received through user interface 112.
  • the memory 114 can include one or more volatile storage components and/or one or more non-volatile storage components, such as optical, magnetic, and/or organic storage devices, and the memory 114 can be fully or partially coupled to the processor 113. integrated.
  • Memory 114 may include instructions 115 (eg, program logic) executable by processor 113 to perform various vehicle functions, including any of the functions or methods described herein.
  • the components of the fusion device 100 can be configured to operate in a manner that is interconnected with other components internal and/or external to their respective systems. To this end, the components and systems of the fusion device 100 can be communicatively linked together by a system bus, network, and/or other connection mechanism.
  • the fusion device 100 executes the following instructions in the processor 113:
  • the vehicle sensing data is obtained by detecting, by the vehicle sensing device, a road environment within a sensing range;
  • roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the vehicle sensing data and the roadside sensing data are fused by a fusion formula to obtain a first fusion result.
  • the fusion device 100 executes the following instructions in the processor 113:
  • the roadside device And receiving, by the roadside device, the first fusion result, where the roadside device fuses the vehicle sensing data of at least one vehicle device and the roadside sensing data by using a fusion formula, where The roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the vehicle sensing data and the first fusion result are fused to obtain a second fusion result.
  • the fusion formula is expressed as:
  • result r is a roadside result set
  • the roadside result set is used to represent the roadside sensing data
  • result v is a vehicle result set
  • vehicle result set is used to represent the vehicle sensing data
  • y is The first fusion result is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • w r is the confidence factor of the roadside sensing device
  • w r (w r1 , w r2 , ..., w rM ), result r (roadside 1 , roadside 2 , ..., roadside M )
  • M is The roadside sensing device senses the number of target objects in the range
  • w ri is a confidence factor corresponding to the target object i within the sensing range of the roadside sensing device
  • the roadside i is the target object within the sensing range of the roadside sensing device.
  • the confidence factor is determined jointly based on the sensing device parameters, the perceived distance of the target object, and the perceived angle of the target object.
  • the confidence factor w can be obtained according to the following formula:
  • S k is the sensing device parameter
  • R i is the sensing distance of the target object
  • ⁇ j is the sensing angle of the target object
  • g is a calibration parameter table obtained by the sensing device.
  • the vehicle result set includes at least one vehicle result unit, the at least one vehicle result unit has a one-to-one correspondence with at least one target object, and each of the at least one vehicle result unit has a vehicle result
  • the unit is used to describe the characteristics of the corresponding target object from a multi-dimensional perspective.
  • one of the at least one vehicle result unit is represented as vehicle j (p vj , v vj , s vj , c vj ), wherein p vj is represented as the vehicle sensing device detection
  • v vj is the speed of the target object j detected by the vehicle sensing device
  • s vj is the size of the target object j detected by the vehicle sensing device
  • N is the number of target objects within the sensing range of the vehicle sensing device
  • j is a natural number, 0 ⁇ j ⁇ N.
  • the roadside result set includes at least one roadside result unit, and the at least one roadside result unit has a one-to-one correspondence with at least one target object, and the at least one roadside result unit is in the unit.
  • Each roadside result unit is used to describe the characteristics of the corresponding target object from a multidimensional perspective.
  • one of the at least one roadside result unit is represented as roadside i (p vi , v vi , s vi , c vi ), wherein p vi is represented as the roadside sensing
  • p vi is represented as the roadside sensing
  • v vi is the speed of the target object i detected by the roadside sensing device
  • s vi is the target object detected by the roadside sensing device.
  • the size of c vi is the color of the target object i detected by the roadside sensing device
  • M is the number of target objects within the sensing range of the roadside sensing device
  • i is a natural number, 0 ⁇ i ⁇ M .
  • the roadside sensing data and the vehicle perception are obtained before the vehicle sensing data and the roadside sensing data are merged by the fusion formula to obtain a first fusion result.
  • the data is matched to obtain a matching result, and the vehicle sensing data and the roadside sensing data are fused according to the matching result to obtain a first fusion result.
  • the matching relationship between the roadside result unit in the roadside result set and the vehicle result unit in the vehicle result set is found by the deviation network.
  • the matching relationship between the roadside result unit in the roadside result set and the vehicle result unit in the vehicle result set is found by the deviation network, and specifically includes:
  • the confidence of the matching result by the inter-frame loopback and/or the multi-frame association manner is performed to obtain the evaluation result, and the deviation network is adjusted according to the evaluation result.
  • the second matching result is a corresponding vehicle of the target object j detected by the vehicle sensing device in the ith frame a result of matching the result unit with a corresponding vehicle result unit of the target object j detected by the vehicle sensing device in the (i+1)th frame
  • the third matching result is that the vehicle sensing device is in the i+1th frame
  • the fourth matching result is Corresponding roadside result unit of the target object j detected by the roadside sensing device in the i+1th frame and the road The matching result of the corresponding roadside result unit of the target object j detected by the side sensing device in the i-th frame.
  • the merging device 200 when the merging device is a roadside device, the merging device 200 includes an RF (Radio Frequency) circuit 210, a memory 220, other input devices 230, a display screen 240, a sensor system 250, and an I/O. Subsystem 270, processor 280, and power supply 290 and the like. It will be understood by those skilled in the art that the roadside sensing device structure shown in FIG. 12 does not constitute a limitation to the roadside sensing device, and may include more or less components than those illustrated, or may combine certain components. Or split some parts, or different parts. Those skilled in the art will appreciate that the display screen 240 can be used to display a User Interface (UI).
  • UI User Interface
  • the RF circuit 210 can be used to send and receive data.
  • RF circuits include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, and the like.
  • RF circuitry 210 can also communicate with the network and other devices via wireless communication.
  • the wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access). , Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), e-mail, SMS (Short Messaging Service), and the like.
  • the memory 220 may include a high speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • Memory 220 can include instructions 222 that processor 280 can run, and can also store map data 224.
  • Other input devices 230 can be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the fusion device 200.
  • other input devices 230 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and light mice (the light mouse is not sensitive to display visual output).
  • function keys such as volume control buttons, switch buttons, etc.
  • trackballs mice, joysticks, and light mice (the light mouse is not sensitive to display visual output).
  • Other input devices 230 are coupled to other input device controllers 271 of I/O subsystem 270 for signal interaction with processor 280 under the control of other device input controllers 271.
  • the display screen 240 can include a display panel 241 and a touch panel 242.
  • the display panel 241 can be configured by using an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.
  • the touch panel 242, also referred to as a touch screen, a touch sensitive screen, etc., can collect contact or non-contact operations on or near the user (eg, the user uses any suitable object or accessory such as a finger, a stylus, etc. on the touch panel 242. Or the operation in the vicinity of the touch panel 242 may also include a somatosensory operation; the operation includes a single point control operation, a multi-point control operation, and the like, and drives the corresponding connection device according to a preset program.
  • Sensor system 250 can include a number of sensors for sensing a road environment within a range of perception of fusion device 200. As shown, the sensors of the sensor system include a GPS 251, a Radio Detection and Radar Ranging (RADAR) unit 255, a Laser Ranging (LIDAR) unit 257, a camera 258, and actuation for modifying the position and/or orientation of the sensor. 259. Sensor system 250 is coupled to sensor controller 272 of I/O subsystem 270 for signal interaction with processor 280 under the control of sensor controller 272.
  • RADAR Radio Detection and Radar Ranging
  • LIDAR Laser Ranging
  • the GPS module 251 can be any sensor for estimating the geographic location of the vehicle.
  • the GPS module 251 may include a transceiver that estimates the position of the fusion device 200 relative to the earth based on satellite positioning data.
  • the fusion device 200 can be used in conjunction with the map data 224 to estimate the location of the fusion device 200 in the road using the GPS module 251.
  • the GPS module 126 can take other forms as well.
  • the RADAR unit 255 can be regarded as an object detecting system for detecting characteristics of a target object using radio waves such as the distance, height, direction or speed of the object.
  • the RADAR unit 255 can be configured to transmit radio waves or microwave pulses that can bounce off any object in the course of the wave.
  • the object may return a portion of the energy of the wave to a receiver (eg, a dish or antenna), which may also be part of the RADAR unit 255.
  • the RADAR unit 255 can also be configured to perform digital signal processing on the received signal (bounce from the object) and can be configured to identify the target object.
  • LIDAR Light Detection and Ranging
  • the LIDAR unit 257 includes a sensor that uses the light sensing or detection fusion device 200 to sense a target object in the road environment within range.
  • LIDAR is an optical remote sensing technique that can measure the distance to a target object or other properties of a target object by illuminating the target with light.
  • LIDAR unit 257 can include a laser source and/or a laser scanner configured to emit laser pulses, and a detector for receiving reflections of the laser pulses.
  • the LIDAR unit 257 can include a laser range finder that is reflected by a rotating mirror and scans the laser around the digitized scene in one or two dimensions to acquire distance measurements at specified angular intervals.
  • LIDAR unit 257 can include components such as light (eg, laser) sources, scanners and optical systems, photodetectors, and receiver electronics, as well as position and navigation systems.
  • the LIDAR unit 257 can be configured to image an object using ultraviolet (UV), visible, or infrared light, and can be used with a wide range of target objects, including non-metallic objects.
  • UV ultraviolet
  • a narrow laser beam can be used to map physical features of an object with high resolution.
  • wavelengths in the range of from about 10 microns (infrared) to about 250 nanometers (UV) can be used.
  • Light is typically reflected via backscattering.
  • Different types of scattering are used for different LIDAR applications such as Rayleigh scattering, Mie scattering and Raman scattering, and fluorescence.
  • LIDAR can thus be referred to as Rayleigh laser RADAR, Mie LIDAR, Raman LIDAR, and sodium/iron/potassium fluorescent LIDAR.
  • Appropriate combinations of wavelengths may allow remote mapping of objects, for example by looking for wavelength dependent changes in the intensity of the reflected signal.
  • Three-dimensional (3D) imaging can be achieved using both a scanned LIDAR system and a non-scanning LIDAR system.
  • "3D gated viewing laser radar” is an example of a non-scanning laser ranging system that uses a pulsed laser and a fast gating camera.
  • the imaging LIDAR can also use a high-speed detector array that is typically built on a single chip using CMOS (Complementary Metal Oxide Semiconductor) and CCD (Charge Coupled Device) manufacturing techniques. And modulating the sensitive detector array to perform.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge Coupled Device
  • each pixel can be locally processed by high speed demodulation or gating such that the array can be processed to represent an image from the camera.
  • thousands of pixels can be acquired simultaneously to create a 3D point cloud representing the object or scene detected by the LIDAR unit 257.
  • a point cloud can include a set of vertices in a 3D coordinate system. These vertices may be defined, for example, by X, Y, Z coordinates, and may represent the outer surface of the target object.
  • the LIDAR unit 257 can be configured to create a point cloud by measuring a large number of points on the surface of the target object, and can output the point cloud as a data file. As a result of the 3D scanning process on the object by the LIDAR unit 257, the point cloud can be used to identify and visualize the target object.
  • the point cloud can be rendered directly to visualize the target object.
  • a point cloud may be converted to a polygonal or triangular mesh model by a process that may be referred to as surface reconstruction.
  • Example techniques for converting a point cloud to a 3D surface may include a Delaunay triangulation, an alpha shape, and a rotating sphere. These techniques include building a network of triangles on existing vertices of a point cloud.
  • Other example techniques may include converting a point cloud to a volumetric distance field, and reconstructing such an implicit surface as defined by a moving cube algorithm.
  • Camera 258 may be any camera (eg, a still camera, video camera, etc.) for acquiring images of the road environment in which the vehicle is located. To this end, the camera can be configured to detect visible light, or can be configured to detect light from other portions of the spectrum, such as infrared or ultraviolet light. Other types of cameras are also possible. Camera 258 can be a two-dimensional detector or can have a three-dimensional spatial extent. In some examples, camera 258 can be, for example, a distance detector configured to generate a two-dimensional image indicative of the distance from camera 258 to several points in the environment. To this end, camera 258 can use one or more distance detection techniques.
  • a distance detector configured to generate a two-dimensional image indicative of the distance from camera 258 to several points in the environment. To this end, camera 258 can use one or more distance detection techniques.
  • camera 258 can be configured to use structured light technology in which fusion device 200 illuminates an object in the environment with a predetermined light pattern, such as a grid or checkerboard pattern, and uses camera 258 to detect a predetermined light pattern from the object. reflection. Based on the distortion in the reflected light pattern, the roadside sensing device 258 can be configured to detect the distance of a point on the object.
  • the predetermined light pattern may include infrared light or light of other wavelengths.
  • the I/O subsystem 270 is used to control external devices for input and output, and may include other device input controllers 271 and sensor controllers 272.
  • one or more other input control device controllers 271 receive signals from other input devices 230 and/or send signals to other input devices 230.
  • Other input devices 230 may include physical buttons (press buttons, rocker buttons, etc.) , dial, slide switch, joystick, click wheel, light mouse (light mouse is a touch-sensitive surface that does not display visual output, or an extension of a touch-sensitive surface formed by a touch screen). It is worth noting that other input control device controllers 271 can be connected to any one or more of the above devices.
  • Sensor controller 272 can receive signals from one or more sensors 250 and/or send signals to one or more sensors 250.
  • the processor 280 is a control center of the fusion device 200 that connects various portions of the entire fusion device 200 using various interfaces and lines, by running or executing software programs and/or modules stored in the memory 220, and by calling them stored in the memory 220.
  • the data, the various functions of the fusion device 200 and the processing data are performed to perform overall monitoring of the fusion device 200.
  • the processor 280 can include one or more processing units; preferably, the processor 280 can integrate 2 modem processors, wherein the modem processor primarily processes wireless communications. It can be understood that the above modem processor may not be integrated into the processor 280.
  • the fusion device 200 also includes a power source 290 (e.g., a battery) that supplies power to the various components.
  • a power source 290 e.g., a battery
  • the power source can be logically coupled to the processor 280 via a power management system to manage functions such as charging, discharging, and power consumption through the power management system.
  • the fusion device 200 executes the following instructions in the processor 280:
  • vehicle sensing data wherein the vehicle sensing data is obtained by detecting, by the vehicle sensing device, a road environment within a sensing range;
  • roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the vehicle sensing data and the roadside sensing data are fused by a fusion formula to obtain a first fusion result.
  • the fusion device 200 executes the following instructions in the processor 280:
  • Vehicle sensing data transmitted by at least one vehicle device wherein the vehicle sensing data is obtained by detecting, by the vehicle sensing device, a road environment within a sensing range
  • the target vehicle device is configured to fuse the vehicle perception data and the first fusion result to obtain a second fusion result, the target vehicle device belonging to Said at least one vehicle device.
  • the fusion formula is expressed as:
  • result r is a roadside result set
  • the roadside result set is used to represent the roadside sensing data
  • result v is a vehicle result set
  • vehicle result set is used to represent the vehicle sensing data
  • y is The first fusion result is used to map the first fusion result according to the roadside result set and the vehicle result set.
  • w r is the confidence factor of the roadside sensing device
  • w r (w r1 , w r2 , ..., w rM ), result r (roadside 1 , roadside 2 , ..., roadside M )
  • M is The roadside sensing device senses the number of target objects in the range
  • w ri is a confidence factor corresponding to the target object i within the sensing range of the roadside sensing device
  • the roadside i is the target object within the sensing range of the roadside sensing device.
  • the confidence factor is determined jointly based on the sensing device parameters, the perceived distance of the target object, and the perceived angle of the target object.
  • the confidence factor w can be obtained according to the following formula:
  • S k is the sensing device parameter
  • R i is the sensing distance of the target object
  • ⁇ j is the sensing angle of the target object
  • g is a calibration parameter table obtained by the sensing device.
  • the vehicle result set includes at least one vehicle result unit, the at least one vehicle result unit has a one-to-one correspondence with at least one target object, and each of the at least one vehicle result unit has a vehicle result
  • the unit is used to describe the characteristics of the corresponding target object from a multi-dimensional perspective.
  • one of the at least one vehicle result unit is represented as vehicle j (p vj , v vj , s vj , c vj ), wherein p vj is represented as the vehicle sensing device detection
  • v vj is the speed of the target object j detected by the vehicle sensing device
  • s vj is the size of the target object j detected by the vehicle sensing device
  • N is the number of target objects within the sensing range of the vehicle sensing device
  • j is a natural number, 0 ⁇ j ⁇ N.
  • the roadside result set includes at least one roadside result unit, and the at least one roadside result unit has a one-to-one correspondence with at least one target object, and the at least one roadside result unit is in the unit.
  • Each roadside result unit is used to describe the characteristics of the corresponding target object from a multidimensional perspective.
  • one of the at least one roadside result unit is represented as roadside i (p vi , v vi , s vi , c vi ), wherein p vi is represented as the roadside sensing
  • p vi is represented as the roadside sensing
  • v vi is the speed of the target object i detected by the roadside sensing device
  • s vi is the target object detected by the roadside sensing device.
  • the size of c vi is the color of the target object i detected by the roadside sensing device
  • M is the number of target objects within the sensing range of the roadside sensing device
  • i is a natural number, 0 ⁇ i ⁇ M .
  • the matching relationship between the roadside result unit in the roadside result set and the vehicle result unit in the vehicle result set is found by the deviation network.
  • the matching relationship between the roadside result unit in the roadside result set and the vehicle result unit in the vehicle result set is found by the deviation network, and specifically includes:
  • the confidence of the matching result by the inter-frame loopback and/or the multi-frame association manner is performed to obtain the evaluation result, and the deviation network is adjusted according to the evaluation result.
  • the second matching result is a corresponding vehicle of the target object j detected by the vehicle sensing device in the ith frame a result of matching the result unit with a corresponding vehicle result unit of the target object j detected by the vehicle sensing device in the (i+1)th frame
  • the third matching result is that the vehicle sensing device is in the i+1th frame
  • the fourth matching result is Corresponding roadside result unit of the target object j detected by the roadside sensing device in the i+1th frame and the road The matching result of the corresponding roadside result unit of the target object j detected by the side sensing device in the i-th frame.
  • FIG. 13 is a schematic structural diagram of a fusion device according to an embodiment of the present invention.
  • the fusion device 300 may include: a first acquisition module 301, a second acquisition module 302, and a fusion module 303.
  • the first acquiring module 301 is configured to acquire vehicle sensing data, where the vehicle sensing data is obtained by detecting, by the vehicle sensing device, a road environment within a sensing range;
  • the second acquiring module 302 is configured to acquire roadside sensing data, where the roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the fusion module 303 is configured to fuse the vehicle sensing data and the roadside sensing data by a fusion formula to obtain a first fusion result.
  • FIG. 14 is a schematic structural diagram of a fusion device according to an embodiment of the present invention.
  • the fusion device 400 of this embodiment includes: a sending module 401, a receiving module 402, and a fusion module 403.
  • the sending module 401 is configured to send the vehicle sensing data to the roadside device, where the vehicle sensing data is obtained by the vehicle sensing device detecting the road environment within the sensing range;
  • the receiving module 402 is configured to receive a first fusion result sent by the roadside device, where the first fusion result is a vehicle sensing data of the roadside device over-fusion formula for at least one vehicle device, and a roadside
  • the roadside sensing data is obtained by the roadside sensing device detecting the road environment within the sensing range;
  • the fusion module 403 is configured to fuse the vehicle sensing data of the at least one vehicle device with the first fusion result to obtain a second fusion result.
  • the roadside sensing data detected by the roadside sensing device and the vehicle sensing data detected by the vehicle sensing device are combined to realize the sensing range of the roadside sensing device and the vehicle sensing device.
  • the sensing range is superimposed to effectively extend the sensing range.
  • the disclosed system, terminal, and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Mining & Analysis (AREA)
  • Electromagnetism (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • Optics & Photonics (AREA)
  • Acoustics & Sound (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)

Abstract

一种数据融合方法以及相关设备，该方法包括：获得车辆感知数据（S101），其中，所述车辆感知数据为车辆传感装置通过车辆传感器对感知范围内的道路环境进行侦测得到的；获取路侧感知数据（S102），其中，所述路侧感知数据为路侧传感装置通过路侧传感器对感知范围内的道路环境进行侦测得到的；通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果（S106）。该方法能够实现将路侧传感装置的感知范围和车辆传感装置的感知范围进行叠加，从而有效地扩展感知范围。

Description

数据融合方法以及相关设备 技术领域
本发明涉及自动驾驶领域，尤其涉及一种数据融合方法以及相关设备。
背景技术
道路环境的感知是实现自动驾驶的首要任务。自动驾驶车辆在感知到道路的环境之后，才能避让道路中的其他车辆或者行人等等，实现安全行驶。为了实现道路环境的感知，现有技术中的自动驾驶车辆通过安装在车上的车辆传感装置来对道路中的其他车辆或者行人等等进行侦测，从而感知道路的环境。但是，在现有技术中车辆传感装置的感知范围比较小，难以满足实现自动驾驶的要求。
发明内容
本申请实施例提供了一种数据融合方法以及相关设备，能够实现将路侧传感装置的感知范围和车辆传感装置的感知范围进行叠加，从而有效地扩展感知范围。
第一方面，提供了一种数据融合方法，可以应用于车辆设备侧或者路侧设备侧，包括：
获得车辆感知数据，其中，所述车辆感知数据为车辆传感装置对感知范围内的道路环境进行侦测得到的；
获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
结合第一方面，所述融合公式表示为：
y＝f(result r，result v)，
其中，result r为路侧结果集，所述路侧结果集用于表示所述路侧感知数据，result v为车辆结果集，所述车辆结果集用于表示所述车辆感知数据，y为所述第一融合结果，函数f用于根据所述路侧结果集和所述车辆结果集映射出所述第一融合结果。
在一具体的实施例中，
Figure PCTCN2019078646-appb-000001
其中，w r为所述路侧传感装置的置信因子，w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为所述路侧传感装置感知范围内目标物体的数量，w ri为所述路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为自然数，0<i≤M；w v为所述车辆传感装置的置信因子，w v＝(w v1，w v2，…，w vN)，result v(vehicle 1，vehicle 2，…，vehicle N)，N为所述车辆传感装置感知范围内目标物体的数量，w vj为所述车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为自然数，0<j≤N。
更具体地，所述置信因子是根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定的。
例如，置信因子w可以根据如下公式获得：
w＝g(S k，R i，θ j)，w∈[0,1]
其中，S k为所述传感装置参数，R i为所述目标物体的感知距离，θ j为所述目标物体的感知角度，g为通过传感装置标定得到的标定参数表。
需要说明的是，当传感装置包括多个传感器时，置信因子可以综合考虑多个传感器的置信度而得到，例如，可以通过加权或求平均等方式综合考虑多个传感器的置信度。
在一具体的实施例中，所述车辆结果集包括至少一个车辆结果单元，所述至少一个车辆结果单元与至少一个目标物体存在一一对应关系，所述至少一个车辆结果单元中每一个车辆结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个车辆结果单元中的任一个车辆结果单元表示为vehicle j(p vj，v vj，s vj，c vj)，其中，p vj表示为所述车辆传感装置侦测到的目标物体j的位置，v vj表示为所述车辆传感装置侦测到的目标物体j的速度，s vj表示为所述车辆传感装置侦测到的目标物体j的大小，c vj表示为所述车辆传感装置侦测到的目标物体j的颜色，N为所述车辆传感装置感知范围内目标物体的数量，j为自然数，0<j≤N。
在一具体的实施例中，所述路侧结果集包括至少一个路侧结果单元，所述至少一个路侧结果单元与至少一个目标物体存在一一对应关系，所述至少一个路侧结果单元中每一个路侧结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个路侧结果单元中的任一个路侧结果单元表示为roadside i(p vi，v vi，s vi，c vi)，其中，p vi表示为所述路侧传感装置侦测到的目标物体i的位置，v vi表示为所述路侧传感装置侦测到的目标物体i的速度，s vi表示为所述路侧传感装置侦测到的目标物体i的大小，c vi表示为所述路侧传感装置侦测到的目标物体i的颜色，M为所述路侧传感装置感知范围内目标物体的数量，i为自然数，0<i≤M。
在一具体的实施例中，在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得融合结果之前，所述方法还包括：
将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果；
所述将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果，包括：
根据所述匹配结果将所述车辆感知数据和所述路侧感知数据进行融合以获得所述第一融合结果。
更具体地，通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来。
例如，通过以下公式：S＝Deviation(roadside i，vehicle j)将所述路侧结果集中的路侧结果单元和所述路侧结果集中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为所述偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
其中，所述偏差网络Deviation用逆向传播BP神经网络进行表示。
在一具体实施例中，在所述将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果之后，所述方法还包括：
通过帧间回环和/或多帧关联的方式对所述匹配结果的置信度进行评价以获得评价结果；
根据所述评价结果对所述偏差网络进行调整。
更具体地，所述帧间回环为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果，第一匹配结果为所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第二匹配结果为所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第三匹配结果为所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与所述路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果，第四匹配结果为所述路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
更具体地，所述多帧关联定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…。
在一具体实施例中，当应用于路侧设备侧时，所述获取车辆感知数据，包括：接收至少一台车辆设备的车辆感知数据；
所述在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果之后，还包括：
向目标车辆设备发送所述第一融合结果，其中，所述目标车辆设备用于将所述目标车辆设备的车辆感知数据和所述第一融合结果进行融合以获得第二融合结果，所述目标车辆设备属于所述至少一台车辆设备。
第二方面，提供了一种数据融合方法，应用于车辆设备侧，包括如下步骤：
向路侧设备发送车辆感知数据，其中，所述车辆感知数据是车辆传感装置对感知范围内的道路环境进行侦测得到的；
接收所述路侧设备发送的第一融合结果，其中，所述第一融合结果为所述路侧设备过融合公式对至少一台车辆设备发送的所述车辆感知数据以及路侧感知数据进行融合得到的，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
将所述车辆感知数据和所述第一融合结果进行融合以获得第二融合结果。
结合第二方面，所述融合公式表示为：
y＝f(result r，result v)，
其中，result r为路侧结果集，所述路侧结果集用于表示所述路侧感知数据，result v为车辆结果集，所述车辆结果集用于表示所述车辆感知数据，y为所述第一融合结果，函数f用于根据所述路侧结果集和所述车辆结果集映射出所述第一融合结果。
在一具体的实施例中，
Figure PCTCN2019078646-appb-000002
其中，w r为所述路侧传感装置的置信因子，w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为所述路侧传感装置感知范围内目标物体的数量，w ri为所述路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为自然数，0<i≤M；w v为所述车辆传感装置的置信因子，w v＝(w v1，w v2，…，w vN)，result v(vehicle 1，vehicle 2，…，vehicle N)，N为所述车辆传感装置感知范围内目标物体的数量，w vj为所述车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为自然数，0<j≤N。
更具体地，所述置信因子是根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定的。
例如，置信因子w可以根据如下公式获得：
w＝g(S k，R i，θ j)，w∈[0,1]
其中，S k为所述传感装置参数，R i为所述目标物体的感知距离，θ j为所述目标物体的感知角度，g为通过传感装置标定得到的标定参数表。
需要说明的是，当传感装置包括多个传感器时，置信因子可以综合考虑多个传感器的置信度而得到，例如，可以通过加权或求平均等方式综合考虑多个传感器的置信度。
在一具体的实施例中，所述车辆结果集包括至少一个车辆结果单元，所述至少一个车辆结果单元与至少一个目标物体存在一一对应关系，所述至少一个车辆结果单元中每一个车辆结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个车辆结果单元中的任一个车辆结果单元表示为vehicle j(p vj，v vj，s vj，c vj)，其中，p vj表示为所述车辆传感装置侦测到的目标物体j的位置，v vj表示为所述车辆传感装置侦测到的目标物体j的速度，s vj表示为所述车辆传感装置侦测到的目标物体j的大小，c vj表示为所述车辆传感装置侦测到的目标物体j的颜色，N为所述车辆传感装置感知范围内目标物体的数量，j为自然数，0<j≤N。
在一具体的实施例中，所述路侧结果集包括至少一个路侧结果单元，所述至少一个路侧结果单元与至少一个目标物体存在一一对应关系，所述至少一个路侧结果单元中每一个路侧结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个路侧结果单元中的任一个路侧结果单元表示为roadside i(p vi，v vi，s vi，c vi)，其中，p vi表示为所述路侧传感装置侦测到的目标物体i的位置，v vi表示为所述路侧传感装置侦测到的目标物体i的速度，s vi表示为所述路侧传感装置侦测到的目标物体i的大小，c vi表示为所述路侧传感装置侦测到的目标物体i的颜色，M为所述路侧传感装置感知范围内目标物体的数量，i为自然数，0<i≤M。
在一具体的实施例中，在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得融合结果之前，所述方法还包括：
将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果；
所述将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果，包括：
根据所述匹配结果将所述车辆感知数据和所述路侧感知数据进行融合以获得所述第一融合结果。
更具体地，通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来。
例如，通过以下公式：S＝Deviation(roadside i，vehicle j)将所述路侧结果集中的路侧结果单元和所述路侧结果集中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为所述偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
其中，所述偏差网络Deviation用逆向传播BP神经网络进行表示。
在一具体实施例中，在所述将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果之后，所述方法还包括：
通过帧间回环和/或多帧关联的方式对所述匹配结果的置信度进行评价以获得评价结果；
根据所述评价结果对所述偏差网络进行调整。
更具体地，所述帧间回环为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果，第一匹配结果为所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第二匹配结果为所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第三匹配结果为所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与所述路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果，第四匹配结果为所述路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
更具体地，所述多帧关联定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…。
第三方面，提供了一种融合装置，包括用于执行第一方面所述的方法的单元。
第四方面，提供了一种融合装置，包括用于执行第二方面所述的方法的单元。
第五方面，提供了一种融合装置，存储器以及与所述存储器耦合的处理器、通信模块，其中：所述通信模块用于发送或者接收外部发送的数据，所述存储器用于存储程序代码，所述处理器用于调用所述存储器存储的程序代码以执行第一方面任一项或者第二方面任一项描述的方法。
第六方面，提供了一种计算机可读存储介质，包括指令，当所述指令在融合装置上运行时，使得所述融合装置执行如第一方面任一项或者第二方面任一项所述的方法。
第七方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行第一方面任一项或者第二方面任一项所述的方法。
通过上述方案，将路侧传感装置侦测得到的路侧感知数据和车辆传感装置侦测得到的车辆感知数据进行融合，可以实现将路侧传感装置的感知范围和车辆传感装置的感知范围进行叠加，从而有效地扩展感知范围。
附图说明
为了更清楚地说明本发明实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例涉及的一种应用场景的示意图；
图2是本申请实施例的涉及的传感装置的安装空间角度的示意图；
图3是本申请实施例的涉及的传感装置的安装坐标的示意图；
图4是本申请实施例的涉及的感知坐标系的示意图；
图5是本申请实施例的涉及的逆向传播神经网络的结构示意图；
图6是本申请实施例的涉及的帧间回环的示意图；
图7是本申请实施例的涉及的多帧关联的示意图；
图8是本申请实施例提供的第一种数据融合方法的流程示意图；
图9是本申请实施例提供的第二种数据融合方法的流程示意图；
图10是本申请实施例提供的第二种数据融合方法的流程示意图；
图11-图14是本申请实施例提供的四种融合装置的结构示意图。
具体实施方式
图1是本申请实施例涉及的一种应用场景的示意图。如图1所示，道路的路边安装有至少一个路侧设备，道路中间行驶的车辆中至少一台车辆安装有车辆设备。其中：
路侧设备用于从路侧的角度对道路环境进行侦测以获得路侧感知数据。路侧设备可以配置有路侧传感装置，所述路侧传感装置可以包括至少一个路侧传感器，例如微波雷达、毫米波雷达等等，能够识别出感知范围内的目标物体(例如，车辆和行人)的位置、速度和大小等等路侧感知数据。路侧传感装置还可以包括摄像头等路侧传感器，摄像头除了能够识别出感知范围内的目标物体的位置、速度和大小等等路侧感知数据之外，还可以识别出感知范围内的目标物体的颜色(例如，车辆的颜色和行人身上衣物的颜色)等等路侧感知数据。可以理解，上述的几个具体例子仅仅是作为路侧传感器的举例，不应该构成具体限定。路侧传感装置可以单独使用所述路侧传感器中的任意一种，也可以同时使用所述路侧传感器中的任意几种。路侧感知数据可以采用路侧结果集的形式进行描述，路侧结果集可以包括多个路侧结果单元，每个路侧结果单元对应一个目标物体。举个例子说明，假设路侧结果单元可以表示为roadside(p r，v r，s r，c r)，其中，p r表示为路侧传感装置侦测到 的目标物体的位置，v r表示为路侧传感装置侦测到的目标物体的速度，s r表示为路侧传感装置侦测到的目标物体的大小，c r表示为路侧传感装置侦测到的目标物体的颜色，则路侧结果集可以表示为result r(roadside 1，roadside 2，…，roadside M)，其中，M为路侧传感装置感知范围内目标物体的数量。更具体地，以矩阵的形式为例，路侧结果集可以表示为：
Figure PCTCN2019078646-appb-000003
车辆设备用于从车辆的角度对道路环境进行侦测以获得车辆感知数据。车辆设备可以配置有车辆传感装置，所述车辆传感装置可以包括至少一个车辆传感器，例如组合惯导、微波雷达、毫米波雷达以及摄像头等等。不同的车辆传感器可以侦测不同的车辆感知数据，例如，车辆传感装置能够通过组合惯导识别出目标物体的位置、速度等等路侧感知数据。车辆传感装置能够通过微波雷达和毫米波雷达识别出感知范围内的目标物体的位置、速度和大小等等路侧感知数据。车辆传感装置能够通过摄像头识别出感知范围内的目标物体的位置、速度、大小和颜色等等路侧感知数据。可以理解，上述的几个具体例子仅仅是作为对车辆传感器的举例，不应该构成具体限定。车辆传感装置可以单独使用所述车辆传感器中的任意一种，也可以同时使用所述车辆传感器中的任意几种。车辆感知数据可以采用车辆结果集的形式进行描述，车辆结果集可以包括多个车辆结果单元，每个车辆结果单元对应一个目标物体，车辆结果单元从多维角度对目标物体的特征进行描述，例如，位置、速度、大小和颜色等等。举个例子说明，假设车辆结果单元可以表示为vehicle(p v，v v，s v，c v)，其中，p v表示为车辆传感装置侦测到的目标物体的位置，v v表示为车辆传感装置侦测到的目标物体的速度，s v表示为车辆传感装置侦测到的目标物体的大小，c v表示为车辆传感装置侦测到的目标物体的颜色，则车辆结果集可以表示为result v(vehicle 1，vehicle 2，…，vehicle N)，其中，N为车辆传感装置感知范围内目标物体的数量。更具体地，以矩阵的形式为例，车辆结果集可以表示为：
Figure PCTCN2019078646-appb-000004
在本申请实施例中，路侧设备和车辆设备可以通过无线的方式进行连接，从而实现数据的通信。
在本申请实施例中，路侧设备和/或车辆设备可以通过融合公式将路侧传感装置在感知范围内侦测到的路侧感知数据和车辆传感装置在感知范围内侦测到的车辆感知数据进行数据融合以获得第一融合结果。以路侧感知数据以路侧结果集表示和车辆感知数据以车辆结果集表示为例，融合公式可以表示为y＝f(result r，result v)，其中，result r为路侧结果集，result v为车辆结果集，y为第一融合结果。f用于根据路侧结果集和车辆结果集映射出第一融合结果。
在一具体的实施方式中，函数f可以表示为：
Figure PCTCN2019078646-appb-000005
其中，w r为路侧传感装置的置信因子，w r可以是多维数据，即w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为路侧传感装置感知范围内目标物体的数量，w ri为路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为小于M的自然数。w v为车辆传感装置的置信因子，w v可以是多维数据，即w v＝(w v1，w v2，…，w vN)，N为车辆传感装置感知范围内目标物体的数量，w vj为车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为小于N的自然数。可以理解，路侧传感装置的置信因子可以划分得更细，以使得路侧结果单元中不同的元素分别对应不同的置信因子。同理，车辆传感装置的置信因子也可以划分得更细，以使得车辆结果单元中不同的元素分别对应不同的置信因子。具体地，
当
Figure PCTCN2019078646-appb-000006
时，
Figure PCTCN2019078646-appb-000007
当
Figure PCTCN2019078646-appb-000008
时，
Figure PCTCN2019078646-appb-000009
不难理解，若路侧传感装置的置信因子与车辆传感装置的置信因子的比值越大，则路侧感知数据在融合结果中所占的比重与车辆感知数据在融合结果中所占的比重的比值越大。简单来说，哪个传感装置的置信因子的值比较大，则哪个传感装置侦测得到的感知数据在融合结果中所占的比重比较大。
需要说明的是，置信因子可以根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定。其中，传感装置参数与传感装置本身的初始精度、安装空间角度以及安装坐标有关。目标物体的感知距离为目标物体与传感装置在感知坐标系中的距离。目标物体的感知角度为目标物体与传感装置在感知坐标系中构成的角度。需要说明的是，当传感装置包括多个传感器时，置信因子可以通过加权或求平均等方式综合考虑多个传感器的置信度而得到。在一具体的实施例中，置信因子可以根据如下公式获得：
w＝g(S k，R i，θ j)，w∈[0,1]
其中，S k为传感装置参数，R i为感知距离，θ j为感知角度，g为通过传感装置标定得到的标定参数表。其中，标定参数表可以通过在传感器装置标定过程中使用大量已知目标物体的数据进行反推得到。不难理解，若传感装置参数的精度越高，则置信因子的值越大，若传感装置参数的精度越低，则置信因子的值越小；若感知距离越小，则置信因子的值越 大，若感知距离越大，则置信因子的值越小；若感知角度越小，则置信因子的值越大，若感知角度越大，则置信因子的值越小。
在一具体的实施例中，传感装置参数可以根据如下公式得到：S k＝h(S 0，A，P)，其中，S k为传感装置参数，S 0为传感装置参数的初始精度，A为传感装置的安装空间角度，即，传感装置安装完毕之后相对于道路的地面的空间角度，P为传感装置的安装坐标，即，传感装置安装完毕之后相对于道路的地面的三维坐标。
可选地，传感装置的安装空间角度可以定义为：A＝(yaw，pitch，roll)，其中，yaw为传感装置相对于道路的地面的偏航角，pitch为传感装置相对于道路的地面的俯仰角，roll为为传感装置相对于道路的地面的翻滚角。如图2所示，以相对于道路的地面建立右手笛卡尔坐标为例，偏航角yaw可以定义为传感装置绕着y轴旋转得到的角度，俯仰角pitch可以定义为传感装置绕着x轴旋转得到的角度，翻滚角roll可以定义为传感装置绕着z轴旋转得到的角度。
可选地，传感器装置的安装坐标可以定义为：P＝(x，y，h)，如图3所示，x和y表示传感装置投影到道路的地面的坐标，h表示传感装置至道路的地面的垂直距离。
在一具体的实施例中，目标物体的感知距离和目标物体的感知角度可以通过如下的方式得到：如图4所示，以传感装置为中心，将传感装置的感知范围划分为不同距离，不同角度的扇形区域，从而构建感知坐标系。传感装置根据目标物体落入感知坐标系中的扇形区域确定目标物体的感知距离R i和目标物体的感知角度θ j。
在将路侧感知数据和车辆感知数据进行数据融合以获得融合结果之前，路侧设备和/或车辆设备需要将路侧感知数据和车辆感知数据进行匹配以获得匹配结果，以使得路侧设备和/或车辆设备可以根据匹配结果将将路侧感知数据和车辆感知数据进行数据融合。
下面通过举例子说明将路侧感知数据和车辆感知数据进行匹配的含义：假设以路侧结果集形式表示的路侧感知数据为：result r(roadside 1，roadside 2，…，roadside M)，以车辆结果集形式表示的车辆感知数据为：result v(vehicle 1，vehicle 2，…，vehicle N)，M为路侧传感装置感知范围内目标物体的数量，N为车辆传感装置感知范围内目标物体的数量，M>N。其中，roadside 1为路侧传感装置对目标物体1进行侦测得到的路侧结果单元，roadside 2为路侧传感装置对目标物体1进行侦测得到的路侧结果单元，…，roadside M为路侧传感装置对目标物体M进行侦测得到的路侧结果单元。vehicle 1为车辆传感装置对目标物体1进行侦测得到的车辆结果单元，vehicle 2为车辆传感装置对目标物体1进行侦测得到的车辆结果单元，…，vehicle N为车辆传感装置对目标物体N进行侦测得到的车辆结果单元。因此，roadside 1和vehicle 1都是对目标物体1进行侦测所得到的结果单元，两者之间存在匹配关系；roadside 2和vehicle 2都是对目标物体2进行侦测所得到的结果单元，两者之间存在匹配关系；…；roadside N和vehicle N都是对目标物体N进行侦测所得到的结果单元，两者之间存在匹配关系。所以，将路侧感知数据和车辆感知数据进行匹配就是将路侧感知数据中路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系都找出来。
在本申请实施例中，路侧设备和/或车辆设备可以通过偏差网络将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来。具体地，将路侧结 果单元和车辆结果单元作为偏差网络的输入，则偏差网络将输出路侧结果单元和车辆结果单元之间的匹配结果。其中，如果偏差网络输出的匹配结果为两者匹配，则可以认为路侧结果单元和车辆结果单元之间存在匹配关系，如果偏差网络输出的匹配结果为两者不匹配，则可以认为路侧结果单元和车辆结果单元之间不存在匹配关系。继续以上述的例子为例，如果将roadside 1和vehicle 1作为偏差网络的输入，偏差网络输出的偏差结果为两者匹配，则可以确定roadside 1和vehicle 1之间存在匹配关系；如果将roadside 1和vehicle 2作为偏差网络的输入，偏差网络输出的偏差结果为两者不匹配，则可以确定roadside 1和vehicle 2之间不存在匹配关系。
在本申请实施例中，可以通过以下公式：S＝Deviation(roadside i，vehicle j)将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
以roadside i＝(p ri，v ri，s ri，c ri)，vehicle j＝(p vj，v vj，s vj，c vj)为例，在一具体的实施方式中，偏差网络Deviation可以用如图5所示的的逆向传播(back propagation，BP)神经网络进行表示：
其中，Δp ij为位置偏差值，Δp ij＝fabs(p ri-p vj)，p ri为路侧传感装置侦测到的目标物体i的位置，p vj为车辆传感器侦测到的目标物体j的位置，fabs为求绝对值函数；
Δv ij为速度偏差值，Δv ij＝fabs(v ri-v vj)，v ri为路侧传感装置侦测到的目标物体i的速度，v vj为车辆传感装置侦测到的目标物体j的速度；
Δs ij为大小偏差值，Δs ij＝fabs(s ri-s vj)，s ri为路侧传感装置侦测到的目标物体i的大小，s vj为车辆传感装置侦测到的目标物体j的大小；
Δc ij为颜色偏差值，Δc ij＝fabs(c ri-c vj)，c ri为路侧传感装置侦测到的目标物体i的颜色，c vj为车辆传感装置侦测到的目标物体j的颜色；
P p为位置偏差因子，
Figure PCTCN2019078646-appb-000010
为路侧传感装置侦测到的目标物体i的位置对应的置信因子，
Figure PCTCN2019078646-appb-000011
为车辆传感装置侦测到的目标物体j的位置对应的置信因子；
P v为速度偏差因子，
Figure PCTCN2019078646-appb-000012
为路侧传感装置侦测到的目标物体i的速度对应的置信因子，
Figure PCTCN2019078646-appb-000013
为车辆传感装置侦测到的目标物体j的速度对应的置信因子；
P s为大小偏差因子，
Figure PCTCN2019078646-appb-000014
为路侧传感装置侦测到的目标物体i的大小对应的置信因子，
Figure PCTCN2019078646-appb-000015
为车辆传感装置侦测到的目标物体j的大小对应的置信因子；
P c为速度偏差因子，
Figure PCTCN2019078646-appb-000016
为路侧传感装置侦测到的目标物体i的颜色对应的置信因子，
Figure PCTCN2019078646-appb-000017
为车辆传感装置侦测到的目标物体j的颜色对应的置信因子；
Figure PCTCN2019078646-appb-000018
为激活函数，其中，激活函数可以是带泄露的修正线性单元(Leaky Rectified Linear Unit，LReLU)、参数化修正线性单元(Parameteric Rectified Linear Unit，PReLU)、随机带泄露的修正线性单元(Randomized Leaky Rectified Linear Unit，RReLU)、ReLUSoftplus函数、Softsign函数、Sigmoid函数、tanh函数。
需要说明的是，
Figure PCTCN2019078646-appb-000019
以及
Figure PCTCN2019078646-appb-000020
的求解方法可以参见上文中置信因子的相关段落，此处不再展开描述。
在上述例子中，偏差网络是以BP神经网络为例进行说明，在其他的实施方式中，偏差网络还可以是长短期记忆网络(Long Short-Term Memory，LSTM)，残差网络(Residential Networking，ResNet)，循环神经网络(Recurrent Neural Networks，RNN)等等，此处不作具体限定。
上述内容仅仅是利用了单帧的路侧感知数据和车辆感知数据实现匹配，匹配结果的置信度不高。为了解决上述问题，可以考虑利用两帧甚至多帧数路侧感知数据和车辆感知数据实现匹配，以提高匹配结果的置信度。进一步地，路侧设备和/或车辆设备还可以通过帧间回环和/或多帧关联的方式对匹配结果的置信度进行评价以获得评价结果，并根据评价结果对偏差网络进行调整。
在本申请实施例中，帧间回环主要是根据相邻帧的路侧感知数据和车辆感知数据之间交叉匹配得到的匹配结果进行计算得到的。以一个结果单元为例，帧间回环是主要是根据相邻帧的帧内匹配结果以及帧间匹配结果得到。其中，帧内匹配结果为将不同的传感装置在同一帧内对同一目标物体进行侦测得到的结果单元进行匹配得到的匹配结果。帧间匹配结果为将相同的传感装置在相邻帧内对同一目标物体进行侦测得到的结果单元进行匹配得到的匹配结果。
以路侧传感装置和车辆传感装置为例，如图6所示，帧间回环可以定义为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果。其中，第一匹配结果为第i帧的帧内匹配结果，即，路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第二匹配结果为车辆传感装置的帧间匹配结果，即，车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第三匹配结果为第i+1帧的帧内匹配结果，即，车辆传感装置在第i+1帧侦测到的目标物体j的对应的车 辆结果单元与路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。第四匹配结果为路侧传感装置的帧间匹配结果，即，路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
在本申请实施例中，多帧关联主要根据多个连续的帧的帧间回环得到。在一具体的实施方式中，如图7所示，多帧关联可以定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多 帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…，等等。
不难理解，路侧设备和/或车辆设备将数据融合的好处至少包括以下两种：
第一种，可以实现将路侧传感装置的感知范围和车辆传感装置的感知范围进行叠加，从而有效扩展路侧传感装置和/或车辆传感装置的感知范围。举个例子说明，假设路侧传感装置在感知范围内侦测到的目标物体的数量为3个(目标物体1、目标物体2、目标物体3)，车辆传感装置的感知范围内侦测到的目标物体的数量为2个(目标物体3、目标物体4)，则在车辆传感装置融合了路侧感知数据和车辆感知数据之后，融合结果的感知范围包括4个目标啊物体(目标物体1、目标物体2、目标物体3、目标物体4)。
第二种，可以实现使用高置信度的数据对低置信度的数据进行纠正，从而有效地提高路侧传感装置和/或车辆传感装置的数据的置信度。举个例子说明，假设路侧传感装置测量得到的目标物体的速度的置信度低于车辆传感装置测量得到的目标物体的速度的置信度，则路侧传感装置可以使用车辆传感装置测量得到的目标物体的速度对自身测量得到的目标物体的速度进行纠正，从而获得高置信度的数据。
文章的上述内容重点介绍了如何实现将路侧传感装置在感知范围内侦测到的路侧感知数据和车辆传感装置在感知范围内侦测到的车辆感知数据进行数据融合的方案。下面将从数据融合方发以及相关设备的角度介绍路侧传感装置和/或车辆传感装置如何利用上述的数据融合方案实现扩展感知范围。
如图8所示，本申请实施例提供的第一种数据融合方法的流程示意图。如图8所示，本申请实施例的数据融合方法，具体包括如下步骤：
S101：车辆设备获取车辆感知数据，其中，所述车辆感知数据为车辆传感装置通过车辆传感器对感知范围内的道路环境进行侦测以获得车辆感知数据。
在本申请实施例中，车辆传感设备可以配置有车辆传感装置，所述车辆传感装置至少包括一个车辆传感器，例如组合惯导、微波雷达、毫米波雷达以及摄像头等等，能够识别出感知范围内的目标物体的车辆感知数据。其中，车辆感知数据可以包括目标物体的位置、速度、大小以及颜色等等。
可以理解，上述的几个具体例子仅仅是作为对车辆传感器的举例，不应该构成具体限定。车辆传感装置可以单独使用所述车辆传感器中的任意一种，也可以同时使用所述车辆传感器中的任意几种。
S102：路侧设备获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置通过路侧传感器对感知范围内的道路环境进行侦测得到的。
在本申请实施例中，路侧设备可以配置有路侧传感装置，所述路侧传感装置至少包括一个路侧传感器，例如微波雷达、毫米波雷达等等，能够识别出感知范围内的目标物体的路侧感知数据。其中，路侧感知数据可以包括目标物体的位置、速度、大小以及颜色等等。
可以理解，上述的几个具体例子仅仅是作为路侧传感器的举例，不应该构成具体限定。路侧传感装置可以单独使用所述路侧传感器中的任意一种，也可以同时使用所述路侧传感器中的任意几种。
S103：路侧设备向车辆设备发送路侧感知数据。相应地，车辆设备接收路侧设备发送的路侧感知数据。
S104：车辆设备将路侧感知数据和车辆感知数据进行匹配以获得匹配结果。
在本申请实施例中，车辆设备可以通过偏差网络将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来。具体地，将路侧结果单元和车辆结果单元作为偏差网络的输入，则偏差网络将输出路侧结果单元和车辆结果单元之间的匹配结果。
在本申请实施例中，可以通过以下公式：S＝Deviation(roadside i，vehicle j)将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
S105：车辆设备通过帧间回环和/或多帧关联的方式对匹配结果的置信度进行评价，并根据评价结果调整偏差网络。
在本申请实施例中，帧间回环可以定义为：T回环＝T1+T2+T3+T4，其中，T回环为帧间回环，T1为第一匹配结果，T2为第二匹配结果，T3为第三匹配结果，T4为第四匹配结果。其中，第一匹配结果为第i帧的帧内匹配结果，即，路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第二匹配结果为车辆传感装置的帧间匹配结果，即，车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第三匹配结果为第i+1帧的帧内匹配结果，即，车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。第四匹配结果为路侧传感装置的帧间匹配结果，即，路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
在本申请实施例中，多帧关联主要根据多个连续的帧的帧间回环得到。在一具体的实施方式中，多帧关联可以定义为：T多帧＝T回环12+T回环23+T回环34+…，其中，T多帧为多帧关联，T回环12为第一帧和第二帧之间的帧间回环，T回环23为第二帧和第三帧之间的帧间回环，T回环34为第三帧和第四帧之间的帧间回环，…，等等。
S106：车辆设备通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获 得第一融合结果。
在本申请实施例中，车辆设备可以通过融合公式将路侧传感装置在感知范围内侦测到的路侧感知数据和车辆传感装置在感知范围内侦测到的车辆感知数据进行数据融合以获得第一融合结果。
在本申请实施例中，融合公式可以表示为y＝f(result r，result v)，其中，result r为路侧结果集，result v为车辆结果集，y为第一融合结果。f用于根据路侧结果集和车辆结果集映射出第一融合结果。
在一具体的实施方式中，函数f可以表示为：
Figure PCTCN2019078646-appb-000021
其中，w r为路侧传感装置的置信因子，w r可以是多维数据，即w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为路侧传感装置感知范围内目标物体的数量，w ri为路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为小于M的自然数。w v为车辆传感装置的置信因子，w v可以是多维数据，即w v＝(w v1，w v2，…，w vN)，N为车辆传感装置感知范围内目标物体的数量，w vj为车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为小于N的自然数。
为了简便起见，本申请实施例不再对路侧感知数据和车辆感知数据的融合进行详细的介绍，详情请参见文章开头关于路侧感知数据和车辆感知数据的融合的介绍。
如图9所示，本申请实施例提供的第二种数据融合方法的流程示意图。如图9所示，本申请实施例的数据融合方法具体包括如下步骤：
S201：路侧设备获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置通过路侧传感器对感知范围内的道路环境进行侦测得到的。
在本申请实施例中，路侧设备可以配置有路侧传感装置，所述路侧传感装置包括至少一个路侧传感器，例如微波雷达、毫米波雷达等等，能够识别出感知范围内的目标物体的路侧感知数据。其中，路侧感知数据可以包括目标物体的位置、速度、大小以及颜色等等。
可以理解，上述的几个具体例子仅仅是作为路侧传感器的举例，不应该构成具体限定。路侧传感装置可以单独使用所述路侧传感器中的任意一种，也可以同时使用所述路侧传感器中的任意几种。
S202：车辆设备获取车辆感知数据，其中，所述车辆感知数据为车辆传感装置通过车辆传感器对感知范围内的道路环境进行侦测以获得车辆感知数据。
在本申请实施例中，车辆设备可以配置有车辆传感装置，所述车辆传感装置至少包括一个车辆传感器，例如组合惯导、微波雷达、毫米波雷达以及摄像头等等，能够识别出感知范围内的目标物体的车辆感知数据。其中，车辆感知数据可以包括目标物体的位置、速度、大小以及颜色等等。
可以理解，上述的几个具体例子仅仅是作为对车辆传感器的举例，不应该构成具体限定。车辆传感装置可以单独使用所述车辆传感器中的任意一种，也可以同时使用所述车辆 传感器中的任意几种。
S203：车辆设备向路侧设备发送车辆感知数据。相应地，路侧设备接收车辆设备发送的车辆感知数据。
S204：路侧设备将路侧感知数据和车辆感知数据进行匹配以获得匹配结果。
在本申请实施例中，路侧传感装置可以通过偏差网络将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来。具体地，将路侧结果单元和车辆结果单元作为偏差网络的输入，则偏差网络将输出路侧结果单元和车辆结果单元之间的匹配结果。
在本申请实施例中，可以通过以下公式：S＝Deviation(roadside i，vehicle j)将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
S205：路侧设备通过帧间回环和/或多帧关联的方式对匹配结果的置信度进行评价以获得评价结果，并根据评价结果调整偏差网络。
在本申请实施例中，帧间回环可以定义为：T回环＝T1+T2+T3+T4，其中，T回环为帧间回环，T1为第一匹配结果，T2为第二匹配结果，T3为第三匹配结果，T4为第四匹配结果。其中，第一匹配结果为第i帧的帧内匹配结果，即，路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第二匹配结果为车辆传感装置的帧间匹配结果，即，车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第三匹配结果为第i+1帧的帧内匹配结果，即，车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。第四匹配结果为路侧传感装置的帧间匹配结果，即，路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
在本申请实施例中，多帧关联主要根据多个连续的帧的帧间回环得到。在一具体的实施方式中，多帧关联可以定义为：T多帧＝T回环12+T回环23+T回环34+…，其中，T多帧为多帧关联，T回环12为第一帧和第二帧之间的帧间回环，T回环23为第二帧和第三帧之间的帧间回环，T回环34为第三帧和第四帧之间的帧间回环，…，等等。
S206：路侧设备通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
在本申请实施例中，路侧传感装置可以通过融合公式将路侧传感装置在感知范围内侦测到的路侧感知数据和车辆传感装置在感知范围内侦测到的车辆感知数据进行数据融合以获得第一融合结果。
在本申请实施例中，融合公式可以表示为y＝f(result r，result v)，其中，result r为路侧结果集，result v为车辆结果集，y为第一融合结果。f用于根据路侧结果集和车辆结果集映射出第一融合结果。
在一具体的实施方式中，函数f可以表示为：
Figure PCTCN2019078646-appb-000022
其中，w r为路侧传感装置的置信因子，w r可以是多维数据，即w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为路侧传感装置感知范围内目标物体的数量，w ri为路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为小于M的自然数。w v为车辆传感装置的置信因子，w v可以是多维数据，即w v＝(w v1，w v2，…，w vN)，N为车辆传感装置感知范围内目标物体的数量，w vj为车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为小于N的自然数。
为了简便起见，本申请实施例不再对路侧感知数据和车辆感知数据的融合进行详细的介绍，详情请参见文章开头关于路侧感知数据和车辆感知数据的融合的介绍。
如图10所示，本申请实施例提供的第三种数据融合方法的流程示意图。如图10所示，本申请实施例的数据融合方法具体包括如下步骤：
S301：路侧设备获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置通过路侧传感器对感知范围内的道路环境进行侦测得到的。
在本申请实施例中，路侧设备可以配置有路侧传感装置，所述路侧传感装置包括至少一个路侧传感器，例如微波雷达、毫米波雷达等等，能够识别出感知范围内的目标物体的路侧感知数据。其中，路侧感知数据可以包括目标物体的位置、速度、大小以及颜色等等。
可以理解，上述的几个具体例子仅仅是作为路侧传感器的举例，不应该构成具体限定。路侧传感装置可以单独使用所述路侧传感器中的任意一种，也可以同时使用所述路侧传感器中的任意几种。
S302：至少一台车辆设备获取车辆感知数据，其中，所述车辆感知数据为车辆传感装置通过车辆传感器对感知范围内的道路环境进行侦测以获得车辆感知数据。
在本申请实施例中，车辆设备可以配置有车辆传感装置，所述车辆传感装置包括至少一个车辆传感器，例如组合惯导、微波雷达、毫米波雷达以及摄像头等等，能够识别出感知范围内的目标物体的车辆感知数据。其中，车辆感知数据可以包括目标物体的位置、速度、大小以及颜色等等。
可以理解，上述的几个具体例子仅仅是作为对车辆传感器的举例，不应该构成具体限定。车辆传感装置可以单独使用所述车辆传感器中的任意一种，也可以同时使用所述车辆传感器中的任意几种。
S303：至少一台车辆设备向路侧设备发送车辆感知数据。相应地，路侧设备接收至少一台车辆设备发送的车辆感知数据。
S304：路侧设备将路侧感知数据和至少一台车辆设备发送的车辆感知数据进行匹配以获得匹配结果。
在本申请实施例中，路侧设备可以通过偏差网络将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来。具体地，将路侧结果单元和车辆 结果单元作为偏差网络的输入，则偏差网络将输出路侧结果单元和车辆结果单元之间的匹配结果。
在本申请实施例中，可以通过以下公式：S＝Deviation(roadside i，vehicle j)将路侧感知数据中的路侧结果单元和车辆感知数据中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
S305：路侧设备通过帧间回环和/或多帧关联的方式对匹配结果的置信度进行评价以获得评价结果，并根据评价结果调整偏差网络。
在本申请实施例中，帧间回环可以定义为：T回环＝T1+T2+T3+T4，其中，T回环为帧间回环，T1为第一匹配结果，T2为第二匹配结果，T3为第三匹配结果，T4为第四匹配结果。其中，第一匹配结果为第i帧的帧内匹配结果，即，路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第二匹配结果为车辆传感装置的帧间匹配结果，即，车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果。第三匹配结果为第i+1帧的帧内匹配结果，即，车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。第四匹配结果为路侧传感装置的帧间匹配结果，即，路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
在本申请实施例中，多帧关联主要根据多个连续的帧的帧间回环得到。在一具体的实施方式中，多帧关联可以定义为：T多帧＝T回环12+T回环23+T回环34+…，其中，T多帧为多帧关联，T回环12为第一帧和第二帧之间的帧间回环，T回环23为第二帧和第三帧之间的帧间回环，T回环34为第三帧和第四帧之间的帧间回环，…，等等。
S306：路侧设备通过融合公式对所述至少一台车辆设备发送的车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
在本申请实施例中，路侧设备可以通过融合公式将路侧传感装置在感知范围内侦测到的路侧感知数据和车辆传感装置在感知范围内侦测到的车辆感知数据进行数据融合以获得第一融合结果。
在本申请实施例中，融合公式可以表示为y＝f(result r，result v)，其中，result r为路侧结果集，result v为车辆结果集，y为第一融合结果。f用于根据路侧结果集和车辆结果集映射出第一融合结果。
在一具体的实施方式中，函数f可以表示为：
Figure PCTCN2019078646-appb-000023
其中，w r为路侧传感装置的置信因子，w r可以是多维数据，即w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为路侧传感装置感知范围内目标物体的数量，w ri为路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为路侧传 感装置感知范围内目标物体i对应的路侧结果单元，i为小于M的自然数。w v为车辆传感装置的置信因子，w v可以是多维数据，即w v＝(w v1，w v2，…，w vN)，N为车辆传感装置感知范围内目标物体的数量，w vj为车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为小于N的自然数。
在本申请实施例中，第一融合结果可以是路侧传感装置通过融合公式对一个或者多个车辆传感装置发送的车辆感知数据和本身的路侧感知数据进行融合得到的，此处不作具体限定。
为了简便起见，本申请实施例不再对路侧感知数据和车辆感知数据的融合进行详细的介绍，详情请参见文章开头关于路侧感知数据和车辆感知数据的融合的介绍。
S307：路侧设备将第一融合结果向目标车辆设备发送。相应地，目标车辆设备接收路侧设备发送的第一融合结果。其中，所述目标车辆设备属于所述至少一台车辆设备。
S308：目标车辆设备将第一融合结果和目标车辆设备的车辆感知数据进行融合以获得第二融合结果。
在本申请实施例中，目标车辆设备可以通过融合公式将第一融合结果和目标车辆设备的车辆感知数据进行数据融合以获得第二融合结果。其中，第一融合结果和车辆感知数据进行融合的过程与路侧感知数据和车辆感知数据进行融合的过程类似，此处不再展开描述。
在上述方案中，路侧设备可以将多个车辆传感装置发送的车辆感知数据和本身的路侧感知数据进行融合以得到感知范围更大的第一融合结果(这里的感知范围为路侧传感装置的感知范围和多个车辆传感装置的感知范围的叠加)，然后，再将第一融合结果发送给目标车辆设备，以供目标车辆设备将第一融合结果和车辆感知数据进行融合以扩大车辆传感装置的感知范围。
基于同一发明构思，本申请实施例提供了一种融合装置，该设备可以应用在车辆设备中，也可以应用于路侧设备中。该融合装置可以芯片、可编程组件、电路组件、设备(即，融合装置即为车辆设备或者路侧设备)或者系统等等，此处不作具体限定。
如图11所示，以融合装置为车辆设备为例，融合装置100包括：传感器系统104、控制系统106、外围设备108、电源110以及计算装置112。计算装置112可包括处理器113和存储器114。计算装置112可以是融合装置100的控制器或控制器的一部分。存储器114可包括处理器113可运行的指令115，并且还可存储地图数据116。融合装置100的组件可被配置为以与彼此互连和/或与耦合到各系统的其它组件互连的方式工作。例如，电源110可向融合装置100的所有组件提供电力。计算装置111可被配置为从传感器系统104、控制系统106和外围设备108接收数据并对它们进行控制。
在其它示例中，融合装置100可包括更多、更少或不同的系统，并且每个系统可包括更多、更少或不同的组件。此外，示出的系统和组件可以按任意种的方式进行组合或划分。
传感器系统104可包括用于感测关于融合装置100感知范围内的道路环境的若干个传感器。如图所示，传感器系统的传感器包括GPS126、IMU(Inertial Measurement Unit，惯性测量单元)128、无线电检测和雷达测距(RADAR)单元130、激光测距(LIDAR)单元132、相机134以及用于为修改传感器的位置和/或朝向的致动器136。
GPS模块126可以为用于估计车辆的地理位置的任何传感器。为此，GPS模块126可能包括收发器，基于卫星定位数据，估计融合装置100相对于地球的位置。在示例中，计算装置111可用于结合地图数据116使用GPS模块126来估计融合装置100可在其上行驶的道路上的车道边界的位置。GPS模块126也可采取其它形式。
IMU 128可以是用于基于惯性加速度及其任意组合来感测车辆的位置和朝向变化。在一些示例中，传感器的组合可包括例如加速度计和陀螺仪。传感器的其它组合也是可能的。
RADAR单元130可以被看作物体检测系统，其用于使用无线电波来检测目标物体的特性，诸如物体的距离、高度、方向或速度。RADAR单元130可被配置为传送无线电波或微波脉冲，其可从波的路线中的任何物体反弹。物体可将波的一部分能量返回至接收器(例如，碟形天线或天线)，该接收器也可以是RADAR单元130的一部分。RADAR单元130还可被配置为对接收到的信号(从物体反弹)执行数字信号处理，并且可被配置为识别目标物体。
其它类似于RADAR的系统已用在电磁波谱的其它部分上。一个示例是LIDAR(光检测和测距)，其可使用来自激光的可见光，而非无线电波。
LIDAR单元132包括传感器，该传感器使用光感测或检测融合装置100感知范围内的道路环境中的目标物体。通常，LIDAR是可通过利用光照射目标来测量到目标物体的距离或目标物体的其它属性的光学遥感技术。作为示例，LIDAR单元132可包括被配置为发射激光脉冲的激光源和/或激光扫描仪，和用于为接收激光脉冲的反射的检测器。例如，LIDAR单元132可包括由转镜反射的激光测距仪，并且以一维或二维围绕数字化场景扫描激光，从而以指定角度间隔采集距离测量值。在示例中，LIDAR单元132可包括诸如光(例如，激光)源、扫描仪和光学系统、光检测器和接收器电子器件之类的组件，以及位置和导航系统。
在示例中，LIDAR单元132可被配置为使用紫外光(UV)、可见光或红外光对物体成像，并且可用于广泛的目标物体，包括非金属物体。在一个示例中，窄激光波束可用于以高分辨率对物体的物理特征进行地图绘制。
在示例中，从约10微米(红外)至约250纳米(UV)的范围中的波长可被使用。光通常经由后向散射被反射。不同类型的散射被用于不同的LIDAR应用，诸如瑞利散射、米氏散射和拉曼散射以及荧光。基于不同种类的后向散射，作为示例，LIDAR可因此被称为瑞利激光RADAR、米氏LIDAR、拉曼LIDAR以及钠/铁/钾荧光LIDAR。波长的适当组合可允许例如通过寻找反射信号的强度的依赖波长的变化对物体进行远程地图绘制。
使用扫描LIDAR系统和非扫描LIDAR系统两者可实现三维(3D)成像。“3D选通观测激光RADAR(3D gated viewing laser radar)”是非扫描激光测距系统的示例，其应用脉冲激光和快速选通相机。成像LIDAR也可使用通常使用CMOS(Complementary Metal Oxide Semiconductor，互补金属氧化物半导体)和CCD(Charge Coupled Device，混合互补金属氧化物半导体/电荷耦合器件)制造技术在单个芯片上构建的高速检测器阵列和调制敏感检测器阵列来执行。在这些装置中，每个像素可通过以高速解调或选通来被局部地处理，以使得阵列可被处理成表示来自相机的图像。使用此技术，可同时获取上千个像素以创建表示LIDAR单元132检测到的物体或场景的3D点云。
点云可包括3D坐标系统中的一组顶点。这些顶点例如可由X、Y、Z坐标定义，并且可表示目标物体的外表面。LIDAR单元132可被配置为通过测量目标物体的表面上的大量点来创建点云，并可将点云作为数据文件输出。作为通过LIDAR单元132的对物体的3D扫描过程的结果，点云可用于识别并可视化目标物体。
在一个示例中，点云可被直接渲染以可视化目标物体。在另一示例中，点云可通过可被称为曲面重建的过程被转换为多边形或三角形网格模型。用于将点云转换为3D曲面的示例技术可包括德洛内三角剖分、阿尔法形状和旋转球。这些技术包括在点云的现有顶点上构建三角形的网络。其它示例技术可包括将点云转换为体积距离场，以及通过移动立方体算法重建这样定义的隐式曲面。
摄像头134可以为用于获取车辆所位于的道路环境的图像的任何摄像头(例如，静态相机、视频相机等)。为此，摄像头可被配置为检测可见光，或可被配置为检测来自光谱的其它部分(诸如红外光或紫外光)的光。其它类型的相机也是可能的。摄像头134可以是二维检测器，或可具有三维空间范围。在一些示例中，摄像头134例如可以是距离检测器，其被配置为生成指示从摄像头134到环境中的若干点的距离的二维图像。为此，摄像头134可使用一种或多种距离检测技术。例如，摄像头134可被配置为使用结构光技术，其中融合装置100利用预定光图案，诸如栅格或棋盘格图案，对环境中的物体进行照射，并且使用摄像头134检测从物体的预定光图案的反射。基于反射的光图案中的畸变，融合装置100可被配置为检测到物体上的点的距离。预定光图案可包括红外光或其它波长的光。
致动器136例如可被配置为修改传感器的位置和/或朝向。传感器系统104可额外地或可替换地包括除了所示出的那些以外的组件。
控制系统106可被配置为控制融合装置100及其组件的操作。为此，控制系统106可包括传感器融合算法144，计算机视觉系统146，导航或路线控制(pathing)系统148以及避障系统150。
传感器融合算法144可以包括例如计算装置111可运行的算法(或者存储算法的计算机程序产品)。传感器融合算法144可被配置为接受来自传感器104的数据作为输入。所述数据可包括例如表示在传感器系统104的传感器处感测到的信息的数据。传感器融合算法144可包括例如卡尔曼滤波器、贝叶斯网络或者另外的算法。传感器融合算法144还可被配置为基于来自传感器系统104的数据来提供各种评价，包括例如对车辆所位于的环境中的个体物体和/或特征的评估、对具体情形的评估和/或基于特定情形的可能影响的评估。其它评价也是可能的。
计算机视觉系统146可以是被配置为处理和分析由相机134捕捉的图像以便识别融合装置100所位于的环境中的物体和/或特征的任何系统，所述物体和/或特征包括例如车道信息、交通信号和障碍物。为此，计算机视觉系统146可使用物体识别算法、从运动中恢复结构(Structure from Motion，SFM)算法、视频跟踪或其它计算机视觉技术。在一些示例中，计算机视觉系统146可以额外地被配置为地图绘制环境、跟随物体、估计物体的速度，等等。
导航和路线控制系统148可以是被配置为确定车辆的驾驶路线的任何系统。导航和路线控制系统148可以额外地被配置为在车辆处于操作中的同时动态地更新驾驶路线。在一 些示例中，导航和路线控制系统148可被配置为结合来自传感器融合算法144、GPS模块126和一个或多个预定地图的数据以便为车辆确定驾驶路线。
避障系统150可以是被配置为识别、评估和避免或者以其它方式越过车辆所位于的环境中的障碍物的任何系统。
控制系统106可以额外地或可替换地包括除了所示出的那些以外的组件。
外围设备108可被配置为允许融合装置100与外部传感器、其它车辆和/或用户交互。为此，外围设备108可包括例如无线通信系统152、触摸屏154、麦克风156和/或扬声器158。
无线通信系统152可以是被配置为直接地或经由通信网络无线耦合至一个或多个其它车辆、传感器或其它实体的任何系统。为此，无线通信系统152可包括用于直接或通过空中接口与其它车辆、传感器或其它实体通信的天线和芯片集。芯片集或整个无线通信系统152可被布置为根据一个或多个其它类型的无线通信(例如，协议)来通信，所述无线通信诸如蓝牙、IEEE 802.11(包括任何IEEE 802.11修订版)中描述的通信协议、蜂窝技术(诸如GSM、CDMA、UMTS(Universal Mobile Telecommunications System，通用移动通信系统)、EV-DO、WiMAX或LTE(Long Term Evolution，长期演进))、紫蜂、DSRC(Dedicated Short Range Communications，专用短程通信)以及RFID(Radio Frequency Identification，射频识别)通信，等等。无线通信系统152也可采取其它形式。
触摸屏154可被用户用来向融合装置100输入命令。为此，触摸屏154可被配置为经由电容感测、电阻感测或者表面声波过程等等来感测用户的手指的位置和移动中的至少一者。触摸屏154可能够感测在与触摸屏表面平行或与触摸屏表面在同一平面内的方向上、在与触摸屏表面垂直的方向上或者在这两个方向上的手指移动，并且还可能够感测施加到触摸屏表面的压力的水平。触摸屏154可由一个或多个半透明或透明绝缘层和一个或多个半透明或透明导电层形成。触摸屏154也可采取其它形式。
麦克风156可被配置为从融合装置100的用户接收音频(例如，声音命令或其它音频输入)。类似地，扬声器158可被配置为向融合装置100的用户输出音频。
外围设备108可以额外地或可替换地包括除了所示出的那些以外的组件。
电源110可被配置为向融合装置100的一些或全部组件提供电力。为此，电源110可包括例如可再充电锂离子或铅酸电池。在一些示例中，一个或多个电池组可被配置为提供电力。其它电源材料和配置也是可能的。在一些示例中，电源110和能量源120可一起实现，如一些全电动车中那样。
包括在计算装置111中的处理器113可包括一个或多个通用处理器和/或一个或多个专用处理器(例如，图像处理器、数字信号处理器等)。就处理器113包括多于一个处理器而言，这种处理器可单独工作或组合工作。计算装置111可实现基于通过用户接口112接收的输入控制车辆100的功能。
存储器114进而可包括一个或多个易失性存储组件和/或一个或多个非易失性存储组件，诸如光、磁和/或有机存储装置，并且存储器114可全部或部分与处理器113集成。存储器114可包含可由处理器113运行的指令115(例如，程序逻辑)，以运行各种车辆功能，包括本文中描述的功能或方法中的任何一个。
融合装置100的组件可被配置为以与在其各自的系统内部和/或外部的其它组件互连的方式工作。为此，融合装置100的组件和系统可通过系统总线、网络和/或其它连接机制通信地链接在一起。
在实施例一中，融合装置100在处理器113内执行如下指令：
获取车辆感知数据，其中，所述车辆感知数据为车辆传感装置对感知范围内的道路环境进行侦测得到的；
获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
在实施例二中，融合装置100在处理器113内执行如下指令：
向路侧设备发送车辆感知数据，其中，所述车辆感知数据是车辆传感装置通过车辆传感器对感知范围内的道路环境进行侦测得到的；
接收路侧设备发送的第一融合结果，其中，所述第一融合结果为所述路侧设备通过融合公式将至少一台车辆设备的车辆感知数据以及路侧感知数据进行融合得到的，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
将所述车辆感知数据和所述第一融合结果进行融合以获得第二融合结果。
结合实施例一或者实施例二，所述融合公式表示为：
y＝f(result r，result v)，
其中，result r为路侧结果集，所述路侧结果集用于表示所述路侧感知数据，result v为车辆结果集，所述车辆结果集用于表示所述车辆感知数据，y为所述第一融合结果，函数f用于根据所述路侧结果集和所述车辆结果集映射出所述第一融合结果。
在一具体的实施方式中，
Figure PCTCN2019078646-appb-000024
其中，w r为所述路侧传感装置的置信因子，w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为所述路侧传感装置感知范围内目标物体的数量，w ri为所述路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为自然数，0<i≤M；w v为所述车辆传感装置的置信因子，w v＝(w v1，w v2，…，w vN)，result v(vehicle 1，vehicle 2，…，vehicle N)，N为所述车辆传感装置感知范围内目标物体的数量，w vj为所述车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为自然数，0<j≤N。
更具体地，所述置信因子是根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定的。
例如，置信因子w可以根据如下公式获得：
w＝g(S k，R i，θ j)，w∈[0,1]
其中，S k为所述传感装置参数，R i为所述目标物体的感知距离，θ j为所述目标物体的感知角度，g为通过传感装置标定得到的标定参数表。
在一具体的实施方式中，所述车辆结果集包括至少一个车辆结果单元，所述至少一个车辆结果单元与至少一个目标物体存在一一对应关系，所述至少一个车辆结果单元中每一个车辆结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个车辆结果单元中的其中一个车辆结果单元表示为vehicle j(p vj，v vj，s vj，c vj)，其中，p vj表示为所述车辆传感装置侦测到的目标物体j的位置，v vj表示为所述车辆传感装置侦测到的目标物体j的速度，s vj表示为所述车辆传感装置侦测到的目标物体j的大小，c vj表示为所述车辆传感装置侦测到的目标物体j的颜色，N为所述车辆传感装置感知范围内目标物体的数量，j为自然数，0<j≤N。
在一具体的实施方式中，所述路侧结果集包括至少一个路侧结果单元，所述至少一个路侧结果单元与至少一个目标物体存在一一对应关系，所述至少一个路侧结果单元中每一个路侧结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个路侧结果单元中的其中一个路侧结果单元表示为roadside i(p vi，v vi，s vi，c vi)，其中，p vi表示为所述路侧传感装置侦测到的目标物体i的位置，v vi表示为所述路侧传感装置侦测到的目标物体i的速度，s vi表示为所述路侧传感装置侦测到的目标物体i的大小，c vi表示为所述路侧传感装置侦测到的目标物体i的颜色，M为所述路侧传感装置感知范围内目标物体的数量，i为自然数，0<i≤M。
结合实施例一或者实施例二，在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果之前，将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果，并根据匹配结果将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
在一具体的实施方式中，通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来。
更具体地，所述通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来，具体包括：
通过以下公式：S＝Deviation(roadside i，vehicle j)将所述路侧结果集中的路侧结果单元和所述路侧结果集中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
在一具体的实施方式中，在所述将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果之后，通过帧间回环和/或多帧关联的方式对匹配结果的置信度进行评价以获得评价结果，并根据评价结果对偏差网络进行调整。
具体地，所述帧间回环为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果，第一匹配结果为所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第二匹配结果为所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与所述车辆传感装置在第i+1帧侦 测到的目标物体j的对应的车辆结果单元的匹配结果，第三匹配结果为所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与所述路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果，第四匹配结果为所述路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
具体地，所述多帧关联定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…。
需要说明的，图11实施例中未提及的内容可参考图8-图10对应的实施例，这里不再赘述。
如图12所示，当融合装置为路侧设备为例，融合装置200包括、RF(Radio Frequency，射频)电路210、存储器220、其他输入设备230、显示屏240、传感器系统250、I/O子系统270、处理器280、以及电源290等部件。本领域技术人员可以理解，图12中示出的路侧传感装置结构并不构成对路侧传感装置的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。本领领域技术人员可以理解显示屏240可用于显示用户界面(UI，User Interface)。
下面结合图12对融合装置200的各个构成部件进行具体的介绍：
RF电路210可用于收发数据。通常，RF电路包括但不限于天线、至少一个放大器、收发信机、耦合器、LNA(Low Noise Amplifier，低噪声放大器)、双工器等。此外，RF电路210还可以通过无线通信与网络和其他设备通信。所述无线通信可以使用任一通信标准或协议，包括但不限于GSM(Global System of Mobile communication，全球移动通讯系统)、GPRS(General Packet Radio Service，通用分组无线服务)、CDMA(Code Division Multiple Access，码分多址)、WCDMA(Wideband Code Division Multiple Access,宽带码分多址)、LTE(Long Term Evolution,长期演进)、电子邮件、SMS(Short Messaging Service，短消息服务)等。
存储器220可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。存储器220可包括处理器280可运行的指令222，并且还可存储地图数据224。
其他输入设备230可用于接收输入的数字或字符信息，以及产生与融合装置200的用户设置以及功能控制有关的键信号输入。具体地，其他输入设备230可包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆、光鼠(光鼠是不显示可视输出的触摸敏感表面，或者是由触摸屏形成的触摸敏感表面的延伸)等中的一种或多种。其他输入设备230与I/O子系统270的其他输入设备控制器271相连接，在其他设备输入控制器271的控制下与处理器280进行信号交互。
显示屏240可包括显示面板241，以及触控面板242。其中显示面板241可以采用LCD(Liquid Crystal Display，液晶显示器)、OLED(Organic Light-Emitting Diode，有机发光二极管)等形式来配置显示面板241。触控面板242，也称为触摸屏、触敏屏等，可收集用 户在其上或附近的接触或者非接触操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板242上或在触控面板242附近的操作，也可以包括体感操作；该操作包括单点控制操作、多点控制操作等操作类型。)，并根据预先设定的程式驱动相应的连接装置。
传感器系统250可包括用于感测关于融合装置200感知范围内的道路环境的若干个传感器。如图所示，传感器系统的传感器包括GPS251、无线电检测和雷达测距(RADAR)单元255、激光测距(LIDAR)单元257、摄像头258以及用于为修改传感器的位置和/或朝向的致动器259。传感器系统250与I/O子系统270的传感器控制器272相连接，在传感器控制器272的控制下与处理器280进行信号交互。
GPS模块251可以为用于估计车辆的地理位置的任何传感器。为此，GPS模块251可能包括收发器，基于卫星定位数据，估计融合装置200相对于地球的位置。在示例中，融合装置200可用于结合地图数据224使用GPS模块251来估计融合装置200在道路中的位置。GPS模块126也可采取其它形式。
RADAR单元255可以被看作物体检测系统，其用于使用无线电波来检测目标物体的特性，诸如物体的距离、高度、方向或速度。RADAR单元255可被配置为传送无线电波或微波脉冲，其可从波的路线中的任何物体反弹。物体可将波的一部分能量返回至接收器(例如，碟形天线或天线)，该接收器也可以是RADAR单元255的一部分。RADAR单元255还可被配置为对接收到的信号(从物体反弹)执行数字信号处理，并且可被配置为识别目标物体。
其它类似于RADAR的系统已用在电磁波谱的其它部分上。一个示例是LIDAR(光检测和测距)，其可使用来自激光的可见光，而非无线电波。
LIDAR单元257包括传感器，该传感器使用光感测或检测融合装置200感知范围内的道路环境中的目标物体。通常，LIDAR是可通过利用光照射目标来测量到目标物体的距离或目标物体的其它属性的光学遥感技术。作为示例，LIDAR单元257可包括被配置为发射激光脉冲的激光源和/或激光扫描仪，和用于为接收激光脉冲的反射的检测器。例如，LIDAR单元257可包括由转镜反射的激光测距仪，并且以一维或二维围绕数字化场景扫描激光，从而以指定角度间隔采集距离测量值。在示例中，LIDAR单元257可包括诸如光(例如，激光)源、扫描仪和光学系统、光检测器和接收器电子器件之类的组件，以及位置和导航系统。
在示例中，LIDAR单元257可被配置为使用紫外光(UV)、可见光或红外光对物体成像，并且可用于广泛的目标物体，包括非金属物体。在一个示例中，窄激光波束可用于以高分辨率对物体的物理特征进行地图绘制。
在示例中，从约10微米(红外)至约250纳米(UV)的范围中的波长可被使用。光通常经由后向散射被反射。不同类型的散射被用于不同的LIDAR应用，诸如瑞利散射、米氏散射和拉曼散射以及荧光。基于不同种类的后向散射，作为示例，LIDAR可因此被称为瑞利激光RADAR、米氏LIDAR、拉曼LIDAR以及钠/铁/钾荧光LIDAR。波长的适当组合可允许例如通过寻找反射信号的强度的依赖波长的变化对物体进行远程地图绘制。
使用扫描LIDAR系统和非扫描LIDAR系统两者可实现三维(3D)成像。“3D选通观测激光RADAR(3D gated viewing laser radar)”是非扫描激光测距系统的示例，其应用脉冲激光和快速选通相机。成像LIDAR也可使用通常使用CMOS(Complementary Metal Oxide  Semiconductor，互补金属氧化物半导体)和CCD(Charge Coupled Device，混合互补金属氧化物半导体/电荷耦合器件)制造技术在单个芯片上构建的高速检测器阵列和调制敏感检测器阵列来执行。在这些装置中，每个像素可通过以高速解调或选通来被局部地处理，以使得阵列可被处理成表示来自相机的图像。使用此技术，可同时获取上千个像素以创建表示LIDAR单元257检测到的物体或场景的3D点云。
点云可包括3D坐标系统中的一组顶点。这些顶点例如可由X、Y、Z坐标定义，并且可表示目标物体的外表面。LIDAR单元257可被配置为通过测量目标物体的表面上的大量点来创建点云，并可将点云作为数据文件输出。作为通过LIDAR单元257的对物体的3D扫描过程的结果，点云可用于识别并可视化目标物体。
在一个示例中，点云可被直接渲染以可视化目标物体。在另一示例中，点云可通过可被称为曲面重建的过程被转换为多边形或三角形网格模型。用于将点云转换为3D曲面的示例技术可包括德洛内三角剖分、阿尔法形状和旋转球。这些技术包括在点云的现有顶点上构建三角形的网络。其它示例技术可包括将点云转换为体积距离场，以及通过移动立方体算法重建这样定义的隐式曲面。
摄像头258可以为用于获取车辆所位于的道路环境的图像的任何摄像头(例如，静态相机、视频相机等)。为此，摄像头可被配置为检测可见光，或可被配置为检测来自光谱的其它部分(诸如红外光或紫外光)的光。其它类型的相机也是可能的。摄像头258可以是二维检测器，或可具有三维空间范围。在一些示例中，摄像头258例如可以是距离检测器，其被配置为生成指示从摄像头258到环境中的若干点的距离的二维图像。为此，摄像头258可使用一种或多种距离检测技术。例如，摄像头258可被配置为使用结构光技术，其中融合装置200利用预定光图案，诸如栅格或棋盘格图案，对环境中的物体进行照射，并且使用摄像头258检测从物体的预定光图案的反射。基于反射的光图案中的畸变，路侧传感装置258可被配置为检测到物体上的点的距离。预定光图案可包括红外光或其它波长的光。
I/O子系统270用来控制输入输出的外部设备，可以包括其他设备输入控制器271、传感器控制器272。可选的，一个或多个其他输入控制设备控制器271从其他输入设备230接收信号和/或者向其他输入设备230发送信号，其他输入设备230可以包括物理按钮(按压按钮、摇臂按钮等)、拨号盘、滑动开关、操纵杆、点击滚轮、光鼠(光鼠是不显示可视输出的触摸敏感表面，或者是由触摸屏形成的触摸敏感表面的延伸)。值得说明的是，其他输入控制设备控制器271可以与任一个或者多个上述设备连接。传感器控制器272可以从一个或者多个传感器250接收信号和/或者向一个或者多个传感器250发送信号。
处理器280是融合装置200的控制中心，利用各种接口和线路连接整个融合装置200的各个部分，通过运行或执行存储在存储器220内的软件程序和/或模块，以及调用存储在存储器220内的数据，执行融合装置200的各种功能和处理数据，从而对融合装置200进行整体监控。可选的，处理器280可包括一个或多个处理单元；优选的，处理器280可集成2调制解调处理器，其中，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器280中。
融合装置200还包括给各个部件供电的电源290(比如电池)，优选的，电源可以通过电源管理系统与处理器280逻辑相连，从而通过电源管理系统实现管理充电、放电、以及 功耗等功能。
在实施例一中，融合装置200在处理器280内执行如下指令：
获得车辆感知数据，其中，所述车辆感知数据为车辆传感装置对感知范围内的道路环境进行侦测得到的；
获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。在实施例二中，融合装置200在处理器280内执行如下指令：
接收至少一台车辆设备发送的车辆感知数据，其中，所述车辆感知数据是车辆传感装置对感知范围内的道路环境进行侦测得到的；
通过融合公式将所述至少一台车辆设备的车辆感知数据和路侧感知数据进行融合以获得第一融合结果，其中，所述路侧感知数据是路侧传感装置对感知范围内的道路环境进行侦测得到的；
向目标车辆设备发送所述第一融合结果，其中，所述目标车辆设备用于将所述车辆感知数据和所述第一融合结果进行融合以获得第二融合结果，所述目标车辆设备属于所述至少一台车辆设备。
结合实施例一或者实施例二，所述融合公式表示为：
y＝f(result r，result v)，
其中，result r为路侧结果集，所述路侧结果集用于表示所述路侧感知数据，result v为车辆结果集，所述车辆结果集用于表示所述车辆感知数据，y为所述第一融合结果，函数f用于根据所述路侧结果集和所述车辆结果集映射出所述第一融合结果。
在一具体的实施方式中，
Figure PCTCN2019078646-appb-000025
其中，w r为所述路侧传感装置的置信因子，w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为所述路侧传感装置感知范围内目标物体的数量，w ri为所述路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为自然数，0<i≤M；w v为所述车辆传感装置的置信因子，w v＝(w v1，w v2，…，w vN)，result v(vehicle 1，vehicle 2，…，vehicle N)，N为所述车辆传感装置感知范围内目标物体的数量，w vj为所述车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为自然数，0<j≤N。
更具体地，所述置信因子是根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定的。
例如，置信因子w可以根据如下公式获得：
w＝g(S k，R i，θ j)，w∈[0,1]
其中，S k为所述传感装置参数，R i为所述目标物体的感知距离，θ j为所述目标物体的 感知角度，g为通过传感装置标定得到的标定参数表。
在一具体的实施方式中，所述车辆结果集包括至少一个车辆结果单元，所述至少一个车辆结果单元与至少一个目标物体存在一一对应关系，所述至少一个车辆结果单元中每一个车辆结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个车辆结果单元中的其中一个车辆结果单元表示为vehicle j(p vj，v vj，s vj，c vj)，其中，p vj表示为所述车辆传感装置侦测到的目标物体j的位置，v vj表示为所述车辆传感装置侦测到的目标物体j的速度，s vj表示为所述车辆传感装置侦测到的目标物体j的大小，c vj表示为所述车辆传感装置侦测到的目标物体j的颜色，N为所述车辆传感装置感知范围内目标物体的数量，j为自然数，0<j≤N。
在一具体的实施方式中，所述路侧结果集包括至少一个路侧结果单元，所述至少一个路侧结果单元与至少一个目标物体存在一一对应关系，所述至少一个路侧结果单元中每一个路侧结果单元用于从多维角度对对应的目标物体的特征进行描述。
更具体地，所述至少一个路侧结果单元中的其中一个路侧结果单元表示为roadside i(p vi，v vi，s vi，c vi)，其中，p vi表示为所述路侧传感装置侦测到的目标物体i的位置，v vi表示为所述路侧传感装置侦测到的目标物体i的速度，s vi表示为所述路侧传感装置侦测到的目标物体i的大小，c vi表示为所述路侧传感装置侦测到的目标物体i的颜色，M为所述路侧传感装置感知范围内目标物体的数量，i为自然数，0<i≤M。
结合实施例一或者实施例二，在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得融合结果之前，将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果，并根据匹配结果将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
在一具体的实施方式中，通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来。
更具体地，所述通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来，具体包括：
通过以下公式：S＝Deviation(roadside i，vehicle j)将所述路侧结果集中的路侧结果单元和所述路侧结果集中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
在一具体的实施方式中，在所述将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果之后，通过帧间回环和/或多帧关联的方式对匹配结果的置信度进行评价以获得评价结果，并根据评价结果调整偏差网络。
具体地，所述帧间回环为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果，第一匹配结果为所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第二匹配结果为所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第三匹配结果为所述车辆传感装置 在第i+1帧侦测到的目标物体j的对应的车辆结果单元与所述路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果，第四匹配结果为所述路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
具体地，所述多帧关联定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…。
需要说明的，图12实施例中未提及的内容可参考图8-图10对应的实施例，这里不再赘述。
基于同一发明构思，图13示出了本发明实施例提供的一种融合装置的结构示意图。如图13所示，融合装置300可以包括：第一获取模块301、第二获取模块302以及融合模块303，
所述第一获取模块301用于获取车辆感知数据，其中，所述车辆感知数据为车辆传感装置对感知范围内的道路环境进行侦测得到的；
所述第二获取模块302用于获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
所述融合模块303用于通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
需要说明，图13实施例中未提及的内容以及各个功能单元的具体实现，请参考图8-图9对应的实施例，这里不再赘述。
基于同一发明构思，图14示出了本发明实施例提供的一种融合装置的结构示意图。如图14所示，本实施例的融合装置400包括：发送模块401、接收模块402以及融合模块403，
所述发送模块401用于向路侧设备发送车辆感知数据，其中，所述车辆感知数据是车辆传感装置对感知范围内的道路环境进行侦测得到的；
所述接收模块402用于接收所述路侧设备发送的第一融合结果，其中，所述第一融合结果为所述路侧设备过融合公式对至少一台车辆设备的车辆感知数据以及路侧感知数据进行融合得到的，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
所述融合模块403用于将所述至少一台车辆设备的车辆感知数据和所述第一融合结果进行融合以获得第二融合结果。
需要说明，图14实施例中未提及的内容以及各个功能单元的具体实现，请参考图10对应的实施例，这里不再赘述。
通过上述方案，将路侧传感装置侦测得到的路侧感知数据和车辆传感装置侦测得到的车辆感知数据进行融合，可以实现将路侧传感装置的感知范围和车辆传感装置的感知范围进行叠加，从而有效地扩展感知范围。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、终端和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本发明实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (38)

  1. 一种数据融合方法，其特征在于，包括：
    获得车辆感知数据，其中，所述车辆感知数据为车辆传感装置对感知范围内的道路环境进行侦测得到的；
    获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
    通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
  2. 根据权利要求1所述的方法，其特征在于，所述融合公式表示为：
    y＝f(result r，result v)，
    其中，result r为路侧结果集，所述路侧结果集用于表示所述路侧感知数据，result v为车辆结果集，所述车辆结果集用于表示所述车辆感知数据，y为所述第一融合结果，函数f用于根据所述路侧结果集和所述车辆结果集映射出所述第一融合结果。
  3. 根据权利要求2所述的方法，其特征在于，
    Figure PCTCN2019078646-appb-100001
    其中，w r为所述路侧传感装置的置信因子，w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为所述路侧传感装置感知范围内目标物体的数量，w ri为所述路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为自然数，0<i≤M；w v为所述车辆传感装置的置信因子，w v＝(w v1，w v2，…，w vN)，result v(vehicle 1，vehicle 2，…，vehicle N)，N为所述车辆传感装置感知范围内目标物体的数量，w vj为所述车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为自然数，0<j≤N。
  4. 根据权利要求3所述的方法，其特征在于，所述置信因子是根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定的。
  5. 根据权利要求4所述的方法，其特征在于，置信因子w可以根据如下公式获得：
    w＝g(S k，R i，θ j)，w∈[0,1]
    其中，S k为所述传感装置参数，R i为所述目标物体的感知距离，θ j为所述目标物体的感知角度，g为通过传感装置标定得到的标定参数表。
  6. 根据权利要求2-5任一权利要求所述的方法，其特征在于，所述车辆结果集包括至少一个车辆结果单元，所述至少一个车辆结果单元与至少一个目标物体存在一一对应关系，所述至少一个车辆结果单元中每一个车辆结果单元用于从多维角度对对应的目标物体的特征进行描述。
  7. 根据权利要求6所述的方法，其特征在于，所述至少一个车辆结果单元中的任一个车辆结果单元表示为vehicle j(p vj，v vj，s vj，c vj)，其中，p vj表示为所述车辆传感装置侦测到的目标物体j的位置，v vj表示为所述车辆传感装置侦测到的目标物体j的速度，s vj表示 为所述车辆传感装置侦测到的目标物体j的大小，c vj表示为所述车辆传感装置侦测到的目标物体j的颜色，N为所述车辆传感装置感知范围内目标物体的数量，j为自然数，0<j≤N。
  8. 根据权利要求2-7任一权利要求所述的方法，其特征在于，所述路侧结果集包括至少一个路侧结果单元，所述至少一个路侧结果单元与至少一个目标物体存在一一对应关系，所述至少一个路侧结果单元中每一个路侧结果单元用于从多维角度对对应的目标物体的特征进行描述。
  9. 根据权利要求8所述的方法，其特征在于，所述至少一个路侧结果单元中的任一个路侧结果单元表示为roadside i(p vi，v vi，s vi，c vi)，其中，p vi表示为所述路侧传感装置侦测到的目标物体i的位置，v vi表示为所述路侧传感装置侦测到的目标物体i的速度，s vi表示为所述路侧传感装置侦测到的目标物体i的大小，c vi表示为所述路侧传感装置侦测到的目标物体i的颜色，M为所述路侧传感装置感知范围内目标物体的数量，i为自然数，0<i≤M。
  10. 根据权利要求2-5任一权利要求所述的方法，其特征在于，在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果之前，所述方法还包括：
    将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果；
    所述将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果，包括：
    根据所述匹配结果将所述车辆感知数据和所述路侧感知数据进行融合以获得所述第一融合结果。
  11. 根据权利要求10所述的方法，其特征在于，所述将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果，包括：
    通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来。
  12. 根据权利要求11所述的方法，其特征在于，所述通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来，具体包括：
    通过以下公式：S＝Deviation(roadside i，vehicle j)将所述路侧结果集中的路侧结果单元和所述路侧结果集中的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为所述偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
  13. 根据权利要求12所述的方法，其特征在于，所述偏差网络Deviation为如图5所示的逆向传播BP神经网络：
    其中，Δp ij为位置偏差值，Δp ij＝fabs(p ri-p vj)，p ri为所述路侧传感装置侦测到的目标物体i的位置，p vj为所述车辆传感装置侦测到的目标物体j的位置，fabs为求绝对值函数；
    Δv ij为速度偏差值，Δv ij＝fabs(v ri-v vj)，v ri为所述路侧传感装置侦测到的目标物体i的速度，v vj为所述车辆传感装置侦测到的目标物体j的速度；
    Δs ij为大小偏差值，Δs ij＝fabs(s ri-s vj)，s ri为所述路侧传感装置侦测到的目标物体i的大小，s vj为所述车辆传感装置侦测到的目标物体j的大小；
    Δc ij为颜色偏差值，Δc ij＝fabs(c ri-c vj)，c ri为所述路侧传感装置侦测到的目标物体i的颜色，c vj为所述车辆传感装置侦测到的目标物体j的颜色；
    P p为位置偏差因子，
    Figure PCTCN2019078646-appb-100002
    Figure PCTCN2019078646-appb-100003
    为所述路侧传感装置侦测到的目标物体i的位置对应的置信因子，
    Figure PCTCN2019078646-appb-100004
    为所述车辆传感装置侦测到的目标物体j的位置对应的置信因子；
    P v为速度偏差因子，
    Figure PCTCN2019078646-appb-100005
    Figure PCTCN2019078646-appb-100006
    为所述路侧传感装置侦测到的目标物体i的速度对应的置信因子，
    Figure PCTCN2019078646-appb-100007
    为所述车辆传感装置侦测到的目标物体j的速度对应的置信因子；
    P s为大小偏差因子，
    Figure PCTCN2019078646-appb-100008
    Figure PCTCN2019078646-appb-100009
    为所述路侧传感装置侦测到的目标物体i的大小对应的置信因子，
    Figure PCTCN2019078646-appb-100010
    为所述车辆传感装置侦测到的目标物体j的大小对应的置信因子；
    P c为速度偏差因子，
    Figure PCTCN2019078646-appb-100011
    Figure PCTCN2019078646-appb-100012
    为所述路侧传感装置侦测到的目标物体i的颜色对应的置信因子，
    Figure PCTCN2019078646-appb-100013
    为所述车辆传感装置侦测到的目标物体j的颜色对应的置信因子；
    Figure PCTCN2019078646-appb-100014
    为激活函数。
  14. 根据权利要求11-13任一权利要求所述的方法，其特征在于，在所述将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果之后，所述方法还包括：
    通过帧间回环和/或多帧关联的方式对所述匹配结果的置信度进行评价以获得评价结果；
    根据所述评价结果对所述偏差网络进行调整。
  15. 根据权利要求14所述的方法，其特征在于，所述帧间回环为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果，第一匹配结果为所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第二匹配结果为所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第三匹配结果为所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与所述路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果，第四匹配结果为所述路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与 所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
  16. 根据权利要求15所述的方法，其特征在于，所述多帧关联定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环 23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…。
  17. 根据权利要求1-16任一权利要求所述的方法，其特征在于，所述获得车辆感知数据，包括：
    接收至少一台车辆设备发送的所述车辆感知数据；
    所述在所述通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果之后，还包括：
    向目标车辆设备发送所述第一融合结果，其中，所述目标车辆设备用于将所述目标车辆设备的车辆感知数据和所述第一融合结果进行融合以获得第二融合结果，所述目标车辆设备属于所述至少一台车辆设备。
  18. 一种数据融合方法，其特征在于，包括：
    向路侧设备发送车辆感知数据，其中，所述车辆感知数据是车辆传感装置对感知范围内的道路环境进行侦测得到的；
    接收所述路侧设备发送的第一融合结果，其中，所述第一融合结果为所述路侧设备通过融合公式对至少一台车辆设备发送的车辆感知数据以及路侧感知数据进行融合得到的，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
    将所述车辆感知数据和所述第一融合结果进行融合以获得第二融合结果。
  19. 一种融合装置，其特征在于，包括：第一获取模块、第二获取模块以及融合模块，
    所述第一获取模块用于获取车辆感知数据，其中，所述车辆感知数据为车辆传感装置对感知范围内的道路环境进行侦测得到的；
    所述第二获取模块用于获取路侧感知数据，其中，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
    所述融合模块用于通过融合公式将所述车辆感知数据和所述路侧感知数据进行融合以获得第一融合结果。
  20. 根据权利要求19所述的装置，其特征在于，所述融合公式表示为：
    y＝f(result r，result v)，
    其中，result r为路侧结果集，所述路侧结果集用于表示所述路侧感知数据，result v为车辆结果集，所述车辆结果集用于表示所述车辆感知数据，y为所述第一融合结果，函数f用于根据所述路侧结果集和所述车辆结果集映射出所述第一融合结果。
  21. 根据权利要求20所述的装置，其特征在于，
    Figure PCTCN2019078646-appb-100015
    其中，w r为所述路侧传感装置的置信因子，w r＝(w r1，w r2，…，w rM)，result r(roadside 1，roadside 2，…，roadside M)，M为所述路侧传感装置感知范围内目标物体的数量，w ri为所述路侧传感装置感知范围内目标物体i对应的置信因子，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，i为自然数，0<i≤M；w v为所述车辆传感装置的置信 因子，w v＝(w v1，w v2，…，w vN)，result v(vehicle 1，vehicle 2，…，vehicle N)，N为所述车辆传感装置感知范围内目标物体的数量，w vj为所述车辆传感装置感知范围内目标物体j对应的置信因子，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，j为自然数，0<j≤N。
  22. 根据权利要求21所述的装置，其特征在于，所述置信因子是根据传感装置参数、目标物体的感知距离以及目标物体的感知角度共同确定的。
  23. 根据权利要求22所述的装置，其特征在于，置信因子w可以根据如下公式获得：
    w＝g(S k，R i，θ j)，w∈[0,1]
    其中，S k为所述传感装置参数，R i为所述目标物体的感知距离，θ j为所述目标物体的感知角度，g为通过传感装置标定得到的标定参数表。
  24. 根据权利要求20-23任一权利要求所述的装置，其特征在于，所述车辆结果集包括至少一个车辆结果单元，所述至少一个车辆结果单元与至少一个目标物体存在一一对应关系，所述至少一个车辆结果单元中每一个车辆结果单元用于从多维角度对对应的目标物体的特征进行描述。
  25. 根据权利要求24所述的装置，其特征在于，所述至少一个车辆结果单元中的任一个车辆结果单元表示为vehicle j(p vj，v vj，s vj，c vj)，其中，p vj表示为所述车辆传感装置侦测到的目标物体j的位置，v vj表示为所述车辆传感装置侦测到的目标物体j的速度，s vj表示为所述车辆传感装置侦测到的目标物体j的大小，c vj表示为所述车辆传感装置侦测到的目标物体j的颜色，N为所述车辆传感装置感知范围内目标物体的数量，j为自然数，0<j≤N。
  26. 根据权利要求20-25任一权利要求所述的装置，其特征在于，所述路侧结果集包括至少一个路侧结果单元，所述至少一个路侧结果单元与至少一个目标物体存在一一对应关系，所述至少一个路侧结果单元中每一个路侧结果单元用于从多维角度对对应的目标物体的特征进行描述。
  27. 根据权利要求26所述的装置，其特征在于，所述至少一个路侧结果单元中的任一个路侧结果单元表示为roadside i(p vi，v vi，s vi，c vi)，其中，p vi表示为所述路侧传感装置侦测到的目标物体i的位置，v vi表示为所述路侧传感装置侦测到的目标物体i的速度，s vi表示为所述路侧传感装置侦测到的目标物体i的大小，c vi表示为所述路侧传感装置侦测到的目标物体i的颜色，M为所述路侧传感装置感知范围内目标物体的数量，i为自然数，0<i≤M。
  28. 根据权利要求20-27任一权利要求所述的装置，其特征在于，所述装置还包括匹配模块，所述匹配模块用于将所述路侧感知数据和所述车辆感知数据进行匹配以获得匹配结果；所述融合模块还用于根据所述匹配结果将所述车辆感知数据和所述路侧感知数据进行融合以获得所述第一融合结果。
  29. 根据权利要求28所述的装置，其特征在于，所述匹配模块还用于通过偏差网络将所述路侧结果集中的路侧结果单元和所述车辆结果集中的车辆结果单元之间的匹配关系找出来。
  30. 根据权利要求29所述的装置，其特征在于，所述匹配模块还用于通过以下公式：S＝Deviation(roadside i，vehicle j)将所述路侧结果集中的路侧结果单元和所述路侧结果集中 的车辆结果单元之间的匹配关系找出来，其中，S为匹配结果，Deviation为所述偏差网络，roadside i为所述路侧传感装置感知范围内目标物体i对应的路侧结果单元，vehicle j为所述车辆传感装置感知范围内目标物体j对应的车辆结果单元，i，j均为自然数。
  31. 根据权利要求30所述的装置，其特征在于，所述偏差网络Deviation为如图5所示的逆向传播BP神经网络：
    其中，Δp ij为位置偏差值，Δp ij＝fabs(p ri-p vj)，p ri为所述路侧传感装置侦测到的目标物体i的位置，p vj为所述车辆传感装置侦测到的目标物体j的位置，fabs为求绝对值函数；
    Δv ij为速度偏差值，Δv ij＝fabs(v ri-v vj)，v ri为所述路侧传感装置侦测到的目标物体i的速度，v vj为所述车辆传感装置侦测到的目标物体j的速度；
    Δs ij为大小偏差值，Δs ij＝fabs(s ri-s vj)，s ri为所述路侧传感装置侦测到的目标物体i的大小，s vj为所述车辆传感装置侦测到的目标物体j的大小；
    Δc ij为颜色偏差值，Δc ij＝fabs(c ri-c vj)，c ri为所述路侧传感装置侦测到的目标物体i的颜色，c vj为所述车辆传感装置侦测到的目标物体j的颜色；
    P p为位置偏差因子，
    Figure PCTCN2019078646-appb-100016
    Figure PCTCN2019078646-appb-100017
    为所述路侧传感装置侦测到的目标物体i的位置对应的置信因子，
    Figure PCTCN2019078646-appb-100018
    为所述车辆传感装置侦测到的目标物体j的位置对应的置信因子；
    P v为速度偏差因子，
    Figure PCTCN2019078646-appb-100019
    Figure PCTCN2019078646-appb-100020
    为所述路侧传感装置侦测到的目标物体i的速度对应的置信因子，
    Figure PCTCN2019078646-appb-100021
    为所述车辆传感装置侦测到的目标物体j的速度对应的置信因子；
    P s为大小偏差因子，
    Figure PCTCN2019078646-appb-100022
    Figure PCTCN2019078646-appb-100023
    为所述路侧传感装置侦测到的目标物体i的大小对应的置信因子，
    Figure PCTCN2019078646-appb-100024
    为所述车辆传感装置侦测到的目标物体j的大小对应的置信因子；
    P c为速度偏差因子，
    Figure PCTCN2019078646-appb-100025
    Figure PCTCN2019078646-appb-100026
    为所述路侧传感装置侦测到的目标物体i的颜色对应的置信因子，
    Figure PCTCN2019078646-appb-100027
    为所述车辆传感装置侦测到的目标物体j的颜色对应的置信因子；
    Figure PCTCN2019078646-appb-100028
    为激活函数。
  32. 根据权利要求29-31任一权利要求所述的装置，其特征在于，所述装置还包括评价 模块以及调整模块，
    所述评价模块用于通过帧间回环和/或多帧关联的方式对所述匹配结果的置信度进行评价以获得评价结果；
    所述调整模块用于根据所述评价结果对所述偏差网络进行调整。
  33. 根据权利要求32所述的装置，其特征在于，所述帧间回环为：T 回环＝T 1+T 2+T 3+T 4，其中，T 回环为帧间回环，T 1为第一匹配结果，T 2为第二匹配结果，T 3为第三匹配结果，T 4为第四匹配结果，第一匹配结果为所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元与所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第二匹配结果为所述车辆传感装置在第i帧侦测到的目标物体j的对应的车辆结果单元与所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元的匹配结果，第三匹配结果为所述车辆传感装置在第i+1帧侦测到的目标物体j的对应的车辆结果单元与所述路测传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元的匹配结果，第四匹配结果为所述路侧传感装置在第i+1帧侦测到的目标物体j的对应的路侧结果单元与所述路侧传感装置在第i帧侦测到的目标物体j的对应的路侧结果单元的匹配结果。
  34. 根据权利要求32或33所述的装置，其特征在于，所述多帧关联定义为：T 多帧＝T 回环12+T 回环23+T 回环34+…，其中，T 多帧为多帧关联，T 回环12为第一帧和第二帧之间的帧间回环，T 回环23为第二帧和第三帧之间的帧间回环，T 回环34为第三帧和第四帧之间的帧间回环，…。
  35. 根据权利要求19-34任一权利要求所述的装置，其特征在于，所述装置还包括发送模块，
    所述第一获取模块用于接收至少一台车辆设备发送的所述车辆感知数据；
    所述发送模块用于向目标车辆设备发送所述第一融合结果，其中，所述目标车辆设备用于将所述目标车辆设备的车辆感知数据和所述第一融合结果进行融合以获得第二融合结果，所述目标车辆设备属于所述至少一台车辆设备。
  36. 一种融合装置，其特征在于，包括发送模块、接收模块以及融合模块，
    所述发送模块用于向路侧设备发送车辆感知数据，其中，所述车辆感知数据是车辆传感装置对感知范围内的道路环境进行侦测得到的；
    所述接收模块用于接收所述路侧设备发送的第一融合结果，其中，所述第一融合结果为所述路侧设备过融合公式对至少一台车辆设备发送的车辆感知数据以及路侧感知数据进行融合得到的，所述路侧感知数据为路侧传感装置对感知范围内的道路环境进行侦测得到的；
    所述融合模块用于将所述车辆感知数据和所述第一融合结果进行融合以获得第二融合结果。
  37. 一种融合装置，其特征在于，包括：存储器以及与所述存储器耦合的处理器、通信模块，其中：所述通信模块用于发送或者接收外部发送的数据，所述存储器用于存储程序代码，所述处理器用于调用所述存储器存储的程序代码以执行如权利要求1-18任一权利要求描述的方法。
  38. 一种计算机可读存储介质，其特征在于，包括指令，当所述指令在融合装置上运行时，使得所述融合装置执行如权利要求1-18任意一项所述的方法。
PCT/CN2019/078646 2018-03-20 2019-03-19 数据融合方法以及相关设备 Ceased WO2019179417A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP19771774.7A EP3754448B1 (en) 2018-03-20 2019-03-19 Data fusion method and related device
JP2020550669A JP7386173B2 (ja) 2018-03-20 2019-03-19 データ融合方法及び関連装置
US17/021,911 US11987250B2 (en) 2018-03-20 2020-09-15 Data fusion method and related device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810232615.5 2018-03-20
CN201810232615.5A CN108762245B (zh) 2018-03-20 2018-03-20 数据融合方法以及相关设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/021,911 Continuation US11987250B2 (en) 2018-03-20 2020-09-15 Data fusion method and related device

Publications (1)

Publication Number Publication Date
WO2019179417A1 true WO2019179417A1 (zh) 2019-09-26

Family

ID=63980561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/078646 Ceased WO2019179417A1 (zh) 2018-03-20 2019-03-19 数据融合方法以及相关设备

Country Status (5)

Country Link
US (1) US11987250B2 (zh)
EP (1) EP3754448B1 (zh)
JP (1) JP7386173B2 (zh)
CN (1) CN108762245B (zh)
WO (1) WO2019179417A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113223297A (zh) * 2021-07-09 2021-08-06 杭州目炬科技有限公司 一种多维度自动车辆识别方法
CN113655509A (zh) * 2021-08-13 2021-11-16 中国科学院国家天文台长春人造卫星观测站 一种高重复率卫星激光测距实时控制系统
WO2022043423A1 (en) * 2020-08-31 2022-03-03 Robert Bosch Gmbh Method for implementing autonomous driving, medium, vehicle-mounted computer, and control system

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108762245B (zh) 2018-03-20 2022-03-25 华为技术有限公司 数据融合方法以及相关设备
CN109583505A (zh) * 2018-12-05 2019-04-05 百度在线网络技术（北京）有限公司 一种多传感器的物体关联方法、装置、设备及介质
CN112183715A (zh) * 2019-07-03 2021-01-05 本田技研工业株式会社 传感器融合
CN110164157A (zh) * 2019-07-16 2019-08-23 华人运通(上海)新能源驱动技术有限公司 路侧设备、用于路侧设备的方法和车路协同系统
CN110276972A (zh) * 2019-07-16 2019-09-24 启迪云控(北京)科技有限公司 一种基于车联网的目标物感知方法及系统
US11787407B2 (en) * 2019-07-24 2023-10-17 Pony Ai Inc. System and method for sensing vehicles and street
US11867798B2 (en) * 2019-09-13 2024-01-09 Samsung Electronics Co., Ltd. Electronic device including sensor and method of determining path of electronic device
CN111079079B (zh) * 2019-11-29 2023-12-26 北京百度网讯科技有限公司 数据修正方法、装置、电子设备及计算机可读存储介质
CN111209327A (zh) * 2020-01-14 2020-05-29 南京悠淼科技有限公司 一种多传感器分布式感知互联与边缘融合处理系统及方法
CN111583690B (zh) * 2020-04-15 2021-08-20 北京踏歌智行科技有限公司 一种基于5g的矿区无人运输系统的弯道协同感知方法
US20220017115A1 (en) * 2020-07-14 2022-01-20 Argo AI, LLC Smart node network for autonomous vehicle perception augmentation
US20220019225A1 (en) * 2020-07-14 2022-01-20 Argo AI, LLC Smart node for autonomous vehicle perception augmentation
CN112085960A (zh) * 2020-09-21 2020-12-15 北京百度网讯科技有限公司 车路协同信息处理方法、装置、设备及自动驾驶车辆
CN112349100B (zh) * 2020-11-09 2021-06-04 紫清智行科技(北京)有限公司 一种基于网联环境的多视角行车风险评估方法及装置
CN114639262B (zh) * 2020-12-15 2024-02-06 北京万集科技股份有限公司 感知设备的状态检测方法、装置、计算机设备和存储介质
CN113012429B (zh) * 2021-02-23 2022-07-15 云控智行(上海)汽车科技有限公司 一种车路多传感器数据融合方法及系统
CN112884892B (zh) * 2021-02-26 2023-05-02 武汉理工大学 基于路侧装置的无人矿车位置信息处理系统和方法
CN112671935A (zh) * 2021-03-17 2021-04-16 中智行科技有限公司 远程控制车辆的方法、路侧设备及云平台
CN115331421B (zh) * 2021-05-10 2024-05-10 北京万集科技股份有限公司 路侧多传感环境感知方法、装置及系统
CN113378704B (zh) * 2021-06-09 2022-11-11 武汉理工大学 一种多目标检测方法、设备及存储介质
CN113655494B (zh) * 2021-07-27 2024-05-10 上海智能网联汽车技术中心有限公司 路侧相机与4d毫米波融合的目标检测方法、设备及介质
CN115690119A (zh) * 2021-07-29 2023-02-03 华为技术有限公司 一种数据处理方法及装置
CN115691099A (zh) * 2021-07-30 2023-02-03 华为技术有限公司 感知能力信息生成方法、使用方法及装置
US11845429B2 (en) * 2021-09-30 2023-12-19 GM Global Technology Operations LLC Localizing and updating a map using interpolated lane edge data
US11987251B2 (en) 2021-11-15 2024-05-21 GM Global Technology Operations LLC Adaptive rationalizer for vehicle perception systems toward robust automated driving control
CN114120256B (zh) * 2021-12-01 2025-11-21 浙江数智交院科技股份有限公司 信息感知方法、装置、系统、服务器及可读存储介质
CN114152942B (zh) * 2021-12-08 2022-08-05 北京理工大学 一种毫米波雷达与视觉二阶融合多分类目标检测方法
CN114596547B (zh) * 2022-01-28 2025-04-29 北京汽车研究总院有限公司 自动驾驶车辆的障碍物检测方法、装置、设备及介质
CN114915940B (zh) * 2022-05-13 2023-07-21 山东高速建设管理集团有限公司 一种车路通信链路匹配方法、系统、设备及介质
DE102022204770A1 (de) * 2022-05-16 2023-11-16 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren zum Steuern einer Robotervorrichtung
EP4438430A1 (en) * 2023-03-30 2024-10-02 TomTom International B.V. Method of controlling vehicle functions, vehicle processing system, vehicle, and machine-readable instruction code

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020099481A1 (en) * 2001-01-22 2002-07-25 Masaki Mori Travel controlling apparatus of unmanned vehicle
US20040215377A1 (en) * 2003-04-22 2004-10-28 Jung-Rak Yun Automated self-control traveling system for expressways and method for controlling the same
CN107063275A (zh) * 2017-03-24 2017-08-18 重庆邮电大学 基于路侧设备的智能车辆地图融合系统及方法
CN107807633A (zh) * 2017-09-27 2018-03-16 北京图森未来科技有限公司 一种路侧设备、车载设备以及自动驾驶感知方法及系统
CN108762245A (zh) * 2018-03-20 2018-11-06 华为技术有限公司 数据融合方法以及相关设备

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6445983B1 (en) * 2000-07-07 2002-09-03 Case Corporation Sensor-fusion navigator for automated guidance of off-road vehicles
US7065465B2 (en) * 2002-03-26 2006-06-20 Lockheed Martin Corporation Method and system for multi-sensor data fusion
US7548184B2 (en) * 2005-06-13 2009-06-16 Raytheon Company Methods and apparatus for processing data from multiple sources
US7460951B2 (en) * 2005-09-26 2008-12-02 Gm Global Technology Operations, Inc. System and method of target tracking using sensor fusion
CN101231340A (zh) * 2007-12-29 2008-07-30 四川川大智胜软件股份有限公司 多雷达系统航迹融合处理时的误差配准方法
JP4565022B2 (ja) 2008-06-30 2010-10-20 日立オートモティブシステムズ株式会社 交通情報システムおよび交通情報処理方法
CN101393264B (zh) * 2008-10-12 2011-07-20 北京大学 基于多激光扫描仪的移动目标跟踪方法及系统
CN101807245B (zh) * 2010-03-02 2013-01-02 天津大学 基于人工神经网络的多源步态特征提取与身份识别方法
CN102508246B (zh) 2011-10-13 2013-04-17 吉林大学 车辆前方障碍物检测跟踪方法
JP2013182490A (ja) 2012-03-02 2013-09-12 Sumitomo Electric Ind Ltd 交通情報変換装置、交通情報システム、中央サーバ、サービスサーバ、及び、交通情報提供方法
JP2013214225A (ja) * 2012-04-03 2013-10-17 Sumitomo Electric Ind Ltd 交通情報システム、中央サーバ、サービスサーバ、及び、交通情報提供方法
CN103105611B (zh) * 2013-01-16 2016-01-20 广东工业大学 一种分布式多传感器智能信息融合方法
CN103226708B (zh) * 2013-04-07 2016-06-29 华南理工大学 一种基于Kinect的多模型融合视频人手分割方法
EP2865575B1 (en) 2013-10-22 2022-08-24 Honda Research Institute Europe GmbH Confidence estimation for predictive driver assistance systems based on plausibility rules
EP2865576B1 (en) 2013-10-22 2018-07-04 Honda Research Institute Europe GmbH Composite confidence estimation for predictive driver assistant systems
WO2015085483A1 (en) * 2013-12-10 2015-06-18 SZ DJI Technology Co., Ltd. Sensor fusion
WO2015134991A1 (en) * 2014-03-07 2015-09-11 Capitalogix, LLC Systems and methods for generating and selecting trading algorithms for big data trading in financial markets
US9563808B2 (en) * 2015-01-14 2017-02-07 GM Global Technology Operations LLC Target grouping techniques for object fusion
CN106503723A (zh) * 2015-09-06 2017-03-15 华为技术有限公司 一种视频分类方法及装置
CN105446338B (zh) * 2015-12-21 2017-04-05 福州华鹰重工机械有限公司 云辅助自动驾驶方法及系统
CN105741545B (zh) * 2016-03-16 2018-07-20 山东大学 一种基于公交车gnss时空轨迹数据判别交通状态的装置及方法
CN105741546B (zh) 2016-03-18 2018-06-29 重庆邮电大学 路侧设备与车传感器融合的智能车辆目标跟踪系统及方法
US10571913B2 (en) * 2016-08-05 2020-02-25 Aptiv Technologies Limited Operation-security system for an automated vehicle
US10317901B2 (en) * 2016-09-08 2019-06-11 Mentor Graphics Development (Deutschland) Gmbh Low-level sensor fusion
CN106447707B (zh) * 2016-09-08 2018-11-16 华中科技大学 一种图像实时配准方法及系统
US10692365B2 (en) * 2017-06-20 2020-06-23 Cavh Llc Intelligent road infrastructure system (IRIS): systems and methods
CN107229690B (zh) 2017-05-19 2019-01-25 广州中国科学院软件应用技术研究所 基于路侧传感器的高精度动态地图数据处理系统及方法
CN107230113A (zh) * 2017-08-01 2017-10-03 江西理工大学 一种多模型融合的房产评估方法
US10852426B2 (en) * 2018-01-12 2020-12-01 Tiejun Shan System and method of utilizing a LIDAR digital map to improve automatic driving
KR102838499B1 (ko) * 2018-02-06 2025-07-28 씨에이브이에이치 엘엘씨 지능형 도로 인프라구조 시스템(iris) 시스템 및 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020099481A1 (en) * 2001-01-22 2002-07-25 Masaki Mori Travel controlling apparatus of unmanned vehicle
US20040215377A1 (en) * 2003-04-22 2004-10-28 Jung-Rak Yun Automated self-control traveling system for expressways and method for controlling the same
CN107063275A (zh) * 2017-03-24 2017-08-18 重庆邮电大学 基于路侧设备的智能车辆地图融合系统及方法
CN107807633A (zh) * 2017-09-27 2018-03-16 北京图森未来科技有限公司 一种路侧设备、车载设备以及自动驾驶感知方法及系统
CN108762245A (zh) * 2018-03-20 2018-11-06 华为技术有限公司 数据融合方法以及相关设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3754448A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022043423A1 (en) * 2020-08-31 2022-03-03 Robert Bosch Gmbh Method for implementing autonomous driving, medium, vehicle-mounted computer, and control system
US12576876B2 (en) 2020-08-31 2026-03-17 Robert Bosch Gmbh Method for implementing autonomous driving, medium, vehicle-mounted computer, and control system
CN113223297A (zh) * 2021-07-09 2021-08-06 杭州目炬科技有限公司 一种多维度自动车辆识别方法
CN113655509A (zh) * 2021-08-13 2021-11-16 中国科学院国家天文台长春人造卫星观测站 一种高重复率卫星激光测距实时控制系统
CN113655509B (zh) * 2021-08-13 2023-07-21 中国科学院国家天文台长春人造卫星观测站 一种高重复率卫星激光测距实时控制系统

Also Published As

Publication number Publication date
CN108762245B (zh) 2022-03-25
EP3754448B1 (en) 2025-06-25
JP7386173B2 (ja) 2023-11-24
CN108762245A (zh) 2018-11-06
US20200409372A1 (en) 2020-12-31
JP2021516401A (ja) 2021-07-01
US11987250B2 (en) 2024-05-21
EP3754448A1 (en) 2020-12-23
EP3754448A4 (en) 2021-04-28

Similar Documents

Publication Publication Date Title
US11987250B2 (en) Data fusion method and related device
US11754721B2 (en) Visualization and semantic monitoring using lidar data
CN110967011B (zh) 一种定位方法、装置、设备及存储介质
US10582121B2 (en) System and method for fusing outputs of sensors having different resolutions
US10024965B2 (en) Generating 3-dimensional maps of a scene using passive and active measurements
TW202115366A (zh) 機率性多機器人slam的系統及方法
US11561553B1 (en) System and method of providing a multi-modal localization for an object
CN106993181A (zh) 多vr/ar设备协同系统及协同方法
JP7166446B2 (ja) ロボットの姿勢を推定するシステムおよび方法、ロボット、並びに記憶媒体
CN103852067A (zh) 调整飞行时间(tof)测量系统的操作参数的方法
CN103852754A (zh) 飞行时间(tof)测量系统中的干扰抑制的方法
US11092690B1 (en) Predicting lidar data using machine learning
JP2016085602A (ja) センサ情報統合方法、及びその装置
WO2021078003A1 (zh) 无人载具的避障方法、避障装置及无人载具
CN111538009A (zh) 雷达点的标记方法和装置
CN114384486B (zh) 一种数据处理方法及装置
CN113887400A (zh) 障碍物检测方法、模型训练方法、装置及自动驾驶车辆
CN117452411A (zh) 障碍物检测方法和装置
Wang et al. Intelligent construction activity identification for all-weather site monitoring using 4D millimeter-wave technology
CN117063218A (zh) 信息处理装置、车辆、路侧机以及信息处理方法
CN115131756A (zh) 一种目标检测方法及装置
TWI843116B (zh) 移動物體檢測方法、裝置、電子設備及存儲介質
US20240219542A1 (en) Auto-level step for extrinsic calibration
Byun et al. Data transmission for infrared images via Earphone Jack on smartphone
CN117055016A (zh) 传感器的探测精度测试方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19771774

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019771774

Country of ref document: EP

Effective date: 20200915

Ref document number: 2020550669

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2019771774

Country of ref document: EP