WO2012139636A1 - Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques - Google Patents

Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques Download PDF

Info

Publication number
WO2012139636A1
WO2012139636A1 PCT/EP2011/055789 EP2011055789W WO2012139636A1 WO 2012139636 A1 WO2012139636 A1 WO 2012139636A1 EP 2011055789 W EP2011055789 W EP 2011055789W WO 2012139636 A1 WO2012139636 A1 WO 2012139636A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
reference points
camera
image
online calibration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2011/055789
Other languages
English (en)
Inventor
Myles Friel
Derek Anthony Savage
Ciaran Hughes
Pantelis ERMILIOS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Application Solutions Electronic and Vision Ltd
Connaught Electronics Ltd
Original Assignee
Application Solutions Electronic and Vision Ltd
Connaught Electronics Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Application Solutions Electronic and Vision Ltd, Connaught Electronics Ltd filed Critical Application Solutions Electronic and Vision Ltd
Priority to PCT/EP2011/055789 priority Critical patent/WO2012139636A1/fr
Publication of WO2012139636A1 publication Critical patent/WO2012139636A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior

Definitions

  • This invention relates generally to an online calibration of a vehicle video system, and particularly to a method for online calibration of the vehicle video system evaluated from image frames of a camera. It is also related to a computer program product for processing data relating to the online calibration of the vehicle video system, the computer program product comprising a computer usable medium having a computer usable program code embodied therewith, the computer program code being configured to perform any of the method steps.
  • the present invention is further related to an online calibration system for a vehicle video system for processing the computer implemented online calibration method.
  • image capture devices such as, for example, digital or analogue video cameras
  • mounts image capture devices on a motor vehicle in order to produce a video image of an aspect of the environment exterior of the vehicle.
  • image capturing devices on respective opposite sides of the vehicle, for example, on side rear view mirror housings which extend sidewardly from the driver and front passenger doors of the vehicle.
  • the image capture devices are mounted in the side rear view mirror housings with the field of view of the image capture devices directed downwardly towards the ground for capturing plan view images of the ground on respective opposite sides of the vehicle adjacent the vehicle.
  • a visual display unit is located in the vehicle, either in or on the dashboard, or in a location corresponding to that of a conventional interiorly mounted rear view mirror.
  • a plan view image of the vehicle with the respective plan view images of the ground on respective opposite sides of the vehicle can be displayed on the visual display unit.
  • the plan view display of the vehicle and the ground on respective opposite sides of the vehicle assists the driver in parking, and in particular, carrying out a parking manoeuvre for parking the vehicle in a parking space parallel to a kerb of a footpath or the like.
  • plan view images of the ground accurately reflect the positions of objects relative to the vehicle, which are captured in the images
  • plan view images of the ground juxtapositioned with the plan view image of the vehicle should accurately represent a top plan view of the ground adjacent the respective opposite sides of the vehicle exactly as would be seen when viewed from above.
  • the edges of the respective plan view images of the ground which extend along the sides of the plan view image of the vehicle must correspond directly with the edge of the ground along the sides of the vehicle when viewed in plan view from a position above the vehicle. Otherwise, the positions of objects in the respective plan view images of the ground will not be accurately positioned relative to the vehicle.
  • the edge of one of the plan view images of the ground adjacent the corresponding side of the plan view image of the vehicle corresponds with a portion of a plan view of the ground which is spaced apart from the side of the vehicle, then the positions of objects in the plan view image of the ground will appear closer to the vehicle in the image than they actually are.
  • one of the image capture devices is mounted on a side mirror housing so that an image of a portion of the ground beneath a side of the vehicle is captured, the positions of objects captured in the plan view image will appear farther away from the vehicle than they actually are, with disastrous results, particularly if a driver is parking the vehicle parallel to a wall or bollards. Similar requirements apply also for front or rear placed image capture devices.
  • plan view images of the ground when displayed on the visual display screen juxtapositioned along with the plan view image of the vehicle must be representative of plan views of the ground on respective opposite sides of the vehicle exactly as would be seen from a top plan view of the vehicle and adjacent ground.
  • the image capture devices would have to be precision mounted on the vehicle. In practice this is not possible. Accordingly, in order to achieve the appropriate degree of exactness and accuracy of the plan view images of the ground relative to the plan view image of the vehicle, it is necessary to calibrate the outputs of the image capture devices.
  • Calibration values determined during calibration of the image capture devices are then used to correct subsequently captured image frames for offset of the image capture devices from ideal positions thereof, so that plan view images of the ground subsequently outputted for display with the plan view image of the vehicle are exact representations of the ground on respective opposite sides of the vehicle.
  • Such calibration can be accurately carried out in a factory during production of the motor vehicle.
  • the image capture devices are relatively accurately fitted in the side mirror housings of the motor vehicle, and by using suitable grid patterns on the ground, calibration can be effected.
  • the environments in which motor vehicles must operate are generally relatively harsh environments, in that side mirror housings are vulnerable to impacts with other vehicles or stationary objects.
  • WO2009/027090 is described a method and system for online calibration of a vehicle video system using vanishing points evaluated from frames of a camera image containing identified markings or edges on a road.
  • US2009/0290032 is described a system and method for calibrating a camera on a vehicle by identifying at least two feature points in at least two camera images from a vehicle that has moved between taking the images. The method then determines a camera translation direction between two camera positions. Following this, the method determines a ground plane in camera coordinates based on the corresponding feature points from the images and the camera translation direction. The method then determines a height of the camera above the ground and a rotation of the camera in vehicle coordinates.
  • a method for online calibration of a vehicle video system is evaluated from image frames of a camera containing features on the road.
  • the method comprises the following steps of capturing by the camera of a portion of a road surface in an image frame. Then, a selection of at least two different features within the image frame is taken, the selected features being chosen as respective reference points.
  • a sequence of at least one more image frame is performed with the camera while locating within the new frame the features chosen as reference points to be tracked. Analysing in a virtual image space the trajectory covered by the reference points during driving interval between image sequence by identifying a geometrical shape obtained by joining the located reference points between each other in each used images and considering the respective trajectories.
  • a deviation of the resulting geometrical object from a parallelogram with corners defined by the reference points from at least two subsequent images is calculated while any measured deviation being used to define an offset correction of the camera.
  • a row of features can be selected within the image frame for the reference points. In such a way, multiple features can be tracked allowing a parallel analysis of different geometrical shape.
  • a specific window size can be used for the selection of features within the image frame, the specific window size being possibly correlated to driving parameters of the vehicle like the speed or the steering angle of the vehicle.
  • a correlation search window is defined for tracking the selected reference points, the size of the correlation search window being adapted possibly according to available processing power. Accordingly, the size of the window could be chosen relatively big in case strong processing power could be used for the tracking step.
  • a filtering can be applied using an image filter with band-pass like properties.
  • a correlation search window is defined for each reference point.
  • a subtraction of possible distortion of the camera on the identified geometrical object is performed such to obtain a quadrilateral to be compared to the defined parallelogram. This is particularly relevant for a fish-eye camera. The performed subtraction corresponds then to a transformation to a non-fish-eye rectilinear image space.
  • driving parameters of the moving vehicle are used for optimizing the tracking of the reference points.
  • speed of the moving vehicle can be used for calculating the distance of the reference points when tracked between image frame sequences allowing determining the height of the camera and in case correcting it.
  • an offset rotation as online calibration is defined with respect to a measured deviation from the parallelism between lines obtained from subsequent image frames by joining the tracked reference points within an image frame for an offset rotation about the x-axis longitudinal to the vehicle.
  • a similar approach can be applied for an offset rotation about the y-axis transverse to the vehicle, this time by considering the trajectory of the tracked reference points at subsequent image frames.
  • a deviation can be measured such that the lines joining the tracked reference points in subsequent image frame being rotated in the frame corresponds to an offset rotation about the vehicle z-axis for a camera located at a wing mirror, at rear or at the front of the vehicle.
  • steering angle information of the vehicle can be taken into account.
  • This information can be of use in the case the vehicle is turning when performing sequence of image frames to transform accordingly the identified geometrical object such to obtain a quadrilateral to be compared to the defined parallelogram.
  • some error function can be used to be minimized to take into account a deviation from a straight trajectory by the vehicle turning when performing the sequence of image frames. This is of advantage when the steering angle is not easily available for the analysis of the trajectory covered by the reference points. And then the resulting geometrical object is compared to a quadrilateral like lying on concentric arcs.
  • a computer program product for processing data relating to online calibration of a vehicle video system is also described and claimed herein.
  • the computer program product comprises a computer usable medium having computer usable program code embodied therewith, the computer usable program code being configured to perform the above summarized methods.
  • an online calibration system for a vehicle video system comprises a computer program product for processing data relating to an online calibration method and an image processing apparatus with a camera for taking image frames to be used by the online calibration method such to perform the above
  • FIG. 1 illustrates a side view and a top view of a vehicle with a 3d co-ordinate system
  • FIG. 2 illustrates a flow diagram according to the invention
  • FIG. 3 illustrates one example of feature tracking for a wing mirror camera
  • FIG. 4 illustrates another example of feature tracking according to the invention
  • FIG. 5 illustrates a further example of feature tracking for a wing mirror fish-eye camera according to the invention
  • FIG. 6 illustrates a way to measure error correction according to the invention
  • FIG. 7 illustrates one example of feature tracking for a rear- view camera.
  • aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
  • the computer readable medium may be a computer readable signal medium or a computer readable storage medium.
  • a computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
  • a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof.
  • a computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
  • Program code embodied on a computer readable medium may be transmitted within the vehicle using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
  • the program code may execute entirely on the vehicle's computer, partly on the vehicle's computer, as a stand-alone software package, partly on the vehicle's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the vehicle's computer through any type of wireless network, including a wireless local area network (WLAN), possibly but not necessarily through the Internet using an Internet Service Provider.
  • WLAN wireless local area network
  • These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
  • the computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • Present invention is a means of calibrating a camera on a vehicle to determine the extrinsic camera rotation parameters relative to the vehicle co-ordinate system when the camera is positioned in an undetermined relevant position on the vehicle body.
  • Calibration allows the vehicle manufacturer to provide a geometrically representative and more importantly a useful view to a vehicle user. Additionally, by extension, the height of the camera off the ground plane (XY-plane) can be determined if the speed of the vehicle is known.
  • Figure 1 a side view and a top view of a vehicle 1 with a defined three dimensional coordinate system XYZ as used in the following.
  • X-axis is chosen along the longitudinal direction
  • Y-axis along the transverse direction
  • Z-axis along the vertical direction of the vehicle 1.
  • the camera whether placed at rear, front or wing-mirror, captures a significant portion of road surfaces in the scene. Such portion of road surfaces is stored in image frames from which at least two different features of the road surfaces are selected. The selected features are then chosen as respective reference points.
  • Fig. 2 shows a flowchart according to present invention.
  • the online calibration method is started 10 at some stage when vehicle is driving.
  • Such starting event 10 can be programmed in different ways e.g. after a predefined time during driving or after travelling some predefined distance or a combination of several parameters.
  • image frame N captured 1 1 i.e., the first frame of the online calibration
  • two features are selected for tracking. Arbitrary positions can therefore be selected though this is extendable to using different methods to select the first calibration feature.
  • At least two different texture features on the road surfaces are chosen as respective reference points 12 to be tracked.
  • a selected feature or texture feature is defined in the present invention as a window of any defined size (m x n pixels) in the image frame that is associated with a given point on the road surface.
  • Some implementation uses a window of 16 x 16 pixels in the luminance channel of the frame. However, this is extendable to a texture feature of any size. It can also include information from the chroma channels, or from the R (red), G (green) and B (blue) channels if RGB color space is utilized. Alternately, information of RGBA (A for alpha) color space could also be considered if used for the definition of the selected feature as reference point on the road surface.
  • the algorithm selects 14 a search window in which it searches for the feature being tracked.
  • the window location is selected based on the location of the texture feature in the previous frame.
  • a cross- correlation can be performed to determine 15 the location of the tracked texture feature in the current frame.
  • any type of correlation could also be used to find the texture in the search window such as cross-correlation, SAD (sum of absolute difference), SSD (sum of squared differences) or other, the choice being possibly taken according to the measured efficiency in terms of processing time and detection performance.
  • a filtering with an image filter with band-pass like properties is advantageously applied on the texture and the search window.
  • the chosen size of the window can impact on the maximum speed of the vehicle at which this algorithm will operate successfully. That is, should the vehicle be traveling at a high speed, the feature will not be found within the chosen window size and location as easily. However, the size of this window is programmable, and is dependent on processing power available. A larger window size will allow for operation at greater speeds.
  • a predictive tracking can be applied using the speed of the vehicle gathered via CAN (Controller area network) bus. This allows to use a relatively small search window placed around the expected location of the feature in the image at each frame.
  • a geometrical shape is then identified 16. Such shape is build in a virtual image space by joining the located reference points between each other in image frame N and N + 1 and joining the same reference point from image N to N + 1 thus considering the respective trajectories of the reference points.
  • the obtained geometrical shape shows a shape which possibly deviates from a parallelogram with corners defined by the reference points from the image frame N and N + 1. Such deviation is directly related to a slight alteration of the rotation from the camera of the video system to be calibrated. Therefore, once a deviation is defined 17, a calibration of the camera which captured the two analyzed image frameN andN + 1 can be performed 18.
  • the online calibration method may be stop here 19 before being restarted at a later time or can be further performed on subsequently captured image frame 20 during some time interval.
  • Fig. 3 shows the combination of successive image frames captured by some camera of the vehicle video system in a virtual image space.
  • the showed example corresponds to a camera which exhibits no fisheye, and no rotation i.e. the camera is looking along the vehicle Z-axis at the ground plane with no rotation about the vehicle X, Y or Z-axes.
  • the two left most visible corners 21 and 22 correspond to the two features within the image frameN selected as reference points. Both reference points are selected within a specific window size 23 defined in the present case by 16 x 16 pixels while other sizes could be applied also.
  • the window size can be correlated to driving parameters e.g. to the speed and/or the steering curvature of the vehicle when performing the online calibration.
  • the two selected reference points are then tracked using some correlation search window 24 for each reference points 21 and 22.
  • the search window 24 includes respectively the initially selected reference points 21, 22 at image frameN and the location where the respective reference points 25, 26 shall be located at the successive image frameN + 1 .
  • the search window will be of a size possibly also related to driving parameters and to the time interval between two successive image frames.
  • the search window can also be defined according to the available processing power.
  • Fig. 3 shows the selected reference points 21, 22 tracked over a sequence of 5 image frames from image frameN tillN + 4 using a side or wing mirror camera therefore capturing not only the ground of the road but also part of the car 30.
  • the two reference points 21, 22 are joined between each other within an image frame, hereN, to obtain a straight line 31 in a virtual image space.
  • Such straight line is compared to the straight lines 32-35 obtained by joining the tracked reference points within successive image frame tillN+ 4. If the straight lines 31-35 are parallel (as it is shown on fig. 3) then no deviation about the X-axis from the ideal position can be measured.
  • a similar comparison is performed by joining this time the reference points 21, 22 from image frameN with the same respective reference points 25, 26 from successive image frame, hereN + 1 tillN + 4.
  • Two portions 36 and 37 of two straight lines are obtained in the virtual image space, the two portions 36, 37 being then compared between each other to determine a possible deviation from a parallel property.
  • Such a comparison is performed using the successive image frameN tillN + 4 allowing measuring a deviation as soon as captured (not the case on Fig. 3) about the Y-axis.
  • Fig. 4 shows the capture of successive image frame using the same camera as in Fig. 3 but this time showing some rotation of the camera.
  • the obtained geometrical shape deviates from a perfect parallelogram in the way that straight lines 41, 42 joining the two reference points 21 , 22 in image frameN and 25, 26 in image frameN + 1 are no more perfectly parallel.
  • This is an indication of a rotation from the camera about the X-axis.
  • the straight line 43, 44 joining the same reference points from image frameN to image frameN + 1 respectively 21, 25 and 22, 26 are no more parallel either.
  • This is an indication of a rotation from the camera about Y-axis.
  • the use of those indications allows to define an offset to apply to calibrate accordingly the camera to correct those deviations.
  • Usual vehicle video system uses fish-eye camera. Accordingly when tracking texture features from the ground using such fish-eye camera will return position in a distorted fish-eye space.
  • Fig. 5 shows such distorted fish-eye space with two references points 51 , 52 selected using some window 53 within image frame N.
  • a correlation search window 54 is defined for each reference points 51, 52 (only the one for reference point 51 is shown on fig. 5) to track the reference points from image frame N to image frame N + 1 till image frame N + 4.
  • the selected reference points Prior to using the geometric properties to estimate the necessary parameters for the online calibration, the selected reference points are transformed to their position in non-fisheye rectilinear image space (resulting in the geometric shape as shown on fig. 4) using previously determined fisheye parameters, possibly determined in factory based calibration.
  • the number of frames across which features are tracked is a programmable parameter and can be limited by the speed of the vehicle and/or other practical considerations.
  • Appropriate angles that describe the rotation of the cameras about the vehicle axes need to be determined such that the tracked features form the desired parallelogram shape. This can be done by using any solver or error minimization method, by
  • Figure 6 outlines the effect of the rotations of the used camera. The rotations described here assume a wing-mirror based camera. Figures 6 shows the selected reference points 60, 61 between the different frames along respectively track 62 and track 63. The error function is then composed using following results:
  • Fig. 7 shows the use of present invention for a rear- view camera.
  • the ground 67 of the road with characteristic markings 68 and borders 69 as well as the rear deck 70 of the vehicle is clearly visible.
  • Two reference points 71 , 72 are selected using some specific window size 73.
  • the selected reference points 71 , 72 are tracked in the subsequent image frame by using some correlation search window 74 to be localized 75, 76.
  • a geometrical shape can be obtained in a virtual image space in a same way as for the side or wing mirror camera by joining the four corners 71, 72, 75 and 76 while performing a projection to take into account the fact that the camera is capturing the road surface with some inclination.
  • Such projection is shown on the top of fig. 7 with the respective projected points 7 ⁇ , 72', 75'and 76' .
  • a deviation (not visible on fig. 7) from a parallelogram allows defining a correction function to be used as offset correction of the camera as for the wing or side mirror camera.
  • the tracks are parallel to the X-axis of the vehicle although this can be eliminated for by using the steering information or by having a range of predefined expected steering curvature.
  • the method can be used by vehicle located at any position on the vehicle, as long as an appropriate portion of the road texture is visible to the camera. If initial estimates of the camera extrinsic parameters are known (e.g. from vehicle mechanical data), they can be used as starting points for the calibration.
  • the speed and steering information possibly available on the vehicular network (e.g.
  • the online calibration is most suitable only when vehicle is moving straight ahead then it is important to continuously monitor steering angle CAN signal and invalidate the tracking when the steering angle is not zero (within acceptable tolerance). It is possible to adapt present invention to recognize if the vehicle or car is travelling straight without dependencies on the steering CAN signal by only analyzing the geometry of the tracks. As a step further, present invention can be adapted to work also when the vehicle comprising the video system to be calibrated is not moving straight ahead.
  • Steering angle information can be used when analyzing the trajectory covered by the reference points to obtain a geometrical shape close to a quadrilateral i.e. removing any bending of the trajectory coming from the known turning action allowing a comparison with the defined parallelogram.
  • an error function can be devised to take into account the fact that the quadrilaterals being the projection in the virtual space of the trajectories lie on concentric arcs (if no distortion like fish-eye distortion is present) when the vehicle is turning.
  • Such error function will take into account the unknown steering angle as well as possible unknown offsets. And this error function will have to be minimized to achieve some calibration of the online vehicle video system.
  • one or more aspects of the present invention can be included in an article of manufacture (e.g., one or more computer program products) having, for instance, computer usable media.
  • the media has embodied therein, for instance, computer readable program code means for providing and facilitating the capabilities of the present invention.
  • the article of manufacture can be included as a part of a computer system or sold separately.
  • at least one program storage device readable by a machine, tangibly embodying at least one program of instructions executable by the machine to perform the capabilities of the present invention can be provided.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

L'invention porte sur un procédé d'étalonnage en ligne d'un système vidéo de véhicule évalué à partir d'images individuelles d'une caméra contenant des caractéristiques présentes sur la route. Une partie d'une surface routière est capturée par la caméra dans une image individuelle. Une sélection d'au moins deux caractéristiques différentes dans l'image individuelle est effectuée, les caractéristiques sélectionnées tant choisies à titre de points de référence respectifs. Une séquence d'au moins une image individuelle supplémentaire est effectuée avec la caméra tout en localisant dans la nouvelle image les caractéristiques choisies à titre de points de référence devant être suivis. La trajectoire couverte par les points de référence durant un intervalle de conduite entre des séquences d'image est analysée dans un espace d'image virtuelle par identification d'une forme géométrique obtenue par jonction l'un à l'autre des points de référence localisés dans chaque image utilisée et considération des trajectoires respectives. Un écart de l'objet géométrique résultant par rapport à un parallélogramme dont les coins sont définis par les points de référence issus d'au moins deux images subséquentes est calculé pendant que tout écart mesuré est utilisé pour définir une correction de décalage de la caméra.
PCT/EP2011/055789 2011-04-13 2011-04-13 Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques Ceased WO2012139636A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2011/055789 WO2012139636A1 (fr) 2011-04-13 2011-04-13 Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2011/055789 WO2012139636A1 (fr) 2011-04-13 2011-04-13 Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques

Publications (1)

Publication Number Publication Date
WO2012139636A1 true WO2012139636A1 (fr) 2012-10-18

Family

ID=44625801

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2011/055789 Ceased WO2012139636A1 (fr) 2011-04-13 2011-04-13 Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques

Country Status (1)

Country Link
WO (1) WO2012139636A1 (fr)

Cited By (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9014904B2 (en) 2004-12-23 2015-04-21 Magna Electronics Inc. Driver assistance system for vehicle
TWI508021B (zh) * 2014-02-24 2015-11-11 Univ Kun Shan 雲端路況偵測系統
US9205776B2 (en) 2013-05-21 2015-12-08 Magna Electronics Inc. Vehicle vision system using kinematic model of vehicle motion
US9357208B2 (en) 2011-04-25 2016-05-31 Magna Electronics Inc. Method and system for dynamically calibrating vehicular cameras
WO2016087298A1 (fr) 2014-12-04 2016-06-09 Connaught Electronics Ltd. Étalonnage en ligne d'un système de caméra de véhicule à moteur
US9491451B2 (en) 2011-11-15 2016-11-08 Magna Electronics Inc. Calibration system and method for vehicular surround vision system
US9491450B2 (en) 2011-08-01 2016-11-08 Magna Electronic Inc. Vehicle camera alignment system
EP2736013A3 (fr) * 2012-11-21 2017-01-25 Fujitsu Limited Étalonnage des caméras multiples sur un objet bougeant
US9563951B2 (en) 2013-05-21 2017-02-07 Magna Electronics Inc. Vehicle vision system with targetless camera calibration
US9688200B2 (en) 2013-03-04 2017-06-27 Magna Electronics Inc. Calibration system and method for multi-camera vision system
WO2017116570A1 (fr) * 2015-12-28 2017-07-06 Intel Corporation Étalonnage automatique de caméra de véhicule en temps réel
US9723272B2 (en) 2012-10-05 2017-08-01 Magna Electronics Inc. Multi-camera image stitching calibration system
US9762880B2 (en) 2011-12-09 2017-09-12 Magna Electronics Inc. Vehicle vision system with customized display
DE102016104729A1 (de) * 2016-03-15 2017-09-21 Connaught Electronics Ltd. Verfahren zur extrinsischen Kalibrierung einer Kamera, Rechenvorrichtung, Fahrerassistenzsystem sowie Kraftfahrzeug
US9834153B2 (en) 2011-04-25 2017-12-05 Magna Electronics Inc. Method and system for dynamically calibrating vehicular cameras
CN107491753A (zh) * 2017-08-16 2017-12-19 电子科技大学 一种基于背景建模的违章停车检测方法
US9916660B2 (en) 2015-01-16 2018-03-13 Magna Electronics Inc. Vehicle vision system with calibration algorithm
JP2018136739A (ja) * 2017-02-22 2018-08-30 クラリオン株式会社 キャリブレーション装置
US10071687B2 (en) 2011-11-28 2018-09-11 Magna Electronics Inc. Vision system for vehicle
WO2018202464A1 (fr) 2017-05-03 2018-11-08 Connaught Electronics Ltd. Étalonnage d'un système de caméra de véhicule dans la direction longitudinale du véhicule ou dans la direction transversale du véhicule
GB2562898A (en) * 2017-04-12 2018-11-28 Bosch Gmbh Robert Method and device for calibrating a vehicle camera of a vehicle
US10179543B2 (en) 2013-02-27 2019-01-15 Magna Electronics Inc. Multi-camera dynamic top view vision system
WO2019012004A1 (fr) * 2017-07-12 2019-01-17 Connaught Electronics Ltd. Procédé de détermination d'incertitude spatiale dans des images d'une région environnementale d'un véhicule automobile, système d'aide à la conduite et véhicule automobile
WO2019021876A1 (fr) 2017-07-28 2019-01-31 クラリオン株式会社 Dispositif et procédé d'étalonnage de caméra embarquée
DE102017123226A1 (de) 2017-10-06 2019-04-11 Connaught Electronics Ltd. Verfahren zum Bestimmen einer kritischen Höhe eines vorausliegenden Streckenabschnitts für ein Fahrzeug, das ein Zugfahrzeug und einen Anhänger umfasst
US10304210B2 (en) * 2017-05-25 2019-05-28 GM Global Technology Operations LLC Method and apparatus for camera calibration
DE102018100667A1 (de) 2018-01-12 2019-07-18 Connaught Electronics Ltd. Computersichtvorfusion und räumlich-zeitliche Verfolgung
US10424081B2 (en) * 2014-05-14 2019-09-24 Volkswagen Aktiengesellschaft Method and apparatus for calibrating a camera system of a motor vehicle
CN111783597A (zh) * 2020-06-24 2020-10-16 中国第一汽车股份有限公司 行车轨迹线的标定方法、装置、计算机设备和存储介质
DE102019114404A1 (de) * 2019-05-29 2020-12-03 Connaught Electronics Ltd. Bilderfassungssystem
US10868974B2 (en) 2010-12-01 2020-12-15 Magna Electronics Inc. Method for determining alignment of vehicular cameras
US10946799B2 (en) 2015-04-21 2021-03-16 Magna Electronics Inc. Vehicle vision system with overlay calibration
EP3901913A1 (fr) * 2020-04-22 2021-10-27 Continental Automotive GmbH Étalonnage de caméra unique
US11877054B2 (en) 2011-09-21 2024-01-16 Magna Electronics Inc. Vehicular vision system using image data transmission and power supply via a coaxial cable
US12407912B2 (en) 2021-12-20 2025-09-02 Samsung Electronics Co., Ltd. Electronic apparatus and controlling method thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050163343A1 (en) * 2002-12-18 2005-07-28 Aisin Seiki Kabushiki Kaisha Movable body circumstance monitoring apparatus
US20090015675A1 (en) * 2007-07-09 2009-01-15 Sanyo Electric Co., Ltd. Driving Support System And Vehicle
WO2009027090A2 (fr) 2007-08-31 2009-03-05 Valeo Schalter Und Sensoren Gmbh Procédé et système d'étalonnage en ligne d'un système vidéo
US20090290032A1 (en) 2008-05-22 2009-11-26 Gm Global Technology Operations, Inc. Self calibration of extrinsic camera parameters for a vehicle camera

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050163343A1 (en) * 2002-12-18 2005-07-28 Aisin Seiki Kabushiki Kaisha Movable body circumstance monitoring apparatus
US20090015675A1 (en) * 2007-07-09 2009-01-15 Sanyo Electric Co., Ltd. Driving Support System And Vehicle
WO2009027090A2 (fr) 2007-08-31 2009-03-05 Valeo Schalter Und Sensoren Gmbh Procédé et système d'étalonnage en ligne d'un système vidéo
US20090290032A1 (en) 2008-05-22 2009-11-26 Gm Global Technology Operations, Inc. Self calibration of extrinsic camera parameters for a vehicle camera

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
FRIEL M ET AL: "Automatic calibration of fish-eye cameras from automotive video sequences", IET INTELLIGENT TRANSPORT SYSTEMS,, vol. 4, no. 2, 4 June 2010 (2010-06-04), pages 136 - 148, XP006035033, ISSN: 1751-9578, DOI: 10.1049/IET-ITS:20090052 *
MICHAEL MIKSCH ET AL: "Automatic extrinsic camera self-calibration based on homography and epipolar geometry", INTELLIGENT VEHICLES SYMPOSIUM (IV), 2010 IEEE, IEEE, PISCATAWAY, NJ, USA, 21 June 2010 (2010-06-21), pages 832 - 839, XP031732209, ISBN: 978-1-4244-7866-8 *

Cited By (93)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11308720B2 (en) 2004-12-23 2022-04-19 Magna Electronics Inc. Vehicular imaging system
US9940528B2 (en) 2004-12-23 2018-04-10 Magna Electronics Inc. Driver assistance system for vehicle
US9193303B2 (en) 2004-12-23 2015-11-24 Magna Electronics Inc. Driver assistance system for vehicle
US10509972B2 (en) 2004-12-23 2019-12-17 Magna Electronics Inc. Vehicular vision system
US9014904B2 (en) 2004-12-23 2015-04-21 Magna Electronics Inc. Driver assistance system for vehicle
US12118806B2 (en) 2004-12-23 2024-10-15 Magna Electronics Inc. Vehicular imaging system
US11553140B2 (en) 2010-12-01 2023-01-10 Magna Electronics Inc. Vehicular vision system with multiple cameras
US10868974B2 (en) 2010-12-01 2020-12-15 Magna Electronics Inc. Method for determining alignment of vehicular cameras
US12244957B2 (en) 2010-12-01 2025-03-04 Magna Electronics Inc. Vehicular vision system with multiple cameras
US11007934B2 (en) 2011-04-25 2021-05-18 Magna Electronics Inc. Method for dynamically calibrating a vehicular camera
US10202077B2 (en) 2011-04-25 2019-02-12 Magna Electronics Inc. Method for dynamically calibrating vehicular cameras
US9357208B2 (en) 2011-04-25 2016-05-31 Magna Electronics Inc. Method and system for dynamically calibrating vehicular cameras
US10919458B2 (en) 2011-04-25 2021-02-16 Magna Electronics Inc. Method and system for calibrating vehicular cameras
US11554717B2 (en) 2011-04-25 2023-01-17 Magna Electronics Inc. Vehicular vision system that dynamically calibrates a vehicular camera
US10654423B2 (en) 2011-04-25 2020-05-19 Magna Electronics Inc. Method and system for dynamically ascertaining alignment of vehicular cameras
US9834153B2 (en) 2011-04-25 2017-12-05 Magna Electronics Inc. Method and system for dynamically calibrating vehicular cameras
US10640041B2 (en) 2011-04-25 2020-05-05 Magna Electronics Inc. Method for dynamically calibrating vehicular cameras
US9491450B2 (en) 2011-08-01 2016-11-08 Magna Electronic Inc. Vehicle camera alignment system
US11877054B2 (en) 2011-09-21 2024-01-16 Magna Electronics Inc. Vehicular vision system using image data transmission and power supply via a coaxial cable
US12143712B2 (en) 2011-09-21 2024-11-12 Magna Electronics Inc. Vehicular vision system using image data transmission and power supply via a coaxial cable
US9491451B2 (en) 2011-11-15 2016-11-08 Magna Electronics Inc. Calibration system and method for vehicular surround vision system
US10264249B2 (en) 2011-11-15 2019-04-16 Magna Electronics Inc. Calibration system and method for vehicular surround vision system
US11305691B2 (en) 2011-11-28 2022-04-19 Magna Electronics Inc. Vehicular vision system
US11142123B2 (en) 2011-11-28 2021-10-12 Magna Electronics Inc. Multi-camera vehicular vision system
US10099614B2 (en) 2011-11-28 2018-10-16 Magna Electronics Inc. Vision system for vehicle
US12541861B2 (en) 2011-11-28 2026-02-03 Magna Electronics Inc. Vehicular vision system
US11787338B2 (en) 2011-11-28 2023-10-17 Magna Electronics Inc. Vehicular vision system
US10071687B2 (en) 2011-11-28 2018-09-11 Magna Electronics Inc. Vision system for vehicle
US11634073B2 (en) 2011-11-28 2023-04-25 Magna Electronics Inc. Multi-camera vehicular vision system
US10640040B2 (en) 2011-11-28 2020-05-05 Magna Electronics Inc. Vision system for vehicle
US12100166B2 (en) 2011-11-28 2024-09-24 Magna Electronics Inc. Vehicular vision system
US11082678B2 (en) 2011-12-09 2021-08-03 Magna Electronics Inc. Vehicular vision system with customized display
US10129518B2 (en) 2011-12-09 2018-11-13 Magna Electronics Inc. Vehicle vision system with customized display
US11689703B2 (en) 2011-12-09 2023-06-27 Magna Electronics Inc. Vehicular vision system with customized display
US9762880B2 (en) 2011-12-09 2017-09-12 Magna Electronics Inc. Vehicle vision system with customized display
US10542244B2 (en) 2011-12-09 2020-01-21 Magna Electronics Inc. Vehicle vision system with customized display
US11265514B2 (en) 2012-10-05 2022-03-01 Magna Electronics Inc. Multi-camera calibration method for a vehicle moving along a vehicle assembly line
US9723272B2 (en) 2012-10-05 2017-08-01 Magna Electronics Inc. Multi-camera image stitching calibration system
US10284818B2 (en) 2012-10-05 2019-05-07 Magna Electronics Inc. Multi-camera image stitching calibration system
US10904489B2 (en) 2012-10-05 2021-01-26 Magna Electronics Inc. Multi-camera calibration method for a vehicle moving along a vehicle assembly line
EP2736013A3 (fr) * 2012-11-21 2017-01-25 Fujitsu Limited Étalonnage des caméras multiples sur un objet bougeant
US10179543B2 (en) 2013-02-27 2019-01-15 Magna Electronics Inc. Multi-camera dynamic top view vision system
US10486596B2 (en) 2013-02-27 2019-11-26 Magna Electronics Inc. Multi-camera dynamic top view vision system
US10780827B2 (en) 2013-02-27 2020-09-22 Magna Electronics Inc. Method for stitching images captured by multiple vehicular cameras
US11572015B2 (en) 2013-02-27 2023-02-07 Magna Electronics Inc. Multi-camera vehicular vision system with graphic overlay
US11192500B2 (en) 2013-02-27 2021-12-07 Magna Electronics Inc. Method for stitching image data captured by multiple vehicular cameras
US9688200B2 (en) 2013-03-04 2017-06-27 Magna Electronics Inc. Calibration system and method for multi-camera vision system
US9979957B2 (en) 2013-05-21 2018-05-22 Magna Electronics Inc. Vehicle vision system with targetless camera calibration
US9701246B2 (en) 2013-05-21 2017-07-11 Magna Electronics Inc. Vehicle vision system using kinematic model of vehicle motion
US9205776B2 (en) 2013-05-21 2015-12-08 Magna Electronics Inc. Vehicle vision system using kinematic model of vehicle motion
US9563951B2 (en) 2013-05-21 2017-02-07 Magna Electronics Inc. Vehicle vision system with targetless camera calibration
US11109018B2 (en) 2013-05-21 2021-08-31 Magna Electronics Inc. Targetless vehicular camera misalignment correction method
US10780826B2 (en) 2013-05-21 2020-09-22 Magna Electronics Inc. Method for determining misalignment of a vehicular camera
US11447070B2 (en) 2013-05-21 2022-09-20 Magna Electronics Inc. Method for determining misalignment of a vehicular camera
US11597319B2 (en) 2013-05-21 2023-03-07 Magna Electronics Inc. Targetless vehicular camera calibration system
US11794647B2 (en) 2013-05-21 2023-10-24 Magna Electronics Inc. Vehicular vision system having a plurality of cameras
US10567748B2 (en) 2013-05-21 2020-02-18 Magna Electronics Inc. Targetless vehicular camera calibration method
US11919449B2 (en) 2013-05-21 2024-03-05 Magna Electronics Inc. Targetless vehicular camera calibration system
US10266115B2 (en) 2013-05-21 2019-04-23 Magna Electronics Inc. Vehicle vision system using kinematic model of vehicle motion
TWI508021B (zh) * 2014-02-24 2015-11-11 Univ Kun Shan 雲端路況偵測系統
US10424081B2 (en) * 2014-05-14 2019-09-24 Volkswagen Aktiengesellschaft Method and apparatus for calibrating a camera system of a motor vehicle
US10163226B2 (en) 2014-12-04 2018-12-25 Connaught Electronics Ltd. Online calibration of a motor vehicle camera system
DE102014117888A1 (de) 2014-12-04 2016-10-13 Connaught Electronics Ltd. Online-Kalibrierung eines Kraftfahrzeug-Kamerasystems
WO2016087298A1 (fr) 2014-12-04 2016-06-09 Connaught Electronics Ltd. Étalonnage en ligne d'un système de caméra de véhicule à moteur
US9916660B2 (en) 2015-01-16 2018-03-13 Magna Electronics Inc. Vehicle vision system with calibration algorithm
US10235775B2 (en) 2015-01-16 2019-03-19 Magna Electronics Inc. Vehicle vision system with calibration algorithm
US10946799B2 (en) 2015-04-21 2021-03-16 Magna Electronics Inc. Vehicle vision system with overlay calibration
US11535154B2 (en) 2015-04-21 2022-12-27 Magna Electronics Inc. Method for calibrating a vehicular vision system
US10694175B2 (en) 2015-12-28 2020-06-23 Intel Corporation Real-time automatic vehicle camera calibration
WO2017116570A1 (fr) * 2015-12-28 2017-07-06 Intel Corporation Étalonnage automatique de caméra de véhicule en temps réel
DE102016104729A1 (de) * 2016-03-15 2017-09-21 Connaught Electronics Ltd. Verfahren zur extrinsischen Kalibrierung einer Kamera, Rechenvorrichtung, Fahrerassistenzsystem sowie Kraftfahrzeug
JP2018136739A (ja) * 2017-02-22 2018-08-30 クラリオン株式会社 キャリブレーション装置
GB2603731B (en) * 2017-04-12 2022-11-23 Bosch Gmbh Robert Method and device for calibrating a vehicle camera of a vehicle
GB2603731A (en) * 2017-04-12 2022-08-10 Bosch Gmbh Robert Method and device for calibrating a vehicle camera of a vehicle
GB2562898A (en) * 2017-04-12 2018-11-28 Bosch Gmbh Robert Method and device for calibrating a vehicle camera of a vehicle
US10438374B2 (en) 2017-04-12 2019-10-08 Robert Bosch Gmbh Method and device for calibrating a vehicle camera of a vehicle
WO2018202464A1 (fr) 2017-05-03 2018-11-08 Connaught Electronics Ltd. Étalonnage d'un système de caméra de véhicule dans la direction longitudinale du véhicule ou dans la direction transversale du véhicule
US10304210B2 (en) * 2017-05-25 2019-05-28 GM Global Technology Operations LLC Method and apparatus for camera calibration
WO2019012004A1 (fr) * 2017-07-12 2019-01-17 Connaught Electronics Ltd. Procédé de détermination d'incertitude spatiale dans des images d'une région environnementale d'un véhicule automobile, système d'aide à la conduite et véhicule automobile
US11373333B2 (en) 2017-07-28 2022-06-28 Clarion Co., Ltd. Calibration apparatus and method for in-vehicle camera
WO2019021876A1 (fr) 2017-07-28 2019-01-31 クラリオン株式会社 Dispositif et procédé d'étalonnage de caméra embarquée
CN107491753A (zh) * 2017-08-16 2017-12-19 电子科技大学 一种基于背景建模的违章停车检测方法
DE102017123226A1 (de) 2017-10-06 2019-04-11 Connaught Electronics Ltd. Verfahren zum Bestimmen einer kritischen Höhe eines vorausliegenden Streckenabschnitts für ein Fahrzeug, das ein Zugfahrzeug und einen Anhänger umfasst
WO2019137912A1 (fr) 2018-01-12 2019-07-18 Connaught Electronics Ltd. Pré-fusion de vision artificielle et suivi spatio-temporel
DE102018100667A1 (de) 2018-01-12 2019-07-18 Connaught Electronics Ltd. Computersichtvorfusion und räumlich-zeitliche Verfolgung
DE102019114404A1 (de) * 2019-05-29 2020-12-03 Connaught Electronics Ltd. Bilderfassungssystem
WO2020239457A1 (fr) 2019-05-29 2020-12-03 Connaught Electronics Ltd. Système d'acquisition d'images
EP3901913A1 (fr) * 2020-04-22 2021-10-27 Continental Automotive GmbH Étalonnage de caméra unique
WO2021213879A1 (fr) * 2020-04-22 2021-10-28 Continental Automotive Gmbh Calibration de caméra unique
US12505576B2 (en) 2020-04-22 2025-12-23 Continental Autonomous Mobility Germany GmbH System and method for calibrating a camera in a vehicle
CN111783597A (zh) * 2020-06-24 2020-10-16 中国第一汽车股份有限公司 行车轨迹线的标定方法、装置、计算机设备和存储介质
CN111783597B (zh) * 2020-06-24 2022-12-13 中国第一汽车股份有限公司 行车轨迹线的标定方法、装置、计算机设备和存储介质
US12407912B2 (en) 2021-12-20 2025-09-02 Samsung Electronics Co., Ltd. Electronic apparatus and controlling method thereof

Similar Documents

Publication Publication Date Title
WO2012139636A1 (fr) Étalonnage de caméra de véhicule en ligne sur la base d'un suivi de texture de surface routière et de propriétés géométriques
CN109155066B (zh) 用于机动车辆的环境区域的两个图像之间的运动估计的方法，计算装置，驾驶员辅助系统以及机动车辆
US10079975B2 (en) Image distortion correction of a camera with a rolling shutter
JP6126809B2 (ja) エゴモーション概算システムおよび方法
JP6107081B2 (ja) 画像処理装置、画像処理方法及びプログラム
US8059887B2 (en) System and method for providing mobile range sensing
US20100020178A1 (en) Calibrating a camera system
US20140037136A1 (en) Method and System for Determining Poses of Vehicle-Mounted Cameras for In-Road Obstacle Detection
WO2012143036A1 (fr) Étalonnage de caméra de véhicule en ligne sur la base de la continuité d'éléments
KR20120126016A (ko) 서라운드뷰 시스템 카메라 자동 보정 전용 외부 변수
CN104883553B (zh) 视差值导出设备及方法、装备控制系统、可移动装置
JP2014074632A (ja) 車載ステレオカメラの校正装置及び校正方法
KR20170139548A (ko) 이미지 라인들로부터의 카메라 외부 파라미터들 추정
KR20230000585A (ko) 센서 캘리브레이션 방법 및 장치
JP2010181919A (ja) 三次元形状特定装置、三次元形状特定方法、三次元形状特定プログラム
CN108629756A (zh) 一种Kinect v2深度图像无效点修复方法
KR20210077792A (ko) 동적 입체 캘리브레이션을 위한 시스템 및 방법
Fraundorfer et al. A constricted bundle adjustment parameterization for relative scale estimation in visual odometry
JP6396499B2 (ja) 三次元情報の規模測定
Ohashi et al. Fisheye stereo camera using equirectangular images
CN107145828B (zh) 车辆全景图像处理方法及装置
CN109074480A (zh) 用于检测机动车辆的环境区域的图像中的滚动快门效应的方法、计算装置、驾驶员辅助系统以及机动车辆
JP2005217883A (ja) ステレオ画像を用いた道路平面領域並びに障害物検出方法
JP2023520189A (ja) 単一カメラ較正
CN111316322A (zh) 路面区域检测装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11714049

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11714049

Country of ref document: EP

Kind code of ref document: A1