WO2021134219A1 - 参数标定方法及装置 - Google Patents

参数标定方法及装置 Download PDF

Info

Publication number
WO2021134219A1
WO2021134219A1 PCT/CN2019/129932 CN2019129932W WO2021134219A1 WO 2021134219 A1 WO2021134219 A1 WO 2021134219A1 CN 2019129932 W CN2019129932 W CN 2019129932W WO 2021134219 A1 WO2021134219 A1 WO 2021134219A1
Authority
WO
WIPO (PCT)
Prior art keywords
calibration
camera device
camera
instruction
calibration pattern
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/129932
Other languages
English (en)
French (fr)
Inventor
陈亦伦
李涵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP19958494.7A priority Critical patent/EP4071713B1/en
Priority to CN201980022070.8A priority patent/CN113330487B/zh
Priority to PCT/CN2019/129932 priority patent/WO2021134219A1/zh
Publication of WO2021134219A1 publication Critical patent/WO2021134219A1/zh
Priority to US17/854,506 priority patent/US12307715B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods

Definitions

  • This application relates to the field of optical technology, and in particular to a parameter calibration method and device.
  • the current camera calibration method is to fix the camera to be calibrated, and then manually hold the calibration board in the field of view of the camera, move the calibration board to change the angle and position of the calibration board, so that the camera can take pictures of the calibration board at different angles and positions. Image; Finally, the internal and external parameters of the camera are solved simultaneously by the real position of the feature points of the calibration plate in the three-dimensional world and the position in the image imaged by the camera.
  • the current calibration board is printed on various types of plates, and its flatness and printing accuracy will affect the accuracy of the world coordinate system.
  • the accuracy of the world coordinate system will be low, which will affect the calibration accuracy.
  • the way of manually moving the calibration plate makes it impossible to accurately determine the position of the calibration plate, which will also affect the calibration accuracy.
  • This application provides a parameter calibration method and device to improve the calibration accuracy of the camera device.
  • this application provides a parameter calibration method.
  • the method includes: generating a first calibration pattern; sending a first instruction, the first instruction being used to instruct at least one set of pose parameters of at least one camera device in a first time period; sending a second instruction to at least one camera device, and second The instruction is used to instruct at least one of the shooting timing and the number of shootings of the at least one camera device in the first time period; obtain at least one shooting result from the at least one camera device; determine according to part or all of the shooting results in the at least one shooting result At least one internal parameter of the at least one imaging device, and any imaging result in part or all of the imaging results includes part or all of the first calibration pattern.
  • the parameter calibration method provided in this application can be applied to the calibration of the internal parameters and external parameters of the camera device.
  • This method accurately adjusts the camera by changing the pose parameters of the camera device under the condition that the position of the calibration pattern is fixed.
  • the position and angle of the device relative to the calibration pattern enable the camera device to accurately photograph the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result.
  • the shooting result can be compared with the pixel coordinate system and the world coordinate system.
  • the precise correspondence of the coordinates makes the internal parameters and external parameters obtained by the corresponding calculation more accurate, and improves the calibration accuracy.
  • sending the first instruction includes: sending the first instruction to the motion device, the motion device carries at least one camera device, and at least one set of pose parameters is used for the motion device to control the pose of the at least one camera device. Therefore, the movement device can accurately control the pose of the camera device according to the pose parameters, and can realize the reproduction of the pose, which is beneficial to improve the calibration accuracy.
  • the shooting result includes at least one photo and/or at least one piece of video taken in response to the second instruction from at least one camera device.
  • the image participating in the calculation of the internal parameters of the camera device can be selected from the photo or from the video frame of the video, in a flexible manner.
  • sending the first instruction includes: sending a plurality of first instructions in a first time period; sending a second instruction to at least one camera device includes: sending a plurality of first instructions to at least one camera device in the first time period.
  • the second instruction wherein, multiple first instructions and multiple second instructions are sent alternately.
  • the first instruction and the second instruction can instruct the camera device to take different photos or videos in different poses according to actual needs, for example, make the first calibration pattern appear in each position of the camera device's field of view, thereby improving the calibration accuracy.
  • At least one of the internal parameters has a corresponding error threshold.
  • the method further includes: adjusting the first calibration pattern to the first calibration pattern. Two calibration patterns, and/or adjusting the number of feature points corresponding to each field of view of the at least one camera device extracted from the at least one shooting result, and/or adjusting the image quality of the first calibration pattern. Therefore, the present application can perform multiple calibrations on the camera device, so that the internal parameter matrix and the external parameter matrix of the camera device are continuously updated iteratively, and the calibration accuracy is continuously improved until the value of the calibration accuracy is less than the corresponding threshold.
  • the number and/or size of the markers in the first calibration pattern and the second calibration pattern are different.
  • adjusting the number of feature points corresponding to each field of view of the at least one camera device extracted from the at least one shooting result includes: determining at least one subregion included in the first calibration pattern, and combining the feature points of the at least one subregion The quantity is adjusted uniformly. As a result, the number of feature points in each sub-region is unified to improve the calibration accuracy.
  • the first calibration pattern is a preset gray scale pattern
  • the internal parameters include at least one white balance parameter of the camera device.
  • the first calibration pattern is a preset monochrome pattern
  • the internal parameter includes a halo parameter of at least one camera device.
  • the first calibration pattern is a preset color chart pattern containing multiple standard colors
  • the internal parameters include color difference parameters of at least one camera device.
  • the first calibration pattern is a preset target pattern containing lines of different angles
  • the internal parameters include at least one modulation transfer function parameter of the camera device.
  • this application provides a parameter calibration device.
  • the device includes: a generating unit for generating a first calibration pattern; a first sending unit for sending a first instruction, the first instruction for instructing at least one set of pose parameters of at least one camera device in a first time period;
  • the second sending unit is used to send a second instruction to the camera device, the second instruction is used to instruct at least one of the shooting timing and the number of times of shooting of the at least one camera device in the first time period;
  • the obtaining unit is used to obtain information from at least one At least one shooting result of the camera device;
  • a processing unit configured to determine at least one internal parameter of the at least one camera device according to part or all of the shooting results in the at least one shooting result, and any camera result in the part or all of the camera results includes the first Part or all of the calibration pattern.
  • the parameter calibration device provided in this application can be applied to the calibration of the internal parameters and external parameters of the camera device.
  • This method accurately adjusts the camera by changing the pose parameters of the camera device under the condition that the position of the calibration pattern is fixed.
  • the position and angle of the device relative to the calibration pattern enable the camera device to accurately photograph the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result.
  • the shooting result can be compared with the pixel coordinate system and the world coordinate system.
  • the precise correspondence of the coordinates makes the internal parameters and external parameters obtained by the corresponding calculation more accurate, and improves the calibration accuracy.
  • the first sending unit is specifically configured to send a first instruction to the sports device
  • the sports device carries at least one camera device
  • at least one set of pose parameters is used for the sports device to control the pose of the at least one camera parameter. Therefore, the movement device can accurately control the pose of the camera device according to the pose parameters, and can realize the reproduction of the pose, which is beneficial to improve the calibration accuracy.
  • the shooting result includes at least one photo and/or at least one piece of video taken in response to the second instruction from at least one camera device.
  • the image participating in the calculation of the internal parameters of the camera device can be selected from the photo or from the video frame of the video, in a flexible manner.
  • the first sending unit is configured to send a plurality of first instructions in a first period; the second sending unit is configured to send a plurality of second instructions to at least one camera device in the first period; wherein , Multiple first instructions and multiple second instructions are sent alternately. Therefore, the first instruction and the second instruction can instruct the camera device to take different photos or videos in different poses according to requirements, for example, make the first calibration pattern appear in each position of the camera's field of view, thereby improving the calibration accuracy.
  • At least one of the internal parameters has a corresponding error threshold; the processing unit is used to determine whether the repetition accuracy of at least one of the internal parameters is greater than the corresponding error threshold; the generating unit is used to repeat When the accuracy is greater than the corresponding error threshold, the first calibration pattern is adjusted to the second calibration pattern, and/or the image quality of the first calibration pattern is adjusted; the processing unit is configured to adjust from at least at least the repetition accuracy to the corresponding error threshold.
  • the number and/or size of the markers in the first calibration pattern and the second calibration pattern are different.
  • the processing unit is used to determine at least one sub-region included in the first calibration pattern, and to uniformly adjust the number of feature points of the at least one sub-region. As a result, the number of feature points in each sub-region is unified to improve the calibration accuracy.
  • the first calibration pattern is a preset gray scale pattern
  • the internal parameters include at least one white balance parameter of the camera device.
  • the first calibration pattern is a preset monochrome pattern
  • the internal parameter includes a halo parameter of at least one camera device.
  • the first calibration pattern is a preset color chart pattern containing multiple standard colors
  • the internal parameters include color difference parameters of at least one camera device.
  • the first calibration pattern is a preset target pattern containing lines of different angles
  • the internal parameters include at least one modulation transfer function parameter of the camera device.
  • this application provides a parameter calibration device.
  • the device includes: a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, when the processor executes the computer instructions, the device executes the first aspect and any implementation thereof The method in the way.
  • this application provides a parameter calibration system.
  • the system includes: a control device for generating a first calibration pattern; a display screen, coupled to the control device, for displaying the first calibration pattern; a motion device for carrying at least one camera device so that the field of view of the at least one camera device faces The display screen, the movement device is coupled to the control device; the control device, is also used to send a first instruction to the movement device, the first instruction is used to instruct at least one set of pose parameters of the at least one camera device in the first time period; the movement device, It is also used to control the pose of at least one camera device according to at least one set of pose parameters; the control device is also used to send a second instruction to at least one camera device, and the second instruction is used to instruct the at least one camera device to be in the first time period.
  • this application also provides a computer storage medium.
  • the computer storage medium computer instruction when the computer instruction runs on the control device, causes the control device to execute the method in the above first aspect and any implementation manner thereof.
  • the present application also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the method in the first aspect and any of its implementations.
  • the present application also provides a chip system that includes a processor for supporting the above-mentioned device or device to implement the functions involved in the above-mentioned first aspect and any of its implementations, for example, generating or processing the above-mentioned The information involved in the method.
  • Figure 1 is a schematic diagram of the basic structure of the camera
  • Figure 2 is a schematic diagram of an object mapping from a three-dimensional scene to a two-dimensional image
  • Figure 3 is a schematic diagram of the conversion relationship of each coordinate system
  • Figure 4 is a schematic diagram of the distortion of the camera
  • Figure 5 is a schematic diagram of a camera calibration method
  • FIG. 6 is a schematic diagram of some calibration boards shown in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a hardware system for parameter calibration provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a hardware system for parameter calibration provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a control device shown in an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of an exercise device shown in an embodiment of the present application.
  • FIG. 11 is a flowchart of a parameter calibration method provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of adjusting the pose of a camera device according to an embodiment of the present application.
  • FIG. 13 is a schematic diagram of adjusting the pose of a camera device provided by an embodiment of the present application.
  • FIG. 14 is a schematic diagram of shooting by a camera device provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of shooting by a camera device provided by an embodiment of the present application.
  • FIG. 16 is a schematic diagram of shooting by a camera device provided by an embodiment of the present application.
  • FIG. 17 is a schematic diagram of shooting by a camera device provided by an embodiment of the present application.
  • Figure 18 is a schematic diagram of feature points
  • Figure 19 is a schematic diagram of the control device adjusting the calibration pattern
  • Figure 20 is a schematic diagram of the control device adjusting the calibration pattern
  • Figure 21 is a schematic diagram of the control device adjusting the calibration pattern
  • Figure 22 is a schematic diagram of other calibration patterns provided by an embodiment of the present application.
  • FIG. 23 is a schematic diagram of a parameter calibration device provided by an embodiment of the present application.
  • machine vision/computer vision cameras are usually used to image objects and scenes in the objective world, so as to reconstruct the objective objects and scenes in three dimensions based on the imaging of the camera, so as to realize the three-dimensional recognition of the objective world by the machine.
  • Machine vision/computer vision technology can be applied to virtual reality, advanced driving assistance systems, drone cruise systems, visual inspection, biometric recognition, distance measurement, three-dimensional reconstruction and other fields.
  • machine vision/computer vision technology has been continuously developed.
  • FIG. 1 is a schematic diagram of the basic structure of the camera.
  • the camera includes an optical lens 10 and a photoelectric sensor 20.
  • the optical lens 10 is used to project the optical image of the space object onto the photoelectric sensor 20, and the photoelectric sensor 20 is used to convert the optical signal projected by the optical lens 10 into an electrical signal, and finally form an image by processing of other devices such as an image signal sensor.
  • the photosensor 20 may include a charge-coupled device (CCD) or a complementary metal oxide semiconductor active pixel sensor (CMOS active pixel sensor).
  • CCD charge-coupled device
  • CMOS active pixel sensor complementary metal oxide semiconductor active pixel sensor
  • the objects in the real space are three-dimensional, and the objects in the corresponding images are two-dimensional. Therefore, from the objects in the three-dimensional scene to the corresponding two-dimensional images, it can be considered that there is a three-dimensional
  • the mapping model makes a three-dimensional to two-dimensional or two-dimensional to three-dimensional conversion between the objects in the three-dimensional scene and their corresponding images. Then, it is easy to understand that when the camera shoots an image of an object in a three-dimensional scene, the camera can be considered as the mapping model, and the camera parameters are the parameters of the mapping model. Therefore, only the camera parameters are required, and the world coordinates of the object in space can be deduced from the pixel coordinates of the object in the image, thereby realizing functions such as vision detection, biometric recognition, distance measurement, and three-dimensional reconstruction.
  • the internal parameters of the camera refers to the parameters related to the characteristics of the camera itself, which can be determined by the installation position of the optical lens and photoelectric sensor in the camera module.
  • the camera internal parameters may include at least one of the focal length, pixel size, optical distortion, white balance, resolution, contrast, vignetting, and vignetting.
  • the external parameters of the camera may include at least one of the following:
  • World coordinate system Also known as the measurement coordinate system, it is a three-dimensional rectangular coordinate system that can describe the spatial position of the camera and the object to be measured in the objective world based on it.
  • the position of the world coordinate system can be freely determined according to the actual situation.
  • the external parameters of the camera and the internal parameters of the camera are collectively referred to as camera parameters.
  • Camera coordinate system is a three-dimensional rectangular coordinate system, the origin is located at the optical center of the optical lens (wherein, the optical center refers to the center point of the lens of the optical lens), the x and y axes are respectively the image of the camera The two sides of the plane are parallel, and the z-axis is the optical axis of the optical lens, which is perpendicular to the imaging plane.
  • Pixel coordinate system It is a two-dimensional rectangular coordinate system that reflects the arrangement of pixels in the camera's CCD/CMOS chip.
  • the origin of the pixel coordinate system is located at the upper left corner of the image, and the u-axis and v-axis are respectively parallel to the two sides of the image plane.
  • the unit of the coordinate axis in the pixel coordinate system is pixel (integer).
  • Image coordinate system is a two-dimensional rectangular coordinate system, the unit of its coordinate axis is usually millimeter (mm), the origin is the intersection of the optical axis of the camera and the image plane (called the principal point), that is, the center of the image Point, X-axis, Y-axis are parallel to the u-axis and v-axis of the pixel coordinate system, respectively.
  • X-axis, Y-axis are parallel to the u-axis and v-axis of the pixel coordinate system, respectively.
  • There is a translation relationship between the image coordinate system and the pixel coordinate system that is, it can be obtained by translation.
  • Fig. 3 is a schematic diagram of the conversion relationship of each coordinate system. The following describes the conversion relationship between the various coordinate systems with reference to FIG. 3, and then specifically describes the camera parameters that need to be calibrated in the embodiment of the present application:
  • R is the 3*3 rotation matrix
  • t is the 3*1 translation vector
  • (x c , y c , z c , 1) T is the coordinates of the camera coordinate system
  • (x w , y w , z w , 1) T is the coordinate of the world coordinate system. Therefore, the conversion from the world coordinate system to the camera coordinate system actually includes a rotation process and a translation process.
  • (X, Y, 1) T is the coordinate of the image coordinate system
  • (u, v, 1) T is the coordinate of the image coordinate system
  • dX, dY are the physical size of the pixel in the X and Y axis directions
  • u 0 and v 0 are the coordinates of the optical center.
  • connection line between any point P in space and the optical center C of the camera is CP
  • intersection point p between CP and the image plane is the projection of the spatial point P on the image plane.
  • the projection can be realized by the following matrix transformation:
  • S is the scale factor (S is not 0)
  • f is the effective focal length of the camera (the distance from the optical center to the image plane)
  • (x c , y c , z c , 1) T is the spatial point P in the camera coordinate system
  • the coordinates in (X, Y, 1) T is the coordinate of the image point p in the image coordinate system.
  • the conversion from the world coordinate system to the pixel coordinate system can be obtained by the following matrix transformation:
  • M is the parameter matrix of the camera, that is, the camera parameters; where M 1 is the internal parameter matrix of the camera, that is, the internal parameters of the camera; M 2 is the external parameter matrix of the camera, that is, the external parameters of the camera.
  • the camera calibration includes solving the above-mentioned internal parameter matrix M 1 and/or external parameter matrix M 2 .
  • the pixel size 1/dX and 1/dY of the camera, the focal length f of the camera, the optical center position of the camera (v 0 , u 0 ), the rotation matrix R and the translation variable and other parameters are obtained by solving.
  • distortion can be specifically divided into tangential distortion and radial distortion.
  • the radial distortion comes from the lens of the camera itself, and it comes from the influence of "fisheye” and "barrel shape", which is more obvious at the edge of the lens imaging;
  • tangential distortion is a defect in lens manufacturing or a deviation in installation makes the lens itself It is not parallel to the image plane.
  • the matrix transformation shown in 1 to 3 above can also be extended to:
  • k 1 , k 2 , k 3 , k 4 , k 5 and k 6 are radial distortion coefficients, and p 1 and p 2 -bit tangential distortion coefficients. Therefore, the process of camera calibration can also determine the distortion parameters of the camera .
  • a camera calibration method is shown in Figure 5.
  • the calibration of industrial-grade cameras generally requires the use of a large robotic arm.
  • the calibration board is hoisted on the large robotic arm, and the angle and position of the calibration board are changed through the swing arm action of the robotic arm.
  • the swing arm and the posture of the swing arm Stabilization takes a lot of time, resulting in low camera calibration efficiency and difficulty in large-scale application.
  • Fig. 6 is a schematic diagram of some calibration plates shown in an embodiment of the present application.
  • the checkerboard pattern, dot pattern, two-dimensional code pattern or other specific patterns can be printed on the calibration board.
  • any of the pattern units such as any one of the checkerboard patterns Any dot in the square or dot pattern can be called a marker.
  • Some feature points can be determined from the markers, which can be used to calculate camera parameters.
  • other forms of calibration boards will be used, for example: the white balance parameters of the camera are calibrated through the calibration board containing the color card, and the uniform diffuse reflection material is used.
  • the calibration plate is matched with a uniformly illuminated light source to calibrate the vignetting parameters of the camera, and the optical resolution modulation transfer function (MTF) of the camera is calibrated through the calibration plate containing a special design pattern.
  • MTF optical resolution modulation transfer function
  • the flatness and printing accuracy of the calibration plate will affect the accuracy of the world coordinate system.
  • the accuracy of the world coordinate system will be low, which will affect the calibration. Accuracy.
  • the way of manually moving the calibration plate makes it impossible to accurately determine the position of the calibration plate, which will also affect the calibration accuracy.
  • To calibrate different parameters it is necessary to print calibration boards with different patterns on different plates. The calibration boards are independent of each other, and the lighting conditions and shapes and sizes of the calibration boards when calibrating different camera parameters are not the same, making the calibration board impossible to achieve Reuse.
  • the calibration board in order to obtain the internal and external parameters of the camera and other parameters such as the lens distortion of the camera, the calibration board needs to be moved to various local positions in the camera's field of view, and the calibration board or robotic arm needs to be moved manually.
  • the method of moving the calibration board is time-consuming and laborious, which reduces the calibration efficiency.
  • the method of manually moving the calibration board or the robot arm moving the calibration board realizes the complete repetition of the calibration process, which makes it difficult to reproduce the problems in the calibration process.
  • the embodiments of the present application provide a parameter calibration method and device.
  • the method and device can be applied to the calibration of various consumer-grade cameras and various industrial-grade cameras to obtain the internal parameters, external parameters, lens distortion parameters, white balance parameters, chromatic aberration, MTF and other parameters of the camera.
  • the above-mentioned consumer-grade cameras may include, for example, various types of cameras installed in mobile phones, tablet computers, cameras, virtual/mixed/augmented reality devices, drones with cameras, and camera-based auxiliary driving devices/systems.
  • Camera array; industrial-grade cameras may include, for example, surveillance cameras, range/speed cameras used in industrial scenes, and cameras or camera arrays included in various industrial cameras.
  • Fig. 7 exemplarily provides a hardware system.
  • the hardware system may include, for example, a control device 100, a display screen 200, a camera device 300, and a sports device 400, and the control device 100 establishes a communication connection with the display screen 200, the camera device 300, and the sports device 400.
  • the display screen 200 can be used to display various calibration patterns, calibration results, and other information in the embodiment of the present application.
  • the movement device 400 includes at least one installation station 410 for installing a camera. Each installation station 410 can be used for installing at least one camera device 300 to be calibrated. Each camera device 300 includes at least one camera to be calibrated.
  • the hardware system can be used to perform binocular or multi-target positioning on two or more cameras, and to calibrate a camera array containing multiple cameras.
  • the motion device 400 can drive the installation station to produce displacement and rotation, thereby changing the pose of the camera.
  • the display screen 200 and the sports device 400 in the hardware system should have the following positional relationship: the display screen 200 is set facing the field of view 310 of the camera, so that when the pose of the sports device 400 driving the camera changes, the display screen 200 can appear on the camera.
  • Various positions of the field of view 310 are examples of the display screen 200.
  • FIG. 7 is only used to illustrate the feasibility of realizing the pose change of the camera device with respect to the display screen, and does not constitute a specific limitation on the method and hardware system of the embodiments of the present application.
  • multiple control devices 100, and/or multiple display screens 200, and/or multiple movement devices 400 may be included.
  • the embodiment of the present application does not limit the specific implementation of the foregoing hardware structure.
  • FIG. 9 is a schematic structural diagram of a control device 100 shown in an embodiment of the present application.
  • the control device 100 may include, for example, a computer, a mobile phone, a tablet computer, and an embedded system (embedded system) and other devices.
  • the control device 100 may include: a processor 110, a memory 120, an interface unit 130, and an input unit 140. Wherein, the processor 110, the memory 120, the interface unit 130 and the input unit 140 are coupled and connected.
  • the memory 120 stores readable program instructions, and the processor 110 is used to read and execute the program instructions in the memory 120 to execute the relevant steps of the parameter calibration method of this embodiment, for example: Send instructions to cause the motion device to drive the camera device to produce displacement, rotation, etc., or send instructions to the camera device through the interface unit 130, so that the camera device captures an image or records a video.
  • the memory 120 may also store document materials corresponding to the calibration pattern, and the processor 110 may read and decode the document materials, and send them to the display screen for display.
  • the storage device 120 may also be used for result files, intermediate result files, logs, debugging information, and other information generated during the calibration process.
  • the processor 110 of the control device 100 may include one or more processing units, such as a system chip (system on a chip, SoC), a central processing unit (CPU), and a microcontroller (microcontroller). , MCU), storage controller, etc.
  • processing units such as a system chip (system on a chip, SoC), a central processing unit (CPU), and a microcontroller (microcontroller). , MCU), storage controller, etc.
  • different processing units may be independent devices, or may be integrated in one or more processors 110.
  • the memory 120 of the control device 100 may include one or more storage units, for example, may include volatile memory (volatile memory), such as dynamic random access memory (DRAM), static memory Random access memory (static random access memory, SRAM), etc.; can also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory (flash memory) , Hard Disk Drive HDD and Solid State Drive SSD, etc.
  • volatile memory such as dynamic random access memory (DRAM), static memory Random access memory (static random access memory, SRAM), etc.
  • NVM non-volatile memory
  • ROM read-only memory
  • flash memory flash memory
  • different storage units may be independent devices, and may also be integrated or packaged in one or more processors 110 or interface units 130 to become a part of the processor 110 or the interface unit 130.
  • the interface unit 130 of the control device 100 may include, for example, a network adapter (network adapter), a network interface card, a local area network receiver (LAN adapter), a network interface controller (NIC), Modem and antenna, etc.
  • the interface unit 130 may be an independent device, or may be partially or fully integrated or packaged in the processor 110 to become a part of the processor 110.
  • the input unit 140 may include, for example, a keyboard, a mouse, a touch screen, a rocker controller, a touch pad, etc. coupled to the processor 110.
  • control device 100 does not constitute a specific limitation on the control device 100.
  • the control device may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • FIG. 10 is a schematic structural diagram of an exercise device 400 shown in an embodiment of the present application.
  • the motion device includes a three-axis moving platform 420, a two-axis rotating platform 430, and an installation station 410.
  • the two-axis rotating platform 430 is installed on the three-axis moving platform 420, and the installation station 410 is fixed at
  • the two-axis rotating platform 430 is equipped with the camera device 300 to be calibrated.
  • the three-axis moving platform 420 can be displaced along the X-axis, Y-axis, and Z-axis directions in the figure to change the position of the camera device 300 in the world coordinate system (ie, real space).
  • the two-axis rotating platform 430 can move along the Y-axis.
  • the Z axis is a rotation axis to generate rotation to change the pitch angle and the horizontal angle of the camera device 300 in the world coordinate system.
  • the displacement and rotation of the movement device 400 can be realized by a motor drive.
  • the motion device 400 shown in FIG. 10 may include at least one motor for driving the three-axis moving platform 420 to generate displacement and at least one motor for driving the two-axis rotating platform 430 to generate rotation.
  • the exercise device 400 may also include a programmable controller, and the controller may be coupled with the motor and the interface unit 130 of the control device 100 for driving the motor to operate according to instructions issued by the control device 100.
  • the displacement and rotation of the motion device 400 can also be achieved in other ways, such as hydraulic drive, electromagnetic drive, etc., which are not specifically limited here.
  • the exercise device 400 provided in the embodiment of the present application can have many different forms and can be implemented in many ways.
  • the structures of the exercise device 400 discussed above are only some preferred implementations, which are used to illustrate the feasibility of the methods and structures described in the embodiments of the present application, and do not limit the protection scope of the embodiments of the present application. Implementing the exercise device 400 of the embodiment of the present application by other methods or sequences is also within the protection scope of the embodiment of the present application.
  • the embodiment of the present application provides a parameter calibration method, which can be applied to any control device shown in FIGS. 7-10, for example.
  • the camera device in this method may include one camera device (for example, FIG. 7) or multiple camera devices (for example, FIG. 8). Therefore, the “camera device” in the implementation of this application can also be described as “at least one camera device”.
  • Step S101 generating a first calibration pattern.
  • the first calibration pattern can be generated programmatically, and the first calibration pattern can be dynamically changed according to the calibration result and the calibration requirement of the previously used calibration pattern in this embodiment.
  • the control device when the first calibration pattern is generated by programming, can obtain the configuration parameters corresponding to the first calibration pattern, generate the first calibration pattern according to the configuration parameters of the first calibration pattern, and combine the first calibration pattern.
  • the image signal corresponding to the calibration pattern is sent to the display screen, so that the display screen displays the first calibration pattern.
  • the control device may pre-store the configuration parameters of multiple calibration patterns, and select the first configuration parameter from the above multiple configuration parameters in accordance with the preset sequence of generating the calibration patterns, or in response to the input action performed by the user on the input unit.
  • Configuration parameters of a calibration pattern to generate the calibration pattern.
  • the control device can display an interactive interface for guiding the user to input configuration parameters on the display screen, so that the user can input and debug the configuration parameters according to the calibration requirements of the camera device; the control device can also display the configuration parameters on the display screen based on the configuration parameters that the user has entered.
  • the corresponding first calibration pattern is generated in real time, and the first calibration pattern is refreshed in real time according to the user's adjustment of the configuration parameters, until the first calibration pattern meets the user's needs.
  • this application does not make specific limitations.
  • the configuration parameters may include, for example, the distribution area of the marker (for example: in the full screen, center, upper left corner, lower left corner, upper right corner, and right corner of the display screen. The distribution of lower corners, etc.), the density of markers (e.g. how many checkers/dots are contained in a unit size display area of the screen), the total number of markers, the size of the checkerboard/dots (e.g.
  • each square of the grid, the diameter of the dot, etc., the above length, width and diameter can be determined by the number of pixels) and the horizontal and vertical spacing between the markers (the above horizontal and vertical spacing can be determined by the pixel The quantity is confirmed).
  • the configuration parameters may include, for example, the number, position, size of the color cards, the gray value and color of each color card, and other parameters.
  • the first calibration pattern may be generated according to the corresponding image file, and at least one image file may be pre-stored in the memory of the control device, and each image file contains a calibration pattern.
  • the above-mentioned image file may include, for example, a checkerboard pattern, a dot pattern, a gray scale pattern, a stripe pattern, a solid color pattern, and the like.
  • the control device may select the first image file corresponding to the first calibration pattern from the at least one image file according to the preset sequence of generating the calibration pattern, and compare the first image file corresponding to the first calibration pattern.
  • An image file is decoded to generate an image signal, and the image signal is output to the display screen for display, so that the display screen displays the first calibration pattern.
  • the display screen can display a graphical operation interface for browsing and selecting image files, so that the user can use the input unit to perform various input actions based on the graphical operation interface and control the device In response to the user's input action on the input unit, select the first image file from at least one image file, generate a preview of the corresponding first calibration pattern and send it to the display screen for display, when the user further performs the input action to confirm the selection
  • the control device causes the display screen to display the first calibration pattern.
  • the calibration patterns of the embodiments of the present application can also be implemented in a variety of ways. Each of the above implementations are only some preferred implementations, and does not limit the protection scope of the embodiments of the present application. Other applied methods for generating calibration patterns are also within the protection scope of the embodiments of the present application.
  • the display screen in the embodiment of the present application may preferably use a self-luminous display screen with an active light source to improve the quality of the calibration pattern, thereby improving the calibration quality of the camera device.
  • the self-luminous display may include, for example, an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, a quantum dot display ( quantum dot display) etc.
  • OLED organic light-emitting diode
  • AMOLED active-matrix organic light-emitting diode
  • quantum dot display quantum dot display
  • the calibration pattern displayed on the display screen is controlled by the control device and has the ability to be reproducible and dynamically adjustable.
  • the calibration pattern at each position of the display screen is not affected by changes in ambient light sources and differences in lighting conditions, and has high consistency, which is conducive to improving the quality of images captured by the camera device and avoiding feature point detection failures in the subsequent image processing. Problems such as the failure of edge detection help to improve the calibration quality.
  • the color mode will be converted from other modes to the printing mode (for example: CMYK), causing some colors to be lost or changed, so it cannot be used to calibrate the color difference, white balance and other parameters of the camera device.
  • the display screen can use the same color mode and color gamut as the image taken by the camera, which can realize the calibration of the camera's color difference and white balance.
  • Step S102 Send a first instruction, where the first instruction is used to instruct at least one set of pose parameters of the camera device in the first time period.
  • the pose of the camera device relative to the control device may be determined by a set of pose parameters, and each set of pose parameters may determine a pose of the camera device.
  • a set of pose parameters may include at least one position parameter, and/or, at least one angle parameter.
  • the position parameter is used to determine the position of the camera device in the world coordinate system.
  • the position parameter may include part or all of the coordinate values of the X-axis, Y-axis and Z-axis coordinates of the camera device in the world coordinate system, such as (X 1 , Y 1 , Z 1 ), (X 2 , Y 2 ), (X 3 , Z 3 ) or (X 4 ), etc.
  • the angle parameter is used to determine the angle of the camera device relative to each coordinate axis of the world coordinate system.
  • the angle parameter may include part or all of the angle value of the camera device relative to the X axis, Y axis, and Z axis in the world coordinate system, such as ( ⁇ 1 , ⁇ 1 , ⁇ 1 ), ( ⁇ 2 , ⁇ 2 ), ( ⁇ 3 , ⁇ 3 ) or ( ⁇ 4 ), etc.
  • a set of pose parameters includes some angle parameters , Indicating that the coordinate value corresponding to the missing angle parameter does not change in the camera device.
  • control device can send a first instruction to the motion device, the motion device can carry at least one camera device, and drive the camera device to translate according to at least one set of pose parameters indicated by the first instruction And rotate to control the pose of at least one camera device.
  • FIG. 12 is a schematic diagram of adjusting the pose of a camera device provided by an embodiment of the present application.
  • the control device may send multiple first instructions to the motion device within the first time period, each of the first instructions indicates a set of pose parameters, and the positions indicated by different first instructions The pose parameters are different or the same.
  • the motion device can receive multiple first instructions in the first time period. Each time a first instruction is received, the motion device will perform a translation and rotation according to the pose parameters indicated by the first instruction to turn the camera device Adjust to the corresponding pose.
  • the sending of a plurality of first instructions in the above-mentioned first time period may be performed sequentially at different times.
  • the control device sends the first instruction 1 to the first instruction N, and the first instruction 1 to the first instruction N to the motion device. Indicate the first group of pose parameters to the Nth group of pose parameters in sequence. Then, at time t1, the control device sends the first instruction 1 to the motion device to adjust the pose of the camera device from the initial pose 0 to pose 1. At time t2, the control device sends the first instruction 2 to the motion device to adjust the camera The pose of the device is from pose 1 to pose 2; at t3, the control device sends the first instruction 3 to the motion device to adjust the pose of the camera device from pose 2 to pose 3; and so on... At time tN, the control device sends a first instruction N to the motion device to adjust the pose of the camera device from pose N-1 to pose N.
  • FIG. 13 is a schematic diagram of adjusting the pose of a camera device provided by an embodiment of the present application.
  • the control device can send only one first instruction in the first period of time.
  • the first instruction contains a plurality of different pose parameters, and the above-mentioned multiple different pose parameters can be Connected into a moving track, and the changing trend of the angle of the camera device. Therefore, after receiving the first instruction, the motion device can drive the camera to move along the movement track indicated by the first instruction, and adjust the angle of the camera device during the movement, so that the camera device has different characteristics in the first period of time. Posture.
  • the control device sends a first instruction 1 to the motion device.
  • the first instruction 1 passes a plurality of different pose parameters of the first group-the Mth group.
  • the pose parameter (M is a natural number) indicates a movement trajectory L1.
  • the motion device moves the camera device along the trajectory L1, so that the pose of the camera device changes from pose 0 to pose M.
  • the motion device can interrupt the movement process along the track L1, and continue to adjust the position of the camera device along the track L2 indicated by the new first instruction 2 Or, if the movement device receives the new first instruction 2, the movement device can continue to move along the trajectory L1, after completing the movement along the trajectory L1, and then continue along the trajectory L2 indicated by the new first instruction 2 Adjust the pose of the camera device.
  • the first instruction instructing the pose parameters of the camera device can be implemented in the following manner: the first instruction may include at least one control parameter of the motion device, for example: three-axis movement The distance, linear velocity and/or displacement time of the platform along the X-axis, Y-axis and Z-axis, the angle, angular velocity and/or rotation time of the two-axis rotating platform around the Y-axis and Z-axis, etc.
  • the above control parameters can be sent to the controller of the motion device, so that the controller determines the rotation direction, speed, and rotation time of the motor according to the above control parameters, and then drives the motor to run, so that the three-axis moving platform and the two-axis rotating platform produce the above Control the displacement or rotation corresponding to the parameter, and then adjust the camera device to the corresponding pose.
  • Step S103 Send a second instruction to the camera device, where the second instruction is used to instruct the camera device to at least one of the shooting timing and the number of shooting times in the first time period.
  • the camera device performs at least one shooting including shooting a photo and/or shooting a video.
  • the control device can alternately send multiple first instructions and multiple second instructions to the camera device during the first time period.
  • the camera device can take multiple photos with different pose parameters.
  • the calibration patterns of the display screen may be all or Part of it appears in the photo, and in different photos, the position where the calibration pattern appears may also be different.
  • the control device sequentially sends the first instruction 1 to the first instruction N to the motion device, and the first instruction 1 to the first instruction N in turn indicate the first group of pose parameters To the Nth group of pose parameters, and alternately with the N first instructions, the control device also sends multiple (for example, N) second instructions to the motion device in turn, so that the camera device shoots under different pose parameters photo.
  • N is a natural number
  • the control device sends the first instruction 1 to the motion device to adjust the pose of the camera device from the initial pose 0 to pose 1, and then the control device sends the second instruction 1 to the camera device so that the camera device is in Take at least one photo with pose 1; at t2, the control device sends a first instruction 2 to the motion device to adjust the pose of the camera device from pose 1 to pose 2, and then the control device sends a second instruction to the camera device 2.
  • the control device sends the first instruction N to the motion device to adjust the camera device's pose from pose N-1 to pose N, and then control The device sends a second instruction N to the camera device, so that the camera device takes at least one photo in the Nth group of pose parameters.
  • the second instruction can be specifically used to instruct the camera device to take a photo time interval ⁇ T. Therefore, the control device may only need to send the second instruction to the camera device once.
  • the camera device can take at least one photo every ⁇ T during the change of the pose parameter.
  • the first command and the second command may be sent at the same time or not at the same time. For example, the first command is sent first and then the second command is sent, or the second command is sent first and then the first command is sent.
  • the control device sends a first instruction to the motion device.
  • the first instruction passes a plurality of different pose parameters of the first group-the M pose parameters (M is a natural number) Indicates a trajectory L1.
  • M is a natural number
  • the motion device moves the camera device along the trajectory L1, so that the pose of the camera device changes from pose 0 to pose M.
  • the control device may send a second instruction to the camera device.
  • the second instruction indicates the time interval ⁇ T, so that the camera device takes a picture of the paper every ⁇ T until the camera device moves along the trajectory L1. The move is complete.
  • the control device can send the second instruction to the camera device only once in the first time period. After the camera device receives the second instruction, it starts to shoot the video.
  • the calibration pattern of the display screen may appear in the video in whole or in part, and the position of the calibration pattern may also be different in different video frames.
  • the first command and the second command may be sent at the same time or not at the same time. For example, the first command is sent first and then the second command is sent, or the second command is sent first and then the first command is sent.
  • control device may also send a termination instruction to the camera device, so that the camera device stops shooting video.
  • the control device sends a first instruction to the motion device.
  • the first instruction passes a plurality of different first group pose parameters-M-th group pose parameters (M is a natural number) Indicates a trajectory L1.
  • M is a natural number
  • the motion device moves the camera device along the trajectory L1, so that the pose of the camera device changes from pose 1 to pose M.
  • the control device may send a second instruction to the camera device, so that the camera device starts to take a video while changing its pose.
  • the control device sends a termination instruction to the camera device, so that the camera device stops shooting video.
  • control device may alternately send the second instruction and the termination instruction multiple times, so that the camera device can shoot multiple video clips during the pose change.
  • the control device sends the first second instruction at time t11, and sends the first termination instruction at time t12, so that the camera device shoots t11- Video clip 1 at t12; the control device sends a second second instruction at t21 and a second termination instruction at t22 to make the camera capture video clip 2 at t21-t22; the control device at tP1 (P It is a natural number) to send the P-th second command, and send the P-th termination command at time tP2 to make the camera capture the video clip P at time tP1-tP2.
  • time t11 may be, for example, before the camera device moves to pose 1
  • time t12 may be, for example, after the camera device moves to pose 1, so that the video segment 1 contains the video frame captured by the camera device in pose 1, for example
  • t21 may be, for example Before the camera device moves to pose 2
  • time t22 may be, for example, after the camera device moves to pose 2, so that video clip 2 contains the video frame captured by the camera device in pose 2
  • time t31 may be, for example, before the camera device moves to pose 3.
  • Time t32 may be, for example, after the camera device moves to pose 3, so that the video segment 3 includes the video frame captured by the camera device in pose 3.
  • Step S104 Acquire at least one photographing result from the photographing device.
  • the above-mentioned shooting result includes photos and/or videos shot by the camera device.
  • the camera device can return the taken photos and videos to the control device, and the control device saves the photos and videos returned by the camera device in the memory for use in calculating the internal parameters of the camera device in subsequent steps.
  • the camera device may send analog signals of photos and videos to a capture card, which performs analog-to-digital conversion of the signals, and then sends the converted digital signals of photos and videos to the control device for storage.
  • Step S105 Determine at least one internal parameter of the camera device according to part or all of the shooting results in the at least one shooting result.
  • any photographing result in the above-mentioned “partial or all photographing results” should include part or all of the patterns of the first calibration pattern. Further optionally, the "part of or all of the imaging results" enables the first calibration pattern to appear in various positions of the photo or video (that is, various positions of the field of view of the camera device), especially positions with severe distortion.
  • the control device can detect and acquire feature points from the above-mentioned "part or all" of the photos.
  • Each feature point has the coordinates of the pixel coordinate system in the photo and the coordinates of the world coordinate system in the display screen.
  • the above-mentioned "world coordinate system conversion to Internal parameters such as M 1 and M 2 in the formula of "pixel coordinate system” (that is, formula (1)).
  • the control device may extract at least one video frame from the video, and each extracted video frame includes a part or all of the first calibration pattern. Then, the control device can detect and acquire feature points from the above at least one video frame. Each feature point has the pixel coordinate system coordinates in the video frame and the world coordinate system coordinates in the display screen. According to the pixel coordinate system coordinates and the world coordinate system coordinates of the above feature points, the above-mentioned "world coordinate system conversion" can be solved.
  • the internal parameters such as M 1 and M 2 in the formula of "the pixel coordinate system” (that is, formula (1)).
  • the characteristic points may be points that are easily recognized from the first calibration pattern.
  • the characteristic point of the checkerboard calibration pattern may be the corner point of each square
  • the characteristic point of the dot calibration pattern may be the center point of each dot.
  • the feature points can be obtained by performing binarization processing and edge detection on the photo or video frame, and will not be repeated here.
  • Step S106 It is judged whether the errors of at least one target internal parameter in the at least one internal parameter all meet its corresponding error threshold. If the error of any target internal parameter does not meet its corresponding error threshold, return to step S102; The target internal parameters of all meet their respective error thresholds, and the calibration results are output.
  • technicians may have different requirements for the calibration of the camera device.
  • the technician wants to calibrate all the internal parameters of the camera device.
  • the technician only wants to calibrate all internal parameters of the camera device. Calibrate part of the internal parameters of the camera device.
  • the internal parameters that need to be calibrated are referred to as target internal parameters here. It is understandable that there may be one or more target internal parameters, and the target internal parameters may be all or part of the above-mentioned "at least one internal parameter".
  • the internal parameters are partially or completely included in the internal parameter matrix. Therefore, the process of solving the internal parameters may actually be the process of solving the internal parameter matrix.
  • the process of solving the internal parameters may actually be the process of solving the internal parameter matrix.
  • the calibration result is output. If the error of any target internal parameter does not meet its corresponding error threshold, it needs to return to step S102 to continue the calibration, so that the internal parameter matrix can be updated iteratively and more accurate internal parameters can be obtained.
  • the error of the internal parameter may include, for example, repetition accuracy, reprojection error, and the like.
  • the internal parameters obtained in the first calibration may be assumed to be inaccurate, so that after the first calibration, return to step S102 to perform the second calibration and subsequent repeated calibrations, so as to calculate the calibration error.
  • the repeatability refers to the deviation between multiple internal parameters obtained by repeatedly performing multiple calibrations under the same calibration environment.
  • the same calibration environment mentioned above refers to the calibration pattern used for at least two calibrations, the pose change trajectory of the camera, the brightness and contrast of the display screen, the number of feature points, the extraction accuracy, and the photos or video frames used to calculate the calibration results. The number of shots and the poses taken are partly or all the same.
  • At least one of the internal parameters may have a corresponding error threshold, and the error threshold may correspond to repeat accuracy or re-projection error, for example: focal length threshold, optical center threshold, pixel size threshold, optical distortion threshold, white balance Threshold etc.
  • the deviation of the focal length of at least two calibrations is greater than the focal length threshold, and/or the deviation of the optical center of the two calibrations is greater than the optical center threshold, and/or, the deviation of the optical center of the two calibrations
  • the deviation of the pixel size is greater than the pixel size threshold, and/or the deviation of the optical distortion of the two calibrations is greater than the optical distortion threshold, and/or the deviation of the white balance of the two calibrations is greater than the white balance threshold, then it is necessary to return to step S102 Continue to calibrate.
  • the projection value of the coordinate point in the pixel coordinate system can be obtained.
  • the coordinate point is in the camera device
  • the shooting result also corresponds to the observation value of a pixel coordinate system, and the error between the projection value and the observation value is the reprojection error. The smaller the reprojection error, the smaller the internal parameter error, and the larger the reprojection error, the greater the internal parameter error.
  • Method 1 The calibration pattern can be changed.
  • the calibration pattern is fixed during a calibration process, that is, the marker
  • the number and size of the product remain unchanged.
  • the calibration accuracy does not meet the accuracy requirements, the number of feature points may be insufficient. Therefore, in the next calibration, a new calibration pattern (for example, the second calibration pattern) can be obtained by reducing the size of the marker and increasing the number of markers, so as to improve the control device from the shooting result in step S105.
  • a new calibration pattern (for example, the second calibration pattern) can be obtained by increasing the size of the marker, so as to improve the accuracy of feature point extraction.
  • Manner 2 Adjusting the number of feature points corresponding to each field of view of the camera device extracted from at least one shooting result.
  • the feature points are not evenly distributed in each area of the camera’s field of view.
  • the feature points are concentrated in a certain area, or a certain area has no feature points or only a few feature points, which affects the weight of the internal parameter matrix algorithm. Therefore, the feature points are concentrated in a certain area, and when repeat calibration is performed, part of the feature points in this area can be deleted, or feature points in other areas can be added.
  • the field of view of the camera device can be divided into K sub-areas, for example, divided into sub-areas 1-sub-areas 9 through a nine-square grid.
  • the control device extracts the feature points from the shooting result, it can count the number of feature points in the sub-region 1-the sub-region 9 respectively.
  • the control device can dynamically adjust the feature points of each sub-region to uniformize, so that the number of feature points in each sub-region tends to be uniform, so as to improve the calibration accuracy.
  • Method 3 The image quality of the calibration pattern can be adjusted.
  • the calibration accuracy when using a checkerboard pattern or a dot pattern to calibrate the internal parameters of the camera device, if the calibration accuracy does not meet the accuracy requirements, it may be that the brightness and contrast of the display screen and other parameters that affect the image quality are set unreasonably, or the shooting result is unreasonable.
  • the relevant thresholds used for binarization and edge detection are unreasonable. Therefore, when performing repeated calibration, the brightness and contrast of the display screen can also be changed, or the relevant thresholds of binarization processing and edge detection can be adjusted to improve the accuracy of feature point extraction.
  • the embodiment of the present application can perform multiple calibrations on the camera device, so that the internal parameter matrix and the external parameter matrix of the camera device are continuously updated iteratively, and the calibration accuracy is continuously improved until the calibration accuracy is improved.
  • the value is less than the corresponding threshold.
  • the method of the embodiments of the present application can dynamically adjust the calibration parameters according to the field of view and depth of field of the camera device during specific implementation.
  • the calibration pattern when the field of view of the camera device is small, the number of markers in the calibration pattern can be appropriately reduced to increase the number of feature points in the field of view; when the field of view of the camera device is large, the calibration pattern can be appropriately increased In order to reduce the number of feature points in the field of view and improve the recognition accuracy of feature points.
  • the calibration pattern in the process of changing the pose of the camera device, the calibration pattern should appear in various positions of the camera device's field of view, especially the edge of the camera device's field of view (the field of view edge is often a severely distorted area), so that the subsequent extracted feature points It can be evenly distributed in the field of view of the imaging device to improve the calibration accuracy.
  • the parameter calibration method provided by the embodiments of the present application can be applied to the calibration of the internal parameters and external parameters of the camera device.
  • This method changes the position of the camera device when the position of the calibration pattern is fixed.
  • the position and angle of the camera device relative to the calibration pattern are accurately adjusted by the method of pose parameters, so that the camera device can accurately photograph the calibration pattern at each predetermined position and angle relative to the calibration pattern, and obtain at least one shooting result.
  • the above-mentioned shooting result can be compared with the calibration pattern.
  • the coordinates of the pixel coordinate system and the world coordinate system correspond accurately, so that the internal parameters and external parameters obtained by the corresponding calculation are more accurate, and the calibration accuracy is improved.
  • the method of the embodiment of the present application can also be used to calibrate different parameters.
  • the method of the embodiment of the present application can be used to calibrate the white balance parameters of the camera device.
  • the first calibration pattern is a monochrome pattern
  • the monochrome pattern appears in each position of the camera device's field of view and the corresponding photos or videos are taken.
  • the vignette of the camera device can be calculated according to the shooting results. parameter.
  • the optical resolution of the camera can be calibrated by adjusting the pose parameters of the camera device so that the lines appear in each position of the camera device's field of view and shooting corresponding photos or videos.
  • Rate modulation transfer function MTF Rate modulation transfer function
  • the first calibration pattern is a color card pattern containing multiple standard colors
  • the color difference parameters of the camera device can be calibrated, and the various calibration patterns described above can be preset in the control device to be generated when needed.
  • each device such as the aforementioned network device and storage device, includes hardware structures and/or software modules corresponding to each function in order to realize the aforementioned functions.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software-driven hardware depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
  • each solution of the parameter calibration method provided in this application is introduced from the perspective of the equipment/device itself and the interaction between the equipment/devices.
  • each device/device such as the aforementioned control device and motion device, in order to realize the aforementioned functions, includes hardware structures and/or software modules corresponding to the respective functions.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software-driven hardware depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
  • control equipment implements corresponding functions through software modules.
  • the control device may include, for example, a computer, a mobile phone, a tablet computer, and an embedded system (embedded system) and other devices.
  • the parameter calibration device for realizing the above-mentioned function of controlling device behavior includes: a generating unit 501, configured to generate a first calibration pattern; and a first sending unit 502, configured to send a first calibration pattern; Instruction, the first instruction is used to indicate at least one set of pose parameters of at least one camera device in the first time period; the second sending unit 503 is used to send a second instruction to the camera device, and the second instruction is used to Indicate the shooting timing and shooting times of the at least one camera device in the first time period; the obtaining unit 504 is configured to obtain at least one shooting result from the at least one camera device; the processing unit 505 is configured to obtain at least one shooting result from the at least one camera device; Part or all of the photographing results in the photographing results determine the internal parameters of the at least one photographing device, and any photographing result in the part or all of the photographing results includes part or all of the patterns of the first calibration pattern.
  • the parameter calibration device provided in this application can be applied to the calibration of the internal parameters and external parameters of the camera device.
  • This method accurately adjusts the camera by changing the pose parameters of the camera device under the condition that the position of the calibration pattern is fixed.
  • the position and angle of the device relative to the calibration pattern enable the camera device to accurately photograph the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result.
  • the shooting result can be compared with the pixel coordinate system and the world coordinate system.
  • the precise correspondence of the coordinates makes the internal parameters and external parameters obtained by the corresponding calculation more accurate, and improves the calibration accuracy.
  • the first sending unit 502 is specifically configured to send a first instruction to a sports device, the sports device carrying the at least one camera device, and the sports device is configured to respond according to the at least one set of pose parameters Controlling the pose of the at least one camera parameter. Therefore, the movement device can accurately control the pose of the camera device according to the pose parameters, and can realize the reproduction of the pose, which is beneficial to improve the calibration accuracy.
  • the shooting result includes at least one photo and/or at least one video taken by the at least one camera in response to the second instruction.
  • the image participating in the calculation of the internal parameters of the camera device can be selected from the photo or from the video frame of the video, in a flexible manner.
  • the first time period includes multiple first instructions and multiple second instructions
  • the first sending unit 502 and the second sending unit 503 are specifically configured to alternately send the multiple instructions.
  • At least one of the internal parameters has a corresponding error threshold; the processing unit 505 is configured to determine whether the repetition accuracy of at least one of the internal parameters is greater than the corresponding error threshold;
  • the generating unit 501 is configured to adjust the first calibration pattern to the second calibration pattern when the repetition accuracy is greater than the corresponding error threshold, and/or adjust the image quality of the first calibration pattern;
  • the processing unit 505 is configured to adjust the number of feature points extracted from the first calibration pattern when the repetition accuracy is greater than the corresponding error threshold. Therefore, the present application can perform multiple calibrations on the camera device, so that the internal parameter matrix and the external parameter matrix of the camera device are continuously updated iteratively, and the calibration accuracy is continuously improved until the value of the calibration accuracy is less than the corresponding threshold.
  • the number and/or size of the markers of the first calibration pattern and the second calibration pattern are different.
  • the processing unit 505 is configured to divide the first calibration pattern into at least one sub-region, and uniformly adjust the number of the feature points in the at least one sub-region. As a result, the number of feature points in each sub-region is unified to improve the calibration accuracy.
  • the first calibration pattern is a preset gray scale pattern
  • the internal parameter includes a white balance parameter of the at least one camera device.
  • the first calibration pattern is a preset monochrome pattern
  • the internal parameter includes a halo parameter of the at least one imaging device.
  • the first calibration pattern is a preset color chart pattern containing multiple standard colors
  • the internal parameters include color difference parameters of the at least one camera device.
  • the first calibration pattern is a preset target pattern including lines with different angles
  • the internal parameters include the at least one imaging device modulation transfer function parameter.
  • the above-mentioned devices implement corresponding functions through hardware modules.
  • the above-mentioned functions of the control device may be realized by the structure shown in FIG. 9.
  • the interface unit 130 is used to send a first instruction to a sports device, and to send a second instruction to at least one camera device, and to obtain at least one shooting result from at least one camera device;
  • the memory 120 is used to store computer program codes,
  • the computer program code includes computer instructions; when the processor 110 executes the computer instructions, the control device executes the methods involved in the above embodiments, for example: generating a first calibration pattern; sending a first instruction, the first instruction being used for Indicate at least one set of pose parameters of at least one camera in the first period; send a second instruction to the at least one camera, and the second instruction is used to instruct the at least one camera in the first period Shooting timing and number of shooting times; acquiring at least one shooting result from the at least one shooting device; determining the internal parameters of the at least one shooting device according to part or all of the shooting results in the at least one shooting result, and the part or Any imaging result in all imaging results
  • the parameter calibration system includes: a control device 100 for generating a first calibration pattern; a display screen 200, coupled to the control device 100, for displaying the first calibration pattern; a motion device 400 for carrying at least One camera device 300 makes the field of view of at least one camera device 300 face the display screen 200, and the motion device 400 is coupled to the control device 100; the control device 100 is also used to send a first instruction to the motion device 400, and the first instruction is used to instruct at least At least one set of pose parameters of one camera device 300 in the first time period; the motion device 400 is also used to control the pose of at least one camera device 300 according to at least one set of pose parameters; the control device 100 is also used to One camera device 300 sends a second instruction, the second instruction is used to instruct at least one camera device 300 in the first period of time and the number of shots; and to obtain at least one shooting result from the at least one camera device 300; and, according to Part or all of the shooting results in the at least
  • the embodiments of the present application also provide a computer storage medium, in which computer instructions are stored, which when run on a computer, cause the computer to execute the methods of the above-mentioned various aspects.
  • the embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to execute the methods of the above-mentioned various aspects.
  • the application also provides a chip system.
  • the chip system includes a processor, which is used to support the above-mentioned device or device to realize the functions involved in the above-mentioned aspects, for example, to generate or process the information involved in the above-mentioned method.
  • the chip system also includes a memory, which is used to store the necessary program instructions and data of the above-mentioned device or equipment.
  • the chip system can be composed of chips, or include chips and other discrete devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)
  • Manipulator (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

一种参数标定方法及装置，可以应用于摄像装置（300）的内部参数和外部参数的标定，在标定图案的位置固定不变的情况下，通过变化摄像装置（300）的位姿参数的方式，精准调整摄像装置（300）相对于标定图案的位置和角度，使摄像装置（300）在相对于标定图案各个既定的位置和角度对标定图案进行精确拍摄，得到至少一个拍摄结果，上述拍摄结果能够与像素坐标系和世界坐标系的坐标精确对应，使得相应计算得到的内部参数和外部参数的准确性更高，提高了标定精度。

Description

参数标定方法及装置 技术领域
本申请涉及光学技术领域，尤其涉及一种参数标定方法及装置。
背景技术
在机器视觉/计算机视觉等技术的实际应用中，为确定空间物体表面某点的三维几何位置与其在摄像头采集的图像中对应点之间的相互关系，必须建立摄像头成像的几何模型，这些几何模型的参数可以被概括为摄像头参数(或相机参数)。在大多数情况下，摄像头参数必须通过实验和计算才能够求解得到，通过实验和计算求解摄像头参数的过程被称为摄像头标定(或相机标定)。
目前的摄像头标定方法，将待标定的摄像头固定，然后，人工手持标定板出现在摄像头的视野中，移动标定板以改变标定板的角度和位置，使得摄像头拍摄不同角度和位置下的标定板的图像；最后，通过标定板的特征点在三维世界中的真实位置和摄像头成像的图像中的位置联立求解摄像头的内部参数和外部参数。
目前的标定板是在各类板材上印制的，其平整度和打印精度会影响到世界坐标系的精度，当平整度和打印精度较低时，世界坐标系的精度较低，影响标定精度。并且，人工移动标定板的方式使得标定板的位置无法精确确定，也会影响标定精度。
发明内容
本申请提供了一种参数标定方法及装置，以提高摄像装置的标定精度。
第一方面，本申请提供了一种参数标定方法。该方法包括：生成第一标定图案；发送第一指令，第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；向至少一个摄像装置发送第二指令，第二指令用于指示至少一个摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个；获取来自至少一个摄像装置的至少一个拍摄结果；根据至少一个拍摄结果中的部分或者全部拍摄结果，确定至少一个摄像装置的至少一个内部参数，部分或者全部摄像结果中的任意摄像结果包含第一标定图案的部分或者全部图案。
本申请提供的参数标定方法，可以应用于摄像装置的内部参数和外部参数的标定，该方法在标定图案的位置固定不变的情况下，通过变化摄像装置的位姿参数的方式，精准调整摄像装置相对于标定图案的位置和角度，使摄像装置在相对于标定图案各个既定的位置和角度对标定图案进行精确拍摄，得到至少一个拍摄结果，上述拍摄结果能够与像素坐标系和世界坐标系的坐标精确对应，使得相应计算得到的内部参数和外部参数的准确性更高，提高了标定精度。
在一种实现方式中，发送第一指令，包括：向运动装置发送第一指令，运动装置承载至少一个摄像装置，至少一组位姿参数用于运动装置控制至少一个摄像装置的位姿。由此，运动装置可以根据位姿参数精确地控制摄像装置的位姿，并可以实现位姿的重现，有利于提高标定精度。
在一种实现方式中，拍摄结果包括来自至少一个摄像装置的、响应于第二指令拍摄的至少一张照片和/或至少一段视频。由此，参与计算摄像装置的内部参数的图像可以从照片中选取，也可以从视频的视频帧中选取，方式灵活。
在一种实现方式中，发送第一指令，包括：在第一时段内发送多个第一指令；向至少一个摄像装置发送第二指令，包括：在第一时段内向至少一个摄像装置发送多个第二指令；其中，多个第一指令和多个第二指令交替发送。由此，通过第一指令和第二指令可以根据实际需求指示摄像装置在不同的位姿下拍摄不同的照片或者视频，例如使得第一标定图案出现在摄像装置视野的各个位置，提高标定精度。
在一种实现方式中，内部参数中的至少一个参数具有对应的误差阈值，当内部参数中的至少一个参数的重复精度大于对应的误差阈值时，该方法还包括：调整第一标定图案为第二标定图案，和/或，调整从至少一个拍摄结果中提取的对应至少一个摄像装置各个视野的特征点数量，和/或，调整第一标定图案的图像质量。由此，本申请可以对摄像装置执行多次标定，使摄像装置的内参矩阵和外参矩阵不断迭代更新，标定精度不断提高，直至标定精度的取值小于对应的阈值。
在一种实现方式中，第一标定图案和第二标定图案中的标志物的数量和/或尺寸不同。
在一种实现方式中，调整从至少一个拍摄结果中提取的对应至少一个摄像装置各个视野的特征点数量，包括：确定第一标定图案包含的至少一个子区域，将至少一个子区域的特征点的数量进行均一化调整。由此，使得各个子区域的特征点的数量区域统一，以提高标定精度。
在一种实现方式中，第一标定图案为预设的灰度板图案，内部参数包含至少一个摄像装置的白平衡参数。
在一种实现方式中，第一标定图案为预设的单色图案，内部参数包含至少一个摄像装置的影晕参数。
在一种实现方式中，第一标定图案为预设的包含多种标准颜色的色卡图案，内部参数包括至少一个摄像装置的色差参数。
在一种实现方式中，第一标定图案为预设的包含不同角度线条的标板图案，内部参数包括至少一个摄像装置调制传递函数参数。
第二方面，本申请提供了一种参数标定装置。该装置包括：生成单元，用于生成第一标定图案；第一发送单元，用于发送第一指令，第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；第二发送单元，用于向摄像装置发送第二指令，第二指令用于指示至少一个摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个；获取单元，用于获取来自至少一个摄像装置的至少一个拍摄结果；处理单元，用于根据至少一个拍摄结果中的部分或者全部拍摄结果，确定至少一个摄像装置的至少一个内部参数，部分或者全部摄像结果中的任意摄像结果包含第一标定图案的部分或者全部图案。
本申请提供的参数标定装置，可以应用于摄像装置的内部参数和外部参数的标定，该方法在标定图案的位置固定不变的情况下，通过变化摄像装置的位姿参数的方式，精准调整摄像装置相对于标定图案的位置和角度，使摄像装置在相对于标定图案各个 既定的位置和角度对标定图案进行精确拍摄，得到至少一个拍摄结果，上述拍摄结果能够与像素坐标系和世界坐标系的坐标精确对应，使得相应计算得到的内部参数和外部参数的准确性更高，提高了标定精度。
在一种实现方式中，第一发送单元具体用于向运动装置发送第一指令，运动装置承载至少一个摄像装置，至少一组位姿参数用于运动装置控制至少一个摄像参数的位姿。由此，运动装置可以根据位姿参数精确地控制摄像装置的位姿，并可以实现位姿的重现，有利于提高标定精度。
在一种实现方式中，拍摄结果包括来自至少一个摄像装置的、响应于第二指令拍摄的至少一张照片和/或至少一段视频。由此，参与计算摄像装置的内部参数的图像可以从照片中选取，也可以从视频的视频帧中选取，方式灵活。
在一种实现方式中，第一发送单元，用于在第一时段内发送多个第一指令；第二发送单元，用于在第一时段内向至少一个摄像装置发送多个第二指令；其中，多个第一指令和多个第二指令交替发送。由此，通过第一指令和第二指令可以根据需求指示摄像装置在不同的位姿下拍摄不同的照片或者视频，例如使得第一标定图案出现在摄像装置视野的各个位置，提高标定精度。
在一种实现方式中，内部参数中的至少一个参数具有对应的误差阈值；处理单元，用于确定内部参数中的至少一个参数的重复精度是否大于对应的误差阈值；生成单元，用于当重复精度大于对应的误差阈值时，调整第一标定图案为第二标定图案，和/或，调整第一标定图案的图像质量；处理单元，用于当重复精度大于对应的误差阈值时，调整从至少一个拍摄结果中提取的对应至少一个摄像装置各个视野的特征点数量。由此，本申请可以对摄像装置执行多次标定，使摄像装置的内参矩阵和外参矩阵不断迭代更新，标定精度不断提高，直至标定精度的取值小于对应的阈值。
在一种实现方式中，第一标定图案和第二标定图案中的标志物的数量和/或尺寸不同。
在一种实现方式中，处理单元用于确定第一标定图案包含的至少一个子区域，将至少一个子区域的特征点的数量进行均一化调整。由此，使得各个子区域的特征点的数量区域统一，以提高标定精度。
在一种实现方式中，第一标定图案为预设的灰度板图案，内部参数包含至少一个摄像装置的白平衡参数。
在一种实现方式中，第一标定图案为预设的单色图案，内部参数包含至少一个摄像装置的影晕参数。
在一种实现方式中，第一标定图案为预设的包含多种标准颜色的色卡图案，内部参数包括至少一个摄像装置的色差参数。
在一种实现方式中，第一标定图案为预设的包含不同角度线条的标板图案，内部参数包括至少一个摄像装置调制传递函数参数。
第三方面，本申请提供了一种参数标定装置。该装置包括：存储器和处理器；存储器和处理器耦合；存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当处理器执行计算机指令时，使该装置执行上述第一方面及其任意实现方式中的方法。
第四方面，本申请提供了一种参数标定系统。该系统包括：控制设备，用于生成第一标定图案；显示屏，耦合至控制设备，用于显示第一标定图案；运动装置，用于承载至少一个摄像装置，使至少一个摄像装置的视野面向显示屏，运动装置耦合至控制设备；控制设备，还用于向运动装置发送第一指令，第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；运动装置，还用于根据至少一组位姿参数控制至少一个摄像装置的位姿；控制设备，还用于向至少一个摄像装置发送第二指令，第二指令用于指示至少一个摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个；以及，获取来自至少一个摄像装置的至少一个拍摄结果；以及，根据至少一个拍摄结果中的部分或者全部拍摄结果，确定至少一个摄像装置的至少一个内部参数，部分或者全部摄像结果中的任意摄像结果包含第一标定图案的部分或者全部图案。
第五方面，本申请还提供了一种计算机存储介质。该计算机存储介质计算机指令，当计算机指令在控制设备上运行时，使得控制设备执行上述第一方面及其任意实现方式中的方法。
第六方面，本申请还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面及其任意实现方式中的方法。
第七方面，本申请还提供了一种芯片系统，该芯片系统包括处理器，用于支持上述装置或设备实现上述第一方面及其任意实现方式中所涉及的功能，例如，生成或处理上述方法中所涉及的信息。
附图说明
图1是摄像头的基本结构示意图；
图2是物体从三维场景映射到二维图像的示意图；
图3是各个坐标系转换关系的示意图；
图4是摄像头的畸变的示意图；
图5是一种摄像头标定方法的示意图；
图6是本申请实施例示出的一些标定板的示意图；
图7是本申请实施例提供的一种用于参数标定的硬件系统的示意图；
图8是本申请实施例提供的一种用于参数标定的硬件系统的示意图；
图9是本申请实施例示出的控制设备的结构示意图；
图10是本申请实施例示出的运动装置的结构示意图；
图11是本申请实施例提供的一种参数标定方法的流程图；
图12是本申请实施例提供的调整摄像装置位姿的示意图；
图13是本申请实施例提供的调整摄像装置位姿的示意图；
图14是本申请实施例提供的摄像装置拍摄的示意图；
图15是本申请实施例提供的摄像装置拍摄的示意图；
图16是本申请实施例提供的摄像装置拍摄的示意图；
图17是本申请实施例提供的摄像装置拍摄的示意图；
图18是特征点的示意图；
图19是控制设备调整标定图案的示意图；
图20是控制设备调整标定图案的示意图；
图21是控制设备调整标定图案的示意图；
图22是本申请实施例提供的其他标定图案的示意图；
图23是本申请实施例提供的一种参数标定装置的示意图。
具体实施方式
在机器视觉/计算机视觉等技术领域中，摄像头通常被用于对客观世界的对象和场景进行成像，以便于根据摄像头的成像对客观对象和场景进行三维重建，实现利用机器对客观世界的三维识别，例如识别对象的形状、尺寸、距离观察点的距离、质地和运动特征等。机器视觉/计算机视觉技术可以被应用到虚拟现实、高级辅助驾驶系统、无人机巡航系统、视觉检测、生物特征识别、距离测量、三维重建等领域。近年来，随着智能手机、平板电脑、虚拟现实设备、搭载辅助驾驶系统的汽车、无人机等产品的不断普及，机器视觉/计算机视觉技术得到不断发展。
在机器视觉/计算机视觉等技术的实际应用中，为确定空间物体表面某点的三维几何位置与其在摄像头采集的图像中对应点之间的相互关系，必须建立摄像头成像的几何模型，这些几何模型参数可以被概括为摄像头参数(或相机参数)。在大多数情况下，摄像头参数必须通过实验和计算才能够求解得到，通过实验和计算求解摄像头参数的过程被称为摄像头标定(或相机标定)。摄像头标定是机器视觉/计算机视觉等领域的重要环节，其标定结果的精度及算法的稳定性直接影响图像推理结果的准确性，因此，做好摄像头标定是做好后续工作的前提，快速准确地完成摄像头标定则是提高后续工作效率的基础。
下面结合一些附图和文字对摄像头标定的基本原理和涉及到的主要技术概念进行具体地解释说明。需要注意的是，在不同的表述方式和习惯中，一些技术概念的表述方式可能不同，例如，将一个概念表述为多个子概念，或者将几个概念通过拆分或组合的方式概括为一个或者多个其他概念，这些表述方式仅仅是习惯上的改变，不够成对本申请实施例的实质区别。
图1是摄像头的基本结构示意图。如图1所示，摄像头包括光学镜头10和光电传感器20。其中，光学镜头10用于将空间物体的光学影像投影到光电传感器20上，光电传感器20用于将光学镜头10投影的光信号转换成电信号，最终借助图像信号传感器等其他器件的处理形成图像。一般来说，光电传感器20可以包括电荷耦合器件(charge-coupled device，CCD)或者互补式金属氧化物半导体有源像素传感器(CMOS active pixel sensor)等。
如图2所示，现实空间中的物体是三维的，而其对应的图像中的物体是二维的，因此，从三维场景中的物体到其对应的二维图像之间可以认为存在一个三维到二维的映射模型，该映射模型使得三维场景中的物体到其对应的图像之间形成从三维到二维或者二维到三维的转换。那么，容易理解的是，在摄像头拍摄对三维场景中的物体拍摄图像时，摄像头就可以被认为是这个映射模型，而摄像头参数就是这个映射模型的参数。因此，只要求得摄像头参数，就可以由物体在图像中的像素坐标反推出物体在空间中的世界坐标，由此实现视觉检测、生物特征识别、距离测量、三维重建等功能。
一般来说，摄像头标定中会涉及到以下概念：
摄像头的内部参数(简称摄像头内参)：是指与摄像头自身特性相关的参数，可以通过光学镜头和光电传感器等在摄像头模组内安装的位置确定。摄像头内参可以包 括：焦距、像素大小、光学畸变、白平衡、分辨率、反差、晕影和暗角等参数中的至少一个。
摄像头的外部参数(简称摄像头外参)可以包含以下中的至少一个：
世界坐标系(world coordinate)：也称为测量坐标系，是一个三维直角坐标系，以其为基准可以描述摄像头和待测物体在客观世界的空间位置。世界坐标系的位置可以根据实际情况自由确定。
摄像头的外部参数和摄像头的内部参数统称为摄像头参数。
摄像头坐标系(camera coordinate)：是一个三维直角坐标系，原点位于光学镜头的光心处(其中，即光学中心，是指光学镜头的透镜的中心点)，x、y轴分别与摄像头的图像平面的两边平行，z轴为光学镜头的光轴，与成像平面垂直。
像素坐标系(pixel coordinate)：是一个二维直角坐标系，反映了相机CCD/CMOS芯片中像素的排列情况。像素坐标系的原点位于图像的左上角，u轴、v轴分别与图像平面的两边平行。像素坐标系中坐标轴的单位是像素(整数)。
图像坐标系(image coordinate)：是一个二维直角坐标系，其坐标轴的单位通常为毫米(mm)，原点是摄像头的光轴与图像平面的交点(称为主点)，即图像的中心点，X轴、Y轴分别与像素坐标系的u轴、v轴平行。图像坐标系和像素坐标系之间是平移关系，即可以通过平移就可得到。
图3是各个坐标系转换关系的示意图。下面结合图3介绍各个坐标系之间的转换关系，进而对本申请实施例需要标定的摄像头参数进行具体说明：
1、世界坐标系转换为摄像头坐标系可以通过以下矩阵变换实现：
Figure PCTCN2019129932-appb-000001
其中，R为3*3的旋转矩阵，t为3*1的平移矢量，(x c，y c，z c，1) T为摄像头坐标系的坐标，(x w，y w，z w，1) T为世界坐标系的坐标。因此，从世界坐标系转换为摄像头坐标系实际上包括了一个旋转过程和一个平移过程。
2、图像坐标系转换为像素坐标系可以通过以下矩阵变换实现：
Figure PCTCN2019129932-appb-000002
其中，(X，Y，1) T为图像坐标系的坐标，(u，v，1) T为图像坐标系的坐标，dX、dY分别为像素在X、Y轴方向上的物理尺寸，u 0，v 0为光心坐标。
3、摄像头坐标系转换为图像坐标系如图3所示是一个透视投影的过程：
例如，空间内任意一点P与摄像头光心C的连线为CP，CP与图像平面的交点p即为空间点P在图像平面上的投影，该投影可以通过以下矩阵变换实现：
Figure PCTCN2019129932-appb-000003
其中，S为比例因子(S不为0)，f为摄像头的有效焦距(光心到图像平面的距 离)，(x c，y c，z c，1) T是空间点P在摄像头坐标系中的坐标，(X，Y，1) T是像点p在图像坐标系中的坐标。
4、世界坐标系转换为像素坐标系：
根据上述1～3示出的各个坐标系之间的转换矩阵，世界坐标系转换为像素坐标系可以通过以下矩阵变换得到：
Figure PCTCN2019129932-appb-000004
其中：
Figure PCTCN2019129932-appb-000005
M＝M 1M 2
在上述矩阵中，M为摄像头的参数矩阵，即摄像头参数；其中，M 1为摄像头的内参矩阵，即摄像头的内部参数；M 2为摄像头的外参矩阵，即摄像头的外部参数。摄像头标定即包括求解上述内参矩阵M 1和/或外参矩阵M 2。例如：求解得到摄像头的像元尺寸1/dX和1/dY、摄像头的焦距f、摄像头的光心位置(v 0，u 0)、旋转矩阵R和平移变量等参数。
需要补充说明的是，摄像头的实际成像通常会有变形，可以称为畸变。如图4所示，畸变具体可以分为切向畸变和径向畸变。其中，径向畸变来自于摄像头的透镜本身，来源于“鱼眼”和“桶形”的影响，在透镜成像的边缘更加明显；切向畸变是透镜制造上的缺陷或者安装存在偏差使得透镜本身与图像平面不平行而产生的。当考虑切向畸变和径向畸变时，上述1～3示出的矩阵变换还可以扩展出：
x′ c＝x c/z c
y′ c＝y c/z c
Figure PCTCN2019129932-appb-000006
Figure PCTCN2019129932-appb-000007
Figure PCTCN2019129932-appb-000008
其中，k 1、k 2、k 3、k 4、k 5和k 6为径向畸变系数，p 1和p 2位切向畸变系数，因此，对摄像头标定的过程还可以确定摄像头的畸变参数。
为了求解摄像头参数，在实验室环境下，一种摄像头标定方法如图5所示，首先，将待标定的摄像头固定在实验室空间环境中的某个位置；然后，人工手持标定板出现在摄像头的视野中，移动标定板以改变标定板的角度和位置，使得摄像头能够相对于标定板以不同的位姿拍摄标定板的图像；最后，通过标定板的特征点在三维世界中的真实位置(即特征点在世界坐标系中的坐标)和摄像头成像的图像中的位置(即特征点在图像的像素坐标系中的坐标)联立求解摄像头的内部参数和外部参数。工业级摄 像头的标定一般需要借助大型机械臂，将标定板吊装在大型机械臂上，通过机械臂的摆臂动作改变标定板的角度和位置，但是，机械臂的摆臂和摆臂后的姿态稳定都需要消耗大量时间，导致摄像头标定的效率很低，难以大规模应用。
图6是本申请实施例示出的一些标定板的示意图。如图6所示，标定板之上可以印制棋盘格图案、圆点图案、二维码图案或者其他特定图案，对于标定板来说，其中的任意一个图案单元，例如棋盘格图案的任意一个方格、圆点图案的任意一个圆点，均可以称作一个标志物。从标志物中可以确定一些特征点，以用于计算摄像头参数。除了图6示出的一些标定板之外，为了标定摄像头的其他参数，还会用到其他形式的标定板，例如：通过包含色卡的标定板标定摄像头的白平衡参数，通过均匀漫反射材质的标定板配合均匀照明的光源标定摄像头的晕影参数，通过包含特殊设计图案的标定板标定摄像头的光学分辨率调制传递函数(modulation transfer function，MTF)。
结合图5和图6示出的参数标定方法在实际应用中会存在以下问题：
由于标定板是在各类板材上印制的，标定板的平整度和打印精度会影响到世界坐标系的精度，当平整度和打印精度较低时，世界坐标系的精度较低，影响标定精度。并且，人工移动标定板的方式使得标定板的位置无法精确确定，也会影响标定精度。标定不同的参数需要在不同的板材上印制不同图案的标定板，标定板之间相互独立，并且标定不同的摄像头参数时的标定板的照明条件和形状尺寸不尽相同，使得标定板无法实现重用。并且，在摄像头标定过程中，为了获取摄像头的内参和外参等集合参数，以及摄像头的镜头畸变等参数，需要将标定板移动到摄像头视野中的各个局部位置，采用手动移动标定板或者机械臂移动标定板的方法费时费力，降低标定效率。另外，采用手动移动标定板或者机械臂移动标定板的方法实现对标定过程的完全重复，使得标定过程中出现的问题难以复现。
为了解决上述技术问题，本申请实施例提供了一种参数标定方法及装置。该方法和装置可以应用于各类消费级摄像头和各类工业级摄像头的标定，以获取摄像头的内部参数、外部参数、镜头畸变参数、白平衡参数和色差和MTF等参数。其中，上述消费级摄像头例如可以包括用于安装在手机、平板电脑、相机、虚拟/混合/增强现实设备、带有摄像头的无人机、基于摄像头的辅助驾驶设备/系统中的各类摄像头或摄像头阵列；工业级摄像头例如可以包括应用于工业场景中的监控摄像头、测距/测速摄像头，以及各类工业相机包含的摄像头或摄像头阵列。
图7示例性地提供了一种硬件系统。如图7所示，该硬件系统例如可以包括：控制设备100、显示屏200、摄像装置300和运动装置400，控制设备100与显示屏200、摄像装置300和运动装置400建立有通信连接。其中，显示屏200在本申请实施例中可以用于显示各种标定图案、标定结果和其他信息。运动装置400包括至少一个用于安装摄像头的安装工位410，每个安装工位410可以用于安装至少一个待标定的摄像装置300，每个摄像装置300包括至少一个待标定的摄像头，当摄像装置300包括两个或者两个以上摄像头时，该硬件系统可以用于对两个或者两个以上摄像头执行双目或者多目标定，以及对包含多个摄像头的摄像头阵列进行标定。运动装置400可以驱动安装工位产生位移和旋转，进而改变摄像头的位姿。另外，该硬件系统中的显示屏200和运动装置400应该具有以下位置关系：显示屏200面向摄像头的视野310设置， 使得运动装置400驱动摄像头的位姿发生变化时，显示屏200能够出现在摄像头视野310的各个位置。
需要补充说明的是，图7示出的硬件结构仅仅用于说明实现摄像装置相对于显示屏产生位姿变化的可行性，不构成对本申请实施例的方法和硬件系统的具体限定。例如，在一些其他的实施方式中，如图8所示，可以包括多个控制设备100，和/或，多个显示屏200，和/或，多个运动装置400。本申请实施例对上述硬件结构的具体实现方式不做限定。
图9是本申请实施例示出的控制设备100的结构示意图。该控制设备100例如可以包括计算机、手机、平板电脑、嵌入式系统(embedded system)等设备。如图9所示，该控制设备100可以包括：处理器110、存储器120、接口单元130和输入单元140。其中，处理器110、存储器120、接口单元130和输入单元140耦合连接。存储器120存中存储有可读的程序指令，处理器110用于读取和执行存储器120中的程序指令，以执行本实施例的参数标定方法的相关步骤，例如：通过接口单元130向运动装置发送指令，使得运动装置驱动摄像装置产生位移和旋转等，或者，通过接口单元130向摄像装置发送指令，使得摄像装置拍摄图像或者录制视频等。存储器120中还可以存储有标定图案对应的文档素材，处理器110可以读取并解码上述文档素材，发送到显示屏进行显示。存储设备120还可以用于标定过程中产生的结果文件、中间结果文件、日志、调试信息和其他信息。
本申请实施例中，控制设备100的处理器110可以包括一个或者多个处理单元，例如系统芯片(system on a chip，SoC)、中央处理器(central processing unit，CPU)、微控制器(microcontroller，MCU)、存储控制器等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器110中。
本申请实施例中，控制设备100的存储器120可以包括一个或者多个存储单元，例如可以包括易失性存储器(volatile memory)，如：动态随机存取存储器(dynamic random access memory，DRAM)、静态随机存取存储器(static random access memory，SRAM)等；还可以包括非易失性存储器(non-volatile memory，NVM)，如：只读存储器(read-only memory，ROM)、闪存(flash memory)、硬盘驱动器HDD和固态硬盘SSD等。其中，不同的存储单元可以是独立的器件，也可以集成或者封装在一个或者多个处理器110或者接口单元130中，成为处理器110或者接口单元130的一部分。
本申请实施例中，控制设备100的接口单元130例如可以包括网络适配器(network adapter)、网卡(network interface card)、局域网接收器(LAN adapter)、网络接口控制器(network interface controller，NIC)、调制解调器(modem)和天线等。其中，接口单元130可以是独立的器件，也可以部分或者全部地集成或者封装在处理器110中，成为处理器110的一部分。
本申请实施例中，输入单元140例如可以包括耦合至处理器110的键盘、鼠标、触控显示屏、摇杆控制器、触摸板等。
可以理解的是，本申请实施例示意的结构并不构成对控制设备100的具体限定。在本申请另一些实施例中，控制设备可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或 软件和硬件的组合实现。
图10是本申请实施例示出的运动装置400的结构示意图。如图10所示，该运动装置包括三轴移动平台420、两轴旋转平台430和安装工位410，其中，两轴旋转平台430安装于三轴移动平台420之上，安装工位410固定于两轴旋转平台430，并且安装有待标定的摄像装置300。三轴移动平台420可以沿着图中的X轴、Y轴和Z轴方向产生位移，以改变摄像装置300在世界坐标系(即真实空间)中的位置，两轴旋转平台430可以以Y轴和Z轴为旋转轴产生旋转，以改变摄像装置300在世界坐标系中的俯仰角度和水平角度。
在一种实施方式中，运动装置400的位移和旋转可以通过电机驱动实现。例如图10所示的运动装置400可以包括至少一个用于驱动三轴移动平台420产生位移的电机和至少一个用于驱动两轴旋转平台430产生旋转的电机。适应性地，运动装置400还可以包括可编程的控制器，控制器可以与电机和控制设备100的接口单元130耦合，用于根据控制设备100发出的指令驱动电机运转。可以理解的是，运动装置400的位移和旋转还可以通过其他方式实现，例如液压驱动、电磁驱动等，此处不做具体限定。
需要注意的是，本申请实施例提供的运动装置400，可以有多种不同的形态，并可以通过多种方式实现。上文所讨论的运动装置400的结构，仅仅是一些优选实施方式，用于阐述本申请实施例所述方法和结构的可行性，不对本申请实施例的保护范围进行限制。通过其它方法或顺序实现本申请实施例的运动装置400，亦在本申请实施例的保护范围之内。
本申请实施例提供了一种参数标定方法，该方法例如可以应用于图7-图10任意所示的控制设备。基于图7-图10所示的硬件系统，本方法中的摄像装置可以包括一个摄像装置(例如图7)，也可以包括多个摄像装置(例如图8)。因此，本申请实施中的“摄像装置”还可以被描述为“至少一个摄像装置”。
如图11所示，该方法步骤S101-步骤S106：
步骤S101，生成第一标定图案。
具体实现中，第一标定图案可以通过编程方式生成，并且第一标定图案可以根据本实施例在先使用的标定图案的标定结果和标定需求动态变化。
在一种实现方式中，当采用编程的方式生成第一标定图案时，控制设备可以获取第一标定图案对应的配置参数，根据第一标定图案的配置参数生成第一标定图案，并将第一标定图案对应的图像信号发送到显示屏，使显示屏显示第一标定图案。
为达到上述目的，控制设备可以预先存储有多个标定图案的配置参数，按照预设的标定图案的生成顺序，或者响应于用户在输入单元执行的输入动作，从上述多个配置参数中选择第一标定图案的配置参数，以生成标定图案。或者，控制设备可以在显示屏显示用于引导用户输入配置参数的交互界面，使用户根据对摄像装置的标定需求输入和调试配置参数；控制设备还可以基于用户已经输入的配置参数，在显示屏实时生成对应的第一标定图案，并根据用户对配置参数的调整，实时刷新第一标定图案，直到第一标定图案满足用户需求。对于标定图案的生成方式，本申请不做具体的限定。
可以理解的是，当第一标定图案不相同时，标定图案的配置参数也不相同。
示例地，当第一标定图案是棋盘格图案或者圆点图案时，配置参数例如可以包括： 标志物的分布区域(例如：在显示屏的全屏、居中、左上角、左下角、右上角和右下角分布等)、标志物的密度(例如：在显示屏的一个单位大小的显示区域内包含多少个棋盘格/圆点)、标志物的总数量、棋盘格/圆点的尺寸(例如：棋盘格的每个正方形的长度和宽度，圆点的直径等，上述长度、宽度和直径可以通过像素数量确定)和标志物之间的水平间距和垂直间距等(上述水平间距和垂直间距可以通过像素数量确定)。
在其他示例中，当第一标定图案是灰度板图案时，配置参数例如可以包括：色卡的数量、位置、尺寸、每个色卡的灰度值和颜色等参数。
具体实现中，第一标定图案可以根据对应的图像文件生成，控制设备的存储器中还可以预先存储有至少一张图像文件，每个图像文件包含一种标定图案。上述图像文件例如可以包含棋盘格图案、圆点点阵图案、灰度板图案、条纹图案、纯色图案等。
在一种实现方式中，当存储器存储有图像文件时，控制设备可以按照预设的标定图案的生成顺序，从上述至少一张图像文件中选择第一标定图案对应的第一图像文件，对第一图像文件进行解码以生成图像信号，将图像信号输出至显示屏进行显示，从而使得显示屏显示第一标定图案。
在另一种实现方式中，当存储器存储有图像文件时，显示屏可以显示用于浏览和选择图像文件的图形操作界面，使得用户可以基于图形操作界面使用输入单元执行各种输入动作，控制设备响应于用户在输入单元的输入动作，从至少一张图像文件中选择第一图像文件，生成对应的第一标定图案的预览并发送到显示屏进行显示，当用户进一步执行输入动作以确认该选择时，控制设备使显示屏显示第一标定图案。
应当理解的是，本申请实施例的标定图案还可以通过多种方式实现，上述各个实现方式仅仅是一些优的选实施方式，不对本申请实施例的保护范围进行限制，能够在本申请实施例应用的其他生成标定图案的方式，亦在本申请实施例的保护范围之内。
需要补充说明的是，本申请实施例中的显示屏可以优选使用具备主动光源的自发光显示屏，以提高标定图案的质量，进而提高摄像装置的标定质量。其中，自发光显示器例如可以包括：有机发光二极管(organic light-emitting diode，OLED)显示屏、有源矩阵有机发光二极管(active-matrix organic light-emitting diode，AMOLED)显示屏、量子点显示屏(quantum dot display)等。与传统的印刷形成的标定板相比，使用自发光的显示屏显示标定图案具有以下优点：
1、显示屏显示的标定图案由控制设备控制，具有可再现、可动态调整的能力。
2、显示屏各个位置的标定图案不受环境光源变化和照明条件差异的影响，一致性高，有利于提高摄像装置拍摄的图像的质量、避免在后续的图像处理过程中出现特征点检测失败、边缘检测失败等问题，有利于提高标定质量。
3、传统标定板在印刷过程中，色彩模式会从其他模式转换成印刷模式(例如：CMYK)，使得一些色彩丢失或改变，因此无法用于标定摄像装置的色差、白平衡等参数。而显示屏可以使用与摄像装置拍的是图片相同的色彩模式和色域，能够实现对摄像装置的色差和白平衡等参数的标定。
步骤S102，发送第一指令，第一指令用于指示摄像装置在第一时段内的至少一组位姿参数。
本申请实施例中，摄像装置相对于控制设备的位姿可以由一组位姿参数确定，每 一组位姿参数可以确定摄像装置的一个位姿。具体实现中，一组位姿参数可以包括至少一个位置参数，和/或，至少一个角度参数。其中，位置参数用于确定摄像装置在世界坐标系中的位置，例如，位置参数可以包括摄像装置在世界坐标系中的X轴、Y轴和Z轴坐标的部分或者全部坐标值，例如(X 1，Y 1，Z 1)、(X 2，Y 2)、(X 3，Z 3)或(X 4)等，可以选择的是，当一组位姿参数包含有部分位置参数时，表明摄像装置在缺失的位置参数所对应的坐标值不发生变化。角度参数用于确定摄像装置相对于世界坐标系的各个坐标轴的角度，例如，角度参数可以包括摄像装置在世界坐标系中相对于X轴、Y轴和Z轴的部分或者全部角度值，例如(α 1，β 1，γ 1)、(α 2，γ 2)、(β 3，γ 3)或(β 4)等，可选择的是，当一组位姿参数包含有部分角度参数时，表明摄像装置在缺失的角度参数所对应的坐标值不发生变化。
具体实现中，基于图7所示的硬件系统，控制设备可以向运动装置发送第一指令，运动装置可以承载至少一个摄像装置，根据第一指令指示的至少一组位姿参数，驱动摄像装置平移和旋转，以控制至少一个摄像装置的位姿。
图12是本申请实施例提供的调整摄像装置位姿的示意图。如图12所示，在一种实现方式中，控制设备可以在第一时段内向运动装置发送多个第一指令，每个第一指令指示一组位姿参数，不同的第一指令指示的位姿参数不同或者相同。由此，运动装置在第一时段内能够接收到多个第一指令，每接收到一个第一指令，运动装置就会根据该第一指令指示的位姿参数执行一次平移和旋转，将摄像装置调整到对应的位姿。可选的，上述第一时段内的多个第一指令的发送可以是依次在不同时刻进行。
示例地，如图12所示，在第一时段内的N个(N为自然数)时刻，控制设备向运动装置发送了第一指令1至第一指令N，第一指令1至第一指令N依次指示第1组位姿参数至第N组位姿参数。那么，在t1时刻，控制设备向运动装置发送第一指令1，调整摄像装置的位姿从初始位姿0至位姿1；在t2时刻，控制设备向运动装置发送第一指令2，调整摄像装置的位姿从位姿1至位姿2；在t3时刻，控制设备向运动装置发送第一指令3，调整摄像装置的位姿从位姿2至位姿3；以此类推……，在tN时刻，控制设备向运动装置发送第一指令N，调整摄像装置的位姿从位姿N-1至位姿N。
图13是本申请实施例提供的调整摄像装置位姿的示意图。如图3所示，在一种实现方式中，控制设备可以在第一时段内只发送一个第一指令，该第一指令包含多个不同的位姿参数，上述多个不同的位姿参数能够连接成一条移动轨迹，以及，摄像装置的角度变化趋势。由此，运动装置在接收了第一指令之后，可以驱动摄像头沿着第一指令指示的移动轨迹进行移动，并在移动过程中调整摄像装置的角度，使得摄像装置在第一时段内具有不同的位姿。
示例地，如图13所示，在第一时段内的某一时刻，控制设备向运动装置发送第一指令1，该第一指令1通过多个不同的第1组位姿参数-第M组位姿参数(M为自然数)指示一条移动轨迹L1。运动装置响应于第一指令1，沿着轨迹L1移动摄像装置，使得摄像装置的位姿从位姿0至位姿M发生变化。在变化过程中，如果运动装置接收到了新的第一指令2，运动装置可以中断沿着轨迹L1的移动过程，转而沿着新的第一指令2指示的轨迹L2，继续调整摄像装置的位姿；或者，如果运动装置接收到了新的第一指令2，运动装置可以继续沿着轨迹L1移动，在沿着轨迹L1移动完成后，再沿 着新的第一指令2指示的轨迹L2，继续调整摄像装置的位姿。
在一种实现方式中，基于图10所示的运动装置，第一指令指示摄像装置的位姿参数可以通过以下方式实现：第一指令可以包含运动装置的至少一个控制参数，例如：三轴移动平台沿X轴、Y轴和Z轴位移的距离、线速度和/或者位移时间，两轴旋转平台围绕Y轴和Z轴旋转的角度、角速度和/或者旋转时间等。上述控制参数可以被发送到运动装置的控制器，使得控制器根据上述控制参数确定电机的旋转方向、转速和旋转时间等信息，进而驱动电机运转，使得三轴移动平台和两轴旋转平台产生上述控制参数对应的位移或旋转，进而将摄像装置调整到对应的位姿。
步骤S103，向摄像装置发送第二指令，第二指令用于指示摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个。
具体实现中，摄像装置执行至少一次拍摄包括拍摄照片和/或拍摄视频。
在一种实现方式中，当摄像装置拍摄照片时，控制设备可以在第一时段内交替向摄像装置发送多个第一指令和多个第二指令，摄像装置每接收到一个第二指令，即对应拍摄一张照片，由于摄像装置在第一时段内的位姿参数在发生变化，摄像装置能够拍摄到位姿参数不同的多张照片，在上述多张照片中，显示屏的标定图案可能全部或者部分地出现在照片中，并且，在不同的照片中，标定图案出现的位置也可能不同。
示例地，如图14所示，在第一时段内的t0时刻，摄像装置位于显示屏的左上方，位姿参数为0。在第一时段内的N个(N为自然数)时刻，控制设备依次向运动装置发送了第一指令1至第一指令N，第一指令1至第一指令N依次指示第1组位姿参数至第N组位姿参数，并且，与N个第一指令交替地，控制设备还依次向运动装置发送了多个(例如N个)第二指令，使得摄像装置在不同的位姿参数下拍摄照片。例如：在t1时刻，控制设备向运动装置发送第一指令1，调整摄像装置的位姿从初始位姿0至位姿1，然后，控制设备向摄像装置发送第二指令1，使得摄像装置在位姿1拍摄至少一张照片；在t2时刻，控制设备向运动装置发送第一指令2，调整摄像装置的位姿从位姿1至位姿2，然后，控制设备向摄像装置发送第二指令2，使得摄像装置在位姿2拍摄至少一张照片；在tN时刻，控制设备向运动装置发送第一指令N，调整摄像装置的位姿从位姿N-1至位姿N，然后，控制设备向摄像装置发送第二指令N，使得摄像装置在第N组位姿参数拍摄至少一张照片。
在一种实现方式中，当摄像装置拍摄照片时，第二指令具体可以用于指示摄像装置拍摄照片的时间间隔ΔT，因此，控制设备可以只需要向摄像装置发送一次第二指令。使得摄像装置在位姿参数变化的过程中，每隔ΔT拍摄至少一张照片。其中，第一指令和第二指令可以同时发送，也可以不同时发送，例如，先发送第一指令后发送第二指令，或者，先发送第二指令后发送第一指令。
示例地，如图15所示，在t0时刻，控制设备向运动装置发送第一指令，该第一指令通过多个不同的第1组位姿参数-第M组位姿参数(M为自然数)指示一条轨迹L1。运动装置响应于第一指令，沿着轨迹L1移动摄像装置，使得摄像装置的位姿从位姿0至位姿M发生变化。在摄像装置沿着轨迹L1移动的过程中，控制设备可以向摄像装置发送第二指令，该第二指令指示时间间隔ΔT，使得摄像装置每隔ΔT拍摄纸一张照片，直至摄像装置沿轨迹L1移动完毕。
在一种实现方式中，当摄像装置拍摄视频时，控制设备可以在第一时段内仅向摄像装置发送一次第二指令，摄像装置在接收到第二指令之后，即开始拍摄视频，随着摄像装置位姿的变化，显示屏的标定图案可能全部或者部分地出现在视频中，并且，在不同的视频帧中，标定图案出现的位置也可能不同。其中，第一指令和第二指令可以同时发送，也可以不同时发送，例如，先发送第一指令后发送第二指令，或者，先发送第二指令后发送第一指令。
可选地，在控制设备向摄像装置发送第二指令之后，控制设备还可以向摄像装置发送终止指令，使得摄像装置停止拍摄视频。
示例地，如图16所示，在时刻t0，控制设备向运动装置发送第一指令，该第一指令通过多个不同的第1组位姿参数-第M组位姿参数(M为自然数)指示一条轨迹L1。运动装置响应于第一指令，沿着轨迹L1移动摄像装置，使得摄像装置的位姿从位姿1至位姿M发生变化。在摄像装置沿着轨迹L1移动的过程中，控制设备可以向摄像装置发送第二指令，使得摄像装置开始一边变化位姿，一边拍摄视频。在之后的某个时刻tM，例如摄像装置沿着轨迹L1移动结束之后，控制设备向摄像装置发送终止指令，使得摄像装置停止拍摄视频。
可选的，当第二指令用于指示摄像装置拍摄视频时，控制设备可以多次交替地发送第二指令和终止指令，使得摄像装置在位姿改变过程中拍摄多个视频片段。
示例地，如图17所示，在摄像装置沿着轨迹L1移动的过程中，控制设备在t11时刻发送第一个第二指令，在t12时刻发送第一个终止指令，使摄像装置拍摄t11-t12时刻的视频片段1；控制设备在t21时刻发送第二个第二指令，在t22时刻发送第二个终止指令，使摄像装置拍摄t21-t22时刻的视频片段2；控制设备在tP1时刻(P为自然数)发送第P个第二指令，在tP2时刻发送第P个终止指令，使摄像装置拍摄tP1-tP2时刻的视频片段P。其中，上述t11时刻例如可以是摄像装置移动到位姿1之前，t12时刻例如可以是摄像装置移动到位姿1之后，使得视频片段1包含摄像装置在位姿1拍摄的视频帧；t21时刻例如可以是摄像装置移动到位姿2之前，t22时刻例如可以是摄像装置移动到位姿2之后，使得视频片段2包含摄像装置在位姿2拍摄的视频帧；t31时刻例如可以是摄像装置移动到位姿3之前，t32时刻例如可以是摄像装置移动到位姿3之后，使得视频片段3包含摄像装置在位姿3拍摄的视频帧。
步骤S104，获取来自摄像装置的至少一个拍摄结果。
具体实现中，其中，上述拍摄结果包括摄像装置拍摄的照片和/或者视频。摄像装置可以将拍摄的照片和视频回传给控制设备，控制设备将摄像装置回传的照片和视频保存在存储器中，以供后续步骤中计算摄像装置的内部参数使用。例如，摄像装置可以将照片和视频的模拟信号发送给采集卡，由采集卡进行信号的模拟-数字转换，然后将转换得到的照片和视频的数字信号发送给控制设备进行存储。
步骤S105，根据至少一个拍摄结果中的部分或者全部拍摄结果，确定摄像装置的至少一个内部参数。
其中，上述“部分或者全部摄像结果”中的任意拍摄结果应当包含第一标定图案的部分或者全部图案。进一步可选的，所述“部分或者全部摄像结果”使得第一标定图案出现在照片或视频的各个位置(即摄像装置视野的各个位置)，尤其是畸变严重 的位置。
具体实现中，当拍摄结果是照片时，控制设备可以从上述“部分或者全部”照片中检测获取特征点。每个特征点都有在照片中的像素坐标系坐标和在显示屏中的世界坐标系坐标，根据上述特征点的像素坐标系坐标和世界坐标系坐标，就可以求解上述“世界坐标系转换为像素坐标系”的公式(即公式(1))中M 1、M 2等内部参数。
具体实现中，当拍摄结果是视频时，控制设备可以从视频中提取至少一个视频帧，所提取的每个视频帧都包含有第一标定图案的部分或者全部图案。然后，控制设备可以从上述至少一个视频帧中检测获取特征点。每个特征点都有在视频帧中的像素坐标系坐标和在显示屏中的世界坐标系坐标，根据上述特征点的像素坐标系坐标和世界坐标系坐标，就可以求解上述“世界坐标系转换为像素坐标系”的公式(即公式(1))中M 1、M 2等内部参数。
如图18所示，特征点可以是容易从第一标定图案中识别的点。例如，棋盘格标定图案的特征点可以是各个方格的角点，圆点标定图案的特征点是可以各个圆点的圆心点。特征点可以通过对照片或者视频帧进行二值化处理和边缘检测等方法得到，此处不再赘述。
步骤S106，判断至少一个内部参数中的至少一个目标内部参数的误差是否均满足其各自对应的误差阈值，如果有任何一个目标内部参数的误差不满足其对应的误差阈值，返回步骤S102；如果所有的目标内部参数均满足其各自对应的误差阈值，输出标定结果。
在具体实现场景中，技术人员可能会对摄像装置的标定有不同的需求，例如：在一些实现场景中，技术人员希望标定摄像装置全部的内部参数，在另一些实现场景中，技术人员只希望标定摄像装置的一部分内部参数。为了便于区分需要标定的内部参数和不需要标定的内部参数，这里将需要标定的内部参数称作目标内部参数。可以理解的是，目标内部参数可以有一个或者多个，并且，目标内部参数可以是上述“至少一个内部参数”的全部参数或者部分参数。
本申请实施例中，如公式(1)所示，内部参数部分或者全部地包含在内参矩阵中，因此，求解内部参数的过程实际上可以是求解内参矩阵的过程。当所有的内部参数或者目标内部参数都满足其对应的误差阈值时，可以认为内参矩阵是准确的，满足输出条件，因此，在步骤S106中，如果所有的目标内部参数均满足其各自对应的误差阈值，输出标定结果，如果有任何一个目标内部参数的误差不满足其对应的误差阈值，则需要返回步骤S102继续标定，使内参矩阵实现迭代更新，得到更多准确的内部参数。
本申请实施例中，内部参数的误差例如可以包括重复精度、重投影误差等。本申请实施例中可以默认第一次标定得到的内部参数是不准确的，从而在第一次标定之后，返回步骤S102执行第二次标定以及后续的重复标定，以便于计算标定误差。
其中，重复精度是指在相同的标定环境下重复执行多次标定所得到的多次内部参数之间的偏差。偏差越小，说明内部参数的误差越小；偏差越大，说明内部参数的误差越大。上述相同的标定环境是指至少两次标定所使用的标定图案、摄像装置的位姿变化轨迹、显示屏的亮度、对比度、特征点的数量、提取精度、用于计算标定结果的照片或者视频帧的数量、拍摄的位姿等部分或者全部相同。
具体实现中，内部参数中的至少一个参数可以具有对应的误差阈值，该误差阈值可以对应重复精度或者重投影误差，例如：焦距阈值、光心阈值、像元尺寸阈值、光学畸变阈值、白平衡阈值等。那么，作为示例地，在计算重复精度时，如果至少两次标定的焦距的偏差大于焦距阈值，和/或，两次标定的光心的偏差大于光心阈值，和/或，两次标定的像元尺寸的偏差大于像元尺寸阈值，和/或，两次标定的光学畸变的偏差大于光学畸变阈值，和/或，两次标定的白平衡的偏差大于白平衡阈值，则需要返回步骤S102继续标定。
另外，使用标定得到的摄像装置的内部参数和外部参数，对世界坐标系中的坐标点进行投影计算，能够得到该坐标点在像素坐标系中的投影值，同时，该坐标点在摄像装置的拍摄结果中还对应一个像素坐标系的观测值，投影值与观测值的误差就是重投影误差。重投影误差越小，说明内部参数的误差越小，重投影误差越大，说明内部参数的误差越大。
具体实现中，例如可以采用以下方式执行重复标定：
方式一：可以改变标定图案。
示例地，如图19所示，当采用棋盘格图案或者圆点图案(例如：第一标定图案)标定摄像装置的内部参数时，标定图案在一次标定过程中是固定不变的，即标志物的数量和大小不变。在这种情况下，如果标定精度不满足精度要求，可能是特征点的数量不足。因此，在下一次标定时，可以通过减小标志物的尺寸、并增加标志物的数量的方式得到新的标定图案(例如：第二标定图案)，以提高控制设备在步骤S105中从拍摄结果中提取到的特征点的数量，以提高标定精度。
示例地，如图20所示，当采用棋盘格图案或者圆点图案(例如：第一标定图案)标定摄像装置的内部参数时，如果标定精度不满足精度要求，可能是标定图案不够清晰，导致了特征点提取精度较差。因此执行重复标定时，可以通过增大标志物的尺寸的方式得到新的标定图案(例如：第二标定图案)，以提高特征点的提取精度。
方式二：调整从至少一个拍摄结果中提取的对应摄像装置各个视野的特征点数量。
示例地，如图21所示，当采用棋盘格图案或者圆点图案标定摄像装置的内部参数时，如果标定精度不满足精度要求，可能是特征点没有均匀地分布在摄像装置视野的各个区域，例如：特征点集中在某个区域，或者，某个区域没有分布特征点或者只分布了极少数的特征点，影响了内参矩阵算法的权重。因此，特征点集中在某个区域，执行重复标定时，可以删除该区域的部分特征点，或者增加其他区域的特征点。
具体实现中，如图21所示，可以将摄像装置的视野切分为K个子区域，例如通过九宫格切分为子区域1-子区域9。控制设备从拍摄结果中提取特征点之后，可以分别统计子区域1-子区域9中的特征点的数量。如果某个子区域，例如子区域5的特征点的数量明显多于其他子区域(例如：子区域5的特征点的数量比其他子区域的特征点的数量多出某个预设的百分比)，则删除子区域5中的部分节点，使子区域5的特征点的数量与其他任意子区域的特征点的数量趋近。如果某个子区域，例如子区域1的特征点的数量明显小于其他子区域，则可以移动摄像装置的位姿，使标定图案位于子区域1内，并再次拍摄视频或者照片，从再次拍摄的视频帧或者照片中提取特征点，以补足子区域1的特征点。需要补充说明的是，在一次调整中，控制设备可以对个子 区域的特征点进行动态的均一化调整，使得各个子区域的特征点的数量趋于统一，以提高标定精度。
方式三：可以调整标定图案的图像质量。
示例地，当采用棋盘格图案或者圆点图案标定摄像装置的内部参数时，如果标定精度不满足精度要求，可能是显示屏的亮度、对比度等影响图像质量的参数设置不合理，或者对拍摄结果进行二值化处理和边缘检测使用的相关阈值不合理。因此执行重复标定时，还可以改变显示屏的亮度和对比度，或者调整二值化处理和边缘检测的相关阈值，以提高特征点的提取精度。
可以理解的是，基于上述示例的重复标定的方法，本申请实施例可以对摄像装置执行多次标定，使摄像装置的内参矩阵和外参矩阵不断迭代更新，标定精度不断提高，直至标定精度的取值小于对应的阈值。
需要补充说明的是，当采用棋盘格图案或者圆点图案标定摄像装置时，不同的摄像装置的视野(field of view，FOV)范围和景深的差异较大，对标定的准确性会带来不同的影响。为了提高各种视野范围和景深的摄像装置的标定精度，本申请实施例的方法在具体实施时可以根据摄像装置的视野范围和景深动态调整标定参数。例如：当摄像装置的视野范围较小时，可以适当减小标定图案中的标志物的数量，以增加视野内的特征点的数量；当摄像装置的视野范围较大时，可以适当增大标定图案中的标志物的数量，以减小视野内的特征点数量并提高特征点的识别精度。另外，在摄像装置改变位姿的过程中，应当使标定图案出现在摄像装置视野的各个位置，尤其是摄像装置的视野边缘(视野边缘往往是畸变严重的区域)，使得后续提取到的特征点能够均匀地分布在摄像装置的视野中，提高标定精度。
由以上技术方案可知，本申请实施例提供的参数标定方法，可以应用于摄像装置的内部参数和外部参数的标定，该方法在标定图案的位置固定不变的情况下，通过变化摄像装置的位姿参数的方式，精准调整摄像装置相对于标定图案的位置和角度，使摄像装置在相对于标定图案各个既定的位置和角度对标定图案进行精确拍摄，得到至少一个拍摄结果，上述拍摄结果能够与像素坐标系和世界坐标系的坐标精确对应，使得相应计算得到的内部参数和外部参数的准确性更高，提高了标定精度。
可以理解的是，当第一标定图案不同时，本申请实施例的方法还可以用于标定不同的参数。
示例地，如图22所示，当第一标定图案是包含不同灰度、不同颜色的灰度板图案时，本申请实施例的方法可以用于标定摄像装置的白平衡参数。当第一标定图案是单色图案时，通过调整摄像装置的位姿参数，使单色图案出现在摄像装置视野的各个位置并拍摄相应的照片或视频，可以根据拍摄结果计算摄像装置的晕影参数。当第一标定图案是包含不同角度线条的标板图案时，通过调整摄像装置的位姿参数，使线条出现在摄像装置视野的各个位置并拍摄相应的照片或视频，可以标定摄像装置的光学分辨率调制传递函数MTF。当第一标定图案是包含多种标准颜色的色卡图案时，可以标定摄像装置的色差参数，上述各种标定图案可以预设在控制设备中，以在需要的时候生成。
上述本申请提供的实施例中，分别从设备本身、以及从设备之间交互的角度对本 申请提供的数据转发方法、数据缓存方法的各方案进行了介绍。可以理解的是，各个设备，例如上述网络设备和存储设备为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
上述本申请提供的实施例中，分别从设备/装置本身、以及从设备/装置之间交互的角度对本申请提供的参数标定方法的各方案进行了介绍。可以理解的是，各个设备/装置，例如上述控制设备和运动装置为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
例如，上述控制设备通过软件模块来实现相应的功能。该控制设备例如可以包括计算机、手机、平板电脑、嵌入式系统(embedded system)等设备。
在一个实施例中，如图23所示，用于实现上述控制设备行为的功能的参数标定装置包括：生成单元501，用于生成第一标定图案；第一发送单元502，用于发送第一指令，所述第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；第二发送单元503，用于向摄像装置发送第二指令，所述第二指令用于指示所述至少一个摄像装置在第一时段内的拍摄时机和拍摄次数；获取单元504，用于获取来自所述至少一个摄像装置的至少一个拍摄结果；处理单元505，用于根据所述至少一个拍摄结果中的部分或者全部拍摄结果，确定所述至少一个摄像装置的内部参数，所述部分或者全部摄像结果中的任意摄像结果包含所述第一标定图案的部分或者全部图案。
本申请提供的参数标定装置，可以应用于摄像装置的内部参数和外部参数的标定，该方法在标定图案的位置固定不变的情况下，通过变化摄像装置的位姿参数的方式，精准调整摄像装置相对于标定图案的位置和角度，使摄像装置在相对于标定图案各个既定的位置和角度对标定图案进行精确拍摄，得到至少一个拍摄结果，上述拍摄结果能够与像素坐标系和世界坐标系的坐标精确对应，使得相应计算得到的内部参数和外部参数的准确性更高，提高了标定精度。
可选的，所述第一发送单元502，具体用于向运动装置发送第一指令，所述运动装置承载所述至少一个摄像装置，所述运动装置用于根据所述至少一组位姿参数控制所述至少一个摄像参数的位姿。由此，运动装置可以根据位姿参数精确地控制摄像装置的位姿，并可以实现位姿的重现，有利于提高标定精度。
可选的，所述拍摄结果包括所述至少一个摄像装置响应于所述第二指令拍摄的至少一张照片和/或至少一段视频。由此，参与计算摄像装置的内部参数的图像可以从照片中选取，也可以从视频的视频帧中选取，方式灵活。
可选的，所述第一时段内包括多个所述第一指令和多个所述第二指令，所述第一 发送单元502和所述第二发送单元503具体用于交替发送所述多个所述第一指令和多个所述第二指令。由此，通过第一指令和第二指令可以根据需求指示摄像装置在不同的位姿下拍摄不同的照片或者视频，例如使得第一标定图案出现在摄像装置视野的各个位置，提高标定精度。
可选的，所述内部参数中的至少一个参数具有对应的误差阈值；所述处理单元505，用于确定所述内部参数中的至少一个参数的重复精度是否大于对应的所述误差阈值；所述生成单元501，用于当所述重复精度大于对应的所述误差阈值时，调整所述第一标定图案为第二标定图案，和/或，调整所述第一标定图案的图像质量；所述处理单元505，用于当所述重复精度大于对应的所述误差阈值时，调整从所述第一标定图案提取的特征点的数量。由此，本申请可以对摄像装置执行多次标定，使摄像装置的内参矩阵和外参矩阵不断迭代更新，标定精度不断提高，直至标定精度的取值小于对应的阈值。
可选的，所述第一标定图案和所述第二标定图案的标志物的数量和/或尺寸不同。
可选的，所述处理单元505，用于将所述第一标定图案切分为至少一个子区域，将所述至少一个子区域的所述特征点的数量进行均一化调整。由此，使得各个子区域的特征点的数量区域统一，以提高标定精度。
可选的，所述第一标定图案为预设的灰度板图案，所述内部参数包含所述至少一个摄像装置的白平衡参数。
可选的，所述第一标定图案为预设的单色图案，所述内部参数包含所述至少一个摄像装置的影晕参数。
可选的，所述第一标定图案为预设的包含多种标准颜色的色卡图案，所述内部参数包括所述至少一个摄像装置的色差参数。
可选的，所述第一标定图案为预设的包含不同角度线条的标板图案，所述内部参数包括所述至少一个摄像装置调制传递函数参数。
另外，例如，上述设备通过硬件模块来实现相应的功能。
在一个实施例中，控制设备的上述功能可以通过图9所示的结构实现。例如，接口单元130用于向运动装置发送第一指令，以及，向至少一个摄像装置发送第二指令，以及，获取来自至少一个摄像装置的至少一个拍摄结果；存储器120用于存储计算机程序代码，该计算机程序代码包括计算机指令；当处理器110执行计算机指令时，使控制设备执行上述各实施例中涉及的方法，例如：生成第一标定图案；发送第一指令，所述第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；向所述至少一个摄像装置发送第二指令，所述第二指令用于指示所述至少一个摄像装置在第一时段内的拍摄时机和拍摄次数；获取来自所述至少一个摄像装置的至少一个拍摄结果；根据所述至少一个拍摄结果中的部分或者全部拍摄结果，确定所述至少一个摄像装置的内部参数，所述部分或者全部摄像结果中的任意摄像结果包含所述第一标定图案的部分或者全部图案。
本申请实施例还提供一种参数标定系统。如图7所示，该参数标定系统包括：控制设备100，用于生成第一标定图案；显示屏200，耦合至控制设备100，用于显示第一标定图案；运动装置400，用于承载至少一个摄像装置300，使至少一个摄像装置300 的视野面向显示屏200，运动装置400耦合至控制设备100；控制设备100，还用于向运动装置400发送第一指令，第一指令用于指示至少一个摄像装置300在第一时段内的至少一组位姿参数；运动装置400，还用于根据至少一组位姿参数控制至少一个摄像装置300的位姿；控制设备100，还用于向至少一个摄像装置300发送第二指令，第二指令用于指示至少一个摄像装置300在第一时段内的拍摄时机和拍摄次数；以及，获取来自至少一个摄像装置300的至少一个拍摄结果；以及，根据至少一个拍摄结果中的部分或者全部拍摄结果，确定至少一个摄像装置300的内部参数，部分或者全部摄像结果中的任意摄像结果包含第一标定图案的部分或者全部图案。
本申请实施例还提供一种计算机存储介质，计算机存储介质中存储有计算机指令，当其在计算机上运行时，使得计算机执行上述各方面的方法。
本申请实施例还提供一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述各方面的方法。
本申请还提供了一种芯片系统。该芯片系统包括处理器，用于支持上述装置或设备实现上述方面中所涉及的功能，例如，生成或处理上述方法中所涉及的信息。在一种可能的设计中，芯片系统还包括存储器，用于保存上述装置或设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包含芯片和其他分立器件。
以上的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本发明的保护范围之内。

Claims (25)

  1. 一种参数标定方法，其特征在于，
    生成第一标定图案；
    发送第一指令，所述第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；
    向所述至少一个摄像装置发送第二指令，所述第二指令用于指示所述至少一个摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个；
    获取来自所述至少一个摄像装置的至少一个拍摄结果；
    根据所述至少一个拍摄结果中的部分或者全部拍摄结果，确定所述至少一个摄像装置的至少一个内部参数，所述部分或者全部摄像结果中的任意摄像结果包含所述第一标定图案的部分或者全部图案。
  2. 根据权利要求1所述的方法，其特征在于，所述发送第一指令，包括：
    向运动装置发送所述第一指令，所述运动装置承载所述至少一个摄像装置，所述至少一组位姿参数用于所述运动装置控制所述至少一个摄像装置的位姿。
  3. 根据权利要求1或2所述的方法，其特征在于，所述拍摄结果包括来自所述至少一个摄像装置的、响应于所述第二指令拍摄的至少一张照片和/或至少一段视频。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，
    所述发送第一指令，包括：在所述第一时段内发送多个所述第一指令；
    所述向所述至少一个摄像装置发送第二指令，包括：在第一时段内向所述至少一个摄像装置发送多个第二指令；
    其中，多个所述第一指令和多个所述第二指令交替发送。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述内部参数中的至少一个参数具有对应的误差阈值，当所述内部参数中的至少一个参数的重复精度大于对应的所述误差阈值时，所述方法还包括：调整所述第一标定图案为第二标定图案，和/或，调整从所述至少一个拍摄结果中提取的对应所述至少一个摄像装置各个视野的特征点数量，和/或，调整所述第一标定图案的图像质量。
  6. 根据权利要求5所述的方法，其特征在于，所述第一标定图案和所述第二标定图案中的标志物的数量和/或尺寸不同。
  7. 根据权利要求5或6所述的方法，其特征在于，所述调整从所述至少一个拍摄结果中提取的对应所述至少一个摄像装置各个视野的特征点数量，包括：确定所述第一标定图案包含的至少一个子区域，将所述至少一个子区域的所述特征点的数量进行均一化调整。
  8. 根据权利要求1-7任一项所述的方法，其特征在于，所述第一标定图案为预设的灰度板图案，所述内部参数包含所述至少一个摄像装置的白平衡参数。
  9. 根据权利要求1-7任一项所述的方法，其特征在于，所述第一标定图案为预设的单色图案，所述内部参数包含所述至少一个摄像装置的影晕参数。
  10. 根据权利要求1-7任一项所述的方法，其特征在于，所述第一标定图案为预设的包含多种标准颜色的色卡图案，所述内部参数包括所述至少一个摄像装置的色差参数。
  11. 根据权利要求1-7任一项所述的方法，其特征在于，所述第一标定图案为预设的包含不同角度线条的标板图案，所述内部参数包括所述至少一个摄像装置调制传递函数参数。
  12. 一种参数标定装置，其特征在于，包括：
    生成单元，用于生成第一标定图案；
    第一发送单元，用于发送第一指令，所述第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；
    第二发送单元，用于向摄像装置发送第二指令，所述第二指令用于指示所述至少一个摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个；
    获取单元，用于获取来自所述至少一个摄像装置的至少一个拍摄结果；
    处理单元，用于根据所述至少一个拍摄结果中的部分或者全部拍摄结果，确定所述至少一个摄像装置的至少一个内部参数，所述部分或者全部摄像结果中的任意摄像结果包含所述第一标定图案的部分或者全部图案。
  13. 根据权利要求12所述的装置，其特征在于，
    所述第一发送单元，具体用于向运动装置所述发送第一指令，所述运动装置承载所述至少一个摄像装置，所述至少一组位姿参数用于所述运动装置控制所述至少一个摄像参数的位姿。
  14. 根据权利要求12或13所述的装置，其特征在于，所述拍摄结果包括来自所述至少一个摄像装置的、响应于所述第二指令拍摄的至少一张照片和/或至少一段视频。
  15. 根据权利要求12-14任一项所述的装置，其特征在于，
    所述第一发送单元，用于在所述第一时段内发送多个所述第一指令；
    所述第二发送单元，用于在第一时段内向所述至少一个摄像装置发送多个第二指令；
    其中，多个所述第一指令和多个所述第二指令交替发送。
  16. 根据权利要求12-15任一项所述的装置，其特征在于，
    所述内部参数中的至少一个参数具有对应的误差阈值；
    所述处理单元，用于确定所述内部参数中的至少一个参数的重复精度是否大于对应的所述误差阈值；
    所述生成单元，用于当所述重复精度大于对应的所述误差阈值时，调整所述第一标定图案为第二标定图案，和/或，调整所述第一标定图案的图像质量；
    所述处理单元，用于当所述重复精度大于对应的所述误差阈值时，调整从所述至少一个拍摄结果中提取的对应所述至少一个摄像装置各个视野的特征点数量。
  17. 根据权利要求16所述的装置，其特征在于，所述第一标定图案和所述第二标定图案中的标志物的数量和/或尺寸不同。
  18. 根据权利要求16或17所述的装置，其特征在于，
    所述处理单元，用于确定所述第一标定图案包含的至少一个子区域，将所述至少一个子区域的所述特征点的数量进行均一化调整。
  19. 根据权利要求12-18任一项所述的装置，其特征在于，所述第一标定图案为预设的灰度板图案，所述内部参数包含所述至少一个摄像装置的白平衡参数。
  20. 根据权利要求12-18任一项所述的装置，其特征在于，所述第一标定图案为预设的单色图案，所述内部参数包含所述至少一个摄像装置的影晕参数。
  21. 根据权利要求12-18任一项所述的装置，其特征在于，所述第一标定图案为预设的包含多种标准颜色的色卡图案，所述内部参数包括所述至少一个摄像装置的色差参数。
  22. 根据权利要求12-18任一项所述的装置，其特征在于，所述第一标定图案为预设的包含不同角度线条的标板图案，所述内部参数包括所述至少一个摄像装置调制传递函数参数。
  23. 一种参数标定装置，其特征在于，包括：存储器和处理器；所述存储器和所述处理器耦合；所述存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述处理器执行所述计算机指令时，使所述装置执行如权利要求1-11中任一项所述的方法。
  24. 一种参数标定系统，其特征在于，包括：
    控制设备，用于生成第一标定图案；
    显示屏，耦合至所述控制设备，用于显示所述第一标定图案；
    运动装置，用于承载至少一个摄像装置，使所述至少一个摄像装置的视野面向所 述显示屏，所述运动装置耦合至所述控制设备；
    所述控制设备，还用于向所述运动装置发送第一指令，所述第一指令用于指示至少一个摄像装置在第一时段内的至少一组位姿参数；
    所述运动装置，还用于根据所述至少一组位姿参数控制所述至少一个摄像装置的位姿；
    所述控制设备，还用于向所述至少一个摄像装置发送第二指令，所述第二指令用于指示所述至少一个摄像装置在第一时段内的拍摄时机和拍摄次数中的至少一个；以及，
    获取来自所述至少一个摄像装置的至少一个拍摄结果；以及，
    根据所述至少一个拍摄结果中的部分或者全部拍摄结果，确定所述至少一个摄像装置的至少一个内部参数，所述部分或者全部摄像结果中的任意摄像结果包含所述第一标定图案的部分或者全部图案。
  25. 一种计算机存储介质，其特征在于，包括计算机指令，当所述计算机指令在控制设备上运行时，使得所述控制设备执行如权利要求1-11中任一项所述的方法。
PCT/CN2019/129932 2019-12-30 2019-12-30 参数标定方法及装置 Ceased WO2021134219A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP19958494.7A EP4071713B1 (en) 2019-12-30 2019-12-30 Parameter calibration method and apapratus
CN201980022070.8A CN113330487B (zh) 2019-12-30 2019-12-30 参数标定方法及装置
PCT/CN2019/129932 WO2021134219A1 (zh) 2019-12-30 2019-12-30 参数标定方法及装置
US17/854,506 US12307715B2 (en) 2019-12-30 2022-06-30 Parameter calibration method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/129932 WO2021134219A1 (zh) 2019-12-30 2019-12-30 参数标定方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/854,506 Continuation US12307715B2 (en) 2019-12-30 2022-06-30 Parameter calibration method and apparatus

Publications (1)

Publication Number Publication Date
WO2021134219A1 true WO2021134219A1 (zh) 2021-07-08

Family

ID=76686996

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/129932 Ceased WO2021134219A1 (zh) 2019-12-30 2019-12-30 参数标定方法及装置

Country Status (4)

Country Link
US (1) US12307715B2 (zh)
EP (1) EP4071713B1 (zh)
CN (1) CN113330487B (zh)
WO (1) WO2021134219A1 (zh)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114073494A (zh) * 2020-08-19 2022-02-22 京东方科技集团股份有限公司 白细胞检测方法、系统、电子设备和计算机可读介质
CN113141518B (zh) * 2021-04-20 2022-09-06 北京安博盛赢教育科技有限责任公司 直播课堂中视频帧图像的控制方法、控制装置
KR20230144591A (ko) * 2021-12-03 2023-10-16 컨템포러리 엠퍼렉스 테크놀로지 씨오., 리미티드 Ccd 카메라 캘리브레이션 시스템, 방법, 장치, 컴퓨팅 디바이스 및 저장 매체
CN114708326B (zh) * 2021-12-14 2025-07-25 燕山大学 自适应调整亮度和模糊度的全自动相机标定系统及其方法
CN116413871A (zh) * 2021-12-31 2023-07-11 深圳市速腾聚创科技有限公司 光学模组的光调装置
CN114022572B (zh) * 2022-01-07 2022-04-08 江西明天高科技股份有限公司 摄像阵列自动校准方法
CN116797651A (zh) * 2022-03-15 2023-09-22 深圳市普渡科技有限公司 红外双目摄像头的标定装置和标定系统
US20240007613A1 (en) * 2022-06-30 2024-01-04 Apple Inc. Camera Rotation Correction
US12221119B2 (en) * 2022-07-12 2025-02-11 Gm Cruise Holdings Llc Drive-through calibration process
KR20240020850A (ko) * 2022-08-09 2024-02-16 한국전자기술연구원 캘리브레이션 패턴의 자동 인식을 이용한 카메라 캘리브레이션 장치 및 방법
CN115423876A (zh) * 2022-08-12 2022-12-02 深圳市道通智能航空技术股份有限公司 图像的标定方法、电子设备及无人机
KR20240082886A (ko) * 2022-12-02 2024-06-11 한국전자통신연구원 듀얼 챠르코보드를 통한 광각-협각 렌즈간 캘리브레이션 장치 및 방법
US20240193815A1 (en) * 2022-12-09 2024-06-13 Google Llc Verification of a multi-camera calibration
CN116797667B (zh) * 2023-06-26 2025-01-10 成都飞机工业（集团）有限责任公司 一种多相机测量系统构建方法、装置、介质及电子设备
US12445721B1 (en) * 2023-12-20 2025-10-14 Zoox, Inc. Determining optimal camera settings
CN121112997B (zh) * 2025-11-17 2026-03-17 中国电建集团西北勘测设计研究院有限公司 海上光伏支架的应变监测方法、装置、程序产品与设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101527040A (zh) * 2008-03-05 2009-09-09 深圳华为通信技术有限公司 图像处理方法及系统
US20100302366A1 (en) * 2009-05-29 2010-12-02 Zhao Bingyan Calibration method and calibration device
CN101998136A (zh) * 2009-08-18 2011-03-30 华为技术有限公司 单应矩阵的获取方法、摄像设备的标定方法及装置
CN106887023A (zh) * 2017-02-21 2017-06-23 成都通甲优博科技有限责任公司 用于双目摄像机标定的标定板及其标定方法和标定系统
CN108942934A (zh) * 2018-07-23 2018-12-07 珠海格力电器股份有限公司 确定手眼标定的方法及装置
CN109416744A (zh) * 2016-06-28 2019-03-01 奇跃公司 改进的相机校准系统、目标和过程

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6527395B1 (en) * 2001-12-10 2003-03-04 Mitsubishi Electric Research Laboratories, Inc. Method for calibrating a projector with a camera
JP4926817B2 (ja) * 2006-08-11 2012-05-09 キヤノン株式会社 指標配置情報計測装置および方法
JP2009017480A (ja) * 2007-07-09 2009-01-22 Nippon Hoso Kyokai <Nhk> カメラキャリブレーション装置およびそのプログラム
JP5830546B2 (ja) * 2011-02-25 2015-12-09 フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ 対象物のモデル変換に基づくモデルパラメータの決定
US8619144B1 (en) * 2012-03-14 2013-12-31 Rawles Llc Automatic camera calibration
US9338447B1 (en) * 2012-03-14 2016-05-10 Amazon Technologies, Inc. Calibrating devices by selecting images having a target having fiducial features
KR20150056980A (ko) * 2013-11-18 2015-05-28 삼성전자주식회사 멀티 디스플레이의 색편차 보정용 카메라의 제어 방법 및 장치
KR101583663B1 (ko) * 2014-04-30 2016-01-12 아진산업(주) 차량용 카메라의 교정판 제공 방법
US20170032529A1 (en) * 2015-07-27 2017-02-02 Csir Method of calibrating a helmet and a system therefor
US20170053405A1 (en) * 2015-08-21 2017-02-23 Samsung Electronics Co., Ltd. Method and system for calibration of a medical imaging system
US10445860B2 (en) * 2015-12-08 2019-10-15 Facebook Technologies, Llc Autofocus virtual reality headset
CN105701827B (zh) * 2016-01-15 2019-04-02 中林信达（北京）科技信息有限责任公司 可见光相机与红外相机的参数联合标定方法及装置
US9965870B2 (en) * 2016-03-29 2018-05-08 Institut National D'optique Camera calibration method using a calibration target
KR102255017B1 (ko) * 2016-06-20 2021-05-24 코그넥스코오포레이션 시간 코딩된 패턴 타겟을 사용하여 적어도 하나의 센서 카메라를 포함하는 이미지 캡처 센서를 캘리브레이션하는 방법
EP3264359B1 (en) * 2016-06-28 2025-02-19 Dassault Systèmes A computer-implemented method of calibrating a camera
EP3264360B1 (en) * 2016-06-28 2025-01-22 Dassault Systèmes Dynamical camera calibration
JP6929123B2 (ja) * 2017-05-10 2021-09-01 日本放送協会 カメラ校正装置及びカメラ校正プログラム
CN108876926B (zh) * 2017-05-11 2021-08-10 京东方科技集团股份有限公司 一种全景场景中的导航方法及系统、ar/vr客户端设备
CN108447097B (zh) * 2018-03-05 2021-04-27 清华-伯克利深圳学院筹备办公室 深度相机标定方法、装置、电子设备及存储介质
GB2572956B (en) * 2018-04-16 2021-09-08 Sony Interactive Entertainment Inc Calibration system and method
US10715705B2 (en) * 2018-06-11 2020-07-14 Pony Ai Inc. Characterizing optical characteristics of optical elements
CN110717942B (zh) * 2018-07-11 2022-06-10 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
US20210124174A1 (en) * 2018-07-17 2021-04-29 Sony Corporation Head mounted display, control method for head mounted display, information processor, display device, and program
WO2020019233A1 (zh) * 2018-07-26 2020-01-30 深圳大学 透明物体光线对应关系采集系统
KR102577448B1 (ko) * 2019-01-22 2023-09-12 삼성전자 주식회사 핸드 아이 캘리브레이션 방법 및 시스템
US11657535B2 (en) * 2019-10-15 2023-05-23 Nvidia Corporation System and method for optimal camera calibration
CN112816949B (zh) * 2019-11-18 2024-04-16 商汤集团有限公司 传感器的标定方法及装置、存储介质、标定系统
CN112907675B (zh) * 2019-11-19 2022-05-24 浙江商汤科技开发有限公司 图像采集设备的标定方法、装置、系统、设备及存储介质
DE102020131808A1 (de) * 2019-12-05 2021-06-10 Denso Wave Incorporated Vorrichtung zur Unterstützung von Kalibrierungsvorgängen bei Industriekameras

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101527040A (zh) * 2008-03-05 2009-09-09 深圳华为通信技术有限公司 图像处理方法及系统
US20100302366A1 (en) * 2009-05-29 2010-12-02 Zhao Bingyan Calibration method and calibration device
CN101998136A (zh) * 2009-08-18 2011-03-30 华为技术有限公司 单应矩阵的获取方法、摄像设备的标定方法及装置
CN109416744A (zh) * 2016-06-28 2019-03-01 奇跃公司 改进的相机校准系统、目标和过程
CN106887023A (zh) * 2017-02-21 2017-06-23 成都通甲优博科技有限责任公司 用于双目摄像机标定的标定板及其标定方法和标定系统
CN108942934A (zh) * 2018-07-23 2018-12-07 珠海格力电器股份有限公司 确定手眼标定的方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4071713A4 *

Also Published As

Publication number Publication date
US12307715B2 (en) 2025-05-20
CN113330487B (zh) 2025-03-28
EP4071713A1 (en) 2022-10-12
EP4071713B1 (en) 2024-06-19
CN113330487A (zh) 2021-08-31
EP4071713A4 (en) 2022-12-21
US20220358679A1 (en) 2022-11-10

Similar Documents

Publication Publication Date Title
CN113330487B (zh) 参数标定方法及装置
US11503275B2 (en) Camera calibration system, target, and process
CN110648274B (zh) 鱼眼图像的生成方法及装置
CN111476876B (zh) 一种三维影像渲染方法、装置、设备及可读存储介质
CN110312111B (zh) 用于图像装置的自动校准的装置、系统和方法
CN114820814A (zh) 摄影机位姿计算方法、装置、设备及存储介质
CN113281723B (zh) 一种基于AR tag的3D激光雷达与相机间结构参数的标定方法
CN109920004A (zh) 图像处理方法、装置、标定物组合、终端设备及标定系统
CN115816833B (zh) 图像校正数据的确定方法、装置、电子设备及存储介质
CN111189415A (zh) 一种基于线结构光的多功能三维测量重建系统及方法
JPWO2018168757A1 (ja) 画像処理装置、システム、画像処理方法、物品の製造方法、プログラム
CN110853102B (zh) 一种新的机器人视觉标定及引导方法、装置及计算机设备
CN115272466A (zh) 手眼标定方法、视觉机器人、手眼标定设备和存储介质
CN109389642A (zh) 视觉系统对机器人的标定方法、系统和具有存储功能装置
CN113298886B (zh) 一种投影仪的标定方法
CN117152244B (zh) 屏幕间位置关系确定方法、装置、电子设备及存储介质
CN117078762B (zh) 一种虚拟现实设备、相机标定装置及方法
CN117252933A (zh) 基于无人机的相机内参自动标定方法、系统及电子设备
CN116883465A (zh) 一种多光谱植物表型三维成像方法、系统及装置
CN108062790B (zh) 应用于物体三维重建的三维坐标系建立方法
CN112102416B (zh) 用于多个相机的自动标定方法及系统
CN119450226A (zh) 一种机器人拍摄定位方法、装置、设备及介质
CN116363209A (zh) 一种图像自动标注方法、终端及可读存储介质
CN102778980A (zh) 一种超大幅面显示触点融合交互系统
CN115984099A (zh) 全景成像设备和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19958494

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019958494

Country of ref document: EP

Effective date: 20220705

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 201980022070.8

Country of ref document: CN