WO2016015547A1 - 一种基于机器视觉的飞机入坞引导和机型识别的方法及系统 - Google Patents

一种基于机器视觉的飞机入坞引导和机型识别的方法及系统 Download PDF

Info

Publication number
WO2016015547A1
WO2016015547A1 PCT/CN2015/083206 CN2015083206W WO2016015547A1 WO 2016015547 A1 WO2016015547 A1 WO 2016015547A1 CN 2015083206 W CN2015083206 W CN 2015083206W WO 2016015547 A1 WO2016015547 A1 WO 2016015547A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
aircraft
unit
front wheel
engine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/083206
Other languages
English (en)
French (fr)
Inventor
邓览
张肇红
向卫
杨月峰
刘海秋
王海彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China International Marine Containers Group Co Ltd
Shenzhen CIMC Tianda Airport Support Ltd
Original Assignee
China International Marine Containers Group Co Ltd
Shenzhen CIMC Tianda Airport Support Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China International Marine Containers Group Co Ltd, Shenzhen CIMC Tianda Airport Support Ltd filed Critical China International Marine Containers Group Co Ltd
Priority to US15/329,994 priority Critical patent/US10290219B2/en
Priority to EP15828078.4A priority patent/EP3196853A4/en
Publication of WO2016015547A1 publication Critical patent/WO2016015547A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/50—Navigation or guidance aids
    • G08G5/51—Navigation or guidance aids for control when on the ground, e.g. taxiing or rolling
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64F—GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
    • B64F1/00—Ground or aircraft-carrier-deck installations
    • B64F1/002—Taxiing aids
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64F—GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
    • B64F1/00—Ground or aircraft-carrier-deck installations
    • B64F1/18—Visual or acoustic landing aids
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/435—Computation of moments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G06V10/7515—Shifting the patterns to accommodate for positional errors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B5/00—Visible signalling systems, e.g. visible personal calling systems or remote indication of seats occupied
    • G08B5/06—Visible signalling systems, e.g. visible personal calling systems or remote indication of seats occupied using hydraulic transmission; using pneumatic transmission
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/20—Arrangements for acquiring, generating, sharing or displaying traffic information
    • G08G5/22—Arrangements for acquiring, generating, sharing or displaying traffic information located on the ground
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30236—Traffic on road, railway or crossing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems

Definitions

  • the invention relates to a method for aircraft docking guidance and model identification, in particular to a method and system for aircraft docking guidance and model recognition based on machine vision.
  • Aircraft docking berth guidance refers to the process of guiding the arriving aircraft from the end of the taxiway to the parking position of the apron and accurately parking it.
  • the purpose of the aircraft berth guidance is to ensure the safe and accurate berthing of the docking aircraft, to facilitate the accurate docking of the aircraft and various ground interfaces, and to enable the boarding bridge to effectively abut the aircraft door, thereby improving the efficiency and safety of the airport operation.
  • the automatic aircraft berth guidance system is mainly divided into different types according to the type of sensor used:
  • the two types of automatic aircraft berth guiding systems are also called visual berth guiding systems.
  • the buried induction coil type automatic guidance system determines the position of the docking aircraft by detecting whether a metal object passes or stays.
  • the advantages of the buried induction coil are fast response speed, low cost, no requirement for weather and illumination, but large error and low anti-interference ability.
  • the buried leads and electronic components are easily crushed, the reliability is not high, the measurement accuracy is not high, the model cannot be identified, and the repairability is poor.
  • the laser scanning and ranging automatic guiding system determines the position, speed and model of the aircraft through laser ranging and laser scanning. It is not affected by the ambient illuminance, and is less affected by the weather. The precision is high and the repairability can be debugged. it is good.
  • the visual perception automatic guidance system acquires the image information of the aircraft docking process through optical imaging, and then determines the position, speed and model information of the docking aircraft through intelligent information processing technology.
  • the system architecture is simple, the cost is low, and the system has high The level of intelligence, adjustability and maintainability are good, but the weather and illumination are required and the adaptability is poor.
  • the visual aircraft berth guiding technology can accurately and quickly acquire the docking information of the docking aircraft, which has been guided at the berth of the airport. Applies in the system.
  • VDGS visual aircraft berth guidance system
  • VDOCKS video berth guidance system
  • the technical problem solved by the invention is that, in a visual manner, the guiding of the docking berth of the aircraft can be realized, and the accuracy of aircraft capturing, tracking and positioning of the aircraft docking process can be effectively improved.
  • model recognition and authentication are implemented.
  • the present invention discloses a method for aircraft docking guidance and model recognition based on machine vision, including:
  • Step S1 an aircraft berth scene setting step, dividing the monitoring scene into different information processing function areas;
  • Step S2 an image preprocessing step of preprocessing the captured image
  • Step S3 an aircraft capturing step of identifying an aircraft in the image by identifying an engine and a front wheel of the aircraft in the image;
  • Step S4 the aircraft tracking step, continuously tracking and updating the image of the engine and the front wheel of the aircraft captured in step S3;
  • Step S5 the aircraft positioning step, realizing the real-time positioning of the aircraft and accurately determining the degree of deviation of the aircraft from the guide line and the distance from the stop line;
  • step S6 the information display outputs and displays the degree of deviation of the aircraft with respect to the guide line and the distance from the stop line in step S5.
  • the image preprocessing step further includes:
  • Step S21 determining whether the image is a low illumination image, a strong illumination image, or a normal illumination image according to an average gray value of the image, performing a low illumination image processing step on the low illumination image, and performing a strong illumination image processing step on the strong illumination image;
  • Step S22 determining whether the normal illumination image is a normal image according to the variance of the image
  • Step S23 judging whether it is a rain or snow image or a fog image for an abnormal image, performing rain on the rain and snow image
  • the snow image processing step performs a fog image processing step on the fog image.
  • the low illumination image processing steps include:
  • f(x, y) is the original image
  • (x, y) is the pixel point coordinate in the image
  • g(x, y) is the image after processing
  • a is the low illumination image processing parameter.
  • the rain and snow image processing steps include:
  • For the pixel to be processed of the current image extract a brightness value of a corresponding pixel of the image adjacent to the current image, and determine, according to the brightness value, whether the corresponding pixel of the image adjacent to the current image is a pixel to be processed, if Yes, taking an average value of luminance values of all adjacent pixels of the pixel to be processed of the current image, and using the average value to replace the luminance value of the pixel to be processed of the current image, and if not, using the image adjacent to the current image
  • the method performs the fog image processing step by a homomorphic filtering method.
  • the aircraft capture step further includes:
  • Step S31 the background elimination step, using a single Gaussian background model to simulate the dynamic distribution of the background in the scene and performing background modeling, and then distinguishing the current image from the background model to eliminate the background and obtain the foreground region;
  • Step S33 the region classification step, establishing a standard frontal aircraft area template, extracting the target area through the change detection and obtaining the vertical projection curve of the area, and then obtaining the vertical projection curve of the vertical projection curve and the standard frontal aircraft area template.
  • Correlation coefficient if the correlation coefficient is greater than or equal to a classification threshold, the target is an aircraft;
  • Step S34 a feature verification step; further verifying whether the target is an aircraft by detecting the engine and the front wheel of the captured aircraft.
  • the feature verification step further includes:
  • Step S341 extracting an extremely black area of the image, performing gray scale histogram statistics on the target area of the current image, and obtaining a maximum gray value and a minimum gray value in a range of 1% to 99% in the middle of the gray level, by using a preset pole
  • the black determination threshold and the maximum gray value and the minimum gray value extract the darkest part of the image to obtain an extremely black area;
  • Step S342 detecting a circle-like shape, extracting all outer boundary boundaries of the extremely black region, and calculating the barycentric coordinates of the boundary using the moment of the boundary for each boundary, and the jith moment of the boundary is defined as follows:
  • the region is considered to be non-circular, and the determination of the next region is performed. Otherwise, The area is considered to be circular, and the center of gravity coordinates and radius of the circular area are recorded;
  • Step S343 detecting the aircraft engine by determining the similarity in the circular region
  • Step S344 detecting the front wheel of the aircraft.
  • step S343 for the detected M circular-like regions, the calculation of the i-th and j-th similarities is:
  • step S343 if the aircraft engine is not detected, iterative detection is performed, and the extreme black determination threshold, the circular determination threshold, and the similarity threshold are respectively increased, and then steps S341-343 are performed; if the aircraft is still not detected For the engine, open the 7*7 circular template for all the extremely black areas, and then proceed to steps S342-343;
  • the increase amounts of the extreme black determination threshold, the circular determination threshold, and the similarity threshold are 0.05, 0.5, and 20, respectively.
  • the step S344 further includes:
  • the gray level of 256 levels is quantized to 64 levels, and the first peak and trough in the gray histogram of the 64-level gray scale are searched, and the gray level histogram of the original 256-level gray scale is
  • the optimal peak position BestPeak, the optimal valley BestValley position is defined as follows:
  • hist 256 (i) is a gray histogram of 256 gray scales, the total number of pixels whose gray scale is i;
  • the optimal valley trough BestValley is used to segment the gray scale, and for the portion smaller than the optimal valley BestValley, the small area is removed, and a flat elliptical structural element is used to close the image;
  • the 7th-order Hu moment feature of the boundary is calculated for all the graphics, and compared with the moment feature of the preset standard front wheel model.
  • the middle one is determined to be the front wheel.
  • the aircraft tracking step further includes:
  • Step S41 after obtaining the engine position of the image of the previous frame, using the flood filling method to track and determine the engine area of the current frame;
  • Step S42 if the filling result of step S41 is invalid, perform a dark environment detection and tracking step, and perform step S341 and step S342 using the parameters of the previous frame to detect the tracking engine area;
  • Step S43 after acquiring the information of the engine area, detecting the front wheel of the aircraft using step S344;
  • Step S44 the front wheel tracks the emergency processing step, and when detecting that the front wheel shape is incorrect or the front wheel position is significantly deviated from the previous multi-frame image, the adjacent two frames are used according to the information of the previous frame image and the current image.
  • the displacement of the engine estimates the front wheel displacement of the frame, and the estimation result is used as the front wheel tracking result. If it is not detected beyond the N frame, an error message is output.
  • the aircraft positioning step further includes:
  • Step S51 the camera calibration and image correction step are used to determine a correspondence between the optical parameters of the camera and the geographic coordinate system;
  • Step S52 the aircraft front wheel deviation degree solving step
  • Step S53 the actual distance calculation step of the front wheel of the aircraft.
  • the step S51 further includes:
  • Step S512 using the cvFindChessboardCorners() function of OpenCV to find the checkerboard corner point, and substituting the read N calibration pictures into the cvFindChessboardCorners() function respectively, if the corner points are successfully found, the function returns 1 and obtains The coordinates of the corner point in the image coordinate system; if not, return 0;
  • Step S513 substituting the coordinates of the corner point successfully found on the calibration template into the function cvCalibrateCamera2(), and returning to obtain the parameter matrix, the distortion coefficient, the rotation vector and the peace of the camera device. Move the vector.
  • the step S52 further includes:
  • the step S53 further includes:
  • Step S7 may further perform step S7, an aircraft identification and identity verification step, and step S7 further includes:
  • Step S71 parameter verification, extracting aircraft parameters in the image and comparing with the model data preset in the database to obtain the model similarity parameter;
  • Step S72 template matching, comparing the image with a template template preset in the database, to obtain a template similarity parameter
  • Step S73 comprehensively determining that the model data similarity parameter and the template similarity parameter are greater than or equal to a verification threshold, and are regarded as being authenticated.
  • Step S71 further includes:
  • Step S711 extracting aircraft engine parameters in the image and comparing with aircraft engine parameters preset in a corresponding model in the database to obtain a first ratio
  • Step S712 extracting aircraft wing parameters in the image and comparing with aircraft wing parameters preset in the database, to obtain a second ratio
  • Step S713 extracting aircraft nose parameters in the image and the aircraft nose preset corresponding to the model in the database The parameters are compared to obtain a third ratio;
  • Step S714 extracting the aircraft tail parameters in the image and comparing with the aircraft tail parameters preset in the database, to obtain a fourth ratio
  • Step S715 taking the minimum value and the maximum value of the first ratio, the second ratio, the third ratio, and the fourth ratio, and taking the minimum value/maximum value as the model similarity parameter.
  • Step S72 further includes:
  • Step S721 the global template is matched, and the whole image is used as the searched image, and the standard aircraft image is used as a template to calculate a global template similarity parameter;
  • Step S722 local template matching, the aircraft engine, the aircraft wing, the aircraft nose and the aircraft tail extracted in steps S711-S714 are respectively searched images, respectively, using standard aircraft image engines, wings,
  • the nose and the tail are templates, and the four similarities between the searched image and the template are calculated, the minimum of the four similarities is removed, and the average of the remaining three similarities of the four similarities is calculated as a partial template. Similarity parameter.
  • the step S73 further includes: if at least two of the model similarity parameter, the global template similarity parameter, and the partial template similarity parameter are greater than or equal to the first verification threshold, it is regarded as being authenticated, or The model similarity parameter, the global template similarity parameter, and the partial template similarity parameter are both greater than the second verification threshold, and are regarded as being authenticated.
  • the invention also discloses a machine vision based aircraft docking guidance and model recognition system, comprising:
  • An aircraft berth scene setting unit is configured to divide the monitoring scene into different information processing function areas
  • An image preprocessing unit for preprocessing the captured image
  • An aircraft capture unit for identifying an engine and a front wheel of the aircraft in the image to confirm the presence of an aircraft in the image
  • An aircraft tracking unit for continuously tracking and real-time updating images of the captured aircraft's engine and front wheels;
  • An aircraft positioning unit for real-time positioning of the aircraft and accurately determining the degree of deviation of the aircraft from the guide line and the distance from the stop line;
  • the information display unit outputs and displays the degree of deviation of the aircraft from the guide line and the distance from the stop line.
  • the present invention accurately realizes the capture, tracking, positioning and identity verification of the aircraft during the docking process of the aircraft, and displays the navigation information of the aircraft berth to provide correctness for the pilot, the co-pilot or other personnel.
  • Effective berth guidance enables the aircraft to achieve safe and effective berths, improve airport operation efficiency and ensure safety.
  • FIG. 1 is a schematic structural view of a machine vision-based aircraft docking guidance and model recognition system according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of an operation of an aircraft berth guiding according to the present invention.
  • FIG. 3 is a flow chart of the aircraft docking guidance and model identification of the present invention.
  • FIG. 4 is a schematic diagram of setting an aircraft berth scene according to the present invention.
  • 5A, 5B are detailed flowcharts of an image preprocessing step
  • Figure 6 is a diagram showing an example of a curve of a homomorphic filter function
  • Figure 7A is a flow chart showing the background elimination of the present invention.
  • Figure 7B is a schematic view of a typical extremely black region
  • FIG. 7C is a schematic flow chart showing the similarity determination
  • FIG. 7D is a diagram showing an example of a gray histogram of 256-level gray scales
  • FIG. 7E is a diagram showing an example of a grayscale histogram of the quantized 64-level gray scale
  • Figure 7F is a diagram showing an example of the effect of closing a image using a flat elliptical structural element
  • FIG. 8A is a schematic flow chart showing an aircraft tracking step
  • Figure 8B is a diagram showing an example of an image of an aircraft engine portion
  • FIG. 9 is a diagram showing an example of a corresponding point of an actual distance and an image distance and a fitting curve
  • Figure 10A shows a flow chart of the aircraft identification and verification algorithm
  • FIG. 10B is a schematic diagram showing the structure of a layered image
  • Figure 10C is a diagram showing an example of an image edge of an aircraft
  • Figure 10D shows an example of a wing profile and an engine profile
  • FIG. 10E is a schematic diagram of the searched image S, the subgraph S ij , and the template T;
  • Figure 11 is a diagram showing an example of a possible display manner displayed in the display device.
  • FIG. 1 is a schematic structural diagram of an aircraft docking guidance and model recognition system based on machine vision according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of an aircraft berth guiding operation according to the present invention.
  • the machine vision based aircraft docking guide and model identification system of the present invention is mainly composed of an image pickup device 1, a central processing device 2, and a display device 3.
  • the camera device 1 is connected to the central processing device 2, and the central processing device 2 is connected to the display device 3.
  • the imaging device 1 transmits the captured image to the central processing device 2, and the central processing device 2 transmits the display content including the guidance information to the display device 3.
  • the camera device 1 is installed behind the stop line 42 of the aircraft berth station 4, and the guide line 41 is suitable, and the installation height is higher than the fuselage of the aircraft 5, preferably about 5-8 m, and the camera in FIG.
  • the dashed line connecting the device 1 indicates that it is placed directly above the place.
  • the central processing device 2 can be a computing device having the capability of accepting data, processing data, storing data, generating display image data, and transmitting data, including performing aircraft berth scene configuration, video image preprocessing, aircraft capture, aircraft tracking, A plurality of functional modules for aircraft positioning, aircraft identification and authentication, and modules for generating information display contents are all installed as software in the central processing unit 2.
  • the display device 3 is preferably a large information display that is installed in the airport for the pilot of the aircraft to watch. In addition, the airport staff can also be equipped with a handheld display device to observe the aircraft.
  • FIG. 3 is a flow chart of aircraft docking guidance and model identification according to an embodiment of the present invention.
  • the invention is based on machine vision for aircraft docking guidance and model identification method, comprising the following steps:
  • Step S1 aircraft berth scene setting.
  • the aircraft Since the aircraft needs to go through a long distance from the beginning of the aircraft to the final stop, it is divided into multiple stages during the aircraft docking guidance process. The monitoring content of each stage is different, that is, the aircraft needs to be advanced in advance. Berth scene settings.
  • step S1 the monitoring scene of the aircraft berth station 4 is divided into different information processing functional areas to narrow the processing area of the picture and improve the processing efficiency.
  • the scene definition needs to be performed in the monitoring scene of the aircraft berth station 4, and a black and white interval ruler is placed next to the guide line 41.
  • the length interval of black and white is the same, and the length interval is at most 1 m, which can be used according to the resolution of the camera.
  • a finer scale with a length interval of 0.5m, 0.25m, etc., the total length of the scale does not exceed the range of distance calculation for the aircraft position, usually 50m.
  • the monitoring scene can be reproduced by software running in the central processing unit 2.
  • the software is turned on to display a picture of the aircraft berth station 4 taken by the camera 1, and the relevant area is marked by manually drawing lines, marquees and dots, and the record is saved.
  • the image pickup apparatus 1 captures a scene image of the aircraft berth station 4 when no aircraft is parked, and transmits it to the central processing unit 2.
  • the schematic diagram of the aircraft berth scene setting is shown in Fig. 4.
  • the frame 40 in the figure indicates the screen displayed when the calibration operation is performed and the area that can be used for drawing.
  • the dotted line frame in the figure can be the position manually drawn, and the line can be manually drawn on the displayed image.
  • the guide line 41 and the stop line 42 are respectively marked, and the position information of the recording guide line 41 and the stop line 42 in the image is saved.
  • the related ground equipment area 8 stores the position information of the record capturing area 6 and the tracking positioning area 7 in the image.
  • the model identification and authentication area, and the tracking location area 7, can correspond to the same section area.
  • the hand animation point marks all the mark points 9 with the maximum interval of 1 m next to the guide line 41, and saves the position information of all the mark points 9 in the image, and each mark point 9 is The distance from the first marker point 91 in the actual scene.
  • the image portion to be marked may be enlarged, and when enlarged to several tens of pixels wide, the middle portion is manually marked to improve the mark accuracy.
  • the position of the marked capture zone 6 and the tracking location zone 7 need not be very strict, the position of the upper edge of the capture zone 6 in the actual scene is about 100 m from the stop line 42, and the position of the lower edge of the capture zone 6 in the actual scene is from the stop line 42. Approximately 50 m, the position of the upper edge of the tracking positioning area 7 in the actual scene is about 50 m from the stop line 42, and the lower edge of the tracking positioning area 7 is below the stop line 42.
  • Step S1 above the broken line in Fig. 3 is performed before the berth guidance is performed after the system installation is completed.
  • the portions below the dashed line are all executed when the berth is booted.
  • the steps in the dashed box need to be executed and updated in real time during the berth boot process.
  • Step S1 is followed by step S2, an image pre-processing step.
  • a detailed flowchart of the image preprocessing step is shown in FIGS. 5A and 5B.
  • the image pickup apparatus 1 takes a picture of the capture area 6 in real time, and performs a step S2 and subsequent steps for each of the captured images.
  • Step S2 further includes:
  • step S21 the captured image is grayscaled.
  • Step S22 the average gray value and the variance of the statistical image are determined, and it is determined whether the average gray value of the image is lower than a minimum threshold. If yes, the image is a low illumination image, and the low illumination image processing step of step S25 is performed. If not, Go to Step 23.
  • the lowest threshold is preset, and the lowest threshold is a value between 50-60.
  • step S23 it is determined whether the average gray value of the image is higher than a highest threshold. If yes, the image is a strong light image, and the step of performing strong light image processing in step S24 is performed. If not, the image is a normal illumination image, and the step is performed. 26.
  • the lowest threshold is preset, and the highest threshold is a value between 150-160.
  • the image with the average gray value between the highest threshold and the lowest threshold is the normal illumination image.
  • Step S24 intense illumination image processing.
  • step S24 the luminance reduction image is processed by the gamma conversion method.
  • Step S25 low illumination image processing.
  • the present invention performs processing by means of nonlinear transformation, and the transformation formula is:
  • f(x, y) is the original image
  • (x, y) is the coordinates of the pixel points in the image
  • g(x, y) is the image after processing
  • a is the low-illumination image processing parameter, which can be a value 0.01.
  • Step S26 determining whether the variance of the normal illumination image is greater than the standard value of the one-way difference. If yes, the image is a rain and snow fog image, and step S27 is performed. If not, it is known that the normal illumination image is a normal image. Then do nothing.
  • Step S27 determining whether the entropy of the normal illumination image is greater than an entropy threshold. If yes, the normal illumination image is a rain and snow image, and performing the step of rain and snow image processing in step S28. If not, the normal illumination image is a fog image. The step of the fog image processing of step S29 is performed.
  • Entropy is a mathematical variable that is usually used to indicate the amount of information. For an image, entropy represents the amount of detail in the image, that is, the amount of information contained in the image. Rain and snow images due to the presence of rain and snow, the raindrops and snowflakes on the image appear in different positions, so that the details of the image are more, and the fog image is less detailed because of the even distribution of the fog, so the rain can be determined by entropy. Snow image with fog image.
  • the neighborhood grayscale mean of the selected image is used as the spatial feature quantity of the grayscale distribution
  • the pixel grayscale of the image is composed of the feature binary group, which is denoted as (i, j)
  • f(i,j) be the feature two-tuple (i, j) the frequency of occurrence
  • N is the scale of the image
  • p ij f(i,j)/N 2
  • the formula for calculating the two-dimensional entropy of the gray image is
  • Step S28 rain and snow image processing.
  • the rain and snow image processing step uses the photometric model of the pixels in the image sequence to determine the linear correlation of the brightness, thereby achieving the purpose of removing the influence of rain and snow on the image.
  • the pixel luminance value I n-1 of the same pixel point P, I n , I n+1 satisfy the following conditions:
  • the luminance value I n-1 of the n-1th frame is equal to the luminance value I n+1 of the n+ 1th frame, and the luminance variation value ⁇ I caused by rain and snow in the nth frame satisfies the following conditions:
  • c represents the minimum threshold of brightness change caused by rain and snow.
  • step S28 the method further includes:
  • step S281 the photometric model is used to find the pixel to be processed contaminated by rain and snow.
  • Step S282 brightness adjustment is performed on the pixel to be processed.
  • Step S282 further includes:
  • Step S2821 for the pixel P to be processed of the image n, extract the first two frames (n-1, n-2) image adjacent to the image n and the corresponding pixels of the last two frames (n+1, n+2) images.
  • the brightness value of P determines whether the pixels P of the extracted four-frame image are all pixels to be processed, and if so, step S2822 is performed, and if no, step S2823 is performed.
  • Step S2822 Take an average value of the luminance values of all the adjacent pixels of the pixel P to be processed, and replace the luminance value of the pixel P to be processed of the image n with the average value to eliminate the influence of rain and snow on the brightness of the image.
  • Step S2823 for the pixel P to be processed of the image n, extract the first two frames (n-1, n-2) image adjacent to the image n and the corresponding pixels of the last two frames (n+1, n+2) images.
  • the luminance value of P is used to extract the luminance values of the same pixel of the four-frame image, and the two minimum luminance values are taken, and the two luminance values are averaged, and the average value is used instead of the pixel P of the image n to be processed.
  • the luminance value of the pixel P to be processed of the image n may be directly replaced by the minimum value of the luminance values of the same pixel of the four-frame image.
  • step S2821 and step S2823 the luminance values of the pixels corresponding to one frame or three or more frames adjacent to the image n may also be extracted.
  • Step S29 fog image processing.
  • the fog image processing step of step S29 may use homomorphic filtering to eliminate the effect of fog on image brightness.
  • the F(u,v) is processed using the homomorphic filter function H(u,v):
  • the shape of the curve of H(u,v) can be approximated by the basic form of any ideal high-pass filter, such as the slightly modified version of the Gaussian high-pass filter:
  • g(x, y) is the result obtained after the fog image processing step.
  • Each frame passes through the image processed by the pre-processing step described in step S3, and a higher picture quality is obtained, and subsequent steps can be performed accordingly.
  • Step S2 is followed by step S3, an aircraft capture step.
  • step S2 In order to achieve the capture of the docking aircraft for subsequent guidance and the like, it is necessary to continuously analyze the preprocessed image in step S2 and accurately identify whether the aircraft has appeared.
  • Step S3 further includes:
  • Step S31 the background elimination step.
  • Step S32 a shadow elimination step.
  • Step S33 a region classification step.
  • Step S34 a feature verification step.
  • the aircraft exists in the foreground of the image. In order to accurately capture the aircraft from the image, it is first necessary to remove the background in the image to eliminate interference.
  • the background elimination step of step S31 is to simulate the dynamic distribution of the background in the scene by using a single Gaussian background model and perform background modeling, and then distinguish the current frame from the background model to eliminate the background.
  • the background elimination flowchart is shown in FIG. 7A.
  • Step S31 further includes:
  • step S311 the background model is initialized.
  • the present invention adopts a single Gaussian background model, which is to treat each pixel in the background model as a one-dimensional normal distribution, and each pixel is independent of each other, and its distribution is normal. The mean and variance of the distribution are determined.
  • the training of the background model is performed using successive N frames of images processed in step S2 to determine the mean and variance of the Gaussian distribution.
  • the N-frame image captures the scene of the capture zone 6 when the aircraft is not present in the capture zone 6, and the N-frame image is also the background image.
  • the positions captured by the N frames of images are the same.
  • the N-frame image can be, for example, a 50-frame image taken by the image pickup apparatus 1.
  • x i is the current pixel value of the pixel
  • ⁇ i is the mean of the current pixel Gaussian model
  • ⁇ i is the mean square of the current pixel Gaussian model.
  • ⁇ (x i , ⁇ i , ⁇ i ) is judged. If ⁇ (x i , ⁇ i , ⁇ i ) ⁇ Tp (Tp is a probability threshold, or a foreground detection threshold), the point is judged as the former Attractions, otherwise background points (also known as x i matching ⁇ (x i , ⁇ i , ⁇ i )). The collected background points form a background model to complete the background model initialization.
  • the probability threshold Tp can also be replaced by an equivalent threshold.
  • d i
  • step S312 the background model is updated.
  • step S311 If the scene changes after the completion of step S311, the background model needs to respond to these changes, and the background model is updated at this time.
  • the background model is updated by using the real-time information provided by the continuous image captured by the camera 1 after the scene changes, as follows:
  • ⁇ is the update rate, indicating how fast the background model is updated. If the pixel is the background, the update rate ⁇ is 0.05. If the pixel is foreground, the update rate ⁇ is generally taken as 0.0025.
  • step S313 is directly executed.
  • Step S313 the current frame image captured by the camera device 1 is processed by the step S2, and then differentiated from the background model to obtain a foreground region of the current frame image.
  • the morphological corrosion and expansion operations may be performed on the difference obtained to obtain a more accurate boundary of the foreground region.
  • the morphological corrosion and expansion operation is in the prior art The steps that the skilled person can master.
  • the shadows in the image can be further eliminated for accurate capture of the aircraft.
  • step S32 the gray value of each pixel in the foreground region obtained by the processing of step 31 is first counted, and the maximum gray value g max and the minimum gray value g min are found, and are aimed at lower gray.
  • the area of the degree value is shaded.
  • the area of the lower gray value is less than the gray value
  • Each frame image includes a foreground area and a background area, and the foreground area and the background area may coincide, and the pixels in the foreground area may have corresponding background pixels at the same position point of the background area.
  • the grayscale ratio between each pixel and the corresponding background pixel is determined. If the ratio is between 0.3 and 0.9, the pixel is considered to be a shadow point.
  • the non-shaded area in the set of shadow points is then removed by multiple morphological erosion and expansion operations to obtain a shaded area.
  • the shadow area is removed from the foreground area, and the voids in the desired foreground area are eliminated by multiple morphological expansion and etching operations, and the areas are connected to obtain the target area.
  • the target area corresponds to an object appearing in the capture zone 6, which may be an airplane or other object such as a vehicle.
  • a template of a standard frontal aircraft area is established in advance, and since the aircraft has a narrow intermediate width characteristic on both sides, the template can be used to distinguish between an airplane and a non-aircraft.
  • the target area is extracted by the change detection, and the vertical projection curve of the target area is obtained. Subsequently, the vertical projection curve of the standard frontal aircraft area is obtained. Determining whether a correlation coefficient between a vertical projection curve of the target area and a vertical projection curve of the standard frontal aircraft area is greater than a classification threshold; if yes, the target area corresponds to an aircraft, and step S34 is continued; if not, the target area corresponds to Not an airplane.
  • the classification threshold is, for example, 0.9.
  • step S33 it is only confirmed by the outline that the target area is an airplane, and then it is further confirmed by the feature verification step of step S34 whether the target area is indeed an airplane.
  • the feature verification step further verifies whether the target is an aircraft by detecting the engine and front wheels of the captured aircraft.
  • Step S34 further includes:
  • step S341 the image is extremely black region extracted.
  • a threshold BlackestJudge is used to extract the region of the gray value in the image between gmin and (gmax-gmin)*BlackestJudge+gmin, that is, the darkest part of the image, to obtain a Extremely black area.
  • the ratio of the maximum (gmax) gray value and the minimum (gmin) gray value it can be determined whether the image is in day or night, and when the ratio is greater than a standard value, it belongs to daytime, and the extreme black determination threshold is selected to be 0.05, otherwise it belongs to night.
  • the polar black decision threshold is selected to be 0.5.
  • FIG. 7B A typical black area diagram is shown in Figure 7B. The inside of each figure is a very black area.
  • Step S342 a circle-like detection.
  • the center of gravity of the boundary is calculated using the moment of the boundary, and the jith moment m ji of the boundary is defined as follows:
  • (x, y) is the coordinate value of the pixel
  • f(x, y) is the image of the black area of the pole.
  • the barycentric coordinates can be calculated from the 00, 10, and 01 moments:
  • the distance from the center of gravity For all the pixels of the current boundary, calculate the distance from the center of gravity. If the ratio of the calculated maximum distance to the minimum distance exceeds a preset value (preferably a circular decision threshold circleJudge of 1.5), it is determined that the boundary corresponds to The area is not a circle. If the ratio of the calculated maximum distance to the minimum distance does not exceed the preset value, it is determined that the area corresponding to the boundary is a circle. The decision is completed for all boundaries according to this rule.
  • a preset value preferably a circular decision threshold circleJudge of 1.5
  • the center of gravity coordinate For the area identified as a circle (referred to as a circular area for short), the center of gravity coordinate, the average distance (ie, the radius) of the boundary to the center of gravity is recorded to perform the similarity determination in the subsequent step S343.
  • Step S343 the similarity determination.
  • FIG. 7C a schematic flowchart of the similarity determination is shown.
  • Step S343 further includes:
  • Step S3431 detecting whether there is an engine in the circular area by calculating the similarity of the circular-like area, if yes, executing step S4, and if no, executing step S3432.
  • step S3432 is performed.
  • step S3432 the threshold is adjusted, and steps S341, 342, and 3431 are re-executed. If the engine area has not been detected, step S3433 is performed.
  • the thresholds BlackestJudge, circleJudge, and similarThresh are respectively increased, and the increments are preferably 0.05, 0.5, and 20, respectively, and the steps of extremely black region extraction, circle-like detection, and engine detection are performed. If the engine area has not been detected, step S3433 is performed.
  • step S3433 the morphological processing is performed using the circular template for all the extremely black regions, and steps S342 and 3431 are re-executed.
  • the circular template may preferably be a 7*7 circular template.
  • N may preferably be 2 times.
  • Step S344 front wheel detection.
  • the line connecting the center of the engine detected in step S343 is taken as the bottom side, and the rectangular area having the height of four half engines below the bottom side is the search area.
  • 256 levels of gray levels are quantized to 64 levels
  • a gray level histogram example of 256 levels of gray is shown in FIG. 7D
  • a quantized gray level histogram of 64 levels of gray is shown in FIG. 7E.
  • the first peak 3001 and the first trough 3002 in the gray histogram of the 64-level gray scale are searched for.
  • the optimal peak position BestPeak and the optimal trough BestValley position in the original 256-level gray scale histogram are defined as follows:
  • hist 256 (i) is the total number of pixels with gray scale i in the gray histogram of 256 gray scales.
  • the 7th-order Hu moment feature of the boundary is calculated for all the graphs after the closed operation, and compared with the Hu moment feature of the preset standard front wheel model (on the HU distance feature: the geometric moment is determined by Hu (Visual pattern recognition) By moment invariants) proposed in 1962, with translation, rotation and scale invariance.
  • Hu uses 7 constant distances constructed by second and third order center distance. Therefore, the 7th order of the 7th order Hu distance feature is uniquely determined.
  • a threshold preferably, the value is 1
  • step S4 is performed.
  • Step S4 the aircraft tracking step.
  • step S4 is directly executed, or after S2 and S3 are executed, step S4 is performed.
  • the engine position of the image of the previous frame has been obtained by the method of feature verification in step S34, the engine position of the current frame image is only slightly moved, so that it is not necessary to re-detect the entire image, only at a small
  • the extended area performs engine extraction of the current frame, and the parameters of the previous frame (BlackestJudge, circleJudge) will be available for target detection of the current frame.
  • FIG. 8A A schematic flow chart of the aircraft tracking step is shown in FIG. 8A.
  • step S41 it is determined whether there is engine information of the previous frame. If yes, step S42 is performed, and if no, step S46 is performed.
  • step S42 the engine position is determined by the flood filling method.
  • the engine Since the engine has a light-colored outer wall, its gray value will be significantly higher than the black area inside the engine.
  • the image of the aircraft engine part is shown in Figure 8B. Therefore, the engine center of the previous frame is the seed point, and the flood filling method is used to obtain the whole. The black area of the engine.
  • step S43 is continued.
  • step S43 it is judged whether or not the filling result of step S42 is valid. If yes, step S46 is performed, and if no, step S44 is performed.
  • Step S44 the dark environment detection tracking step.
  • steps S341 and 342 are re-executed to detect the engine area.
  • step S45 it is determined whether the detection result is valid. If yes, the information of the engine area is output. If not, the engine information of the previous frame is blanked, and step S41 is performed.
  • step S46 the feature verification step of step S34 is directly performed, and the information of the engine area is output.
  • step S46 cannot be more than twice in the image sequence of a group of aircraft berths. Further, after the dark frame detection tracking step of the step S44 is continuously used to detect a specific frame image (for example, 20 frames), the feature verification step of step S34 is used for detection regardless of the detection result.
  • Step S47 the front wheel tracking step.
  • the front wheel of the aircraft is detected using the method of front wheel detection of step S344 for the subsequent aircraft positioning step.
  • step S48 the front wheel tracks the emergency processing steps.
  • step S47 When the detection result obtained in step S47 is obviously incorrect, that is, when the area of the wheel is determined to be inaccurate and the position is significantly deviated from the previous 5 to 10 frames, the information according to the previous frame and the current frame is determined. The displacement of the adjacent two frames is used to estimate the front wheel displacement of the frame, and the estimation result is returned as the front wheel tracking result. If the N (takes 20 to 30) frame is exceeded, the front wheel tracking result is obviously related to the aircraft forward parameter. If it does not match, an error message is output.
  • Step S5 the aircraft positioning step. This step is used to generate the correct berth guide information.
  • Step S5 further includes:
  • Step S51 the camera calibration and image correction step.
  • Step S52 the aircraft front wheel deviation degree solving step.
  • Step S53 the actual distance calculation step of the front wheel of the aircraft.
  • the imaging device calibration process of step S51 is to determine the geometric and optical parameters of the imaging device 1, and the orientation of the imaging device 1 with respect to the world coordinate system.
  • Camera calibration is based on OpenCV implementation, and the calibration process uses a black and white flat checkerboard as the calibration template.
  • the imaging device 1 needs to capture a plurality of pictures of the planar calibration template at different angles to realize calibration of the imaging device 1.
  • at least 10 pairs of 7*8 or larger chessboards are required for the calibration process. Images, and you need to grab as many different angles as possible to calibrate the image.
  • the process is implemented as follows:
  • step S511 N calibration pictures are read.
  • Step S512 using OpenCV function cvFindChessboardCorners() to find the checkerboard corner point, and substituting the read N calibration pictures into the cvFindChessboardCorners() function. If all the corner points are found, the function returns 1 to indicate success, and the corner point is obtained. The coordinates in the image coordinate system; if not successful, return 0.
  • Step S513 if the finding corner point is successful, the coordinates of the corner point on the calibration template are substituted into the function cvCalibrateCamera2(), and the parameter matrix, the distortion coefficient, the rotation vector and the translation vector of the camera device 1 are obtained.
  • the distortion coefficient returned by the function cvCalibrateCamera2() includes the radial distortion coefficient and the tangential distortion coefficient, and they are taken into the OpenCV function cvUndistort2() to mathematically remove the lens distortion.
  • the aircraft front wheel deviation degree calculation of step S52 is for judging whether the front wheel of the aircraft is on the guide line, or is left or right with respect to the guide line.
  • the relevant position information of the guide line, the stop line, and the like can be obtained by the scene definition, and then the knowledge of the relationship between the midpoint of the algebra and the straight line is utilized, that is, The judgment of the degree of deviation of the front wheel of the aircraft can be achieved.
  • it may include:
  • the installation position is known as the x-axis from left to right and the y-axis from bottom to top, using the modified point-to-line distance equation:
  • d 2 ⁇ 0 means that the front wheel of the aircraft exceeds the stop line 41, and d 2 ⁇ 0 means that the front wheel of the aircraft has not reached the stop line 41.
  • d 1 >0 means the aircraft is to the left
  • d 1 ⁇ 0 means the aircraft is to the right
  • d 1 ⁇ 0 means the aircraft is to the left
  • d 1 >0 means the aircraft is to the right
  • the left and right are both aircraft The driver's perspective is determined.
  • any case of d 1 ⁇ 0 results in the conclusion of “off-guide line”, and the judgment condition
  • the actual front wheel solving step of the aircraft in step S53 is used to solve the real distance of the aircraft from the stop line in real time.
  • the coordinates of the marker points set in the scene in S1 are all image coordinates, taking 20 points with an interval of 1 m as an example, respectively: ⁇ point1, point2, ..., point20 ⁇ , and the relative stop line of each point is obtained.
  • the actual distance of each point from the end point point 1 is: ⁇ 0m, 1m, ..., 19m ⁇ , which gives ⁇ dis1, dis2, ..., dis20 ⁇ and ⁇ 0m, 1m,... , one-to-one correspondence of 19m ⁇ .
  • the airport Before the aircraft enters the platform, the airport sends the model information to the machine vision-based aircraft docking guide and the model identification system, and after performing step S3, the next step S6, the aircraft identification and the identity verification step can be performed.
  • This model information is verified by analyzing the image. That is, steps S4, S5 can be performed in synchronization with step S6.
  • Fig. 10B is a schematic diagram showing the structure of a layered image.
  • the detection method of the aircraft is detected by a detection method of coarse to fine multi-level visual features, including:
  • low-noise environments such as rain, snow, fog, weather, nights, etc.
  • lower resolution layered images are used, while in fine weather conditions, higher resolution layered images are used for higher accuracy.
  • the region segmentation result with the mosaic effect on the edge can be obtained.
  • the coarse-to-fine multi-level visual feature detection method is used in the case of bad weather, such as rain, snow, fog, and night, the image noise will become larger, so reducing the resolution can improve the detection effect, and then The mapping will have the maximum resolution for the identification and verification of the aircraft model. It is a method for detecting aircraft characteristics in harsh environments. The system automatically analyzes the best resolution based on image quality to extract the aircraft contour.
  • Step S6 specifically includes:
  • Step S61 parameter verification.
  • step S62 the template matches.
  • step S63 the judgment is comprehensive.
  • step S61 further comprises:
  • the step of extracting the aircraft engine parameters may be specifically implemented by using the foregoing steps S341-S343.
  • the extracted aircraft engine parameters are in pixels.
  • step S612 the aircraft wing parameters are extracted and compared with the aircraft wing parameters preset in the database.
  • the steps of extracting aircraft wing parameters include:
  • step S6121 the edge of the aircraft image is extracted using the Canny operator.
  • An example of an airplane image edge is shown in Figure 10C.
  • Step S6122 extracting the edge pixels of the image of the aircraft, and enumerating the pixels in the axial direction of the aircraft engine on the side of the boarding bridge far away from the aircraft (the left side in FIG. 10C), and drawing multiple slopes for each pixel point a straight line with an inclination angle of 0-20°, and count the number of Canny edge pixels passing through each of the straight lines;
  • step S6123 the edge of the aircraft wing is determined, and two straight lines passing through the edge pixel points are taken as the edge of the aircraft wing.
  • step S6124 the wing tip of the aircraft is determined, and the edge pixels of the area around the wing tip are taken and recorded as wing tip features.
  • FIG. 10D An example of a wing profile and engine profile is shown in Figure 10D.
  • Step S6125 parameter comparison, measuring the length of the aircraft wing by the position of the wing tip, and calculating the ratio of the length of the aircraft wing to the wing length data corresponding to the model information received by the system in the airport model parameter database. This ratio is called the second ratio.
  • step S613 the aircraft nose feature parameters are extracted and compared with the aircraft nose feature parameters preset to the corresponding models in the database.
  • Step S6131 determining the boundary of the aircraft nose, determining the central axis position of the aircraft through the determined aircraft engine parameters, enumerating the points on the central axis as the center of the circle, enumerating 2 to 4 times the radius of the aircraft engine radius as a radius drawing Circle, taking the circle with the most pixels on the Canny edge as the boundary of the aircraft nose.
  • Step S6132 determining an aircraft nose window.
  • the depth-first search method is used to find the longest edge of the upper semicircle of the boundary of the aircraft head that does not adhere to the edge of the aircraft head boundary. The position of the nose window of the aircraft nose.
  • Step S6133 parameter comparison, measuring the nose radius of the aircraft, and calculating the ratio of the aircraft nose radius to the aircraft nose radius corresponding to the model information received by the system in the airport model parameter database, the ratio is called the third ratio.
  • the measured nose radius of the aircraft can be in pixels.
  • Step S614 extracting the parameters of the aircraft tail and entering the parameters of the aircraft tails corresponding to the corresponding models in the database. Line comparison.
  • Step S6141 using the depth-first search method, the portion of the protrusion along the upper edge of the boundary of the aircraft nose is the aircraft tail.
  • Step S6142 parameter comparison, measuring the height of the aircraft tail, in units of pixels, calculating the ratio of the aircraft tail height to the aircraft tail parameter corresponding to the model information received by the system in the airport model parameter database, This ratio is called the fourth ratio.
  • step S615 the minimum value and the maximum value of the first ratio, the second ratio, the third ratio, and the fourth ratio are taken, and the minimum/maximum value is taken as the model similarity parameter as the coefficient 1.
  • the template matching step S62 includes:
  • step S621 the global template is matched, and the current image is taken as the searched image, and the global template similarity parameter is calculated by using the standard aircraft image in the database of the system as the template, and the global template similarity parameter is used as the coefficient 2.
  • FIG. 10E is a schematic diagram of the searched image S, the subgraph S ij , and the template T.
  • the global template matching calculation process is: the searched image S, the mark 6001 in the figure, and the width and height are W*H.
  • the subgraph S ij of the searched image S, the mark 6002 in the figure has a width and height of n*m, j pixels from the left edge of the figure, and i pixels from the lower edge of the figure.
  • Template T, marked 6003 in the figure has a width and height of n*m.
  • the similarity parameter R(i,j) of the template T and the subgraph S ij :
  • M ij is a maximum subgraph S in the height direction can be achieved, N being the maximum subgraph S ij in the width direction can be achieved.
  • Step S622 partial template matching, respectively, the aircraft engine, the aircraft wing, the aircraft nose and the aircraft tail position extracted in step S61 are searched images, respectively, at the airport according to the model information received by the system.
  • the engine, the wing, the nose and the tail of the standard aircraft image corresponding to the model parameter database are templates, and the aircraft engine, the aircraft wing, the aircraft nose and the said are respectively calculated by using the calculation formula in step S621
  • the four similarity parameters R of the aircraft tail are removed, and the minimum of the four similarity parameters is removed, and the average of the remaining three similarity parameters of the four similarity parameters is calculated as a partial template similarity parameter, Local template similar
  • the degree parameter is taken as a factor of 3.
  • Step S63 comprehensively determining that at least two of the coefficients 1, 2, and 3 are greater than or equal to a first verification threshold, or, when the coefficients 1, 2, and 3 are all greater than a second verification threshold, the currently captured aircraft and the advance The obtained model information matches, and the authentication is passed, otherwise the verification fails.
  • Step S7 the information display step.
  • the display device 3 is a large display screen installed in the airport for the pilot of the aircraft to watch during the docking process of the aircraft. At the same time, the airport staff can also observe the situation of the aircraft.
  • FIG. 11 is a diagram showing an example of a possible display manner displayed in the display device 3.
  • 7000 represents an area on the display device 3 for displaying guidance information
  • 7002 represents a "T" shape formed by the guide line and the stop line to conveniently indicate the relative positions of the aircraft and the guide line and the stop line.
  • the berth guidance information such as the specific position of the aircraft determined by the aircraft positioning step of step S5, including: left or right 7001, distance stop line distance 7003 are displayed on the display device in real time.
  • the aircraft model information 7004 verified by the aircraft identification and identity verification step of step S6 is also displayed on the display device in real time. For the pilot to observe the flight path of the aircraft, improve the safety of the aircraft docking.
  • the present invention accurately captures, tracks, locates, and authenticates an aircraft during docking, and displays accurate and effective aircraft berth guidance information on the display device 3 for the pilot and co-pilot.
  • the staff or other personnel provide correct and effective berth guidance to enable the aircraft to achieve safe and effective berths, improve airport operation efficiency and ensure safety.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Mechanical Engineering (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Acoustics & Sound (AREA)
  • Image Analysis (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)

Abstract

一种基于机器视觉的飞机入坞引导和机型识别的方法及系统，该方法包括：51、飞机泊位场景设置步骤，将监测场景划分为不同的信息处理功能区；52、图像预处理步骤，对所拍摄的图像进行预处理；53、飞机捕获步骤，通过在该图像中识别飞机的引擎和前轮，以确认该图像中出现飞机；54、飞机跟踪步骤，对步骤53所捕获到的飞机的引擎和前轮的图像进行连续跟踪和实时更新；55、飞机定位步骤，实现对飞机实时定位并准确判断飞机相对于引导线的偏离程度和相对于停止线的距离；56、信息显示，输出并显示步骤55的飞机相对于引导线的偏离程度和相对于停止线的距离。

Description

一种基于机器视觉的飞机入坞引导和机型识别的方法及系统
本申请基于申请号为201410378566.8、申请日为2014年8月1日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本发明涉及一种飞机入坞引导和机型识别的方法，特别是涉及一种基于机器视觉的飞机入坞引导和机型识别的方法及系统。
背景技术
飞机入坞泊位引导是指将到港飞机从滑行道末端导引至机坪的停机位置并准确停泊的过程。飞机泊位引导的目的是保障入坞飞机安全准确停泊，能方便飞机与各种地勤接口的准确对接，并使登机桥能有效靠接飞机舱门，提高机场运行效率和安全。
自动飞机泊位引导系统按使用传感器的类型不同主要分为：
(1)地埋线圈类；(2)激光扫描测距类；(3)视觉感知类。
由于激光扫描测距类和视觉感知类自动引导系统能有效获取入坞飞机的可视化信息，因此该两类自动飞机泊位引导系统又称为可视化泊位引导系统。
地埋感应线圈类自动引导系统通过探测是否有金属物体经过或停留来确定入坞飞机的位置。地埋感应线圈的优点是响应速度快、成本低，对天气和照度无要求，但误差较大、抗干扰能力低。同时，埋在地下的引线和电子元件容易被压坏、可靠性不高，测量精度不高，不能识别机型，可调试可维修性差。
激光扫描测距类自动引导系统通过激光测距和激光扫描来确定飞机位置、速度和机型等信息，不受环境照度的影响、且受天气影响较小，精度较高，可调试可维修性好。
视觉感知类自动引导系统通过光学成像方式获取飞机入坞过程的图像信息，进而通过智能化信息处理技术确定入坞飞机的位置、速度和机型等信息，系统架构简单、成本低，具有高的智能化水平，可调性可维护性较好，但对天气和照度有要求、适应性较差。
随着视觉感知成像技术、智能化信息处理技术和计算机技术的不断深入发展，可视化飞机泊位引导技术能精确、快速获取入坞飞机的入坞信息，已在机场的泊位引导 系统中得到应用。
美国Honeywell公司研制的可视化飞机泊位引导系统(VDGS)和德国西门子公司研制的视频泊位引导系统(VDOCKS)作为国际领先水平的视觉引导设备也在国际上一些机场得到应用。
但是，在飞机入坞过程的飞机捕获、跟踪与定位、机型识别与身份验证方面的精确性还有待有进一步的提高。
发明内容
本发明解决的技术问题在于，以可视化的方式，实现飞机入坞泊位引导，能有效提高飞机入坞过程的飞机捕获、跟踪与定位的精确性。
进一步的，实现机型识别与身份验证的功能。
进一步的，能有效提高民航机场自动化、智能化和运营管理的水平。
为解决上述问题，本发明公开了一种基于机器视觉的飞机入坞引导和机型识别的方法，包括：
步骤S1，飞机泊位场景设置步骤，将监测场景划分为不同的信息处理功能区；
步骤S2，图像预处理步骤，对所拍摄的图像进行预处理；
步骤S3，飞机捕获步骤，通过在该图像中识别飞机的引擎和前轮，以确认该图像中出现飞机；
步骤S4，飞机跟踪步骤，对步骤S3所捕获到的飞机的引擎和前轮的图像进行连续跟踪和实时更新；
步骤S5，飞机定位步骤，实现对飞机实时定位并准确判断飞机相对于引导线的偏离程度和相对于停止线的距离；
步骤S6，信息显示，输出并显示步骤S5的飞机相对于引导线的偏离程度和相对于停止线的距离。
该图像预处理步骤进一步包括：
步骤S21，根据该图像的平均灰度值判断该图像为低照度图像、强光照图像还是正常光照图像，对低照度图像执行低照度图像处理步骤，对强光照图像执行强光照图像处理步骤；
步骤S22，根据该图像的方差判断该正常光照图像是否为正常图像；
步骤S23，对于非正常图像，判断其为雨雪图像还是雾图像，对雨雪图像执行雨 雪图像处理步骤，对雾图像执行雾图像处理步骤。
该低照度图像处理步骤包括：
g(x,y)＝f(x,y)+af(x,y)(255-f(x,y))
f(x,y)为原图像，(x，y)为图像中的像素点坐标，g(x,y)为处理之后的图像，a为低照度图像处理参数。
该雨雪图像处理步骤包括：
利用光度测定模型寻找被雨雪污染的待处理像素；
对于当前图像的待处理像素，提取与该当前图像前后相邻的图像的相应像素的亮度值，根据该亮度值判断与该当前图像前后相邻的图像的相应像素是否均为待处理像素，如果是，取该当前图像的待处理像素的所有相邻像素的亮度值的平均值，用该平均值代替该当前图像的待处理像素的亮度值，如果否，利用该当前图像前后相邻的图像的相应像素的亮度值中的最小值或最小的两个值的平均值，代替该当前图像的待处理像素的亮度值。
所述方法通过同态滤波的方法进行该雾图像处理步骤。
该飞机捕获步骤进一步包括：
步骤S31，背景消除步骤，利用单高斯背景模型来模拟场景中背景的动态分布并进行背景建模，然后将当前图像与背景模型作差分以消除背景，得到前景区域；
步骤S32，阴影消除步骤，统计该前景区域的灰度值，找出最大灰度值gmax和最小灰度值g min，然后在灰度值小于T＝g min+(g max-g min)*0.5的区域进行阴影消除；
步骤S33，区域分类步骤，建立一个标准正面飞机区域模板，经过变化检测提取目标区域并求取该区域的垂直投影曲线，然后求取该垂直投影曲线与所述标准正面飞机区域模板的垂直投影曲线的相关系数，若该相关系数大于或等于一分类阈值，则该目标为飞机；
步骤S34，特征验证步骤；通过检测捕获到的飞机的引擎和前轮来进一步验证该目标是否为飞机。
该特征验证步骤进一步包括：
步骤S341，图像极黑区域提取，对当前图像的目标区域进行灰度直方图统计，在灰度级中间1％～99％范围内获得最大灰度值、最小灰度值，借助预设的极黑判定阈值以及该最大灰度值、最小灰度值提取图像中最黑的部分，得到一幅极黑区域；
步骤S342，类圆形检测，提取该极黑区域的所有外层边界，对每一个边界使用边界的矩计算边界的重心坐标，边界的第ji阶矩定义如下：
Figure PCTCN2015083206-appb-000001
重心坐标
Figure PCTCN2015083206-appb-000002
Figure PCTCN2015083206-appb-000003
对于当前边界的所有像素点，计算其与该重心的距离，若计算得到的最大距离与最小距离的比值大于一圆形判定阈值，则认为该区域非圆形，进行下一区域的判定，否则认为该区域为类圆形，记录类圆形区域的重心坐标和半径；
步骤S343，在类圆形区域中通过判断相似度检测飞机引擎；
步骤S344，检测飞机前轮。
在步骤S343中，对于检测到的M个类圆形区域，其中第i个和第j个的相似度的计算为：
Similarityij＝|Heighti-Heightj|*|Radiusi-Radiusj|
其中，Height为重心高度，Radius为半径，当相似度Similarityij小于预设的相似度阈值时，则认为类圆形区域i和j为飞机引擎。
在步骤S343中，若没有检测出飞机引擎，则进行迭代检测，将所述极黑判定阈值、圆形判定阈值、相似度阈值分别增大，再进行步骤S341-343；若仍然没有检测出飞机引擎，则对所有的极黑区域使用7*7的圆形模板进行开操作，再进行步骤S342-343；
若仍然没有检测出飞机引擎，则再进行2次上述迭代检测；
若仍然没有检测出飞机引擎，则判定图像中无引擎存在。
所述极黑判定阈值、圆形判定阈值、相似度阈值的增加量分别为0.05、0.5、20。
该步骤S344进一步包括：
在图像的搜索区域中，将256级的灰度级量化至64级，搜索量化为64级的灰度直方图中的第一个波峰和波谷，原始256级灰度的灰度直方图中的最优波峰位置BestPeak、最优波谷BestValley位置定义如下：
Figure PCTCN2015083206-appb-000004
Figure PCTCN2015083206-appb-000005
其中hist256(i)为256级灰度的灰度直方图中，灰度为i的像素总数；
以此最优波谷BestValley对灰度进行分割，对小于最优波谷BestValley的部分，除去面积较小的杂点，使用一个扁平椭圆型结构元素对图像进行闭操作；
接着对所有图形计算边界的7阶Hu矩特征，与预置的标准前轮模型的矩特征进行比对，当相似度低于一阈值时则判定中间一个为前轮。
该飞机跟踪步骤进一步包括：
步骤S41，在获得上一帧图像的引擎位置后，采用洪水填充法跟踪确定当前帧的引擎区域；
步骤S42，如果步骤S41的填充结果无效，执行阴暗环境检测跟踪步骤，使用上一帧的参数进行步骤S341和步骤S342来检测跟踪引擎区域；
步骤S43，在获取到引擎区域的信息之后，使用步骤S344检测飞机前轮；
步骤S44，前轮跟踪应急处理步骤，在检测前轮形状不正确或前轮位置与之前多帧图像相比发生明显偏离时，根据上一帧图像和当前图像的信息，利用相邻两帧图像引擎的位移对该帧的前轮位移进行估计，将估计结果作为前轮跟踪结果，如果超出N帧仍检测不到，则输出错误信息。
该飞机定位步骤进一步包括：
步骤S51，摄像装置标定与图像矫正步骤，用于确定摄像装置的光学参数与地理坐标系之间的对应关系；
步骤S52，飞机前轮偏离程度解算步骤；
步骤S53，飞机前轮实际距离解算步骤。
该步骤S51进一步包括：
步骤S511，读取N幅标定图片；
步骤S512，使用OpenCV的cvFindChessboardCorners()函数寻找棋盘角点，将读取的所述N幅标定图片分别代入所述cvFindChessboardCorners()函数，如果成功寻找到所有的角点，则函数返回1，并得到角点在图像坐标系下坐标；如果不成功则返回0；
步骤S513，将成功寻找到的所述角点在标定模板上的坐标代入函数cvCalibrateCamera2()中，返回得到摄像装置的参数矩阵、畸变系数、旋转向量和平 移向量。
该步骤S52进一步包括：
根据由步骤S43得到前轮的位置坐标点(x0,y0)，利用所述位置坐标点与引导线和停止线的关系，求得引导线的直线方程为：y1＝k1x1+b1，停止线的直线方程为：y2＝k2x2+b2，所述位置坐标点到直线的距离为：
Figure PCTCN2015083206-appb-000006
将坐标点(x0,y0)代入两个直线方程分别求得d1和d2，d2≥0表示飞机前轮超出停止线，d2＜0表示飞机前轮未到达停止线，此时若k1＞0，d1＞0则表示飞机偏左，d1＜0表示飞机偏右；若k1＜0，则d1＜0表示飞机偏左，d1＞0表示飞机偏右。
判断|d1|＞width/2是否成立，width为一等于检测的飞机前轮的宽度的阈值，如果成立，认为飞机已经偏离引导线。
该步骤S53进一步包括：
建立图像坐标与大地坐标的对应关系；
由步骤S1的场景设置中的标记点得到图像坐标，采用最小二乘法对该图像坐标进行二次曲线拟合，得到曲线方程y＝ax2+bx+c，x是图像上的距离，y是实际距离；
对于飞机前轮在图像上的位置，沿停止线方向将其投影到引导线上，计算投影点到停止点的欧氏距离作为x，则通过y＝ax2+bx+c可得到飞机前轮到停止线的实际距离。
该步骤S3之后还可执行步骤S7，飞机识别及身份验证步骤，步骤S7进一步包括：
步骤S71，参数验证，提取图像中的飞机参数并与预置于数据库中的机型数据进行比对，得到机型相似度参数；
步骤S72，模板匹配，将图像与预置于所述数据库中的机型模板进行比对，得到模板相似度参数；
步骤S73，综合判断，所述机型数据相似度参数与所述模板相似度参数大于或等于一验证阈值时，视为通过身份验证。
步骤S71进一步包括：
步骤S711，提取图像中的飞机引擎参数并与预置于数据库中对应机型的飞机引擎参数进行比对，得到第一比值；
步骤S712，提取图像中的飞机机翼参数并与预置于数据库中对应机型的飞机机翼参数进行比对，得到第二比值；
步骤S713，提取图像中的飞机机头参数并与预置于数据库中对应机型的飞机机头 参数进行比对，得到第三比值；
步骤S714，提取图像中的飞机尾翼参数并与预置于数据库中对应机型的飞机尾翼参数进行比对，得到第四比值；以及
步骤S715，取第一比值、第二比值、第三比值、第四比值这四者中的最小值以及最大值，将最小值/最大值，作为该机型相似度参数。
步骤S72进一步包括：
步骤S721，全局模板匹配，以整幅图像为被搜索图像，标准飞机图像为模板，计算全局模板相似度参数；
步骤S722，局部模板匹配，分别以步骤S711-S714中提取得到的所述飞机引擎、飞机机翼、飞机机头和所述飞机尾翼为被搜索图像，分别以标准飞机图像的引擎、机翼、机头和尾翼为模板，计算被搜索图像与模板的4个相似度，去掉所述4个相似度中的最小值，计算所述4个相似度中其余3个相似度的平均数为局部模板相似度参数。
步骤S73进一步包括：若所述机型相似度参数、全局模板相似度参数和所述局部模板相似度参数中至少有2个大于或等于第一验证阈值，视为通过身份验证，或，所述机型相似度参数、全局模板相似度参数和所述局部模板相似度参数都大于第二验证阈值，视为通过身份验证。
本发明还公开了一种基于机器视觉的飞机入坞引导和机型识别的系统，包括：
飞机泊位场景设置单元，用于将监测场景划分为不同的信息处理功能区；
图像预处理单元，用于对所拍摄的图像进行预处理；
飞机捕获单元，用于在该图像中识别飞机的引擎和前轮，以确认该图像中出现飞机；
飞机跟踪单元，用于对所捕获到的飞机的引擎和前轮的图像进行连续跟踪和实时更新；
飞机定位单元，用于实现对飞机的实时定位并准确判断飞机相对于引导线的偏离程度和相对于停止线的距离；
信息显示单元，输出并显示飞机相对于引导线的偏离程度和相对于停止线的距离。
综上所述，本发明在飞机入坞过程中，准确实现对飞机的捕获、跟踪、定位和身份验证，并对飞机泊位引导信息进行显示，为飞机驾驶员、副驾驶员或其他人员提供正确有效的泊位引导，使飞机实现安全有效的泊位，提高机场运行效率，保障安全。
附图说明
图1为本发明一实施例的基于机器视觉的飞机入坞引导和机型识别系统的结构示意图；
图2为本发明的飞机泊位引导工作原理图；
图3为本发明的飞机入坞引导和机型识别流程图；
图4为本发明的飞机泊位场景设置示意图；
图5A、5B所示为图像预处理步骤的详细流程图；
图6所示为同态滤波器函数的曲线示例图；
图7A所示为本发明的背景消除流程图；
图7B所示为一幅典型的极黑区域示意图；
图7C所示为相似度判定的流程示意图；
图7D所示为256级灰度的灰度直方图示例图；
图7E所示为量化后的64级灰度的灰度直方图示例图；
图7F所示为使用一个扁平椭圆型结构元素对图像进行闭操作的效果示例图；
图8A所示为飞机跟踪步骤的流程示意图；
图8B所示为飞机引擎部分的图像示例图；
图9是实际距离与图像距离的对应点及拟合曲线示例图；
图10A所示为飞机识别及验证算法流程图；
图10B所示为分层图像结构示意图；
图10C所示为飞机图像边缘示例图；
图10D所示为机翼轮廓与引擎轮廓示例图；
图10E所示为被搜索图像S、子图Sij、模板T的示意图；
图11所示为显示于该显示设备中的一种可行的显示方式示例图。
具体实施方式
下面结合附图对本发明的结构原理和工作原理作具体的描述：
参见图1及图2，图1为本发明一实施例的基于机器视觉的飞机入坞引导和机型识别系统的结构示意图，图2为本发明飞机泊位引导工作原理图。
本发明的该基于机器视觉的飞机入坞引导和机型识别系统主要由摄像装置1、中央处理设备2和显示设备3组成。摄像装置1与中央处理设备2连接，中央处理设备 2与显示设备3连接。摄像装置1将拍摄的图像发送给中央处理设备2，中央处理设备2将包含引导信息的显示内容发送给显示设备3。
其中，摄像装置1安装在飞机泊位站坪4的停止线42后方，正对引导线41为宜，安装高度要高于飞机5的机身，在5-8m左右为宜，图2中与摄像装置1相连的虚线表示其设置在地方正上方。中央处理设备2可以是一种拥有接受数据、处理数据、储存数据、生成显示图像数据、发送数据能力的计算装置，包括用于执行飞机泊位场景配置、视频图像预处理、飞机捕获、飞机跟踪、飞机定位、飞机识别及身份验证的多个功能模块，以及生成用于信息显示内容的模块，全部作为软件安装在中央处理设备2中。显示设备3优选为安装于机场中可供飞机驾驶员观看的大型信息显示屏，另外，机场工作人员也可配备手持式显示设备以观察飞机情况。
参见图3，图3为本发明一实施例的飞机入坞引导和机型识别流程图。本发明基于机器视觉的飞机入坞引导和机型识别方法，包括如下步骤：
步骤S1、飞机泊位场景设置。
由于飞机从开始进入机位到最终停止需要经历一个较长的距离，故而在飞机入坞引导过程中，需分为多个阶段，每个阶段的监测内容不同，也就是说，需要提前进行飞机泊位场景设置。
在步骤S1中，将飞机泊位站坪4的监测场景划分为不同的信息处理功能区，以缩小图片的处理区域范围，提高处理效率。
首先需要在飞机泊位站坪4的监测场景中进行场景定义，紧邻该引导线41铺设一条黑白间隔的标尺，黑色与白色的长度间隔相同，长度间隔最大1m，可根据摄像装置的分辨率，使用长度间隔为0.5m、0.25m等更精细的标尺，标尺的总长度不超过对飞机位置进行距离解算的范围，通常为50m。
通过运行于中央处理设备2中的软件可再现该监测场景。开启该软件可显示摄像装置1拍摄的关于飞机泊位站坪4的画面，并通过手动绘制线条、选框和点，来标记相关区域，并保存记录。
摄像装置1拍摄没有飞机停靠时的飞机泊位站坪4的场景图像，并传送至中央处理设备2。飞机泊位场景设置示意图见图4，图中边框40表示进行标定操作时所显示的画面和可用于描绘的区域，图中虚线线框可以是手动描绘的位置，可以在显示的图像上手动绘制线条，分别标记出引导线41和停止线42，保存记录引导线41和停止线42在图像中的位置信息。通过手动绘制选框，分别标记出捕获区6、跟踪定位区7和 相关地勤设备区8，保存记录捕获区6和跟踪定位区7在图像中的位置信息。机型识别与身份验证区，以及，跟踪定位区7，可以对应同一段区域。再根据场景中铺设的标尺，手动画点，标记出紧邻引导线41旁边的最大间隔为1m的所有标记点9，保存记录所有标记点9在图像中的位置信息，以及每个标记点9在实际场景中距离第一标记点91的距离。
其中，在标记引导线41、停止线42和标记点9的时候，可将需要标记的图像部分放大，放大到数十像素宽时，手动在其中间部分标记，以提高标记精度。标记的捕获区6和跟踪定位区7的位置不需要非常严格，捕获区6上边缘在实际场景中的位置距离停止线42大约100m，捕获区6下边缘在实际场景中的位置距离停止线42大约50m，跟踪定位区7上边缘在实际场景中的位置距离停止线42大约50m，跟踪定位区7下边缘在停止线42以下即可。
图3中虚线以上的步骤S1为在系统安装完成后，进行泊位引导之前执行。虚线以下的部分均在泊位引导时执行。其中虚线框中的步骤需要在泊位引导过程中实时执行和更新。
步骤S1之后执行步骤S2，图像预处理步骤。如图5A、5B所示为图像预处理步骤的详细流程图。
摄像装置1实时的对捕获区6进行拍照，对于拍摄到的每幅图像，均执行步骤S2以其后的步骤。
步骤S2进一步包括：
步骤S21，对于拍摄的图像进行灰度化。
步骤S22，统计图像的平均灰度值和方差，判断图像的平均灰度值是否低于一最低阈值，如果是，该图像为低照度图像，执行步骤S25的低照度图像处理步骤，如果否，执行步骤23。
该最低阈值为预先设置，该最低阈值为处于50-60之间的一个数值。
步骤S23，判断图像的平均灰度值是否高于一最高阈值，如果是，该图像为强光照图像，执行步骤S24的强光照图像处理的步骤，如果否，该图像为正常光照图像，执行步骤26。
该最低阈值为预先设置，该最高阈值为处于150-160之间的一个数值。平均灰度值位于最高阈值与最低阈值之间的图像为正常光照图像。
步骤S24，强光照图像处理。
该步骤S24采用gamma变换的方式对该强光照图像进行亮度降低的处理。
步骤S25，低照度图像处理。
对于低照度图像，本发明采用非线性变换的方式进行处理，变换公式为：
g(x,y)＝f(x,y)+af(x,y)(255-f(x,y))
其中，f(x,y)为原图像，(x，y)为图像中的像素点坐标，g(x,y)为处理之后的图像，a为低照度图像处理参数，该参数可取值0.01。
步骤S26，对正常光照图像判断其方差是否大于一方差标准值，如果是，该图像为雨雪雾图像，执行步骤S27，如果否，可知该正常光照图像非雨雪雾图像，为正常图像，则不做任何处理。
步骤S27，判断该正常光照图像的熵是否大于一熵阈值，如果是，该正常光照图像为雨雪图像，执行步骤S28的雨雪图像处理的步骤，如果否，该正常光照图像为雾图像，执行步骤S29的雾图像处理的步骤。
熵是一个数学变量，通常用于表示信息量的大小，对于图像来说，熵表示图像的细节的多少，也就是图像所含信息量的多少。雨雪图像由于雨雪的存在，图像上的雨点和雪花在不同位置出现，使得图像的细节比较多，而雾的图像则因为雾的均匀分布而显得细节较少，所以可以通过熵来判别雨雪图像与雾图像。
在一实施例中，对于灰度图像来说，选择图像的邻域灰度均值作为灰度分布的空间特征量，与图像的像素灰度组成特征二元组，记为(i,j)，其中i表示像素的灰度值(0<＝i<＝255)，j表示邻域灰度均值(0<＝j<＝255)：令f(i,j)为特征二元组(i,j)出现的频数，N为图像的尺度，pij＝f(i,j)/N2，灰度图像的二维熵的计算公式为
Figure PCTCN2015083206-appb-000007
步骤S28，雨雪图像处理。
该雨雪图像处理步骤使用图像序列中像素的光度测定模型来判断亮度的线性相关性，从而实现去除雨雪对图像的影响的目的。
受雨雪天气影响所拍摄的图像具有如下的光度测定模型：
在当前为雨雪天气且背景固定不变情况下，对同一位置连续拍摄的三帧图像(n-1、n、n+1帧)中，同一像素点P的像素亮度值In-1、In、In+1，满足如下条件：
第n-1帧的亮度值In-1与第n+1帧的亮度值In+1是相等的，并且在第n帧中由雨雪引起的亮度变化值ΔI满足以下条件：
ΔI＝In-In-1＝In-In+1≥c
c代表由雨雪引起的亮度变化最小阈值。
故而，在步骤S28中，进一步包括：
步骤S281，利用光度测定模型寻找被雨雪污染的待处理像素。
即，对当前图像n的像素点P，判断In-1与In+1是否相等，且，ΔI是否大于等于c，如果两个判断的结果均为是，则认为该图像n的像素点P是待处理像素。对图像n中的所有像素均进行上述判断，直至找到所有待处理像素。
步骤S282，对待处理像素进行亮度调节。
步骤S282进一步包括：
步骤S2821，对于图像n的待处理像素P，提取与该图像n相邻的前两帧(n-1、n-2)图像和后两帧(n+1、n+2)图像的相应像素P的亮度值，判断所提取的四帧图像的像素P是否均为待处理像素，如果是，执行步骤S2822，如果否，执行步骤S2823。
步骤S2822，取该待处理像素P的所有相邻像素的亮度值的平均值，用该平均值代替图像n的待处理像素P的亮度值，以消除雨雪对图像亮度的影响。
步骤S2823，对于图像n的待处理像素P，提取与该图像n相邻的前两帧(n-1、n-2)图像和后两帧(n+1、n+2)图像的相应像素P的亮度值，共提取四帧图像的同一像素点的亮度值，取其中最小的两个亮度值，用这两个亮度值取平均值，用该平均值代替图像n的待处理像素P的亮度值，以消除雨雪对图像亮度的影响。在又一实施例中，可直接利用四帧图像的同一像素点的亮度值中的最小值代替图像n的待处理像素P的亮度值。
该步骤S2821以及步骤S2823中，还可提取图像n相邻的前后一帧或三帧或更多图像对应像素的亮度值。
步骤S29，雾图像处理。
步骤S29的雾图像处理步骤可以使用同态滤波，以消除雾对图像亮度的影响。
具体地，对图像f(x,y)，将其表达成照明和反射两部分乘积形式：
f(x,y)＝i(x,y)r(x,y)
其中0≤i(x,y)≤+∞为照明分量，0≤r(x,y)≤1为反射分量，在对其两边取自然对数，得到：
ln f(x,y)＝ln i(x,y)+ln r(x,y)
再进行傅里叶变换，得到：
F(u,v)＝I(u,v)+R(u,v)
对F(u,v)使用同态滤波器函数H(u,v)进行处理：
S(u,v)＝H(u,v)F(u,v)＝H(u,v)I(u,v)+H(u,v)R(u,v)
其中H(u,v)的曲线形状能用任何一种理想高通滤波器的基本形式近似，例如采用高斯型高通滤波器稍微修改过的如下形式：
Figure PCTCN2015083206-appb-000008
如图6所示为同态滤波器函数的曲线示例图。
再进行傅里叶逆变换：
s(x,y)＝F-1[H(u,v)I(u,v)]+F-1[H(u,v)R(u,v)]
最后做指数运算，得到处理结果：
g(x,y)＝exp(s(x,y))
g(x，y)为经过雾图像处理步骤后得到的结果。
每一帧经过步骤S3所述预处理步骤处理过的图像，获得了较高的画面质量，可以据以进行后续的步骤。
步骤S2之后执行步骤S3，飞机捕获步骤。
为了实现对入坞飞机的捕获，以进行后续的引导等操作，需持续对步骤S2中经过预处理的图像进行分析，并从中准确识别出飞机是否已出现。
步骤S3进一步包括：
步骤S31，背景消除步骤。
步骤S32，阴影消除步骤。
步骤S33，区域分类步骤。
步骤S34，特征验证步骤。
飞机存在于图像的前景中，为了准确的从图像中捕获到飞机，首先需要去除图像中的背景，消除干扰。
步骤S31的背景消除步骤是利用单高斯背景模型来模拟场景中背景的动态分布并进行背景建模，然后将当前帧与背景模型作差分以消除背景，背景消除流程图见图7A。
步骤S31进一步包括：
步骤S311，背景模型初始化。
本发明采用的是单高斯背景模型，单高斯背景模型是把背景模型中的每一个像素都看成是一个一维正态分布，而且每个像素之间是相互独立的，其分布由正态分布的均值和方差来确定。
利用经过步骤S2处理的连续的N帧图像进行背景模型的训练，以确定高斯分布的均值和方差。该N帧图像拍摄到的是，在飞机未出现在捕获区6时，捕获区6的场景，该N帧图像也就是背景图像。该N帧图像所拍摄到的位置均相同。该N帧图像可例如为摄像装置1拍摄的50帧图像。
计算该连续的N帧图像f(x,y)中每一个像素的平均灰度值μ0以及像素灰度的方差
Figure PCTCN2015083206-appb-000009
由μ0和
Figure PCTCN2015083206-appb-000010
组成具有高斯分布η(x,μ0,σ0)的初始背景图像B0，
Figure PCTCN2015083206-appb-000011
其中：
Figure PCTCN2015083206-appb-000012
的帧序号，xi为像素点的当前像素值，μi为当前像素点高斯模型的均值，σi为当前像素点高斯模型的均方差。
随后对η(xi,μi,σi)进行判断，若η(xi,μi,σi)≤Tp(Tp为概率阈值，或称前景检测阈值)，则该点被判定为前景点，否则为背景点(这时又称xi与η(xi,μi,σi)匹配)。所收集到的背景点组成背景模型，完成背景模型初始化。
在实际应用时，也可以用等价的阈值代替概率阈值Tp。记di＝|xi-μi|，在常见的一维情形中，则常根据di/σi的取值来设置前景检测阈值：若di/σi＞T(T值在2到3之间)，则该点被判定为前景点，否则为背景点。
步骤S312，背景模型更新。
若在步骤S311完成后，场景发生变化，则背景模型需要响应这些变化，此时就要对背景模型进行更新。
利用摄像装置1在场景发生变化后拍摄的连续图像提供的实时信息对背景模型进行更新，如下式：
Figure PCTCN2015083206-appb-000013
其中α为更新率，表示背景模型更新的快慢程度，若该像素为背景，则更新率α取0.05，若该像素为前景，则更新率α一般取0.0025。
若在步骤S311完成后，场景未发生变化，则直接执行步骤S313。
步骤S313，摄像装置1拍摄的当前帧图像经步骤S2处理后，与该背景模型做差分，以得到该当前帧图像的前景区域。
该步骤中，在该差分步骤后，还可包括对差分得到的结果进行形态学腐蚀和膨胀操作，以得到更为准确的前景区域的边界。该形态学腐蚀和膨胀操作为现有技术中本 领域技术人员可以掌握的步骤。
在消除了图像中的背景之后，为准确的捕获飞机，还可进一步消除图像中的阴影。
在步骤S32的阴影消除步骤中，首先统计经过步骤31的处理而得到的前景区域中各像素的灰度值，找出最大灰度值g max和最小灰度值g min，并针对较低灰度值的区域进行阴影消除。该较低灰度值的区域为灰度值小于
g min+(g max-g min)*0.5的区域。
每帧图像包括前景区域和背景区域，而前景区域和背景区域可能重合，则前景区域中的像素，在背景区域的同一位置点可能具有对应的背景像素。
在该较低灰度的区域中，求取每个像素与对应的背景像素之间的灰度比值，若这一比值在0.3与0.9之间，则这个像素被认为是阴影点。
接着通过多次形态学腐蚀和膨胀操作，除去阴影点集合中的非阴影区域，从而得到阴影区域。
从前景区域中去除该阴影区域，再通过多次形态学膨胀和腐蚀操作消除需要的前景区域中的空洞以及把各个区域连通起来，得到了目标区域。该目标区域对应着出现在捕获区6的物体，可能为飞机，也可能为车辆等其他物体。
在步骤S33的区域分类步骤中，事先建立一个标准正面飞机区域的模板，由于飞机具有两边窄中间宽的特性，故而该模板可用于区别飞机与非飞机。
经过变化检测提取该目标区域，并求取该目标区域的垂直投影曲线。随后，求取该标准正面飞机区域的垂直投影曲线。判断该目标区域的垂直投影曲线与该标准正面飞机区域的垂直投影曲线的相关系数是否大于一分类阈值，如果是，该目标区域对应一飞机，继续执行步骤S34；如果否，该目标区域对应的不是一飞机。该分类阈值例如为0.9。
步骤S33仅通过外形轮廓大致确认目标区域是否为飞机，之后还需要通过步骤S34的特征验证步骤来进一步确认，该目标区域是否的确为飞机。该特征验证步骤是通过检测捕获到的飞机的引擎和前轮来进一步验证该目标是否为飞机。
步骤S34进一步包括：
步骤S341，图像极黑区域提取。
对当前帧图像的目标区域进行灰度直方图统计，在灰度级1％～99％的范围内(通常也就是2～253的灰度级)获取最大(gmax)灰度值和最小(gmin)灰度值，获得像素数不为0的最大(gmax)灰度值/最小(gmin)灰度值这一灰度值比值，通过该比值判断 是白天还是夜晚。结合该比值使用预设的阈值提取图像中最黑的部分，得到一幅极黑区域。
具体来说，使用一个阈值BlackestJudge(极黑判定阈值)，来提取图像中灰度值在gmin到(gmax-gmin)*BlackestJudge+gmin之间的区域，也就是图像中最黑的部分，得到一幅极黑区域。
而根据最大(gmax)灰度值和最小(gmin)灰度值的比值可以判断图像处于白天还是夜晚，当该比值大于一标准值时属于白天，该极黑判定阈值选择0.05，否则属于夜晚，该极黑判定阈值选择0.5。
一幅典型的极黑区域示意图见图7B，图中的各个图形内部是极黑区域。
步骤S342，类圆形检测。
提取该极黑区域的所有外层边界，对每一个边界，计算其重心坐标。
具体地，使用边界的矩计算边界的重心，边界的第ji阶矩mji定义如下：
Figure PCTCN2015083206-appb-000014
(x，y)为像素点的坐标值，f(x，y)为该极黑区域的图像。
重心坐标可由00、10、01阶矩计算得到：
Figure PCTCN2015083206-appb-000015
对于当前边界的所有像素点，计算其与重心的距离，若计算得到的最大距离与最小距离的比值超过了一预设值(优选为1.5的圆形判定阈值circleJudge)，则判定该边界所对应的区域不是圆形，若计算得到的最大距离与最小距离的比值未超过了该预设值，则判定该边界所对应的区域是圆形。依据这一规则对所有边界完成判定。
对于认定为圆形的区域(简称类圆形区域)，记录其重心坐标、边界到重心的平均距离(即半径)，以进行后续步骤S343的相似度判定。
步骤S343，相似度判定。参见图7C所示为相似度判定的流程示意图。
步骤S343进一步包括：
步骤S3431，通过对类圆形区域的相似度计算，检测该类圆形区域中是否存在引擎，如果是，执行步骤S4，如果否，执行步骤S3432。
假设一共检测到了M个类圆形区域，其中第i个类圆形区域和第j个类圆形区域的相似度的计算公式为：
Similarityij＝|Heighti-Heightj|*|Radiusi-Radiusj|
其中Height为重心高度，Radius为边界到重心的平均距离(即半径)。
当相似度Similarityij小于相似度阈值similarThresh(优选预设为40)时，则认为类圆区域i和j为引擎区域。如果没有一个Similarityij小于阈值similarThresh，则视为未检测到引擎区域，执行步骤S3432。
步骤S3432，调整阈值，重新执行步骤S341、342、3431，如果仍未检测到引擎区域，执行步骤S3433。
将阈值BlackestJudge、circleJudge、similarThresh分别增大，增加量可分别优选取0.05、0.5、20，再进行极黑区域提取、类圆形检测和引擎检测的步骤。如果仍未检测到引擎区域，执行步骤S3433。
步骤S3433，对所有的极黑区域使用圆形模板进行形态学处理的开操作，重新执行步骤S342、3431。
该圆形模板可优选为7*7的圆形模板，开操作后再进行步骤S342的类圆形检测和步骤S3431的引擎检测，如果仍未检测到引擎区域，迭代执行步骤S3432。
如迭代N次后仍未检测到引擎区域，判定图像中并无引擎。N可优选为2次。
在对后续帧图像进行检测时，若其前一帧图像使用的迭代步数为n，则直接从第n-1步开始迭代。
步骤S344，前轮检测。
将步骤S343中检测到的引擎圆心的连线作为底边，底边下方4个半引擎高度的矩形区域为搜索区域。
在搜索区域中，将256级的灰度级量化至64级，256级灰度的灰度直方图示例图见图7D，量化后的64级灰度的灰度直方图示例图见图7E。
以图7E为例，搜索量化后为64级灰度的灰度直方图中的第一个波峰3001和第一个波谷3002。
设量化后的第一个波峰位置为peak，第一个波谷位置为valley，则原始256级灰度的灰度直方图中的最优波峰位置BestPeak、最优波谷BestValley位置定义如下：
Figure PCTCN2015083206-appb-000016
Figure PCTCN2015083206-appb-000017
其中hist256(i)为256级灰度的灰度直方图中，灰度为i的像素总数。
以此最优波谷BestValley对灰度进行分割，对小于最优波谷BestValley的部分， 除去面积较小的杂点，使用一个扁平椭圆型结构元素对图像进行闭操作，效果示例图如图7F。
接着对经过闭操作后的所有图形计算其边界的7阶Hu矩特征，并与预置的标准前轮模型的Hu矩特征进行比对(关于HU距特征：几何矩是由Hu(Visual pattern recognition by moment invariants)在1962年提出的，具有平移、旋转和尺度不变性。Hu利用二阶和三阶中心距构造的7个不变距。故此，7阶Hu距特征的7阶是唯一确定的)，当相似度低于一阈值(优选取值1)时则判定为轮子。这样会得到多组轮子的位置，中间靠下的轮子即为前轮。
一旦确认检测到引擎和前轮即可认为捕获成功，执行步骤S4。
步骤S4，飞机跟踪步骤。
该步骤中，为实现对飞机实时定位并准确得到飞机相对引导线的偏离程度，根据引擎外壁和内部之间亮度的巨大差异和引擎的圆形结构，提取引擎的位置和半径，然后通过空间位置关系找到飞机前轮，对飞机进行定位。
具体而言，在实现对飞机的捕获后，摄像装置1继续进行图像拍摄。则在上一帧图像已经实现了飞机捕获后，对于当前帧图像，在执行过步骤S2的图像预处理步骤后，直接执行步骤S4，或者，执行过S2、S3后，执行步骤S4。
由于利用步骤S34的特征验证的方法已经获得了上一帧图像的引擎位置，则当前帧图像的引擎位置只会进行微小的移动，因此并不需要对全图进行重新检测，只在一个较小的扩展区域进行当前帧的引擎提取，上一帧的参数(BlackestJudge,circleJudge)将可以用于当前帧的目标检测。
如图8A所示为飞机跟踪步骤的流程示意图。
步骤S41，判断是否有上一帧的引擎信息，如果是，执行步骤S42，如果否，执行步骤S46。
步骤S42，利用洪水填充法确定引擎位置。
由于引擎具有浅色的外壁，其灰度值会明显高于引擎内部的黑色区域，飞机引擎部分图像示例图如图8B，因此以上一帧的引擎中心为种子点，使用洪水填充法，获得整个引擎的黑色区域。
在天色阴暗或低照度时，引擎边界的灰度值可能不会比中心高太多，加上一些噪声点，可能使洪水填充法可能出现溢出，导致结果无效，使得到的引擎区域明显过大，且不再是圆形，故而，继续执行步骤S43。
步骤S43，判断步骤S42的填充结果是否有效，如果是，执行步骤S46，如果否，执行步骤S44。
步骤S44，阴暗环境检测跟踪步骤。
该步骤中，直接使用处理上一帧图像时所采用的参数，重新执行步骤S341和342，检测引擎区域。
步骤S45，判断检测结果是否为有效，如果是，输出引擎区域的信息，如果否，将该上一帧的引擎信息置空，执行步骤S41。
步骤S46，直接执行步骤S34的特征验证步骤，并输出引擎区域的信息。
执行步骤S46的情况在一组飞机泊位的图像序列中不能超过两次。另外，在连续使用步骤S44的阴暗环境检测跟踪步骤检测特定帧图像(例如20帧)之后，无论检测结果如何，都将使用步骤S34的特征验证步骤进行检测。
步骤S47，前轮跟踪步骤。
在获取到引擎区域的信息之后，在本步骤中，使用步骤S344的前轮检测的方法检测飞机前轮，以用于后续的飞机定位步骤。
步骤S48，前轮跟踪应急处理步骤。
当步骤S47中得到的检测结果明显不正确时，即，判定为轮子的区域存在形状不正确、位置与之前5～10帧相比发生明显偏离的问题时，根据上一帧和当前帧的信息，利用相邻两帧引擎的位移对该帧的前轮位移进行估计，将估计结果作为前轮跟踪结果返回，如果超出N(取20～30)帧的前轮跟踪结果均明显与飞机前进参数不符，则输出错误信息。
步骤S5，飞机定位步骤。该步骤用于产生正确的泊位引导信息。
步骤S5进一步包括：
步骤S51，摄像装置标定与图像矫正步骤。
步骤S52，飞机前轮偏离程度解算步骤。
步骤S53，飞机前轮实际距离解算步骤。
步骤S51的摄像装置标定过程就是确定摄像装置1的几何和光学参数、摄像装置1相对于世界坐标系的方位。
摄像装置标定基于OpenCV实现，标定过程采用黑白相间的平面棋盘格作为标定模板。摄像装置1需在不同的角度抓取多张平面标定模板的图片，来实现对摄像装置1的标定。为了使标定的结果更加精确，标定过程中至少需要10副7*8或更大棋盘的 图像，并且需要抓取尽量多的不同角度的标定图片。实现的流程如下：
步骤S511，读取N幅标定图片。
步骤S512，使用OpenCV的函数cvFindChessboardCorners()寻找棋盘角点，将读取的N幅标定图片分别代入cvFindChessboardCorners()函数，如果寻找到所有的角点，则函数返回1表示成功，并得到角点在图像坐标系下坐标；如果不成功则返回0。
步骤S513，如果寻找角点成功，则将角点在标定模板上的坐标代入函数cvCalibrateCamera2()中，返回得到摄像装置1的参数矩阵、畸变系数、旋转向量和平移向量。
由于实际的镜头有不同程度的畸变，主要是径向畸变，也有轻微的切向畸变。故而在函数cvCalibrateCamera2()返回的畸变系数中包含了径向畸变系数和切向畸变系数，将它们带入OpenCV的函数cvUndistort2()，即可在数学上去掉透镜畸变。
步骤S52的飞机前轮偏离程度计算是用于判断飞机前轮是否处于引导线上，或者，相对于引导线偏左或偏右。
通过步骤S47的前轮跟踪步骤的结果可以得到前轮的位置坐标点，则通过场景定义可以得到引导线、停止线等的相关位置信息，于是利用代数中点与直线之间关系的知识，即可实现对飞机前轮偏离程度的判断。具体可包括：
通过前轮跟踪的结果得到前轮的位置坐标点(x0,y0)，通过场景定义得到引导线上任意两点坐标(xG1,yG1)、(xG2,yG2)和停止线上任意两点坐标(xS1,yS1)、(xS2,yS2)。如果引导线中两点的x坐标满足：xG1＝xG2，则不能用点斜式表示引导线41的直线方程1，此时直线方程1为：x1＝xG1，直线的斜率k1→∞；当xG1≠xG2时，直线方程1为：y1＝k1x1+b1。同理，求得停止线42的直线方程2为：y2＝k2x2+b2，所述位置坐标点到直线的距离为，坐标系的建立是在图像上，依照摄像装置1的安装位置可知，从左到右为x轴，从下到上为y轴，使用修改后的点到直线的距离方程：
Figure PCTCN2015083206-appb-000018
不同于常用的距离方程，该方程得到的结果可正可负。将当前飞机前轮的坐标点(x0,y0)代入两个直线方程分别求得d1和d2：
Figure PCTCN2015083206-appb-000019
Figure PCTCN2015083206-appb-000020
d2≥0表示飞机前轮超出停止线41，d2＜0表示飞机前轮未到达停止线41，此时若k1＞0(其中包括了k1→+∞的情况)，d1＞0则表示飞机偏左，d1＜0表示飞机偏右；若k1＜0，则d1＜0表示飞机偏左，d1＞0表示飞机偏右，该偏左、偏右均以飞机驾驶员的视角确定。更进一步的，为了使算法不会过于敏感，任何d1≠0的情况都得出“偏离引导线”的结论，可加入判断条件|d1|＞width/2，width为一阈值，该阈值可取等于检测的飞机前轮的宽度，当满足该判断条件时，才认为飞机已经偏离引导线。
由参数判定偏离情况见表1。
Figure PCTCN2015083206-appb-000021
步骤S53的飞机前轮实际距离解算步骤，用于实时解算飞机距离停止线的真实距离。
首先建立图像坐标与大地坐标的对应关系。在场景中紧邻引导线处铺设黑白相间的标尺，随后在场景定义中根据标尺，每隔最大间隔1m描点得到标记点，并记录每个标记点在实际场景中距离第一标记点的距离。
S1中的场景设置的标记点的坐标都是图像坐标，以间隔1m的20个点为例，分别为：{point1,point2,...,point20}，求得每个点相对停止线上的终点(point1)的相对坐标：{relativepoint1,relativepoint2,...,relativepoint20}，其中relativepoint1的坐标为(0,0)，每个点距离终点relativepoint1的距离为{dis1,dis2,...,dis20}，每个点距离终点point1的实际距离又分别为：{0m,1m,...,19m}，这样得到了{dis1,dis2,...,dis20}与{0m,1m,...,19m}的一一对应关系。
因为地面上等间距的点在图像中的表现应为二次曲线关系，即随距离的增大，两点间距呈等差数列，所以采用最小二乘法对所描的点进行二次曲线拟合，得到曲线方 程y＝ax2+bx+c，x是图像上的距离，y是实际距离，实际距离与图像距离的对应点及拟合曲线示例图如图9，图中横轴是实际距离，单位为m，纵轴是图像距离，单位为像素；
对于飞机前轮在图像上的位置，沿所述停止线方向将其投影到所述引导线上，计算投影点到停止点的欧氏距离作为x，代入方程y＝ax2+bx+c求得y，得到飞机前轮到停止线的实际距离(单位m)。据此产生正确的泊位引导信息。
在飞机进入站坪前，机场向所述基于机器视觉的飞机入坞引导和机型识别系统发送机型信息，执行完步骤S3之后，接下来即可执行的步骤S6，飞机识别及身份验证步骤就是通过对图像的分析来验证这一机型信息。也就是说，步骤S4、S5可以与步骤S6同步执行。
参见图10A所示为飞机识别及验证算法流程图。图10B为分层图像结构示意图。优选采用由粗到精的多级视觉特征的检测方法来检测飞机轮廓，具体包括：
i＝0时原始图像S0具有最高的分辨率，随着i的增大图像分辨率下降，而i＝L时图像SL的分辨率最低，分层图像结构示意图见图10B。在噪声较大的环境(例如雨雪雾天气、夜晚等)，采用较低分辨率的分层图像，而在天气晴朗的条件下，采用较高分辨率的分层图像以获得更高的精度。在低分辨率图像中获取飞机区域后再映射回原始图像S0之后，可以得到边缘具有马赛克效果的区域分割结果。
该由粗到精的多级视觉特征的检测方法用于在天气不好的情况下，比如雨雪雾天合夜晚，图像的噪声会变大，因此降低分辨率可以提高检测的效果，然后再映射会最大分辨率，进行飞机机型的识别验证。是在恶劣环境下检测飞机特征的一个方法，系统自动根据图像质量分析最佳的分辨率，以提取飞机轮廓。
步骤S6具体包括：
步骤S61，参数验证。
步骤S62，模板匹配。
步骤S63，综合判断。
其中，步骤S61进一步包括：
S611，提取飞机引擎参数，并与预置于系统的数据库中对应机型的飞机引擎参数进行比对。
其中，提取飞机引擎参数的步骤可具体采用前述步骤S341-S343来实现。提取的飞机引擎参数以像素为单位即可。
计算提取到的飞机引擎参数中的飞机引擎半径与系统所收到的机型信息在机场机型参数数据库中所对应的引擎半径数据的比值，该比值称为第一比值。
步骤S612，提取飞机机翼参数，并与预置于数据库中对应机型的飞机机翼参数进行比对。
该提取飞机机翼参数的步骤包括：
步骤S6121，使用Canny算子提取飞机图像的边缘。飞机图像边缘示例图见图10C。
步骤S6122，提取飞机图像边缘像素点，沿远离飞机的登机桥一侧(图10C中为左侧)的飞机引擎中轴向上枚举像素点，对每个像素点，各画多条斜率倾角为0-20°的直线，统计每条所述直线所经过的Canny边缘像素点数；
步骤S6123，确定飞机机翼边缘，取经过边缘像素点数最多的两条直线作为飞机机翼的边缘。
步骤S6124，确定飞机翼尖，取翼尖周围区域的边缘像素，作为翼尖特征记录下来。
机翼轮廓与引擎轮廓示例图见图10D。
步骤S6125，参数比对，通过翼尖位置测量飞机机翼的长度，计算该飞机机翼的长度与系统所收到的机型信息在机场机型参数数据库中所对应的机翼长度数据的比值，该比值称为第二比值。
步骤S613，提取飞机机头特征参数，并与预置于数据库中对应机型的飞机机头特征参数进行比对。
步骤S6131、确定飞机机头边界，通过已经确定的飞机引擎参数，确定飞机的中轴位置，枚举中轴上的点作为圆心，枚举2至4倍所述飞机引擎半径的长度为半径画圆，取经过Canny边缘像素点最多的圆作为飞机机头的边界。
步骤S6132、确定飞机机头窗。
由于窗户在机头的上半圆周，所以采用深度优先搜索的方法，寻找所述飞机机头的边界的上半圆中与所述飞机机头边界的边缘不相粘连的最长边缘，为所述飞机机头的机头窗所在位置。
步骤S6133、参数比对，测量飞机机头半径，计算飞机机头半径与系统所收到的机型信息在机场机型参数数据库中所对应的飞机机头半径的比值，该比值称为第三比值。测量到的该飞机机头半径以像素为单位即可。
步骤S614，提取飞机尾翼参数，并与预置于数据库中对应机型的飞机尾翼参数进 行比对。
步骤S6141，利用深度优先搜索的方法，沿所述飞机机头的边界的上边缘寻找突起的部分为飞机尾翼。
步骤S6142，参数比对，测量所述飞机尾翼高度，以像素为单位即可，计算飞机尾翼高度与系统所收到的机型信息在机场机型参数数据库中所对应的飞机尾翼参数的比值，该比值称为第四比值。
步骤S615，取第一比值、第二比值、第三比值、第四比值这四者中的最小值以及最大值，将最小值/最大值，作为机型相似度参数，作为系数1。
所述模板匹配步骤S62包括：
步骤S621，全局模板匹配，以当前拍摄的整幅图像为被搜索图像，以系统的数据库中的标准飞机图像为模板，计算全局模板相似度参数，以该全局模板相似度参数作为系数2。
图10E为被搜索图像S、子图Sij、模板T的示意图。具体地，全局模板匹配计算过程为：被搜索图像S，图中标记6001，宽高为W*H。被搜索图像S的子图Sij，图中标记6002，宽高为n*m，距离图左边缘j个像素，距离图下边缘i个像素。模板T，图中标记6003，宽高为n*m。模板T与子图Sij的相似度参数R(i,j)：
Figure PCTCN2015083206-appb-000022
M为子图Sij在高度方向能够取得的最大值，N为子图Sij在宽度方向能够取得的最大值。
在所有结果R(i,j)中找出R(i,j)的最大值Rmax(im,jm)，其对应的子图Sij即为匹配目标，Rmax(im,jm)也就是该子图Sij的全局模板相似度参数。
步骤S622，局部模板匹配，分别以步骤S61中提取得到的所述飞机引擎、飞机机翼、飞机机头和所述飞机尾翼位置为被搜索图像，分别以系统所收到的机型信息在机场机型参数数据库中所对应的标准飞机图像的引擎、机翼、机头和尾翼为模板，利用步骤S621中的计算公式，分别计算出所述飞机引擎、飞机机翼、飞机机头和所述飞机尾翼的4个相似度参数R，去掉所述4个相似度参数中的最小值，计算所述4个相似度参数中其余3个相似度参数的平均数为局部模板相似度参数，以该局部模板相似 度参数作为系数3。
步骤S63，综合判断，系数1、2、3中至少有2个大于等于一第一验证阈值，或，系数1、2、3全部大于一第二验证阈值时，当前所拍摄到的飞机与预先获得的该机型信息相符，通过身份验证，否则验证失败。
步骤S7，信息显示步骤。
显示设备3为安装于机场中的可供飞机驾驶员在飞机入坞过程中观看的大型显示屏，同时，也可供机场工作人员观察飞机情况。
如图11所示为显示于该显示设备3中的一种可行的显示方式示例图。
图中7000代表显示设备3上用于显示引导信息的区域，7002代表引导线和停止线所形成的“T”形状，以方便表示飞机和引导线、停止线的相对位置。
步骤S5的飞机定位步骤所确定的飞机的具体位置等泊位引导信息，包括：偏左或偏右7001、距离停止线距离7003均实时显示在该显示设备上。
步骤S6的飞机识别及身份验证步骤所验证的飞机机型信息7004也实时显示在该显示设备上。以供飞行员观察飞机的行进线路，提高飞机入坞的安全性。
综上所述，本发明在飞机入坞过程中，准确实现对飞机的捕获、跟踪、定位和身份验证，并在显示设备3上显示准确有效的飞机泊位引导信息，为飞机驾驶员、副驾驶员或其他人员提供正确有效的泊位引导，使飞机实现安全有效的泊位，提高机场运行效率，保障安全。
当然，本发明还可有其它多种实施例，在不背离本发明精神及其实质的情况下，熟悉本领域的技术人员当可根据本发明作出各种相应的改变和变形，但这些相应的改变和变形都应属于本发明所附的权利要求的保护范围。

Claims (42)

  1. 一种基于机器视觉的飞机入坞引导和机型识别的方法，其特征在于，包括：
    步骤S1，飞机泊位场景设置步骤，将监测场景划分为不同的信息处理功能区；
    步骤S2，图像预处理步骤，对所拍摄的图像进行预处理；
    步骤S3，飞机捕获步骤，通过在该图像中识别飞机的引擎和前轮，以确认该图像中出现飞机；
    步骤S4，飞机跟踪步骤，对步骤S3所捕获到的飞机的引擎和前轮的图像进行连续跟踪和实时更新；
    步骤S5，飞机定位步骤，实现对飞机实时定位并准确判断飞机相对于引导线的偏离程度和相对于停止线的距离；
    步骤S6，信息显示，输出并显示步骤S5的飞机相对于引导线的偏离程度和相对于停止线的距离。
  2. 如权利要求1所述的方法，其特征在于，该图像预处理步骤进一步包括：
    步骤S21，根据该图像的平均灰度值判断该图像为低照度图像、强光照图像还是正常光照图像，对低照度图像执行低照度图像处理步骤，对强光照图像执行强光照图像处理步骤；
    步骤S22，根据该图像的方差判断该正常光照图像是否为正常图像；
    步骤S23，对于非正常图像，判断其为雨雪图像还是雾图像，对雨雪图像执行雨雪图像处理步骤，对雾图像执行雾图像处理步骤。
  3. 如权利要求2所述的方法，其特征在于，该低照度图像处理步骤包括：
    g(x,y)＝f(x,y)+af(x,y)(255-f(x,y))
    f(x,y)为原图像，(x，y)为图像中的像素点坐标，g(x,y)为处理之后的图像，a为低照度图像处理参数。
  4. 如权利要求2所述的方法，其特征在于，该雨雪图像处理步骤包括：
    利用光度测定模型寻找被雨雪污染的待处理像素；
    对于当前图像的待处理像素，提取与该当前图像前后相邻的图像的相应像素的亮度值，根据该亮度值判断与该当前图像前后相邻的图像的相应像素是否均为待处理像素，如果是，取该当前图像的待处理像素的所有相邻像素的亮度值的平均值，用该平均值代替该当前图像的待处理像素的亮度值，如果否，利用该当前图像前后相邻的图像的相应像素的亮度值中的最小值或最小的两个值的平均值，代替该当前图像的待处 理像素的亮度值。
  5. 如权利要求2所述的方法，其特征在于，通过同态滤波进行该雾图像处理步骤。
  6. 如权利要求1所述的方法，其特征在于，该飞机捕获步骤进一步包括：
    步骤S31，背景消除步骤，利用单高斯背景模型来模拟场景中背景的动态分布并进行背景建模，然后将当前图像与背景模型作差分以消除背景，得到前景区域；
    步骤S32，阴影消除步骤，统计该前景区域的灰度值，找出最大灰度值g max和最小灰度值g min，然后在灰度值小于T＝g min+(g max-g min)*0.5的区域进行阴影消除；
    步骤S33，区域分类步骤，建立一个标准正面飞机区域模板，经过变化检测提取目标区域并求取该区域的垂直投影曲线，然后求取该垂直投影曲线与所述标准正面飞机区域模板的垂直投影曲线的相关系数，若该相关系数大于或等于一分类阈值，则该目标为飞机；
    步骤S34，特征验证步骤，通过检测捕获到的飞机的引擎和前轮来进一步验证该目标是否为飞机。
  7. 如权利要求6所述的方法，其特征在于，该特征验证步骤进一步包括：
    步骤S341，图像极黑区域提取，对当前图像的目标区域进行灰度直方图统计，在灰度级中间1％～99％范围内获得最大灰度值、最小灰度值，借助预设的极黑判定阈值以及该最大灰度值、最小灰度值提取图像中最黑的部分，得到一幅极黑区域；
    步骤S342，类圆形检测，提取该极黑区域的所有外层边界，对每一个边界使用边界的矩计算边界的重心坐标，边界的第ji阶矩定义如下：
    Figure PCTCN2015083206-appb-100001
    重心坐标
    Figure PCTCN2015083206-appb-100002
    Figure PCTCN2015083206-appb-100003
    对于当前边界的所有像素点，计算其与该重心的距离，若计算得到的最大距离与最小距离的比值大于一圆形判定阈值，则认为该区域非圆形，进行下一区域的判定，否则认为该区域为类圆形，记录类圆形区域的重心坐标和半径；
    步骤S343，在类圆形区域中通过判断相似度检测飞机引擎；
    步骤S344，检测飞机前轮。
  8. 如权利要求7所述的方法，其特征在于，在步骤S343中，对于检测到的M个类圆形区域，其中第i个和第j个的相似度Similarityij为：
    Similarityij＝|Heighti-Heightj|*|Radiusi-Radiusj|
    其中，Height为重心高度，Radius为半径，当相似度Similarityij小于预设的相似度阈值时，则认为类圆形区域i和j为飞机引擎。
  9. 如权利要求8所述的方法，其特征在于，在步骤S343中，若没有检测出飞机引擎，则进行迭代检测，将所述极黑判定阈值、圆形判定阈值、相似度阈值分别增大，再进行步骤S341-343；若仍然没有检测出飞机引擎，则对所有的极黑区域使用7*7的圆形模板进行开操作，再进行步骤S342-343；
    若仍然没有检测出飞机引擎，则再进行2次上述迭代检测；
    若仍然没有检测出飞机引擎，则判定图像中无引擎存在。
  10. 如权利要求9所述的方法，其特征在于，所述极黑判定阈值、圆形判定阈值、相似度阈值的增加量分别为0.05、0.5、20。
  11. 如权利要求7所述的方法，其特征在于，该步骤S344进一步包括：
    在图像的搜索区域中，将256级的灰度级量化至64级，搜索量化为64级的灰度直方图中的第一个波峰和波谷，原始256级灰度的灰度直方图中的最优波峰位置BestPeak、最优波谷BestValley位置定义如下：
    Figure PCTCN2015083206-appb-100004
    Figure PCTCN2015083206-appb-100005
    其中hist256(i)为256级灰度的灰度直方图中，灰度为i的像素总数；
    以此最优波谷BestValley对灰度进行分割，对小于最优波谷BestValley的部分，除去面积较小的杂点，使用一个扁平椭圆型结构元素对图像进行闭操作；
    接着对所有图形计算边界的7阶Hu矩特征，与预置的标准前轮模型的矩特征进行比对，当相似度低于一阈值时则判定中间一个为前轮。
  12. 如权利要求7所述的方法，其特征在于，该飞机跟踪步骤进一步包括：
    步骤S41，在获得上一帧图像的引擎位置后，采用洪水填充法跟踪确定当前帧的引擎区域；
    步骤S42，如果步骤S41的填充结果无效，执行阴暗环境检测跟踪步骤，使用上 一帧的参数进行步骤S341和步骤S342来检测跟踪引擎区域；
    步骤S43，在获取到引擎区域的信息之后，使用步骤S344检测飞机前轮；
    步骤S44，前轮跟踪应急处理步骤，在检测前轮形状不正确或前轮位置与之前多帧图像相比发生明显偏离时，根据上一帧图像和当前图像的信息，利用相邻两帧图像引擎的位移对该帧的前轮位移进行估计，将估计结果作为前轮跟踪结果，如果超出N帧仍检测不到，则输出错误信息。
  13. 如权利要求12所述的方法，其特征在于，该飞机定位步骤包括：
    步骤S51，摄像装置标定与图像矫正步骤，用于确定摄像装置的光学参数与地理坐标系之间的对应关系；
    步骤S52，飞机前轮偏离程度解算步骤；
    步骤S53，飞机前轮实际距离解算步骤。
  14. 如权利要求13所述的方法，其特征在于，该步骤S51进一步包括：
    步骤S511，读取N幅标定图片；
    步骤S512，使用OpenCV的cvFindChessboardCorners()函数寻找棋盘角点，将读取的所述N幅标定图片分别代入所述cvFindChessboardCorners()函数，如果成功寻找到所有的角点，则函数返回1，并得到角点在图像坐标系下坐标；如果不成功则返回0；
    步骤S513，将成功寻找到的所述角点在标定模板上的坐标代入函数cvCalibrateCamera2()中，返回得到摄像装置的参数矩阵、畸变系数、旋转向量和平移向量。
  15. 如权利要求13所述的方法，其特征在于，该步骤S52进一步包括：
    根据由步骤S43得到前轮的位置坐标点(x0,y0)，利用所述位置坐标点与引导线和停止线的关系，求得引导线的直线方程为：y1＝k1x1+b1，停止线的直线方程为：y2＝k2x2+b2，所述位置坐标点到直线的距离为：
    Figure PCTCN2015083206-appb-100006
    将坐标点(x0,y0)代入两个直线方程分别求得d1和d2，d2≥0表示飞机前轮超出停止线，d2＜0表示飞机前轮未到达停止线，此时若k1＞0，d1＞0则表示飞机偏左，d1＜0表示飞机偏右；若k1＜0，则d1＜0表示飞机偏左，d1＞0表示飞机偏右。
  16. 如权利要求15所述的方法，其特征在于，该步骤S52还包括，判断|d1|＞width/2是否成立，width为一等于检测的飞机前轮的宽度的阈值，如果成立，认为飞机已经 偏离引导线。
  17. 如权利要求13所述的方法，其特征在于，该步骤S53进一步包括：
    建立图像坐标与大地坐标的对应关系；
    由步骤S1的场景设置中的标记点得到图像坐标，采用最小二乘法对该图像坐标进行二次曲线拟合，得到曲线方程y＝ax2+bx+c，x是图像上的距离，y是实际距离；
    对于飞机前轮在图像上的位置，沿停止线方向将其投影到引导线上，计算投影点到停止点的欧氏距离作为x，则通过y＝ax2+bx+c可得到飞机前轮到停止线的实际距离。
  18. 如权利要求1所述的方法，其特征在于，该步骤S3之后还可执行步骤S7，飞机识别及身份验证步骤，步骤S7进一步包括：
    步骤S71，参数验证，提取图像中的飞机参数并与预置于数据库中的机型数据进行比对，得到机型相似度参数；
    步骤S72，模板匹配，将图像与预置于所述数据库中的机型模板进行比对，得到模板相似度参数；
    步骤S73，综合判断，所述机型数据相似度参数与所述模板相似度参数大于或等于一验证阈值时，视为通过身份验证。
  19. 如权利要求18所述的方法，其特征在于，步骤S71进一步包括：
    步骤S711，提取图像中的飞机引擎参数并与预置于数据库中对应机型的飞机引擎参数进行比对，得到第一比值；
    步骤S712，提取图像中的飞机机翼参数并与预置于数据库中对应机型的飞机机翼参数进行比对，得到第二比值；
    步骤S713，提取图像中的飞机机头参数并与预置于数据库中对应机型的飞机机头参数进行比对，得到第三比值；
    步骤S714，提取图像中的飞机尾翼参数并与预置于数据库中对应机型的飞机尾翼参数进行比对，得到第四比值；以及
    步骤S715，取第一比值、第二比值、第三比值、第四比值这四者中的最小值以及最大值，将最小值/最大值，作为该机型相似度参数。
  20. 如权利要求19所述的方法，其特征在于，步骤S72进一步包括：
    步骤S721，全局模板匹配，以整幅图像为被搜索图像，标准飞机图像为模板，计算全局模板相似度参数；
    步骤S722，局部模板匹配，分别以步骤S711-S714中提取得到的所述飞机引擎、飞机机翼、飞机机头和所述飞机尾翼为被搜索图像，分别以标准飞机图像的引擎、机 翼、机头和尾翼为模板，计算被搜索图像与模板的4个相似度，去掉所述4个相似度中的最小值，计算所述4个相似度中其余3个相似度的平均数为局部模板相似度参数。
  21. 如权利要求20所述的方法，其特征在于，步骤S73进一步包括：若所述机型相似度参数、全局模板相似度参数和所述局部模板相似度参数中至少有2个大于或等于第一验证阈值，视为通过身份验证，或，所述机型相似度参数、全局模板相似度参数和所述局部模板相似度参数都大于第二验证阈值，视为通过身份验证。
  22. 一种基于机器视觉的飞机入坞引导和机型识别的系统，其特征在于，包括：
    飞机泊位场景设置单元，用于将监测场景划分为不同的信息处理功能区；
    图像预处理单元，用于对所拍摄的图像进行预处理；
    飞机捕获单元，用于在该图像中识别飞机的引擎和前轮，以确认该图像中出现飞机；
    飞机跟踪单元，用于对所捕获到的飞机的引擎和前轮的图像进行连续跟踪和实时更新；
    飞机定位单元，用于实现对飞机的实时定位并准确判断飞机相对于引导线的偏离程度和相对于停止线的距离；
    信息显示单元，输出并显示飞机相对于引导线的该偏离程度和相对于停止线的距离。
  23. 如权利要求22所述的系统，其特征在于，该图像预处理单元进一步包括：
    图像判断单元，根据该图像的平均灰度值判断该图像为低照度图像、强光照图像还是正常光照图像；
    低照度图像处理单元，用于对低照度图像进行低照度图像处理；
    强光照图像处理单元，用于对强光照图像进行强光照图像处理；
    正常图像判断单元，用于根据该图像的方差判断该正常光照图像是否为正常图像；
    雨雪雾图像判断单元，用于对于非正常图像，判断其为雨雪图像还是雾图像；
    雨雪图像处理单元，用于对雨雪图像执行雨雪图像处理；
    雾图像处理单元，用于对雾图像执行雾图像处理。
  24. 如权利要求23所述的系统，其特征在于，该低照度图像处理单元进一步包括：
    进行g(x,y)＝f(x,y)+af(x,y)(255-f(x,y))处理的单元；
    f(x,y)为原图像，(x，y)为图像中的像素点坐标，g(x,y)为处理之后的图像，a为低照度图像处理参数。
  25. 如权利要求23所述的系统，其特征在于，该雨雪图像处理单元包括：
    待处理像素确定单元，用于利用光度测定模型寻找被雨雪污染的待处理像素；
    亮度处理单元，用于对于当前图像的待处理像素，提取与该当前图像前后相邻的图像的相应像素的亮度值，根据该亮度值判断与该当前图像前后相邻的图像的相应像素是否均为待处理像素，如果是，取该当前图像的待处理像素的所有相邻像素的亮度值的平均值，用该平均值代替该当前图像的待处理像素的亮度值，如果否，利用该当前图像前后相邻的图像的相应像素的亮度值中的最小值或最小的两个值的平均值，代替该当前图像的待处理像素的亮度值。
  26. 如权利要求23所述的系统，其特征在于，该雾图像处理单元通过同态滤波进行该雾图像处理。
  27. 如权利要求22所述的系统，其特征在于，该飞机捕获单元进一步包括：
    背景消除单元，利用单高斯背景模型来模拟场景中背景的动态分布并进行背景建模，然后将当前图像与背景模型作差分以消除背景，得到前景区域；
    阴影消除单元，统计该前景区域的灰度值，找出最大灰度值g max和最小灰度值g min，然后在灰度值小于T＝g min+(g max-g min)*0.5的区域进行阴影消除；
    区域分类单元，建立一个标准正面飞机区域模板，经过变化检测提取目标区域并求取该区域的垂直投影曲线，然后求取该垂直投影曲线与所述标准正面飞机区域模板的垂直投影曲线的相关系数，若该相关系数大于或等于一分类阈值，则该目标为飞机；
    特征验证单元，通过检测捕获到的飞机的引擎和前轮来进一步验证该目标是否为飞机。
  28. 如权利要求27所述的系统，其特征在于，该特征验证单元进一步包括：
    图像极黑区域提取单元，对当前图像的目标区域进行灰度直方图统计，在灰度级中间1％～99％范围内获得最大灰度值、最小灰度值，借助预设的极黑判定阈值以及该最大灰度值、最小灰度值提取图像中最黑的部分，得到一幅极黑区域；
    类圆形检测单元，提取该极黑区域的所有外层边界，对每一个边界使用边界的矩计算边界的重心坐标，边界的第ji阶矩定义如下：
    Figure PCTCN2015083206-appb-100007
    重心坐标
    Figure PCTCN2015083206-appb-100008
    Figure PCTCN2015083206-appb-100009
    对于当前边界的所有像素点，计算其与该重心的距离，若计算得到的最大距离与最小距离的比值大于一圆形判定阈值，则认为该区域非圆形，进行下一区域的判定，否则认为该区域为类圆形，记录类圆形区域的重心坐标和半径；
    引擎检测单元，用于在类圆形区域中通过判断相似度检测飞机引擎；
    前轮检测单元，用于检测飞机前轮。
  29. 如权利要求28所述的系统，其特征在于，在该引擎检测单元中，对于检测到的M个类圆形区域，其中第i个和第j个的相似度Similarityij为：
    Similarityij＝|Heighti-Heightj|*|Radiusi-Radiusj|
    其中，Height为重心高度，Radius为半径，当相似度Similarityij小于预设的相似度阈值时，则认为类圆形区域i和j为飞机引擎。
  30. 如权利要求29所述的系统，其特征在于，在该引擎检测单元中进一步包括，若没有检测出飞机引擎，则进行迭代检测，将所述极黑判定阈值、圆形判定阈值、相似度阈值分别增大，再调用图像极黑区域提取单元、类圆形检测单元和引擎检测单元；若仍然没有检测出飞机引擎，则对所有的极黑区域使用7*7的圆形模板进行开操作，再调用类圆形检测单元和引擎检测单元的单元；
    若仍然没有检测出飞机引擎，则再进行2次上述迭代检测的单元；
    若仍然没有检测出飞机引擎，则判定图像中无引擎存在的单元。
  31. 如权利要求30所述的系统，其特征在于，所述极黑判定阈值、圆形判定阈值、相似度阈值的增加量分别为0.05、0.5、20。
  32. 如权利要求28所述的系统，其特征在于，该前轮检测单元进一步包括：在图像的搜索区域中，将256级的灰度级量化至64级，搜索量化为64级的灰度直方图中的第一个波峰和波谷，原始256级灰度的灰度直方图中的最优波峰位置BestPeak、最优波谷BestValley位置定义如下：
    Figure PCTCN2015083206-appb-100010
    Figure PCTCN2015083206-appb-100011
    其中hist256(i)为256级灰度的灰度直方图中，灰度为i的像素总数；
    以此最优波谷BestValley对灰度进行分割，对小于最优波谷BestValley的部分，除去面积较小的杂点，使用一个扁平椭圆型结构元素对图像进行闭操作；
    接着对所有图形计算边界的7阶Hu矩特征，与预置的标准前轮模型的矩特征进行比对，当相似度低于一阈值时则判定中间一个为前轮的单元。
  33. 如权利要求28所述的系统，其特征在于，该飞机跟踪单元进一步包括：
    洪水填充单元，在获得上一帧图像的引擎位置后，采用洪水填充法跟踪确定当前帧的引擎区域；
    阴暗环境检测跟踪单元，如果洪水填充单元的填充结果无效，阴暗环境检测跟踪单元使用上一帧的参数调用图像极黑区域提取单元和类圆形检测单元来检测跟踪引擎区域；
    该前轮检测单元，在获取到引擎区域的信息之后，检测飞机前轮；
    前轮跟踪应急处理单元，在检测前轮形状不正确或前轮位置与之前多帧图像相比发生明显偏离时，根据上一帧图像和当前图像的信息，利用相邻两帧图像引擎的位移对该帧的前轮位移进行估计，将估计结果作为前轮跟踪结果，如果超出N帧仍检测不到，则输出错误信息。
  34. 如权利要求33所述的系统，其特征在于，该飞机定位单元包括：
    摄像装置标定与图像矫正单元，用于确定摄像装置的光学参数与地理坐标系之间的对应关系；
    飞机前轮偏离程度解算单元；
    飞机前轮实际距离解算单元。
  35. 如权利要求34所述的系统，其特征在于，该摄像装置标定与图像矫正单元进一步包括：
    读取N幅标定图片的单元；
    使用OpenCV的cvFindChessboardCorners()函数寻找棋盘角点，将读取的所述N幅标定图片分别代入所述cvFindChessboardCorners()函数，如果成功寻找到所有的角点，则函数返回1，并得到角点在图像坐标系下坐标,如果不成功则返回0的单元；
    将成功寻找到的所述角点在标定模板上的坐标代入函数cvCalibrateCamera2()中，返回得到摄像装置的参数矩阵、畸变系数、旋转向量和平移向量的单元。
  36. 如权利要求34所述的系统，其特征在于，该飞机前轮偏离程度解算单元进一步包括：
    引导线的直线方程为：y1＝k1x1+b1，停止线的直线方程为：y2＝k2x2+b2，所述前轮的位置坐标点到直线的距离为：
    Figure PCTCN2015083206-appb-100012
    将前轮的位置坐标点(x0,y0)代入两个直线方程分别求得d1和d2，d2≥0表示飞机前轮超出停止线，d2＜0表示飞机前轮未到达停止线，此时若k1＞0，d1＞0则表示飞机偏左，d1＜0表示飞机偏右；若k1＜0，则d1＜0表示飞机偏左，d1＞0表示飞机偏右的单元。
  37. 如权利要求36所述的系统，其特征在于，该飞机前轮偏离程度解算单元进一步包括：判断|d1|＞width/2是否成立，width为一等于检测的飞机前轮的宽度的阈值，如果成立，认为飞机已经偏离引导线的单元。
  38. 如权利要求34所述的系统，其特征在于，该飞机前轮实际距离解算单元进一步包括：
    建立图像坐标与大地坐标的对应关系的单元；
    由场景设置中的标记点得到图像坐标，采用最小二乘法对该图像坐标进行二次曲线拟合，得到曲线方程y＝ax2+bx+c，x是图像上的距离，y是实际距离的单元；
    对于飞机前轮在图像上的位置，沿停止线方向将其投影到引导线上，计算投影点到停止点的欧氏距离作为x，则通过y＝ax2+bx+c可得到飞机前轮到停止线的实际距离的单元。
  39. 如权利要求22所述的系统，其特征在于，该系统还包括飞机识别及身份验证单元，该飞机识别及身份验证单元进一步包括：
    参数验证单元，提取图像中的飞机参数并与预置于数据库中的机型数据进行比对，得到机型相似度参数；
    模板匹配单元，将图像与预置于所述数据库中的机型模板进行比对，得到模板相似度参数；
    综合判断单元，所述机型数据相似度参数与所述模板相似度参数大于或等于一验证阈值时，视为通过身份验证。
  40. 如权利要求39所述的系统，其特征在于，参数验证单元进一步包括：
    第一比值取得单元，用于提取图像中的飞机引擎参数并与预置于数据库中对应机型的飞机引擎参数进行比对，得到第一比值；
    第二比值取得单元，用于提取图像中的飞机机翼参数并与预置于数据库中对应机 型的飞机机翼参数进行比对，得到第二比值；
    第三比值取得单元，用于提取图像中的飞机机头参数并与预置于数据库中对应机型的飞机机头参数进行比对，得到第三比值；
    第四比值取得单元，用于提取图像中的飞机尾翼参数并与预置于数据库中对应机型的飞机尾翼参数进行比对，得到第四比值；以及
    机型相似度参数获取单元，用于取第一比值、第二比值、第三比值、第四比值这四者中的最小值以及最大值，将最小值/最大值，作为该机型相似度参数。
  41. 如权利要求40所述的系统，其特征在于，该模板匹配单元进一步包括：
    全局模板匹配单元，用于以整幅图像为被搜索图像，标准飞机图像为模板，计算全局模板相似度参数；
    局部模板匹配单元，分别以第一比值取得单元、第二比值取得单元、第三比值取得单元和第四比值取得单元中提取得到的所述飞机引擎、飞机机翼、飞机机头和所述飞机尾翼为被搜索图像，分别以标准飞机图像的引擎、机翼、机头和尾翼为模板，计算被搜索图像与模板的4个相似度，去掉所述4个相似度中的最小值，计算所述4个相似度中其余3个相似度的平均数为局部模板相似度参数。
  42. 如权利要求41所述的系统，其特征在于，综合判断单元进一步包括：若所述机型相似度参数、全局模板相似度参数和所述局部模板相似度参数中至少有2个大于或等于第一验证阈值，视为通过身份验证，或，所述机型相似度参数、全局模板相似度参数和所述局部模板相似度参数都大于第二验证阈值，视为通过身份验证的单元。
PCT/CN2015/083206 2014-08-01 2015-07-02 一种基于机器视觉的飞机入坞引导和机型识别的方法及系统 Ceased WO2016015547A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/329,994 US10290219B2 (en) 2014-08-01 2015-07-02 Machine vision-based method and system for aircraft docking guidance and aircraft type identification
EP15828078.4A EP3196853A4 (en) 2014-08-01 2015-07-02 Machine vision-based method and system for aircraft docking guidance and aircraft type identification

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410378566.8 2014-08-01
CN201410378566.8A CN105373135B (zh) 2014-08-01 2014-08-01 一种基于机器视觉的飞机入坞引导和机型识别的方法及系统

Publications (1)

Publication Number Publication Date
WO2016015547A1 true WO2016015547A1 (zh) 2016-02-04

Family

ID=55216748

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/083206 Ceased WO2016015547A1 (zh) 2014-08-01 2015-07-02 一种基于机器视觉的飞机入坞引导和机型识别的方法及系统

Country Status (4)

Country Link
US (1) US10290219B2 (zh)
EP (1) EP3196853A4 (zh)
CN (1) CN105373135B (zh)
WO (1) WO2016015547A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110110115A (zh) * 2019-03-28 2019-08-09 西南电子技术研究所（中国电子科技集团公司第十研究所） 人机交互筛选识别机载目标图像的方法
CN110569764A (zh) * 2019-08-28 2019-12-13 北京工业大学 一种基于卷积神经网络的手机型号识别方法
CN110908387A (zh) * 2019-12-13 2020-03-24 齐鲁工业大学 一种动态环境水面无人艇路径规划方法、介质及电子设备
CN113420693A (zh) * 2021-06-30 2021-09-21 成都新潮传媒集团有限公司 一种门状态检测方法、装置、轿厢乘客流量统计方法及设备
CN114547763A (zh) * 2020-11-27 2022-05-27 北京理工大学 飞行器指挥控制计算机系统
CN116503768A (zh) * 2023-06-29 2023-07-28 中国科学院心理研究所 一种飞行设备空中对接方法及装置
CN116758271A (zh) * 2023-05-09 2023-09-15 中国航空无线电电子研究所 飞行指引符号图像识别方法

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101556598B1 (ko) * 2014-11-25 2015-10-13 연세대학교 산학협력단 주요 픽셀 정보에 기반한 물체 검출 장치 및 그 방법
CN107710278A (zh) * 2015-06-18 2018-02-16 日本电气方案创新株式会社 图像处理设备，图像处理方法和计算机可读记录介质
CN106205217B (zh) * 2016-06-24 2018-07-13 华中科技大学 基于机器视觉的无人机自动检测方法及无人机管制方法
CN109477888B (zh) * 2016-07-22 2023-10-20 波音公司 低分辨率自适应距离显示
US20190096246A1 (en) * 2017-09-25 2019-03-28 Continental Automotive Systems, Inc. Compact modular wireless sensing radar infrastructure device
US11188800B1 (en) * 2018-04-03 2021-11-30 Robert Edwin Douglas Method and apparatus for improved analysis of CT scans of bags
FR3082329B1 (fr) * 2018-06-06 2020-05-15 Thales Procede de securisation du fonctionnement d'un systeme de vision synthetique d'un aeronef, produit programme d'ordinateur et systeme associes
EP3584172B1 (en) * 2018-06-18 2020-10-07 ADB Safegate Sweden AB A method and a system for guiding a pilot of an approaching aircraft to a stop position at a stand
CN108921844A (zh) * 2018-07-06 2018-11-30 许继集团有限公司 一种绝缘子缺陷检测方法与装置
TWI684996B (zh) * 2018-07-27 2020-02-11 台灣迅聯通信股份有限公司 智慧駐停輔具
EP3691277A1 (en) * 2019-01-30 2020-08-05 Ubimax GmbH Computer-implemented method and system of augmenting a video stream of an environment
CN109887343B (zh) * 2019-04-04 2020-08-25 中国民航科学技术研究院 一种航班地服保障节点自动采集监测系统及方法
CN110188601B (zh) * 2019-04-16 2022-07-15 昆明理工大学 一种基于学习的机场遥感图像检测方法
CN112034831B (zh) * 2019-06-04 2021-11-09 深圳中集天达空港设备有限公司 登机桥接靠飞机的方法及电子设备以及存储介质
PT3757968T (pt) * 2019-06-28 2022-04-12 Adb Safegate Sweden Ab Disposição de plataforma de estacionamento de aeroporto e método
US11308646B2 (en) * 2020-05-13 2022-04-19 Walmart Apollo, Llc Extracting color from item images
CN112139854B (zh) * 2020-09-28 2022-04-19 深圳数马电子技术有限公司 一种钥匙齿识别方法以及装置
CN112966636A (zh) * 2021-03-19 2021-06-15 捻果科技(深圳)有限公司 一种用于民航机场飞行区客梯车靠接航空器的自动识别方法
KR102567249B1 (ko) * 2021-04-21 2023-08-21 (주)안세기술 3차원 레이저 스캐너를 이용한 항공기 주기유도 시스템 및 이를 이용한 항공기 주기 제어방법
EP4177864A1 (en) * 2021-11-09 2023-05-10 TK Airport Solutions, S.A. Visual docking guidance system
CN114360296B (zh) * 2021-12-15 2024-04-09 中国飞行试验研究院 基于地基光电设备的全自动飞机进近降落过程监视方法
US20230267753A1 (en) * 2022-02-24 2023-08-24 Honeywell International Inc. Learning based system and method for visual docking guidance to detect new approaching aircraft types
CN114811339B (zh) * 2022-03-14 2023-08-01 柳州工学院 一种计算机视觉识别装置
EP4559817A4 (en) * 2022-07-22 2026-03-25 Shinmaywa Ind Ltd AIRPORT GROUND SUPPORT EQUIPMENT
CN115661110B (zh) * 2022-11-08 2023-08-01 西南交通大学 一种透明工件识别与定位的方法
CN115690743B (zh) * 2022-11-10 2025-12-16 青岛高重信息科技有限公司 一种基于深度学习的机场停机位入侵的检测方法
CN116039941B (zh) * 2022-11-27 2023-07-18 燕山大学 适用于海上作业直升机的全自动牵引校正入库系统及方法
JP2024077118A (ja) * 2022-11-28 2024-06-07 キヤノン株式会社 画像処理装置、撮像装置、制御方法およびプログラム
CN115790610B (zh) * 2023-02-06 2023-04-25 北京历正飞控科技有限公司 一种无人机精准定位系统及方法
CN116091488B (zh) * 2023-03-07 2023-07-14 西安航天动力研究所 一种发动机摇摆试验的位移测试方法及位移测试系统
CN116823808B (zh) * 2023-08-23 2023-11-17 青岛豪迈电缆集团有限公司 基于机器视觉的电缆绞线智能检测方法
CN117011284B (zh) * 2023-09-01 2025-09-16 国网江苏省电力有限公司泰州供电分公司 一种输电线路异物检测方法及装置
CN118608463B (zh) * 2024-05-23 2025-10-17 成都飞机工业（集团）有限责任公司 一种图像圆孔计数和分类方法
CN121505490B (zh) * 2026-01-12 2026-04-03 环宇佳诚科技(北京)有限公司 复杂战场环境下高超声速飞行器目标类型识别方法和系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030160709A1 (en) * 2001-12-20 2003-08-28 Jan Westlund Centreline identification in a docking guidance system
CN1300750C (zh) * 2005-03-07 2007-02-14 张积洪 飞机泊位机型自动识别与指示系统
WO2013141605A1 (ko) * 2012-03-21 2013-09-26 (주)안세기술 항공기의 기종판별 및 주기유도를 위한 시스템 및 방법

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6324489B1 (en) * 1999-10-29 2001-11-27 Safegate International Ab Aircraft identification and docking guidance systems
US5734736A (en) * 1994-06-17 1998-03-31 Trw Inc. Autonomous rendezvous and docking system and method therefor
JP2713172B2 (ja) * 1994-07-25 1998-02-16 日本電気株式会社 移動体の駐機誘導装置
US5675661A (en) * 1995-10-12 1997-10-07 Northrop Grumman Corporation Aircraft docking system
BR9707716A (pt) * 1996-02-29 2000-01-04 Siemens Ag Sistema de condução de aeroportos, especialmente sistema de condução de tráfego no solo de tráfego no solo de aeroportos
FR2763727B1 (fr) * 1997-05-20 1999-08-13 Sagem Procede et systeme de guidage d'un avion vers un poste d'accostage
US6542086B2 (en) 1997-09-22 2003-04-01 Siemens Aktiengesellschaft Docking system for airport terminals
JP3015875B2 (ja) * 1998-05-19 2000-03-06 工業技術院長 自動車運転時の車線逸脱検出方法及び検出装置
US6564142B2 (en) * 1999-02-01 2003-05-13 Aero Modifications & Consulting, L.L.C. System and apparatus for determining the center of gravity of an aircraft
US6353793B1 (en) * 1999-02-01 2002-03-05 Aero Modifications & Consulting, Llc System and apparatus for determining the center of gravity of an aircraft
ES2206016B1 (es) * 2002-04-04 2005-07-16 Team Tecnologia Europea Aplicada Al Movimiento, S.L. Procedimiento y aparato para el control de la posicion de aeronaves en maniobras en tierra despues del aterrizaje.
KR100933483B1 (ko) * 2008-01-28 2009-12-23 국방과학연구소 영상 내의 표적 인식 방법
CN101739694B (zh) * 2010-01-07 2011-10-05 北京智安邦科技有限公司 基于图像分析的高压输电线的超高检测的方法及装置
EP2736872A1 (de) * 2011-07-27 2014-06-04 Basf Se Verfahren zur herstellung von formamiden und ameisensäureestern
US9280155B2 (en) * 2011-12-02 2016-03-08 Borealis Technical Limited Aircraft ground travel traction control system and method
CN103049788B (zh) * 2012-12-24 2015-11-18 南京航空航天大学 基于计算机视觉的待过行人数目的检测系统及方法
CN103177586A (zh) * 2013-03-05 2013-06-26 天津工业大学 一种基于机器视觉的城市交叉路口多车道车流量检测方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030160709A1 (en) * 2001-12-20 2003-08-28 Jan Westlund Centreline identification in a docking guidance system
CN1300750C (zh) * 2005-03-07 2007-02-14 张积洪 飞机泊位机型自动识别与指示系统
WO2013141605A1 (ko) * 2012-03-21 2013-09-26 (주)안세기술 항공기의 기종판별 및 주기유도를 위한 시스템 및 방법

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
GUO, YANYING: "Research on Key Technologies in Visual-based Aircraft Docking Automatic Guided", CHINA DOCTORAL DISSERTATIONS FULL-TEXT DATABASE, 30 June 2014 (2014-06-30), pages 12 - 37, XP008185316 *
See also references of EP3196853A4 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110110115A (zh) * 2019-03-28 2019-08-09 西南电子技术研究所（中国电子科技集团公司第十研究所） 人机交互筛选识别机载目标图像的方法
CN110110115B (zh) * 2019-03-28 2023-05-23 西南电子技术研究所（中国电子科技集团公司第十研究所） 人机交互筛选识别机载目标图像的方法
CN110569764A (zh) * 2019-08-28 2019-12-13 北京工业大学 一种基于卷积神经网络的手机型号识别方法
CN110569764B (zh) * 2019-08-28 2023-12-22 北京工业大学 一种基于卷积神经网络的手机型号识别方法
CN110908387A (zh) * 2019-12-13 2020-03-24 齐鲁工业大学 一种动态环境水面无人艇路径规划方法、介质及电子设备
CN114547763A (zh) * 2020-11-27 2022-05-27 北京理工大学 飞行器指挥控制计算机系统
CN113420693A (zh) * 2021-06-30 2021-09-21 成都新潮传媒集团有限公司 一种门状态检测方法、装置、轿厢乘客流量统计方法及设备
CN113420693B (zh) * 2021-06-30 2022-04-15 成都新潮传媒集团有限公司 一种门状态检测方法、装置、轿厢乘客流量统计方法及设备
CN116758271A (zh) * 2023-05-09 2023-09-15 中国航空无线电电子研究所 飞行指引符号图像识别方法
CN116503768A (zh) * 2023-06-29 2023-07-28 中国科学院心理研究所 一种飞行设备空中对接方法及装置
CN116503768B (zh) * 2023-06-29 2023-11-07 中国科学院心理研究所 一种飞行设备空中对接方法及装置

Also Published As

Publication number Publication date
US20170263139A1 (en) 2017-09-14
CN105373135B (zh) 2019-01-01
US10290219B2 (en) 2019-05-14
EP3196853A1 (en) 2017-07-26
CN105373135A (zh) 2016-03-02
EP3196853A4 (en) 2018-07-11

Similar Documents

Publication Publication Date Title
WO2016015547A1 (zh) 一种基于机器视觉的飞机入坞引导和机型识别的方法及系统
CN109522793B (zh) 基于机器视觉的多人异常行为检测与识别方法
CN105302151B (zh) 一种飞机入坞引导和机型识别的系统及方法
KR101184097B1 (ko) 얼굴 정면포즈 판단 방법
CN103714541B (zh) 一种利用山体轮廓区域约束识别定位建筑物的方法
US12131485B2 (en) Object tracking device and object tracking method
CN107133973B (zh) 一种桥梁防撞系统中的船舶检测方法
CN111461023A (zh) 基于三维激光雷达的四足机器人自主跟随领航员的方法
US9275472B2 (en) Real-time player detection from a single calibrated camera
CN103810491B (zh) 融合深度和灰度图像特征点的头姿估计兴趣点检测方法
CN105447431B (zh) 一种基于机器视觉的入坞飞机跟踪定位方法及系统
CN111144207B (zh) 一种基于多模态信息感知的人体检测和跟踪方法
CN105279772B (zh) 一种红外序列图像的可跟踪性判别方法
CN108197604A (zh) 基于嵌入式设备的快速人脸定位跟踪方法
CN109063669B (zh) 一种基于图像识别的桥区船舶航行态势分析方法及装置
CN107153819A (zh) 一种排队长度自动检测方法及排队长度控制方法
CN108416798A (zh) 一种基于光流的车辆距离估计方法
CN110414340A (zh) 一种船闸监控系统中的船舶识别方法
CN107067752A (zh) 基于无人机航拍图像的车速估算系统及方法
CN112683228A (zh) 单目相机测距方法及装置
CN105335985A (zh) 一种基于机器视觉的入坞飞机实时捕获方法及系统
CN105335764B (zh) 一种入坞飞机机型识别验证系统及方法
CN105447496B (zh) 一种基于机器视觉的入坞飞机机型识别验证方法和系统
CN108985216B (zh) 一种基于多元logistic回归特征融合的行人头部检测方法
Zhang et al. Morphology-based building target detection from forward-looking infrared imagery in dense urban areas

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15828078

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15329994

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015828078

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015828078

Country of ref document: EP