WO2002005207A2 - Systeme d'analyse d'images pour le traitement d'images - Google Patents

Systeme d'analyse d'images pour le traitement d'images Download PDF

Info

Publication number
WO2002005207A2
WO2002005207A2 PCT/IL2001/000597 IL0100597W WO0205207A2 WO 2002005207 A2 WO2002005207 A2 WO 2002005207A2 IL 0100597 W IL0100597 W IL 0100597W WO 0205207 A2 WO0205207 A2 WO 0205207A2
Authority
WO
WIPO (PCT)
Prior art keywords
feature
classifier according
image classifier
filter
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IL2001/000597
Other languages
English (en)
Other versions
WO2002005207A3 (fr
Inventor
Joseph Shamir
Offer Har
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Technion Research and Development Foundation Ltd
Original Assignee
Technion Research and Development Foundation Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Technion Research and Development Foundation Ltd filed Critical Technion Research and Development Foundation Ltd
Priority to AU2001267813A priority Critical patent/AU2001267813A1/en
Publication of WO2002005207A2 publication Critical patent/WO2002005207A2/fr
Publication of WO2002005207A3 publication Critical patent/WO2002005207A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/88—Image or video recognition using optical means, e.g. reference filters, holographic masks, frequency domain filters or spatial domain filters

Definitions

  • the present invention relates to a classifier for image processing and more particularly but not exclusively to a classifier for measuring the presence- of features within image parts and thereby inferring a classification of the image part.
  • processing may include some or all of the following steps:
  • the units to identify are easy to find, and algorithms exist for distinguishing basic shapes in the letters. Once the letters are identified the original text can be reconstructed.
  • the main obstacle cited was the computation and processing time required for the classification algorithm.
  • the matched filter technique is optimal for detecting a known signal that is corrupted by additive White Gaussian Noise, which is the case with a communication signal.
  • additive white Gaussian noise is not generally a problem in optics.
  • an optical correlator usually the objective of an optical correlator is not to identify the existence of a specific signal (image) but to discriminate between several different signals (images) which may be similar in many aspects.
  • the noise in an optical system is not additive, and does not posses a White Gaussian distribution.
  • optical signals (images) which the correlator is required to recognize are derived from three-dimensional objects, and not a one- dimensional signal as in communication theory.
  • signals (images) which are of the same type may appear differently due, for example, to in plane rotation and out of plane tilt of the 3D object.
  • SLM Spatial Light Modulator
  • the original correlator introduced by VanderLugt is known as the 4F correlator.
  • the correlator's name is simply derived from its physical length which is four times the lens' focal length.
  • the basic architecture of the correlator 10 is shown in Figure- 1 below which shows an input plane 12, a first lens 14, a fourier plane 16, a second lens 18 and a correlation plane 20.
  • the simple architecture of the correlator is misleading, and when used two main problems are encountered.
  • the spatial filter used in the Fourier plane is holographic in nature and therefore the correlation data rides on a carrier frequency, this means that the second part of the correlator should be tilted from the system's main optical axis.
  • the Fourier transform created by a single converging lens leads to a high demagnification of the Fourier transform and therefore to a loss of resolution.
  • the required off-axis measurement of the correlation can be overcome by using a low carrier frequency.
  • a low carrier frequency puts a strong limitation on the signal's Space-Bandwidth Product (SBP), which is derived from Nyquist' s criteria.
  • SBP Space-Bandwidth Product
  • POF Phase Only Filters
  • phase modulator SLM further help to overcome this basic limitation of the 4F correlator.
  • the high demagnification of the Fourier transform causes a high resolution of the information storage in the Fourier plane. Filtering such high- resolution information requires the use of a very high resolution SLM, a wide dynamic range, and a stringent alignment tolerance.
  • JTC Joint Transform Correlator
  • SF Synthetic Filter
  • the output of the first plane is taken from the Fourier plane 34, by CCD 38 and projected onto the SLM 28, where it forms a filter onto which the reference image or a derivation thereof is projected as a filter, as will be explained in more detail below.
  • the correlation plane thus receives the result of filtering the main image through the Fourier based derivation of the reference image.
  • the result is detected by CCD 40.
  • the filter may be constructed optically
  • JTC's architecture is its compatibility with electronic computers, basically it is possible to connect a computer
  • the idea behind the design of the JTC is to use a reference image identical to the input image, so as to give a positive identification of the input image by means of the correlation.
  • Use of the JTC does not necessarily involve computing and creating a specific spatial filter.
  • the JTC architecture suffers from a further reduction of the SBP due to the partitioning of the input plane to hold both the image to be correlated and the reference image. JTC performance is also limited by the resolution of the SLM and the CCDs.
  • the JTC's architecture as shown in Figure-2 suffers from difficulty in integrating a complex Synthetic Reference Function (SRF).
  • SRF Synthetic Reference Function
  • Fig. 3 shows a mathematical equivalent, according to first order optics, of the arrangement of Fig. 1.
  • a so-called 2F correlator due to its length being twice that of the lenses' focal length, comprises an input plane 42, a Fourier plane 44 and a
  • the Fourier plane is surrounded by lenses on either side.
  • the MSFs are very sensitive to small changes in the reference signal.
  • the property restricts the usefulness of the MSF.
  • the MSF are light - inefficient because their magnitude response is usually much smaller than one at most frequencies. This magnitude response problem can be overcome by the use of POF. However, at this time the technology does not permit the extensive use of such filters with a high resolution.
  • Figures 1 to 3 each show different types of Fourier correlators, and all three types of Fourier Plane correlators presented in Figures 1-3 are equivalent mathematically.
  • the basic idea is that these correlators compute the correlation between a reference image that is placed at the input plane and a function (the Spatial Filter) that is placed at the Fourier plane, while the result of the correlation is presented at the output plane.
  • f(x,y) is the input signal, the image placed at the input plane.
  • F ⁇ x, y) is the Fourier transform of the input image.
  • h[x, ) is the image of the spatial filter.
  • H[x,y) is the Fourier transform of the spatial filter image.
  • the output function c ⁇ x,y) represents the cross-correlation between the
  • input image which in our case are the image of the object to be classified, and the spatial filter's image.
  • Such a choice of a spatial filter h x,y) preferably yields the same output
  • the output for an object's image that is not a part of the training set may be in the vicinity of such a constant.
  • the construction of the spatial filter may be achieved by multiple exposure of a film
  • Kallman's Composite Filter Design - aimed at the design of distortion invariant correlation filters.
  • Circular Harmonic Expansion Based Filters which are important techniques in the design of distortion invariant correlation filters.
  • Quadratic Filters These possess the quality of both distortion invariance and clutter rejection that make them more tolerable to input noise
  • Pattern recognition has evolved as part of the field of Artificial Intelligence, in parallel to its evolution in optics.
  • the field of pattern recognition has been explored and covered by many different papers, surveys, and books which provide a lot of information on the subject. Following is a
  • the traditional Al approach has been to divide the pattern classification problem into two sub-problems.
  • the first deals with feature extraction, while the second is a decision-making problem.
  • This is probably the most common approach used by the Al community in pattern recognition since it makes use of already present decision making algorithms.
  • the most basic decision making paradigm utilized is a Decision-Tree.
  • the decision tree lies on the boundary between conventional methods and Al.
  • a decision tree is described by a set of questions, "if ... then" statements, ordered according to some logic about the problem. When the question sequence is followed it allows the problem solver to converge to the correct decision.
  • the decision making process is called a decision tree since, when presented in a diagram, the flow through the problem solving process is presented as a tree. In the decision tree every node (that is not a leaf) presents a question about the object, and every leaf presents the class, i.e. the decision which would be reached.
  • Figure A represents a simple decision tree.
  • the decision tree comprises nodes and leaves, and
  • Bayes' Theorem when Bayes first presented what was later to be known as Bayes' Theorem.
  • the main advantage in Bayes' theorem is that it allows making "logical decision" when only partial data is known about the situation. However, it is required for the algorithm to know the entire conditional probability of the problem. For example consider Bayes' formula:
  • Wong and Chang present a Bayesian based algorithm for the recognition of Chinese characters, through the work they present the classification problem as a Bayesian problem and show an off-line algorithm that can be used for classification.
  • Fuzzy logic techniques try to overcome the need to know the accurate conditional probability function by employing a fuzzy decision rule that substitutes Bayes' Theorem.
  • fuzzy logic decision making In the last few years a lot of work has been devoted to the research of fuzzy logic decision making. Fuzzy logic research has resulted in the development of a wide mathematical foundation for fuzzy decision making which is very similar to the probability theory.
  • Fuzzy logic Another problem on which Fuzzy logic has been put to use is the decision about the probability of detecting an image or a feature in an image.
  • the work relates to the feature extraction process.
  • Neural networks were presented as a computer problem-solving scheme that simulates human brain activities. As Neural networks were introduced to Al many attempts have been made to use them for decision making and specifically for pattern recognition. Taver presents a long review of the use of these techniques.
  • Neural networks The main idea behind Neural networks was to build a net composed of simple elements, perceptrons, which work in concert to yield a solution. Each perceptron receives any number of inputs and generates an output that is usually a binary function. Figure-5 presents a single perceptron having such a series of inputs, leading to a function and from which is produced an output.
  • the Neural network is usually made by a non-linear function /(•) .
  • the Neural network is usually made by a non-linear function /(•) .
  • pattern classification and recognition is the use of Associative Memory.
  • the idea in this problem-solving scheme is that the algorithm learns a set of classes and basically given an input image (or a sub-set of an image), finds the class that is closest to it among all the base classes. Basically the results of the algorithm are similar to the matched correlator only that instead of using a matched filter with which the correlation is computed the system utilizes the network to perform a similar task.
  • One of the more common human problem solving techniques is to guess a solution, test it and, according to the test results, update the guess according to any test error, until the guessed solution is close enough to the required answer, i.e. the error is small enough.
  • humans make use of logic to guide the guessing scheme, that is we actually search the solution space in accordance with some search strategy (the logic), until an acceptable solution is found.
  • classification problems and such search techniques may be used to guide two different actions in the classification process.
  • One such technique involves guiding the feature selection, or the tests
  • a second technique involves guessing a solution and trying to correct the guess following a comparison between the guessed pattern and reference pattern, which comparison the algorithm tries to classify.
  • Algorithms which follow the first approach presented include most heuristic searches such as: A*, RTA*, LRTA*, which are presented by Korf. Other types of algorithm which can be utilized are Bi-directional and Moving Target Searches, the details of which are known to the skilled person.
  • the main methodology that is followed is to construct in real time the shortest path
  • Implementations of the technique may for example utilize Genetic Algorithms, Simulated annealing, and other solution oriented search techniques.
  • the technique involves guessing a solution and trying to correct it according to the differences obtained from comparing the guess with the reference pattern.
  • All the decision-making algorithms considered above require, as an input, a set of properties of the pattern or shape to be classified. These properties are referred to herein as features, and the features can be used to describe the pattern or shape in the solution space.
  • One of the more complicated problems in computer based pattern classification and recognition problems is to measure the properties that are present in the pattern or shape to be classified or, more precisely to extract the features.
  • the first problem encountered is the image segmentation task.
  • image segmentation the computer is required to separate the pattern to be classified from the background scenery and other objects and patterns which may be a
  • the second problem that is encountered in computer based feature extraction is the amount of computation needed for the process. Assuming that the image presented contains a single pattern, i.e. ignoring the segmentation issue, it is possible to perform a frequency domain filtering technique in order to extract a feature. Such a procedure requires the performance of several 2D Fourier transforms, and matrix operations. Such transforms and operations are computationally intensive. Furthermore, with a digital computer it is not
  • Sclaroff and Pentland present a modal matching correspondence and recognition algorithm that finds a correspondence (the correlation) between an object model and an image thereof.
  • the classification is reached by finding the model that is least deformed by matching to the reference object. This is actually the computation of a correlation and the choice of the closest object.
  • the computation is not performed as a direct correlation between the images but the idea is still similar, and makes use of the computational advantages of the computer in mathematical computation and not only bitmap matching.
  • Ben Arie and Wang present another algorithm that performs object classification.
  • the algorithm presented makes use of Affine correspondence in the frequency domain.
  • the basic idea behind the algorithm is to find a correlation between the object's image presented and basic properties of the object which are known. In this case those properties (features) are all in the frequency domain, and the correlation is computed by finding an Affine transformation between the requested feature and the property of the object at hand.
  • He and Kundu present an algorithm that assumes that objects can be described using a Markov model.
  • the algorithm utilizes a hidden Markov model and uses it for the classification process. Again the method relies on the idea of measuring the correlation between the object and some reference set of
  • Optical components are very good at performing massive computation such as Correlation computation, Fourier analysis, etc, since these can be performed at the speed of light on an entire image or image part, relying on the properties of wave propagation and those of the optical system.
  • Electronic computers are used for the result analysis of the optical measurements and for the control of the entire system including the optical system.
  • Hybrid optical systems have been developed as Hybrid optical systems and more specifically as hybrid classifiers. Nevertheless, the current state of the hybrid optical classifier is unsatisfactory in terms of the time taken to converge on a classification decision, and particularly the ability to cope with uncertainty.
  • an image classifier for classifying objects of an image comprising:
  • a feature measurer for measuring objects to determine whether said objects comprise features useful in classifying said objects
  • conditionality network associated with said feature measurer for using conditionality to select said features useful in classifying interactively with
  • said feature measurer comprises an optical processor.
  • said feature measurer comprises a hybrid electro-optical processor.
  • said feature measurer comprises a digital processor.
  • said features useful in classifying are arranged to form a classification feature language.
  • a preferred embodiment further comprises a feature builder for building a set of features useful in classifying into a classification feature language.
  • said conditionality network comprises a Bayesian network.
  • said Bayesian network is controllable to use back propagation of said measurement outputs to select a next feature for measurement.
  • said conditionality network comprises a decision tree.
  • said decision tree is controllable to use back propagation of said measurement outputs to select a next feature for measurement.
  • said feature measurer comprises a 4f optical correlator, connected to a Fourier transform unit, for performing correlation on said object against a filter representing said feature.
  • said feature measurer comprises a 2f optical correlator, connected to a Fourier transform unit, for performing correlation on said object against a filter representing said feature.
  • said feature measurer comprises a JTC, connected to a
  • said feature measurer comprises an electronic correlator, connected to a Fourier transform unit, for performing correlation on said object against a filter representing said feature.
  • said feature measurer further comprises a Fourier transform unit connected to said correlator, which may be an optical correlator, for forming a Fourier transform of said object.
  • a Fourier transform unit connected to said correlator, which may be an optical correlator, for forming a Fourier transform of said object.
  • said Bayesian network comprises an inference engine.
  • said inference engine comprises electronic circuitry.
  • said inference engine is connected to said correlator, and comprises parameter control functionality to control the features useful for classification to be measured by the optical correlator.
  • said electronic circuitry further comprises a classifier to conduct classification computations on the output of said correlator, thereby to perform image classification.
  • said electronic circuitry comprises digital electronic circuitry.
  • said inference engine comprises a Bayesian network.
  • said inference engine comprises a feature selector for
  • a preferred embodiment further comprises a feature register for holding a plurality of features for selection.
  • said plurality of features comprising any of the following:
  • features for detection may include any feature of an object that can be measured using the techniques described herein.
  • the above described feature of rotational symmetry includes any rotational symmetry having an order number (that is the number of lines of symmetry) between 2 and infinity.
  • rotational symmetries having order numbers of 2, 3, 4, 6, 10, and infinity respectively are especially favored.
  • the feature selector comprises a classification probability estimator for estimating a probability of successful classification of said object given recognition of a given feature, said feature selector being operable to use
  • the optical feature extractor comprises a first imaging device
  • a preferred embodiment further comprises a duplicator connected between the output of the Fourier transform unit and the input of the optical correlator, said duplicator being controllable from said inference engine, said duplicator being operable to generate a spatial filter for use in correlation of said image.
  • said duplicator is operable to use said Fourier transform output as an input to said generation of said filter.
  • a preferred embodiment receives an instruction from said inference engine to measure a given symmetric property and generates said filter by duplicating said Fourier transform, thereby to form a filter having said given symmetric property.
  • said duplicator is operable to generate an arbitrary filter.
  • said arbitrary filter is selectable by said inference engine from any of a low pass filter, a selection of band pass filters and a high pass filter.
  • said arbitrary filter is selectable from a low pass filter, a selection of band pass filters, a high pass filter, circular sine filters, circular cosine filters, sine filters and circular sine filters.
  • the prestored set of filters comprises at least
  • duplicator is operable to generate a filter in accordance with input from said inference engine.
  • said duplicator is connected to display said spatial filter in said correlator.
  • a preferred embodiment further comprises a second imaging device at the output of said correlator, for forming an image of a response of said feature to said spatial filter.
  • a preferred embodiment further comprises an evaluator connected to said second imaging device for evaluating said imaged response and ascribing a value thereto, said value being transferable to said inference engine.
  • said features are object dependent features.
  • a preferred embodiment further includes object-dependent features which are any of geometric features, relational features and frequency features.
  • said geometric features are any ones of a group comprising features containing symmetry and features containing internal primitive features.
  • a preferred embodiment further comprises the ability to identify an internal primitive feature within a feature by setting said duplicator to generate a spatial filter of a suspected internal feature and measuring a correlation of
  • the feature extractor further comprises a spatial angle adjustment unit for rotating said Fourier transform to produce a plurality of spatial correlations with said filter.
  • a preferred embodiment is operable to identify an internal primitive feature within a feature by setting said duplicator to select from a prestored set of spatial filters at least one spatial filter of an internal feature to be tested and measuring a correlation of said feature with said spatial filter.
  • said feature extractor further comprises a spatial angle adjustment unit for rotating said Fourier transform to produce a plurality of spatial correlations with said filter.
  • a preferred embodiment is operable to identify a geometric feature by setting said duplicator to generate a spatial filter of a synthetic image related to said geometric feature and measuring a correlation of said geometric feature with said spatial filter.
  • a further preferred embodiment is operable to test said geometric feature for rotational symmetry by setting said duplicator to form said spatial filter by taking a rotational segment of said Fourier transform and duplicating said rotational segment to synthetically complete said transform.
  • a preferred embodiment is operable to test said geometric feature for axial symmetry by setting said duplicator to form said spatial filter by taking an axial segment of said Fourier transform and duplicating said axial segment to
  • a preferred embodiment is operable to identify frequency features of an object by setting said duplicator to produce a high pass filter and a low pass filter, each for respective intensity measurements of said feature, and calculating a ratio between said high frequency and said low frequency measurements.
  • said duplicator is operable to produce at least one of a high pass, a low pass and a band pass filter, and to output said filter to said correlator for correlation with said object, said correlator further comprising an evaluator for comparing said correlated output with an uncorrelated output to ascribe a value to said correlation.
  • a preferred embodiment is operable to identify a relational feature by separately identifying two features, determining a spatial relationship therebetween and applying a best fit relational description to said determined spatial relationship.
  • a preferred embodiment comprises a picture dependence analyzer for determining object features that are specific to a current orientation of an input camera to said object, thereby to screen out said object features.
  • said inference engine is operable to control said duplicator to
  • said inference engine comprising an analyzer for using said series of correlations to determine a most likely object.
  • said inference engine uses a Bayesian classification to select said series of filters to generate for correlating with each given object.
  • the Bayesian classifier is constructed to divide classification uncertainty of said objects into complementary sets such that one of said sets can be substantially ruled out after each correlation.
  • said optical processor comprises a spatial light modulator electronically controllable to produce filters.
  • an optical feature extractor comprising an input for receiving a visual object, a part extractor for extracting a part of said visual object and building a filter from said part, and a correlator for correlating between said input visual object and said filter, thereby to determine the presence of a feature in said visual object.
  • said feature is symmetry and said part extractor is operable to duplicate said part at least once to construct said filter.
  • said feature is n th degree symmetry
  • said part is an n ft part of said object and said part extractor is operable to duplicate said part n times to
  • the part extractor is electronically controllable. According to a third aspect of the present invention there is provided an
  • optical feature classification decision mechanism comprising:
  • a Bayesian network linking a first layer of nodes and a final layer of classes via probability links, at least said first layer of nodes representing measurable features of visual objects and said classes representing potential classification groups of said visual objects, said probability links comprising probability numbers representing potential classifications given feature measurement results,
  • said mechanism being operable to use said probabilities to guide a visual object measurement and classification process.
  • a preferred embodiment is operable to receive a result of a first measurement and to use said probabilities to determine whether to request a second measurement or to output a classification decision.
  • a thresholder for comparing said calculated possibility with a predetermined threshold, thereby to decide whether to output said classification decision or whether to request said further measurement.
  • Fig. 1 is a simplified schematic diagram of a prior art 4F correlator
  • Fig. 2 is a simplified diagram of a prior art joint transform correlator (JTC),
  • Fig. 3 is a simplified schematic diagram of a 2F correlator
  • Fig. 4 is a simplified diagram of a decision tree useful for classification between five classes
  • Fig. 5 is simplified diagram of a perceptron of a neural network
  • Fig. 6 is an overall block diagram of a classifier according to a first
  • Fig. 7 is a simplified block diagram of the optical feature extractor of Fig. 6,
  • Fig. 8 is a schematic diagram of the optical feature extractor of Fig. 7,
  • Figs. 9 - 11 are diagrams showing the construction of a simplified Bayesian network
  • Fig. 12 shows a set of five objects useful for training and testing embodiments of the present invention
  • Figs. 13-15 are graphs showing experimental results obtained with an embodiment of the present invention and showing a number of features that needed to be measured before convergence on the result,
  • Fig. 16 is a schematic representation of a multiple layer Bayesian network for use in embodiments of the present invention.
  • Fig. 6 is a simplified block diagram
  • a hybrid classifier 60 receives as an input an object to be classified 62.
  • the object 62 is preferably the output of a segmentation process
  • the hybrid classifier itself comprises two parts, an optical feature extractor 64 which comprises optical components arranged as will be described in more detail below and
  • the hybrid classifier 60 further comprises a digital inference engine 66 which is a digital processing device able to receive outputs from the optical feature extractor regarding features present or absent and use the received input to do two things, firstly to decide on a next feature to measure for optimal convergence on a decision, and secondly to reach a decision as to the classification of the object 62.
  • the inference engine 66 selects features according to estimated probabilities for reaching a rapid decision from the current measurement given the previous measurements.
  • a classification output 68 is produced indicating a class of objects to which the present object 62 has been inferred to belong.
  • the optical feature extractor 64 preferably comprises a Fourier transform system 70 for optically carrying out a Fourier transform on an input object, which transform is detected by a first CCD 72.
  • the optical feature extractor 64 furthermore
  • the optical correlator 74 comprises an optical correlator 74, whose structure will be discussed in greater detail below.
  • the optical feature extractor is able to produce a Fourier transform of an input object in the Fourier transform system 70 and is able within the correlator to produce a correlation result of the input object with any
  • the filter that may be used as an input into the correlator.
  • sources for such filters include the Fourier transform system and the inference engine.
  • the correlation output is processed by computer 78, either to make a classification decision or to choose a new feature to measure.
  • Fig. 8 is a simplified schematic diagram showing the optical processor of Fig. 7 in greater detail. Parts that are identical to those shown above are given the same reference numerals and are not referred to again except as necessary for an understanding of the present embodiment.
  • An input signal bearing a pattern or object to be classified firstly arrives at a beamsplitter 80, which splits the input signal between the Fourier transform system 70 and the correlator 74.
  • the Fourier transform system comprises a lens 82 and a Fourier plane 84.
  • a lens 86 and mirror 88 direct the signal to the correlator 74, which comprises a spatial light modulator (SLM)
  • SLM spatial light modulator
  • the SLM 90 is electronically controllable to hold a filter.
  • the SLM 90 is followed by a system of two lenses 92 and 94, the second being placed in what is know as the correlation plane.
  • the duplicator is preferably an electronic device for modifying a
  • the duplicator is able to generate any desired filter under electronic control. Specifically, the duplicator is able to take the Fourier analysis output of part of the input signal and duplicate it to form a filter covering a whole signal comprising symmetry.
  • SLM 90 may then perform a correlation between the real image and the synthetic image as defined by the filter, thereby to determine whether the object represented by the signal is symmetrical or not. As will be described in more
  • symmetry there are numerous types of symmetry that can be tested, and the presence or absence of one or more of such types of symmetry can be used to classify the object. Aside from symmetry there are other features that may be considered for testing, as will be described below and such filters are also preferably provided via the duplicator, although generally without reference to the generated Fourier transform.
  • a feature is an atomic piece of information that is used in order to describe the state/object.
  • optical features There are different types of optical features that could be used for the identification of the objects. These features may be categorized as follows:
  • Object dependent object dependent features depend solely on the object to be classified, and are not effected by the surroundings, nor by the manner the object was photographed. In the embodiments herein, treatment is mainly confined to object-dependent features, however this is by no means intended as a restriction on the invention, which extends to any feature that is optically measurable.
  • Object dependent features may be considered to include geometric features, relation features, and frequency features, although frequency features may be considered to be affected by overall scaling.
  • Surrounding dependent Surrounding dependent features depend on the surroundings and the background of the object, for example a ship is presented against a blue background which is the sea. 3. Picture dependent - The picture dependent features are features
  • Geometric features are the basic features that describe an object.
  • the Geometric features to be used throughout this work include:
  • I[x, y) is the intensity (gray level) of the pattern.
  • the objects contain a pattern of a primitive object, for example, within the object we can locate a circle, or within the
  • the present embodiments preferably use different primitives in order to describe as wide a range of objects as possible.
  • the first three types of features ⁇ Is symmetric in "Y”, Is symmetric in "X”, Has rotational symmetry of order k ⁇ may be extracted by measuring the correlation between the original image and a synthetic image reproduced from the original image. For example we can produce a k order symmetric Fourier transform by duplicating a section of the original Fourier transform, then by measuring the correlation between the original image and the synthetic transform we can infer about the symmetry features of the object.
  • the forth type of feature Contains a primitive is measured by measuring the correlation between the original image and an image of the primitive. For example we can produce via the Duplicator the Fourier transform of a circle and then measure the correlation. Considering the correlation peaks and strength, it is possible to infer whether there is a circle inside the original
  • the extraction of the above features is preferably done by measuring the correlation of the image with that of a synthetic image produced by the Duplicator.
  • the Duplicator creates a synthetic Fourier transform of an object given some part of the real object's image for example
  • the frequency features represent information about the ratio between the object's energy in a specified range of frequencies. We can define the following:
  • a pattern has sufficient energy in the frequency range [p p 2 ] , if the
  • Th is some arbitrary threshold
  • 3 p ⁇ is the Fourier transform
  • the pattern is mainly low-pass. In this case we state that most of the pattern's energy is concentrated in low frequencies. That is to say
  • p LP is the highest frequency that is still considered as a low
  • Th is some arbitrary threshold
  • 3 pjp is the Fourier transfor ⁇ i
  • the pattern is mainly high-pass. In this case we state that most of the pattern's energy is concentrated in high frequencies. That is to say
  • p HP is the lowest frequency that is still considered as a high
  • the measurement of the features is done by measuring the band-pass and whole energy of the object's image and computing the ratio between them. This is done be creating a low/high pass filter via the Duplicator and measuring the response of the image.
  • Relational features refer to the relation between primitives that are present in the image.
  • An example for such primitives could be: ⁇ "Is there a triangle above a square?”, “Are there at least two triangles in the image?”, "A circle , a triangle and a rectangle should form a 60-60-60 triangle", etc. ⁇
  • the measurement of these features is a very complicated task, since such a "vocabulary" of relations is very large. That is to say we need to create a complex language in order to describe objects using such features. Furthermore, the process of feature extraction becomes more complicated.
  • h Contains an ellipse - Different types of ellipses may be tested where the difference used in testing may be the eccentricity of the ellipse.
  • Arbitrary Spatial Filters - a set of predetermined spatial filters may be made available to the system so that the system may check the response of the object to filters that do not depend on the object.
  • Polarization Sometimes the polarization of the light arriving from an object or its background may carry characteristic information. Polarization features are not discussed explicitly in the present embodiments except to say that they may be taken as additional cues for the inference engine
  • Feature measurement requires an object to be input to the optical feature extractor, which object is now measured for the presence of one or the other of the features that are being considered.
  • an image of the object may be
  • the obj ect' s Fourier transform may be captured using the Fourier transform system. 2. Via the duplicator a section of a degrees of the picture may be
  • the synthetic Fourier transform may be displayed on the SLM.
  • Axial symmetry features may be measured in a similar manner to the Rotation symmetry features. Via the Duplicator one may create a synthetic Fourier transform of an object possessing the requested symmetry and measure the correlation between the original object and the symmetric object.
  • the object Fourier transform may be captured using the Fourier transform system.
  • a synthetic Fourier transform is preferably created via the duplicator.
  • the synthetic transform possesses the symmetry around the requested axis X, or Y respectively.
  • the synthetic Fourier transform is then preferably displayed on the SLM.
  • the existence of the axis symmetry feature may be decided according to the correlation value.
  • the containment features are features that indicate whether a primitive object is contained within the object to be identified.
  • the Fourier transform is a Linear Operator, therefore:
  • the containment features may be measured as follows:
  • a set of primitive objects and corresponding Fourier transforms are preferably set up.
  • the Fourier transform is preferably rotated several times in order to allow for some tilt and rotation of the primitive within the original image.
  • the synthetic Fourier transform of the primitive is sent, using the duplicator, to the SLM.
  • the existence or otherwise of the primitive in the original image is then decided according to the correlation value.
  • Frequency features depend on the amount of energy the object has in low frequencies v. the energy in possesses in high frequencies.
  • Frequency features may be measured in the following manner: Using the duplicator, it is possible to create High pass, Low pass, and Band pass, spatial filters.
  • the spatial filters are preferably displayed on the SLM.
  • the energy of the reconstructed filtered object is measured and compared to the energy of the unfiltered object.
  • the existence of the requested frequency feature will be determined according to the ratio between the correlation value of the object with itself and the BP-filtered object.
  • the measurement of arbitrary features may be carried out by displaying some arbitrary (predefined) spatial filter on the SLM and measuring the response of the object to the filter.
  • the arbitrary features may be measured in the following manner:
  • Arbitrary spatial filters are created, preferably using the duplicator.
  • the spatial filters are presented on the SLM.
  • the energy of the reconstructed filtered object is measured and compared to the energy of the unfiltered object.
  • the outputs of the correlations need not be binary results. Rather numerical values may be ascribed to correlations and feature recognition can be probabilistic.
  • the algorithm preferably makes use of a Bayesian network that is constructed automatically from a training set
  • the construction of the Bayesian network via the training set that is presented to the specific optical system provides the algorithm with the ability to be adaptive to both the classes that have to be classified, and to the specific optical system. This allows the algorithm to provide a robust classification solution to a wide variety of classification problems using optical systems whose fidelity might vary.
  • the algorithm implemented in this work preferably comprises the following steps:
  • Bayesian networks are a tool that enables a representation of the relationship between cause and effect, thereby to use such a relationship in a reasoning process.
  • a short discussion about Causes, Effects, and the manner in which they are presented by a Bayesian network is provided.
  • Bayesian network Another part of the Bayesian network is the conditional probability associated between the causes and the effects.
  • Bayesian network specifies each of the observed features and via the relationships represented by the network one may calculate the probability of each of the causes. Such a calculation is performed using Bayes' theorem.
  • a Bayesian network may be the wrong tool for solving the problem. It is well known that not all problems are Bayesian in nature. If one tries to apply a Bayesian solution to a non-Bayesian problem one will probably get a meaningless result. For example if one tries to solve the radar detection problem
  • Bayesian model incorporates both the causality and the mathematical aspects of the problem in a single reasoning model. Such may be achieved by using two mathematical tools, probability, which is the main mathematical tool which deals with uncertainty, and the Graph theory, or more precisely the directed acyclic graph (DAG), which is a tool with which causally related events may be represented.
  • DAG directed acyclic graph
  • Security Company decides to build an expert system that will help decide whether a break-in is in progress.
  • the observer is a watch officer who listens to the alarm. He produces the input that is his belief that he has heard the alarm. Given this input the system has to decide whether a break-in is in progress.
  • the graph is a directed acyclic graph (DAG).
  • DAG directed acyclic graph
  • Fig. 11 is a simplified diagram of a two-level Bayesian network, in which a series of features, such as the features that may be measured in the image classification process, are associated using probabilistic connections with a series of classes that are intended to classify the objects.
  • the present embodiments use a two level Bayesian network to represent the knowledge required for the classification process, although the invention is of course in no such way limited and any number of levels may be used.
  • Bayesian network as represented in Figure- 11, the first step in the construction of the classifier is to learn the conditional distribution of the features given the different classes of objects.
  • the learning of the conditional distribution is preferably achieved by presenting the hybrid classifier with a training set.
  • the training set preferably contains several different observations of objects from each class and the classifier measures each of the features that can be used in the classification process. That is, for each observation of an object the classifier measures all the features composing the Generic Classification Language.
  • the classifier preprocess the knowledge acquired from the training set in order to compute the following:
  • the a-priori probability of the classes is a parameter that is used in the initializing phase of the Bayesian classification process. Basically, this parameter influences the following:
  • the training set is selected from the entire set of objects in a manner that the probabilities of the occurrence of any given class remain unchanged.
  • the initial probability of occurrence of a class as long as it is not zero or unity need not effect the final classification result, but rather simply require further features to be measured, thereby extending the time that is required for the classification to take place.
  • the a-priori probability of the classes is calculated by summing the
  • the a priori probability to encounter classy is given by:
  • N- is the number of elements in the training set.
  • One such parameter is the distribution of a feature -
  • the distribution is used in order to compute the probability of measuring a specific value for a feature. This computation is essential for the use of Bayes' theorem.
  • Another such parameter is the conditional distribution of a feature given any of the classes - This is important for both the computation of the probability of a feature given a hypothesis (an assumed class), and the computation of the information residing in a feature, i.e. its entropy.
  • a further parameter is the conditional entropy of a feature -
  • the conditional entropy of a feature is computed directly from the conditional distribution of a feature.
  • Yet another parameter is the entropy of a feature -
  • the entropy of a feature is computed directly from the distribution of a feature. Note that the entropy of a feature changes as the classification process is performed this is due to the fact that the probabilities of the classes change and therefore the probability distribution of the features also change.
  • a parameter which is saved by the algorithm is the conditional probability of the features given the classes. This parameter is saved in an N*M matrix where each element is a function representing the features distribution,
  • N is the number of classes while M is the number of features).
  • the conditional distribution of the features is represented by a distribution function.
  • the distribution function is preferably modeled by a Gaussian distribution convolved with a chain of Delta functions each representing a single occurrence of the feature, as follows:
  • the updated distribution function of the features may be computed as follows:
  • the entropy of a feature is computed from the features probability distribution function.
  • the entropy is defined by:
  • the value chosen is the value of the probability distribution function in the center of the bin.
  • Such a form of computation is actually a simple form of numerical integration, and may be regarded as sufficiently accurate for the classification process.
  • the internal entropy of a feature depends on the current probabilities of the classes.
  • the internal entropy parameter allows for the dynamic adaptation of the algorithm to the observed object.
  • the classifier is preferably dynamically adaptive in respect to the observed class. Thus it is neither desirable nor possible to a-priori define the sequence by which the features would be measured. Therefore, the decision on which feature to measure next, is preferably made in real-time during the classification procedure.
  • the feature that conveys the most information may be selected by taking the feature with the highest ratio between its external entropy and internal entropy. That is to say that we would like to find a feature that changes over the classes but is constant within each class.
  • the external entropy of a feature is:
  • the total internal entropy of a feature is calculated by
  • the current probability of the J a ⁇ class is Pi C ⁇ .
  • the a priori probability of the I th feature is P(f) ⁇
  • the classification may be considered a result, if this is not the case then classification is continued with another feature.
  • Fig. 13 shows attempts to classify a square, and it is seen that after three measurements the results are unambiguous.
  • Fig. 14 shows attempts to classify a triangle and, again, it is seen that three measurements are sufficient.
  • Fig. 15 shows attempts to classify a star of David, and again reliable classification is achieved after three measurements.
  • the classifier always began by measuring for the feature of "Has a 72° Rotational symmetry" and then proceeded to a new feature depending on the result.
  • the choice of measurement reflects the fact that the above-mentioned feature is the most discriminatory between the object classes considered.
  • the second feature to be measured may now depend on the class of the test object that is assumed by the classifier as a result of updating the posteriori probabilities given the first measurement.
  • the process continues until a classification decision is reached, which is to say that a classification is made with an error probability that is below a predetermined threshold.
  • the mean number of features that was required by the classifier in order to reach the correct classification of the objects in the above experiment was 2.3474.
  • Fig. 16 is a simplified diagram showing a Bayesian network for use with the present invention.
  • a series of measurable features are arranged in two layers with interconnecting probabilities therebetween.
  • the use of a second layer of features allows the Bayesian network to better represent interrelationship probabilities between the features and therefore to lead to faster and more reliable convergeance.
  • the above-described embodiments thus comprise a hybrid optical classifier that makes use of a Bayesian network inference engine and an optical feature extractor.
  • the correct usage of the Bayesian inference engine has been found to provide a robust classifier that can adapt to different types of optical systems even optical system with a very low fidelity.
  • optical system in the present embodiments is similar to the most basic 4F correlator introduced by VanderLugt in the early 1960s.
  • SLM that is fully controllable by the computer allows the system to display different spatial filters that are generated in real time.
  • SLM allows measuring of features related to the specific object presented. (Example of such features are the rotation symmetry features).
  • the embodiments provide the use of real-time designed filters to
  • the feature selection process used in this work recalculates both the internal and external entropy of the features prior to every decision, while most other algorithms only recalculate the external information. This recalculation provides for a higher degree of adaptability in the feature selection process yielding a faster conversion to the correct classification.
  • classification features used herein are not applicable only to the classifier herein described but are also applicable to other classifiers and other optical systems.
  • the generic nature of the classification features allows the development of further optical analyzing tools for object classification, without the need to specifically tailor the tools at the development stage to the specific classification task.
  • classifiers described herein in the preferred embodiments may be used as training devices for more general use of the recognition system.
  • the classification features are of generic application to image recognition, and training of the system allows for refining of the feature set.
  • the features in the feature set may thus be built up into a generic language for feature classification, individual shapes and other features

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un système d'analyse d'images permettant la classification d'objets d'une image, comprenant: un processeur optique et/ou électronique et/ou hybride, c'est-à-dire optoélectronique, pour mesurer des objets afin de déterminer si lesdits objets présentent des caractéristiques utiles pour leur classification, et un réseau à conditionnalité associé au processeur optique pour l'utilisation d'une conditionnalité afin de sélectionner lesdites caractéristiques utiles dans la classification interactive à l'aide de signaux de mesures de sortie dudit processeur optique, et ainsi classifier lesdits objets.
PCT/IL2001/000597 2000-06-29 2001-06-28 Systeme d'analyse d'images pour le traitement d'images Ceased WO2002005207A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001267813A AU2001267813A1 (en) 2000-06-29 2001-06-28 Classifier for image processing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US21512000P 2000-06-29 2000-06-29
US60/215,120 2000-06-29

Publications (2)

Publication Number Publication Date
WO2002005207A2 true WO2002005207A2 (fr) 2002-01-17
WO2002005207A3 WO2002005207A3 (fr) 2002-05-02

Family

ID=22801739

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2001/000597 Ceased WO2002005207A2 (fr) 2000-06-29 2001-06-28 Systeme d'analyse d'images pour le traitement d'images

Country Status (2)

Country Link
AU (1) AU2001267813A1 (fr)
WO (1) WO2002005207A2 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2533435A1 (fr) * 2011-06-06 2012-12-12 Alcatel Lucent Égalisation de gains en mode spatial
US8620078B1 (en) 2009-07-14 2013-12-31 Matrox Electronic Systems, Ltd. Determining a class associated with an image
CN112367923A (zh) * 2018-07-13 2021-02-12 古野电气株式会社 超声波拍摄装置、超声波拍摄系统、超声波拍摄方法、以及超声波拍摄程序

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0500315B1 (fr) * 1991-02-18 1999-07-21 Sumitomo Cement Co. Ltd. Procédé de reconnaissance et classification optique de formes
US5835633A (en) * 1995-11-20 1998-11-10 International Business Machines Corporation Concurrent two-stage multi-network optical character recognition system

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8620078B1 (en) 2009-07-14 2013-12-31 Matrox Electronic Systems, Ltd. Determining a class associated with an image
US8873856B1 (en) 2009-07-14 2014-10-28 Matrox Electronic Systems, Ltd. Determining a class associated with an image
EP2533435A1 (fr) * 2011-06-06 2012-12-12 Alcatel Lucent Égalisation de gains en mode spatial
CN112367923A (zh) * 2018-07-13 2021-02-12 古野电气株式会社 超声波拍摄装置、超声波拍摄系统、超声波拍摄方法、以及超声波拍摄程序
US11948324B2 (en) 2018-07-13 2024-04-02 Furuno Electric Company Limited Ultrasound imaging device, ultrasound imaging system, ultrasound imaging method, and ultrasound imaging program

Also Published As

Publication number Publication date
AU2001267813A1 (en) 2002-01-21
WO2002005207A3 (fr) 2002-05-02

Similar Documents

Publication Publication Date Title
CA2536864C (fr) Dispositif de memoire associative et procede base sur la propagation d'onde
US7711157B2 (en) Artificial intelligence systems for identifying objects
Chiang et al. Hybrid fuzzy-neural systems in handwritten word recognition
Amanatiadis et al. Understanding deep convolutional networks through Gestalt theory
Voronov et al. Designing a neural network identification subsystem in the hardware-software complex of face recognition
Murino Structured neural networks for pattern recognition
Gupta et al. Non-linear alignment of neural net outputs for partial shape classification
Won Nonlinear correlation filter and morphology neural networks for image pattern and automatic target recognition
WO2002005207A2 (fr) Systeme d'analyse d'images pour le traitement d'images
Neelima Social ski driver-jaya optimization-enabled deep convolution neural network for signature verification
Sjahputera Object registration in scene matching based on spatial relationships
Troxel Position, Scale, and Rotation Invariant Target Recognition Using Range Imagery.
Nutter 3-D modelling and classification in automated target recognition
Schwartz et al. Machine learning of computer vision algorithms
Kaméus et al. Comparing Convolutional Neural Networks to traditional methods and the human eye for copy-move forgery detection
Tang et al. Invariant object recognition using a neural template classifier
Kassaw Visual Recognition Models for Scenes of Partial Object Occlusion
Han A New Method to Solve Same-Different Problems with Few-Shot Learning
Koch et al. Feature discovery in gray level imagery for one-class object recognition
Wagenknecht Topological Data Analysis of Weight Spaces in Convolutional Neural Networks
Marsic Selective visual perception for object recognition
Mittal Semi-supervised Learning for Real-world Object Recognition using Adversarial Autoencoders
Khotanzad et al. Neural network system for 3D object recognition and pose estimation from a single arbitrary 2D view
CN121058046A (zh) 自主代理中的视觉记忆
Wang et al. Development of gazing algorithms for tracking-oriented recognition

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

ENP Entry into the national phase

Country of ref document: RU

Kind code of ref document: A

Format of ref document f/p: F

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP