WO2004015630A2 - Procede et dispositif de perception visuelle active pour caracterisation et reconnaissance par analyse de parametres mono/multidimentionnels dans des unites multiclasses de calcul et traitement d'histogramme, recrutement dynamique d'unites - Google Patents
Procede et dispositif de perception visuelle active pour caracterisation et reconnaissance par analyse de parametres mono/multidimentionnels dans des unites multiclasses de calcul et traitement d'histogramme, recrutement dynamique d'unites Download PDFInfo
- Publication number
- WO2004015630A2 WO2004015630A2 PCT/FR2003/002478 FR0302478W WO2004015630A2 WO 2004015630 A2 WO2004015630 A2 WO 2004015630A2 FR 0302478 W FR0302478 W FR 0302478W WO 2004015630 A2 WO2004015630 A2 WO 2004015630A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- module
- signal
- barycenter
- memory
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/571—Depth or shape recovery from multiple images from focus
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Definitions
- the present invention relates to a method and a device for active visual perception for object characterization and recognition, in particular for identification and localization purposes, by analysis of mono and / or multidimensional parameters in possibly multiclass calculation units and histogram processing with dynamic recruitment of such units.
- the invention essentially implements, after a rapid phase of reduction in the spatial resolution of the perception of an object observed, a phase of progressive increase, in particular step by step, of this resolution.
- the progressive increase in spatial resolution between a minimum resolution (at the end of the resolution reduction phase and before the start of the resolution increase phase) and a maximum resolution (at the end of the resolution increase phase when the basic resolution has been found before beginning of the reduction phase)
- a minimum resolution at the end of the resolution reduction phase and before the start of the resolution increase phase
- a maximum resolution at the end of the resolution increase phase when the basic resolution has been found before beginning of the reduction phase
- a representation of these details in their hierarchical order of appearance during the increase in resolution for example in the form of a tree illustrating the father ⁇ son relationships between an area and a portion of this area, makes it possible to characterize an object in giving it a tag or label.
- the object is observed by a sensor, in particular a sensor, for example of the MOS or CCD type, debiting a digital signal of the television type, that is to say comprising a succession of frames or sequences which represent an image of the object in its environment at a given time, each of these frames or sequences being constituted by a succession of horizontal lines which each comprise a succession of image points or pixels.
- the modification, in particular the increase, of the resolution can be carried out either optically by acting on the optical part at the front of the sensor, that is to say before the transformation of an optical image into an electronic image. , (namely a frame), either electronically by means of a filter processing the digital signal output by the sensor (namely successive frames).
- the treatment is Gaussian or quasi-Gaussian.
- the invention implements three successive stages: • substantially Gaussian incremental increase in resolution,
- the third operation mentioned above implements modules, called STN (spatiotemporal neurons) which are the subject, for example, of the international publication WO-01/63557 and of the patent application FR-01/02539 having both the same inventor as the present invention.
- STN spatialotemporal neurons
- a one-dimensional (or unilinear) module to be processed and a module with a two-dimensional (or linear) input parameter with two input parameters (for example two coordinates, Cartesian x and y or polar p and ⁇ ).
- two-dimensional (or linear) input parameter with two input parameters (for example two coordinates, Cartesian x and y or polar p and ⁇ ).
- three-dimensional or trilinear STN modules or three input parameters for example the three color components.
- such STN modules operate, on the input parameter or parameters, a logical operation represented by a function f 0 g debiting an output signal available at D using one or more registers (reg) under the control of an API command.
- the mono and multidimensional parameters processed by the STN modules carry characteristics of objects from a space-time space and are represented by sequences and sub-sequences of data evolving over time.
- the invention can in particular be applied to the analysis of images in the form of video data for the purposes for identifying and locating objects on a scene.
- the analysis allows the identification of an object by its shape and / or its dimension and / or its orientation and / or its respective position in relation to the scene and / or to other objects of the scene.
- An application of the invention to active visual perception is more particularly detailed in the present document.
- the invention can also be implemented with other types of data which can be represented in the form of sequences and sub-sequences of data such as, for example, sounds.
- each of these histogram calculation and processing units constitutes and updates a register analysis output providing statistical information on the corresponding parameter.
- the choice of the parameter analyzed by each calculation block and histogram processing, the content of the analysis output register as well as the function (f 0 g) that it fulfills, are determined by software executed in a programming interface. application API (Application Program Interface).
- the maximum RMAX of the histogram is determined from the histogram calculated and stored in a memory. position of said maximum POSRMAX, a number of NBPTS points in the histogram. Classification limits are also determined which make it possible to delimit an area of interest for the parameter and it has been proposed to take as a criterion for determining the limits, a ratio of the maximum of the histogram, for example RMAX / 2, and d '' get the limits by scanning the memory data from the origin in search of the zone limits corresponding to the criterion.
- An application of the STN blocks is more particularly detailed in application FR-01/02539 (cited above) where it has been proposed to hierarchically decompose the object to be identified according to its properties, which makes it possible, for example, to determine first the general outline of a moving object in relation to a relatively stable background, then search within this outline for characteristic elements by their tint, their color, their relative position ...
- Such an approach allows the development rapid multiple applications involving the location of an object.
- These applications can be developed, either from an earlier formalization that has identified the significant characteristics of the object, or, thanks to a learning function by examining a scene in which the object in question is present. , the device allowing him- even to extract characteristic parameters of the object.
- an area of interest in space is identified as a function of a statistical criterion applied to a temporal parameter, b) the main area thus identified is inhibited, c) it is repeated steps a) and b) so as to identify other zones of interest within an area of uninhibited space, d) the process is stopped when a remaining area, uninhibited, of the space no longer produces an area of interest corresponding to the statistical criterion, in other words, when, in the area of interest, the number of points is too low (less than a threshold), e) we increment by valid frame consecutive, a counter for each area of interest thus identified, the barycenter of its point cloud, f) the barycenter of its point cloud is recovered for each area of interest thus identified.
- the advantage has also been shown of associating with this device a third subset receiving a signal carrying a second time parameter, this third subset having an operation similar to the first and replacing it when it validates points in space whose number is n 2 , n being greater than nj.
- One of the aims of the invention is therefore to propose an improved method and device implementing histogram calculation and processing blocks which allow greater efficiency and more flexibility in histogram analysis.
- the invention firstly relates to a method of perceiving an object in its environment, consisting in deducing therefrom a representative digital input signal, constituted by a succession of sequences representative of successive views of the object. in its environment and therefore falling within the time domain, each of said sequences being constituted by a succession of sub-sequences, each representative of locations arranged one after the other in said sequences and therefore falling within the spatial domain, and characterized in that, over a period of several sequences, a temporal variation of the spatial resolution of the object is carried out in said digital input signal, comprising a phase of substantially Gaussian increase in resolution from a value reduced to at an optimal base value, differentiation is carried out, with smoothing, ejiter two successive sequences of said substantially Gaussian increase in resolution, in order to obtain a derived digital signal representative of the variability of the difference of Gaussians between these two sequences when the difference, in absolute value, for each same spatial location of said derived signal, exceeds a threshold, and from said derivative signal is
- said digital input signal is a video signal coming from a video sensor, said sequences being constituted by the successive images of the video signal, said subsequences being constituted by the lines of the video signal and the locations being constituted by the pixel locations in the video signal, and in that the characteristics of the information conveyed by said digital signal relate to an object present in a scene observed by said sensor.
- This process also has other characteristics, possibly combined, which are: - that said substantially Gaussian increase in spatial resolution is produced in stages during steps of increase between two successive sequences, without modification of resolution during the course of the sequence, - that the variation of the spatial resolution includes a preliminary phase of abrupt reduction of the spatial resolution, from its basic value to its reduced value, before said phase of substantially Gaussian increase, ' and in that said phase of increase, from its reduced value to its basic value, is relatively slow, - that the variation in spatial resolution includes a substantially constant resolution phase between the abrupt decrease phase and the slow increase in resolution phase, - that said substantially Gaussian increase in spatial resolution is achieved by electronic filtering of the video signal input charged by said sensor,
- said sensor is provided with a variable focal length objective and that the variation of the resolution is effected by acting on the focal distance of said objective
- At least one lens of said objective is moved in order to modify the focal length thereof, during the steps of substantially Gaussian increase in resolution, the movement of said lens being carried out at constant speed between two bearings,
- shape of at least one lens of said objective is modified by an electric control in order to modify the focal length thereof at constant speed during the steps of substantially Gaussian increase in resolution between two levels,
- a binary signal PD the two values of which respectively represent the exceeding and not exceeding of a threshold by the difference, in absolute value, of the resolution for each same spatial location in the sequences, between two consecutive sequences
- a digital signal CO with a low number of bits, constituting said derived digital signal and representing an adaptive time constant reinjected into said processing in order to reduce the variation, between two consecutive sequences and for the same location in the sequences, of the processed signal of difference in resolution, the successive values of CO, when DP has the value representative of an overstepping of the threshold, being representative, as a function of the successive sequences, of the increase in resolution
- the active perception method to perform not only the perception, but also the recognition and the localization of an object in its environment, we deduce from said derived signal, details less and less characteristic of the object. by forming at least two histograms of said derived digital signal, at least one of which relates to the digital magnitude of said signal in the various locations, which provides a information relating to the characteristic details of the object, and at least one of which relating to the location of the locations in said signal, which provides information relating to the location of said characteristic details.
- This previous method of active perception also has other characteristics, possibly combined, which are:
- a classification processing consisting in producing at least one pair of histograms classifying these values, when DP has the value corresponding to a crossing of the threshold, one by frequency of appearance of different CO values and the other by localization of CO,
- a new zone is divided into several angular sectors centered on the barycenter of the previously determined zone, that one searches in which sector, among said sectors, is the barycenter of said new zone, that one divides said sector in several sub-sectors, which one searches in which sub-sectors, among said sectors, is the barycenter of this new zone and that one defines the polar coordinates of this latter barycenter, relative to the barycenter of the zone determined previously , from the angle characterizing the sub-area containing the new barycenter and the distance between the two barycenters,
- the label of the new object is determined by the preceding means and the label of this new object is compared with the labels previously stored
- the invention also relates to a device for perceiving an object in its environment, comprising means for deducing therefrom a representative digital input signal, constituted by a succession of sequences representative of successive views of the object in its environment and therefore falling within the time domain, each of said sequences consisting of a succession of sub-sequences, each representative of locations arranged one after the other in said sequences and therefore falling within the spatial domain, and characterized by which it further comprises - means for carrying out, over a period of several temporal sequences, a temporal variation of the spatial resolution of the object in said digital input signal, comprising a phase of substantially Gaussian increase in resolution from a reduced value to an optimal base value, means for differentiating, with smoothing between two successive sequences of said substantially Gaussian increase in resolution, in order to obtain a derived digital signal representative of the variability of the difference of Gaussians between successive sequences, when the difference, in absolute value, for each same spatial location of said derived signal , exceeds a threshold, and - means
- said means for deducing an digital input signal are constituted by a video sensor observing an object present in a scene to deduce therefrom a video signal constituting said digital input signal, said sequences being constituted by successive images of the signal video and the locations being constituted by the pixel replacements in the video signal.
- This active perception device also has other characteristics, possibly combined, which are:
- said means for achieving a substantially Gaussian increase in spatial resolution produce the increase thereof in stages during increments between two groups of successive sequences, without modification of resolution during the sequence
- the device comprises means for moving at least one lens of said objective in order to modify the focal length thereof during the steps of substantially Gaussian increase in resolution, the displacement of said lens being effected at constant speed between two bearings, - that the device comprises electrical means for modifying the shape at least one lens of said objective in order to modify the focal length thereof at constant speed during the steps of substantially Gaussian increase in the resolution between two levels, - that said means for differentiating the substantially Gaussian increase are constituted by means, known in themselves, for performing recursive temporal and spatial processing for r deduct, from the substantially Gaussian variation of the resolution, - on the one hand, a binary signal DP, the two values of which respectively represent the exceeding and non-exceeding of a threshold by the difference, in absolute value, of the resolution for each same spatial location in the sequences, between two consecutive sequences, and
- a digital signal CO with a low number of bits, constituting said derived digital signal and representing an adaptive time constant reinjected into said processing in order to reduce the variation, between two consecutive sequences and for a same location in the sub-sequences, of the processed signal of difference in resolution, the successive values of CO, when DP has the value representative of exceeding the threshold, being representative, as a function of the successive sequences, of the increase in the resolution,
- the device comprises means for forming, during said preliminary phase of abrupt reduction in spatial resolution, the histogram of the absolute values of the differences between the sequence at basic resolution and the sequence at subsequent reduced resolution, means for choosing a terminal L in said histogram such that, for said absolute values below the threshold, the number of points in the histogram comprises a fraction of at least 75% of the total number of points in the histogram, means for choose, for said phase of increasing the spatial resolution in stages, a number of increment steps p not greater than the lower integer value by default of the L / S ratio, S being the threshold of the detection sensitivity, and means for imposing on the digital signal CO a zero value at the end of the decreasing phase and the value p at the beginning of the phase of increasing the resolution.
- the active perception device to allow not only the perception, but also the recognition and the localization of an object in its environment, furthermore comprises means for deducing, from said derived signal, details less and less characteristic of the object by forming at least two histograms of said derived digital signal, at least one of which relates to said signal in the various locations, which provides information relating to the characteristics of the object, and at least one of which relates to the location of locations in said signal, which provides information relating to the location of said characteristic details.
- This previous active perception device also has other characteristics, possibly combined, which are:
- the device comprises means for subjecting the successive values of the signal CO to a classification processing consisting in producing at least one pair of histograms classifying these values, when DP has the value corresponding to a threshold being exceeded, one by frequency appearance of the different CO values and the other by CO localization,
- the device comprises means for associating a dominant color with each characteristic of the zone and means for defining each successive zone both by its coordinates and its dominant color,
- the device comprises means for determining successive zones corresponding to details less and less characteristic of the object, as well as the barycenter of each of these zones, and means for defining, from the order of appearance of these zones, an analysis tree which has a common origin corresponding to the barycenter of the initial zone at the initial reduced resolution, from which branches branch out to points corresponding to the barycenters of the zones of increased resolutions and this up to the basic resolution,
- the device also comprises means for performing an axis rotation operation, in the reference plane, between the axes x, y before rotation and the axes X, Y after rotation, according to the matrix formula
- the device comprises means for dividing a new area into several angular sectors centered on the barycenter of the previously determined area, means for searching in which sector, among said sectors, is the barycenter of said new area, means for dividing said sector into several sub-sectors, means for finding in which sub-zone, among said zones, is the barycenter of this new zone and means for defining the polar coordinates of this latter barycenter, relative to the barycenter of the previously determined area, from the angle characterizing the sub-area containing the new barycenter and the distance between the two barycenters,
- the device comprises means for determining the polar coordinates of several barycenters of zones appearing successively with respect to the initial zone representing the object at the time of the minimum resolution and means for storing these polar coordinates in the form of a label representative of the object,
- the device comprises means for determining the distance D of the object or sensor, means for calculating the logarithm LD of this distance D, means for carrying out a sectoring, followed by a sub-sectoring, LD and p coordinates calculated according to the preceding means and means for determining the sub-area in which all the LD values measured for the same object are found and in which the transformed value of p, namely Cp 'is constant.
- the invention also relates to a method of operating a histogram calculation and processing module, called STN, for analysis of a parameter of a space-time space represented in the form of data in sequences and in sequences evolving in time, the parameter being carried by a digital DATA (A) input signal in the form of a sequence Ajj ... t, A'jj ... t, A "jj ... t, ... of binary numbers associated with synchronization signals making it possible to define a moment t of the space and a position i, j ...
- STN histogram calculation and processing module
- said parameter being processed in the STN module by a function in order to produce an output value, the output values of a set of modules forming a retroannotation available on a retroannotation bus, the calculation involving retroannotation, said module having means for calculating in an addressable memory a histogram representative of the parameter, means for processing said histogram and prod Read at least the following values stored in module registers:
- a pair of classification terminals determined as a function of a criterion applied to the histogram by scanning the memory and detection of said criterion.
- a decreasing address scan of the addressable memory is carried out from the position of the maximum POSRMAX in order to to produce a first terminal of the pair of terminals as a function of the criterion and, on the other hand, a scanning of increasing addresses of the addressable memory from the position of the maximum POSRMAX in order to produce a second terminal of the pair of terminals depending on the criterion, said limits being thus determined by bilateral scanning of the addressable memory from the position of the maximum POSRMAX of the histogram.
- - the terminals are produced in a single address scan, a counting signal being alternately added and subtracted from / from the position of the maximum POSRMAX for each counting step,
- the terminals are produced in two address scans, first the first terminal then the second terminal, a count signal being subtracted from the position of the maximum POSRMAX during a first scan then a new count signal being added to the position POSRMAX maximum during a second scan,
- the terminals are produced in two address scans, first the second terminal then the first terminal, a count signal being added to the position of the maximum POSRMAX during a first scan then a new count signal being subtracted from the position POSRMAX maximum during a second scan,
- a selection means is used to select the criterion as a function of at least one of the following values: - the maximum RMAX value,
- the criterion is selected from RMAX / 2, THRESHOLD, NBPTS / THRESHOLD,
- an adder / subtractor making it possible, according to the binary value of a SENS signal of scanning direction, to add or subtract an offset value to a COUNTER counting signal, the offset value possibly being at least either zero or equal to the POSRMAX position value of the maximum of the histogram, to generate the address signal of the addressable memory,
- a comparator and logic circuits receiving on the one hand a datum addressed from the addressable memory and, on the other hand, the criterion, and intended to generate a terminal update signal to allow validation in a register of one of the terminals in the scanning direction,
- a selection multiplexer arranged at an address input of the addressable memory, said multiplexer having three inputs receiving respectively an input signal carrying the parameter, the counting signal COUNTER and the output of adder / subtractor,
- - means are implemented in the STN module for anticipation with calculation of a mean POSMOY position relative to said parameter and calculation of a differential ⁇ A of the parameter by difference between two successive averages of said parameter and in that the parameter is subtracted from the differential signed before implementation in an operating sub-unit intended to produce the output signal for retro-annotation as a function of the determined pair of terminals, a barycentric unit intended * to produce a barycenter output signal for retroannotation with a first binary state when the parameter corresponds to the mean POSMOY position and with a second binary state in the opposite case is used,
- the parameter analyzed by the STN module is complex and that it is obtained by combination of at least two elementary parameters, each of the binary numbers of the input signal DATA (A ⁇ , A 2 , A 3 ... A p ) support of the complex parameter A ⁇ A 2 A 3 ... A p comprising P fields each corresponding to an elementary parameter Ai, A 2 , A 3 , ..., A p and in that one implements means in the STN module for producing and storing in registers P pairs of terminals each corresponding to an elementary parameter by bilateral scanning of the memory from the position of the maximum POSRMAX of the histogram of the complex parameter,
- - P is greater than three, the module being said to be multilinear or multidimensional
- a shift unit is used, said unit making it possible to introduce into the counting signal COUNTER a digital shift making it possible to address in the memory data corresponding to a particular field of the complex parameter,
- - means are implemented for anticipation with calculation of a complex average position POSMOYAIA2 ... relating to the set of p elementary parameters of the complex parameter and calculation of differentials ⁇ Ai, ⁇ A 2 , ... ⁇ A P of each of the p elementary parameters of the complex parameter ⁇ 2 A 3 ... p by difference between two successive complex means and in that each of the elementary parameters is subtracted from the corresponding signed differential before being implemented in an operating subunit intended to produce the output signal for retroannotation as a function of the determined terminal pairs,
- a barycentric unit intended to produce a barycenter output signal for retroannotation with a first binary state is also implemented when the complex parameter A ⁇ A 2 A 3 ...
- a p corresponds to the complex mean position POSMOY A iA2 .. . and with a second binary state in the opposite case,
- an operating subunit which includes means allowing the choice of generating a classification space for the output signal by AND or OR combination of classification zones, respectively Z e t and Z or - each of the basic parameters,
- the existence or not of a class is stored in the form of binary data with two states in a classification memory whose number of words corresponds to the size of the histogram stored in the addressable memory of the STN module, the first state corresponding to the detection of the criterion and the second state to the non-detection of the criterion on the histogram, a class with values of first state corresponding to a zone of histogram comprised between the terminals and in that the signal is sent classification memory output on the feedback bus,
- the addressable memory of the STN module is initialized by a function calculation means during a given calculation cycle with initialization values as a function of the stored values in the addressable memory at the end of the previous calculation cycle
- a function is implemented for calculating an initialization value equal to (Km-1) / Km times the stored value, Km being of the form 2 m with m greater than or equal to zero, m greater than zero allowing obtain a memory effect and m equal to zero corresponding to an absence of memory effect,
- the method is implemented in a recognition system and the variation of m depends on a recognition criterion, the memory effect being reduced as the recognition proceeds,
- a set of class data is determined and stored for a given histogram, each class corresponding to a vertex of said histogram and said data comprising at least the amplitude and the position of said vertex, one of the classes corresponding to the maximum vertex RMAX, POSRMAX of the histogram,
- - We distribute means for determining and storing a set of class data between several STN modules, a first ST1 module determining and storing a first class corresponding to the maximum vertex of the histogram, and a second ST2 module determining and storing a second class corresponding to the second largest vertex of the histogram and so on ST3 ..., the output of the module from a higher vertex class inhibiting all of the following modules determining and memorizing the lower vertices of the histogram, - means are distributed making it possible to determine and store a set of class data between several STN modules, a first ST'O module determining and storing all the classes and sending class by class in descending order from top of histogram to a module of a set of modules ST'1, ST'2, ST'3 ...
- the means are distributed allowing to determine and store a set of class data between an STN module and an API application programming interface, said means comprising a memory MO of histogram values determined by the module, a memory M1 of addresses ordered in amplitude of vertices of the histogram, a memory M2 allowing to store the sequence number of the class, a memory M3 of thresholds of the classes and a register RC of number of classes, the API making it possible to perform: - a cycle of initialization resetting the memories
- the update cycle comprising:
- (C) - a validation of the classes of the memory M2 by comparison of the value of the memory MO at the address of the class considered of M2 with the threshold of M3 corresponding to the class considered,
- the memories MO and M1 correspond to the addressable memory of the STN module and the corresponding data are grouped together in a single functional table of amplitude and memory position pairs RMAX 0 / POSRMAX 0 ; RMAX *, / POSRMAX ⁇ ; RMAX 2 / POSRMAX 2 ... vertices of the histogram in descending order of amplitude, the functional table performing an automatic material sorting of the classes during the calculation stage, - we group in a multi-class functional unit within an STN module:
- said multi-class functional unit receiving at least one simple DATA (A) or complex DATA (A ⁇ ... A p ) parameter, a VALIDATION validation signal linear combination of retroannotation signals and sequencing signals INIT, CALCULATION, END, CLOCK, said multiclass functional unit returning at least one set of output signals each corresponding to a class Cl ⁇ ... Cl k on the feedback bus,
- each of the classes of the classification signal is produced in relation to a set of parameter values for which the amplitude of the histogram is greater than a threshold criterion
- the API is a programmable sequencer of the memory microprocessor or microcontroller type
- the method is implemented in an object recognition system comprising at least two multiclass modules, a first module operating in the time domain TD, determining at least one class and recruiting for said class at least a second module operating in the SD space domain,
- the method is implemented in an object recognition system comprising a set of STN modules for calculating and processing a histogram by sectoring, the modules determining zones and barycenters, and in that a zone is divided determined in several angular sectors centered on the corresponding barycenter of the zone and that one searches in which sector among the sectors, a new barycenter appears and that one divides said sector into several sub-sectors, the process being able to continue in order to continue to progressively refine the sectorization,
- At least one angle and one module are determined, the angle being given with reference to the line joining the two barycenters and the module corresponding to the distance along said line between said two barycenters,
- the method is implemented in at least one STN module with at least two orientation units p p ⁇ of input axes for rotation of reference axes of at least two Cartesian coordinates of input parameters, the module (s) further determining the barycenter for the input parameters, and in the system a first space Z ⁇ comprising a barycenter BarZi is determined by an association of a monolinear module processing a first parameter and a second bilinear module processing the coordinates, a second association determining inside said first space Z-, the appearance of a second barycenter BarZj + i, said first space being divided into distinct angular sectors regularly distributed Zrio , Zr ⁇ , Zri 2 ..., each sector being processed by a bilinear sector module receiving Zj, BarZj and also the signal from the second barycenter BarZi + i, the bilinear sector module corresponding to the second barycenter BarZj + i being set in relation to a set of bilinear sector modules of subsequent rank allowing to split the
- the object can be observed at different distances and that at least one size invariance unit is implemented in the system, said size invariance unit receiving at least one input, firstly, a value of the logarithm of a distance LD between a reference point and at least one point of the object and, secondly, the module p j of distance between two barycenters BarZj and BarZj + i, said unit determining at least one substantially constant projection value Cp 'corresponding to an angle ⁇ of rotation relative to the reference frame p and LD, - we calculate pj, ⁇ , C ⁇ ,
- the distance control means are implemented in the size invariance unit allowing the choice of the use of an external distance measurement or the internal determination of the distance,
- the object can be observed at different angles and at least one rotation correction unit is used in the system, said rotation correction unit making it possible to correct the angle value ⁇ d ' reference axis with respect to a pair of values ⁇ , p of angle and modulus previously determined,
- the rotation correction unit also makes it possible to determine the angle of rotation ⁇ ,
- an object recognition system comprising a set of STN modules for calculating and processing a histogram with at least one unit for transforming the reference frame of parameters by rotation of angle ⁇ , the reference frame being at least on two parameter dimensions, the rotation in the case of a two-dimensional coordinate system for parameters of polar coordinates of pixel X, Y, corresponding to the following matrix operation:
- the angle ⁇ or ⁇ is chosen so that at least the projection of one of the parameters on the corresponding axis p ⁇ p ⁇ has a reduced rate of variation after rotation
- the parameters are two in number and are chosen from pairs X, Y of pixel coordinates or LogD, p logarithm of distance of a pixel relative to a reference point and of angle relative to said point and to a reference line,
- the recognition data of the new object are determined and it is compared with recognition data of previously stored labels
- - recognition data from a first label are associated with those from at least a second label in order to form a new label corresponding to a higher level of recognition
- the labels are analyzed by a histogram calculation and processing module capable of determining and storing a set of data for categorizing said labels.
- the invention likewise relates to a device intended to operate according to one or more of the preceding functional characteristics relating to the method.
- This invention therefore relates to a device with a type of histogram calculation and processing module, called STN, for analysis of a parameter of a space-time space represented under form of data in sequences and in sequences evolving over time, the parameter being carried by a digital DATA (A) input signal in the form of a sequence Ajj ... t , A' ⁇ j ... t, A " i j ... t , ...
- said parameter being processed in the STN module by a function in order to produce an output value, the output values of a set of units forming a retroannotation available on a retroannotation bus, the calculation involving retroannotation, said module having means for calculating in an addressable memory a histogram representative of the parameter, means for processing said histogram and producing at least the following values stored in registers of the module:
- a pair of classification terminals determined as a function of a criterion applied to the histogram by scanning the memory and detection of said criterion.
- the device comprises means for processing the histogram and producing the terminal pair by, on the one hand, a scanning of addresses decreasing from the addressable memory from the position of the maximum POSRMAX in order to produce a first terminal of the pair of terminals according to the criterion and, on the other hand, an increasing address scan of the addressable memory from the position of the maximum POSRMAX in order to produce a second terminal of the pair of terminals according to the criterion, said limits being thus determined by bilateral scanning of the addressable memory from the position of the maximum POSRMAX of the histogram.
- the device invention can also be implemented with the following characteristics, possibly combined according to all technically feasible possibilities:
- the means for processing the histogram and producing the pair of terminals allow a determination according to one of the following methods:
- the device comprises a selection means making it possible to select the criterion as a function of at least one of the following values:
- the criterion is selected from RMAX / 2, THRESHOLD, NBPTS / THRESHOLD,
- the device comprises at least: - an adder / subtractor allowing according to the binary value of a SENS sweep direction signal, to add or subtract an offset value to a COUNTER count signal, the offset value can be either zero or equal to the POSRMAX position value of the maximum of the histogram, to generate the address signal from the addressable memory,
- a comparator and logic circuits receiving on the one hand a datum addressed from the addressable memory and, on the other hand, the criterion, and intended to generate a terminal update signal to allow validation in a register of one of the terminals in the scanning direction,
- the device comprises means for anticipation with calculation of a mean POSMOY position relative to said parameter and calculation of a differential ⁇ A of the parameter by difference between two successive averages of said parameter and in that the parameter is subtracted from the differential signed before implementation in a operating sub-unit intended to produce the output signal for retroannotation as a function of the determined pair of terminals, - the device further comprises a barycentric unit intended to produce an output signal of barycenter for retroannotation with a first binary state when the parameter corresponds to the average POSMOY position and with a seco nd binary state in the opposite case, - the parameter analyzed by the STN module is complex and that it is obtained by combination of at least two elementary parameters, each of the binary numbers of the input signal DATA (A ⁇ , A 2 , A 3 ...
- a p support of the complex parameter A ⁇ A A 3 ...
- a p comprising P fields each corresponding to an elementary parameter A * ⁇ , A 2 , A 3 A p and in that it comprises means in the module for producing and storing in registers P pairs of terminals each corresponding to an elementary parameter by bilateral scanning of the memory from the position of the maximum POSRMAX of the histogram of the complex parameter,
- the device comprises an offset unit, said unit making it possible to introduce into the counting signal COUNTER a digital offset making it possible to address in the memory data corresponding to a particular field of the complex parameter,
- the device includes means for anticipation with calculation of a complex mean position POSMOY AI A2 ... relating to all of the p elementary parameters of the complex parameter and calculation of differentials ⁇ A-i, ⁇ A 2 , ... ⁇ A P of each of the p elementary parameters of the complex parameter A ⁇ A 2 A 3 ... A p by difference between two successive complex means and in that each of the elementary parameters is subtracted from the corresponding signed differential before implementation in a subunit of operation intended to produce the output signal for retro-annotation as a function of the determined terminal pairs,
- the device comprises a sub-unit of operation with means allowing the choice of generating a classification space for the output signal by AND or OR combination of classification zones, respectively Z e t and Z or . of each of the elementary parameters,
- the device comprises a classification memory, the number of words of which corresponds to the size of the histogram stored in the addressable memory of the STN module, the memory storing the existence or not of a class in the form of binary data with two states , the first state corresponding to the detection of the criterion and the second state to the non-detection of the criterion on the histogram, a class with values of first state corresponding to a zone of histogram comprised between the terminals and in that the signal of the classification memory is sent to the retroannotation b.us,
- the device comprises a function calculation means making it possible to initialize the addressable memory of the STN module during a given calculation cycle with initialization values as a function of the values stored in the addressable memory at the end of the preceding calculation cycle ,
- Km the product initialization value (Km- 1) / Km times the stored value, Km being of the form 2 m with m greater than or equal to zero, m greater than zero making it possible to obtain a memory effect and m equal to zero corresponding to an absence of memory effect,
- the device is implemented in a recognition system and the variation of m depends on a recognition criterion, the memory effect being reduced as and recognition measure,
- the device comprises means for determining and storing for a given histogram a set of class data, each class corresponding to a vertex of said histogram and said data comprising at least the amplitude and the position of said vertex, one of the classes corresponding to the vertex maximum RMAX, POSRMAX of the histogram,
- the means making it possible to determine and store a set of class data are distributed between several STN modules, a first module ST1 determining and storing a first class corresponding to the maximum vertex of the histogram, and a second module ST2 determining and storing a second class corresponding to the second largest vertex of the histogram and so on ST3 ..., the output of the module from a higher vertex class inhibiting all of the following modules determining and memorizing the lower vertices of the histogram, - the means for determining and storing a
- “Set of class data are distributed between several modules, a first module ST'O determining and memorizing the set of classes and sending class by class in descending order from the top of the histogram to a module of a set of modules ST ' 1, ST'2, ST'3 ... the data corresponding to one of said classes, the classes being thus distributed between the modules, each of the modules ST'1, ST'2, ST'3 ... determining and storing the class he received,
- the means making it possible to determine and store a set of class data are distributed between an STN module and an interface.
- API application programming said means comprising a memory MO of histogram values determined by the module, a memory M1 of addresses ordered in amplitude of vertices of the histogram, a memory M2 making it possible to store the class order number, a class threshold memory M3 and a class number register RC, the API making it possible to perform:
- the updating cycle comprising: (A) sorting the amplitudes in decreasing order in the memory MO and memorizing the corresponding addresses in the memory M1,
- (C) - a validation of the classes of the memory M2 by comparison of the value of the memory MO at the address of the class considered of M2 with the threshold of M3 corresponding to the class considered,
- the memories MO and M1 correspond to the addressable memory of the STN module and the corresponding data are grouped together in a single functional table of amplitude and memory position pairs RMAX 0 / POSRMAXo; RMAX 1 / POSRMAX1; RMAX2 / POSRMAX2 ... of the vertices of the histogram in decreasing order of amplitude, the functional table performing an automatic material sorting of the classes during the calculation step,
- the memory M2 making it possible to store the sequence number of the class, the memory M3 of the threshold of the classes,
- the multi-class functional unit receiving at least one simple DATA (A) or complex DATA (A ⁇ ... A p) parameter, a VALIDATION validation signal linear combination of retro-annotation signals and IN1T sequencing signals , CALCULATION, END, CLOCK, said multiclass functional unit returning at least one set of output signals each corresponding to a class Cl ⁇ ... Cl k on the feedback bus,
- the API is a programmable sequencer of the memory microprocessor type or microcontroller
- the device is in an object recognition system comprising at least two multiclass modules, a first module operating in the time domain TD, determining at least one class and recruiting for said class at least a second module operating in the space domain SD ,
- the device is in an object recognition system comprising a set of STN calculation and histogram processing modules by sectorization, the modules determining zones and barycenters, and that it makes it possible to divide a determined zone into several angular sectors centered on the corresponding barycenter of the zone and to search in which sector among the sectors, a new barycenter appears and to divide said sector into several sub-sectors in order to gradually refine the sectorization,
- the device comprises at least one STN module with at least two orientation units p ⁇ p ⁇ with axes 150, 15) at input for rotation of reference axes of at least two Cartesian coordinates of input parameters, the / the modules further determining the barycenter for the input parameters, and in the system a first space Zj is determined comprising a barycenter BarZj by an association of a monolinear module processing a first parameter and a second bilinear module processing the coordinates , a second association determining inside said first space Zj the appearance of a second barycenter BarZj + i, said first space being divided into distinct angular sectors regularly distributed Zno, Zr, Zrj 2 ..., each of the sectors being processed by a bilinear sector module 300, 301 ...
- the system also comprises at least one size invariance unit, said size invariance unit receiving at least as input, on the one hand, a value of the logarithm a distance LD between a reference point and, on the other hand, at least one point of the object and the module pj of distance between two barycenters BarZj and BarZ i + ⁇ , said unit determining at least one value Cp 'of projection substantially constant and corresponding to an angle ⁇ of rotation relative to the reference frame p and LD,
- the size invariance unit includes distance control means allowing the choice of the use of an external distance measurement or the internal determination of the distance, - the object is observed at different distances by displacement physical or by zoom effect,
- the system also comprises at least one rotation correction unit, said rotation correction unit making it possible to correct the angle value ⁇ of the reference axis with respect to to a couple of values ⁇ , p of angle and modulus previously determined,
- the rotation correction unit also makes it possible to determine the angle of rotation ⁇ , - the values ⁇ LD, ⁇ , Cp ', are calculated,
- an object recognition system comprising a set of STN modules for calculating and processing a histogram, comprises at least one unit for transforming the reference frame of parameters by angle rotation ⁇ , the reference frame being at least on two dimensions of parameters, the rotation in the case of a two-dimensional coordinate system for parameters of polar coordinates of pixel X, Y, corresponding to the following matrix operation:
- the angle ⁇ or ⁇ is chosen so that at least one of the parameters on the corresponding axis p ⁇ p ⁇ i has a reduced rate of variation after rotation
- the parameters are two in number and chosen from the pairs X, Y of pixel coordinates or LogD, p of logarithm of distance of a pixel with respect to a reference point and of angle with respect to said point and with a right of reference,
- the device includes means for determining whether or not a new object observed corresponds to an object with a label previously stored by determining the recognition data for the new object and comparison with the recognition data for previously stored labels,
- the label recognition data is also associated with a rotation angle ⁇ and an average distance LD 'by means making it possible to determine said rotation angle ⁇ and said average distance LD',
- - recognition data of a first label are associated with those of at least a second label in order to form a new label corresponding to a higher level of recognition
- the labels are analyzed by a histogram calculation and processing module capable of determining and storing a set of data for categorizing said labels.
- the invention as a method or device according to one or more of the preceding characteristics, possibly combined, preferably works on image video data.
- the data space in which the object must be located is preferably a space-time space, that is to say that, on the one hand, it evolves over time and that, on the other hand, the parameters of l object can be represented at a given time on one (point of space) or two (surface) or three (volume or hue + saturation + light or other) or more dimensions as the case may be.
- the invention also relates to a method of operating a multi-class functional unit for calculating and processing a histogram for analyzing a parameter carried by a digital DATA (A) input signal in the form of a Ajj sequence. . t , A '* j ... t, A "jj ... tl ... of binary numbers associated with synchronization signals making it possible to define a moment t of space and a position i, j ... in this space, the histogram possibly comprising several vertices and the processing consisting in producing a set of characterization results of the histogram comprising at least the data pair maximum amplitude of the histogram and position of said maximum amplitude corresponding to the maximum vertex of the histogram.
- the histogram of the parameter is calculated as a function of a validation signal (VALIDATION) during a calculation cycle, and that a set of amplitude pairs is determined during said calculation cycle.
- VALIDATION a validation signal
- RMAXj and POSRMAXj position, the RMAX-, and POSRMAX being automatically classified in descending order of vertex and stored in a functional memory with automatic material sorting of the multi-class functional unit during said calculation cycle.
- the invention can also be implemented with the following characteristics, possibly combined according to all the technically feasible possibilities:
- a set of results is further produced in the form of k classification signals (CI-
- each of the classes of the classification signal is produced in relation to a set of values of the parameter for which the amplitude of the histogram is greater than a threshold criterion
- the threshold criterion is a function of the value of the amplitude of the vertex considered in the histogram
- a multi-class functional unit which includes an application programming interface (API), a memory MO storing the ordered amplitudes RMAXj of the vertices, a memory M1 of addresses ordered in amplitude POSRMAXj of vertices, the memories MO and M1 being grouped together in a single functional table of amplitude and memory position pairs (RMAXo / POSRMAXo; RMAX-i / POSRMAX-,; RMAX 2 / POSRMAX 2 ,
- API application programming interface
- the functional table performing an automatic material sorting of the classes during the calculation cycle
- the multiclass functional unit further comprising a M2 memory allowing to store the sequence number of the class, a memory M3 of threshold of the classes and a register RC of number of classes, - the API allows to perform: - an initialization cycle setting the memories MO, M1, M2, M3 and the register to zero,
- the update cycle comprising:
- (B) - a step of searching for the classes with labeling of the classes in the memory M2 and memorization of the corresponding thresholds in the memory M3, the number of classes thus determined being stored in the register RC,
- (C) - a step of validating the classes of the memory M2 by comparing the value of the memory MO at the address of the class considered of M2 with the threshold of M3 corresponding to the class considered, the API is a sequencer programmable of the memory microprocessor or microcontroller type,
- a set of sorting and storage units B0 ... Bn is implemented with memory registers, comparators and associated logic circuits allowing automated classification and storage of amplitudes and positions vertices of the histogram,
- a selection means is used to select a threshold criterion for a given class as a function of at least one of the following values:
- each of the classes of the classification signal being produced in relation to a set of values of the parameter for which the amplitude of the histogram is greater than said threshold criterion
- the criterion is selected from RMAX / 2, THRESHOLD, NBPTS / THRESHOLD,
- the parameter analyzed by the multiclass functional unit is complex and is obtained by combining at least two elementary parameters, each of the binary numbers of the input signal DATA (A ⁇ , A 2 , A 3 ... A p ) support for the complex parameter (A ⁇ A 2 A 3 ... A p ) comprising P fields each corresponding to an elementary parameter Ai, A 2 , A 3 A p ,
- the multiclass functional unit is implemented in a histogram calculation and processing module, called STN, the output signals forming a retroannotation being sent on a retroannotation bus and the validation signal (VALIDATION) involving the retroannotation .
- STN histogram calculation and processing module
- the multi-class functional unit receives at least one parameter, a validation signal (VALIDATION), a linear combination of retro-annotation signals in the case of an STN module, and sequencing signals (IN IT, CALCULATION, END, CLOCK) ,
- the multiclass functional unit returns at least one set of signals each corresponding to a class
- an object recognition system comprising a set of modules for calculating and processing a histogram, at least two STN modules are used per unit functional multi-class, a first module operating in a TD time domain, determining at least one class and recruiting for said class at least a second module operating in a SD space domain, - the module operating in the TD time domain receives a speed parameter MVT or L / T / S color,
- the method is implemented in an object recognition system comprising a set of STN modules for calculating and processing a histogram by sectorization with at least one module with a multiclass functional unit, the modules determining zones and barycenters, and in that one divides a determined zone into several angular sectors centered on the corresponding barycenter of the zone and that one searches in which sector among the sectors, a new barycenter appears and that one divides said sector into several sub - sectors, the said process being able to continue in order to continue to progressively refine the sectorization,
- At least one angle and one module are determined, the angle being given with reference to the line joining the two barycenters and the module corresponding to the distance along said line between said two barycenters,
- the method is implemented in at least one STN module with at least two orientation units p ⁇ p ⁇ of input axes for rotation of reference axes of at least two Cartesian coordinates of input parameters, the / the modules further determining the barycenter for the input parameters, and in the system a first space Zj is determined comprising a barycenter BarZj by a association of a monolinear module processing a first parameter and of a second bilinear module processing the coordinates, a second association determining inside said first space Z ⁇ the appearance of a second barycenter BarZ i + ⁇ , said first space being split into distinct angular sectors regularly distributed (Z ⁇ o, Zn-i, Zrj 2 ...), each of the sectors being processed by a bilinear sector module receiving Zj, BarZ, and also the signal of the second barycenter BarZj + i, the module bilinear sector corresponding to the second barycenter BarZj +1 being connected with a set of bilinear sector modules of subsequent rank making it possible to
- the size invariance unit receiving at least one input value on the one hand logarithm of a distance LD between a reference point and at least one point of the object and, on the other hand, the module p j of distance between two barycenters BarZj and BarZj + i, said unit determining at least one value Cp 'of substantially constant projection and corresponding to an angle ⁇ of rotation relative to the reference frame p and LD,
- the distance control means are used in the size invariance unit allowing the choice the use of an external distance measurement or the internal determination of the distance,
- the object can be observed at different angles and at least one rotation correction unit (900) is implemented in the system, said rotation correction unit making it possible to correct the angle value ⁇ d reference axis with respect to a pair of values ⁇ , p of angle and modulus previously determined,
- the rotation correction unit (900) also makes it possible to determine the angle of rotation ⁇ ,
- the method is implemented with at least one parameter reference transformation unit by rotation of angle ⁇ , the reference being at least on two parameter dimensions, the rotation in the case of a two-dimensional reference for parameters of polar coordinates of pixel X, Y, corresponding to the following matrix operation:
- the angle ⁇ (or ⁇ ) is chosen so that at least the projection of one of the parameters on the corresponding axis (p ⁇ ) (p ⁇ ) has a reduced rate of change after rotation,
- the parameters are two in number and are chosen from pairs X, Y of pixel coordinates or LogD, p of logarithm of distance of a pixel with respect to a reference point and of angle with respect to said point and with a reference line,
- an analysis tree is created linking the different barycenters as a function of their order of appearance, - to determine whether a new object observed corresponds or not to an object with label previously stored, the recognition data of the new object are determined and it is compared with label recognition data previously stored, - the label recognition data is also associated with a rotation angle ⁇ and an average distance LD 'by means making it possible to determine said rotation angle ⁇ and said average distance LD '
- the label recognition data is associated with a dominant color C by means making it possible to determine said color
- - recognition data from a first label are associated with those from at least a second label in order to form a new label corresponding to a higher level of recognition
- the label recognition data is analyzed by a histogram calculation and processing module capable of determining and storing a set of data for categorizing said labels.
- the invention likewise relates to a device which is a multi-class functional unit intended to operate according to one or more of the preceding functional characteristics relating to the method.
- This invention therefore relates to a multi-class functional unit for calculating and processing a histogram for analyzing a parameter carried by a digital DATA (A) input signal in the form of a sequence Ajj ... t, A'jj .. .t, A "jj ... t, ... of binary numbers associated with synchronization signals making it possible to define a moment t of space and a position i, j ...
- the histogram being able to have multiple vertices and the processing consisting in producing a set of histogram characterization results comprising at least the data pair maximum amplitude of the histogram and position of said maximum amplitude corresponding to the maximum vertex of the histogram.
- the unit comprises means making it possible to calculate the histogram of the parameter as a function of a validation signal (VALIDATION) during a calculation cycle, and to determine during said calculation cycle a set of amplitude couples RMAXj and position POSRMAX , the couples RMAXj and POSRMAXj being automatically classified in descending order of vertex and stored in a functional memory with automatic material sorting of the multiclass functional unit during said calculation cycle.
- VALIDATION validation signal
- the unit can also be implemented with the following characteristics, possibly combined according to all the technically feasible possibilities:
- the means also make it possible to produce as output a set of results in the form of k classification signals (CI I ... CI R ) with k greater than or equal to 1, each classification signal corresponding to a class related to a top of the histogram,
- each of the classes of the classification signal is produced in relation to a set of values of the parameter for which the amplitude of the histogram is greater than a threshold criterion
- the unit comprises an application programming interface (API), a memory MO storing the ordered amplitudes RMAXj of the vertices, a memory M1 of addresses ordered in amplitude POSRMAXj of vertices, the memories MO and M1 being grouped together in a table single functional pair of amplitude and memory position (RMAX 0 / POSRMAX 0 ; RMAX1./POSRMAX 1 ;
- API application programming interface
- the update cycle comprising:
- (B) - a step of searching for the classes with labeling of the classes in the memory M2 and memorization of the corresponding thresholds in the memory M3, the number of classes thus determined being stored in the register RC,
- (C) - a step of validating the classes of the memory M2 by comparing the value of the memory MO at the address of the class considered of M2 with the threshold of M3 corresponding to the class considered
- the API is a sequencer programmable of the memory microprocessor or microcontroller type, - in the case of a functional table, a set of sorting and storage units B0 ... Bn is implemented with memory registers, comparators and associated logic circuits allowing the automated classification and memorization of the amplitudes and positions of the vertices of the histogram, the unit makes it possible to further determine and store for each class, in a memory M4, the number of points NBPTSj of the histogram corresponding to said class,
- the unit makes it possible to determine and store in addition for each class, in an M5 memory, the average position
- the unit further comprises a barycentric unit intended to produce a barycenter output signal with a first binary state when the parameter corresponds to the mean position POSMOYj and with a second binary state otherwise for the class considered.
- the unit comprises a selection means making it possible to select for a given class a threshold criterion as a function of at least one of the following values: the value of the amplitude of the vertex of the histogram,
- each of the classes of the classification signal being produced in relation to a set of values of the parameter for which the amplitude of the histogram is greater than said threshold criterion
- the criterion is selected from RMAX / 2, THRESHOLD, NBPTS / THRESHOLD,
- the parameter analyzed is complex and that it is obtained by combination of at least two elementary parameters, each of the binary numbers of the input signal DATA (A ⁇ , A 2 , A 3 ... A p ) support for the parameter complex (A ⁇ A 2 A 3 ... A p ) comprising P fields each corresponding to an elementary parameter Ai, A 2 , A 3 A p , - the unit is in a histogram calculation and processing module, called STN, the output signals forming a retroannotation being sent on a retroannotation bus and the validation signal (VALI DATION) involving the retroannotation, - the multi-class functional unit receives at least one parameter, a validation signal (VALIDATION), a linear combination of feedback signals in the case of an STN module, and sequencing signals (INIT, CALCULATION, END, CLOCK),
- the multiclass functional unit returns at least one set of signals each corresponding to a class (CI *] ... CI k ) on the feedback bus in the case of an STN module, - the unit is in a system of object recognition comprising a set of modules for calculating and processing a histogram comprising at least two STN modules with a multiclass functional unit, a first module operating in a TD time domain, determining at least one class and recruiting for said class at least a second module operating in an SD space domain.
- the module operating in the TD time domain receives an MVT speed or L / T / S color parameter
- the unit is in an object recognition system comprising a set of STN modules for calculating and processing a histogram by sectorization with at least one module with a multi-class functional unit, and the modules make it possible to determine zones and barycenters and to divide a determined zone into several angular sectors centered on the corresponding barycenter of the zone and to search in which sector among the sectors, a new barycenter appears and to divide said sector into several sub-sectors, said process being able to continue in order to continue to progressively refine the sectorization, - two levels of sectorization are carried out, a first dividing an initial zone into sectors and a second dividing one of the sectors. having a new barycenter in sub sectors,
- At least one angle and one module are determined, the angle being given with reference to the right joining the two barycenters and the module corresponding to the distance along said straight line between said two barycenters,
- the sectorization is carried out in four sequences, - the unit is in the system comprising at least one STN module with at least two orientation units p ⁇ p ⁇ of input axes for rotation of reference axes of at least two Cartesian coordinates of input parameters, the module (s) further determining the barycenter for the input parameters, and in the system a first space Zj is determined comprising a barycenter BarZj by an association of a monolinear module processing a first parameter and a second bilinear module processing the coordinates, a second association determining inside said first space Zj the appearance of a second barycenter BarZj + i, said first space being divided into distinct angular sectors regularly distributed (Zno, Zrn , Zri 2 ...), each of the sectors being processed by a bilinear sector module receiving Zj, BarZj and also the signal from the second barycenter BarZ i + ⁇ , the module bilinear sector corresponding to the second BarZj +1 barycenter being connected to a set
- the system also comprises at least one invariance unit in size, said invariance unit in size receiving at least as input, on the one hand, a value of the logarithm of a distance LD between a reference point and at least one point of the object and, on the other hand , the module pj of distance between two barycenters BarZj and BarZj + i, said unit determining at least one value Cp 'of projection that is substantially constant and corresponding to an angle ⁇ of rotation relative to the reference frame p and LD,
- the size invariance unit includes distance control means allowing the choice of the use of an external distance measurement or the internal determination of the distance ,
- the system also comprises at least one rotation correction unit, said rotation correction unit making it possible to correct the angle value ⁇ of the reference axis with respect to a pair of values ⁇ , p of angle and modulus previously determined,
- the rotation correction unit also makes it possible to determine the angle of rotation ⁇ ,
- the unit is associated with at least one unit for transforming the coordinate system of parameters by rotation of angle ⁇ , the coordinate system being at least on two parameter dimensions, the rotation in the case of a two-dimensional coordinate system for coordinate parameters pixels of pixel X, Y, corresponding to the following matrix operation:
- the angle ⁇ (or ⁇ ) is chosen so that at least the projection of one of the parameters on the corresponding axis (p ⁇ ) (p ⁇ ) has a reduced rate of change after rotation,
- the parameters are two in number and are chosen from pairs X, Y of pixel coordinates or LogD, p of logarithm of distance of a pixel with respect to a reference point and of angle with respect to said point and with a reference line,
- means make it possible to determine over time several barycenters of zones appearing successively with respect to a first of an initial zone and to memorize the coordinates of barycenter in relation to a label in the form of recognition data
- the system makes it possible to determine the recognition data for the new object and to compare it with recognition data for previously stored labels,
- the label recognition data is associated with a dominant color C by means making it possible to determine said color
- the system makes it possible to analyze the label recognition data by a module calculation and histogram processing capable of determining and storing a set of data for categorizing said labels.
- the invention as a method or device according to one or more of the preceding characteristics, possibly combined, preferably works on image video data.
- the data space in which the object must be located is preferably a space-time space, that is to say that on the one hand it evolves over time and that on the other hand the parameters of the object can be represented at a given time on one (point in space) or two (surface) or three (volume or hue + saturation + light or other) or more dimensions as appropriate.
- the invention finally relates to a process for dynamic recruitment of calculation and processing modules of histograms for perception and recognition of an object in a space-time space and represented by parameters carried by data in sequences and in evolving sequences. in time.
- the modules determine, for at least one parameter and under the control of an application programming interface (API), the associated zones and barycenters according to a repository of representation, and in that one divides at least one of the zones thus determined into several angular sectors centered on the corresponding barycenter of the zone and that one searches in which sector among said sectors a new barycenter appears and that one divides said sector into several sub-sectors, the process being able to continue in order to continue to progressively refine the sectorization.
- API application programming interface
- the representation repository is of any dimension, it can correspond to a plan, a volume or a hyperspace.
- the present invention of the recruitment process also relates to the characteristics which will emerge during the description which follows and which will have to be considered in isolation or according to all their technically possible combinations: - two levels of sectorization are carried out, a first dividing an initial zone into sectors and a second dividing one of the sectors with a new barycenter in sub-sectors,
- the segmentation is refined to a single level and that the initial zone is simply divided into sectors, the sector containing the second barycenter not subject to new sectorization,
- At least one angle and one module are determined, the angle being given with reference to the line joining the two barycenters and the module corresponding to the distance along said line between said two barycenters,
- an initial area and an associated barycenter in the representation reference frame are determined by at least a first module for at least one predetermined parameter
- the first module is associated, on the one hand, with at least one second module intended to determine at least one second barycenter in said initial zone, and, on the other hand, a set n of sectoring modules intended for divide the initial area into n angular sectors, said sectoring modules receiving the initial area and its barycenter, - in a third sequence, the second module intended to determine at least a second barycenter having determined a second barycenter, said second barycenter is sent to all of the sectorization modules and the validated sectorization module is determined, the sector of which includes the second centroid
- a zone and an associated barycenter by at least two modules are determined, a first module operating in the time domain TD, recruiting at least a second module operating in the space domain SD,
- the representation repository is for example X, Y or LTS,
- the representation frame of reference is polar, the barycenter corresponding to the pair p, LD, respectively of modulus of distance between barycenters and of logarithm of the distance between the object and a sensor,
- the process can be used with any other type of representation repository, for example for a face: color, distance or for a sound intensity, frequency, harmonic,
- the method is implemented with at least one STN module with at least two orientation units p ⁇ p ⁇ of input axes for rotation of reference axes of at least two Cartesian coordinates of input parameters, the / the modules further determining the barycenter for the input parameters, and in that a first space Zj is determined comprising a barycenter BarZj by an association of a monolinear module processing a first parameter and a second bilinear module processing the coordinates, a second association determining within said first space Zj the appearance of a second barycenter BarZj + 1 , said first space being divided into angular sectors of regularly distributed istincts Z ⁇ JO, Z ⁇ M, Zn2 ..., each of the sectors being treated by a bilinear sector module receiving Zj, BarZj and also the signal from the second barycenter BarZj + i, the bilinear sector module corresponding to the second barycenter BarZj + i being connected to a set of bilinear sector modules of subsequent rank allowing split the sector with the second Bar
- the object can be observed at different distances and that at least one size invariance unit is also used, said size invariance unit receiving at least one input, on the one hand, value of the logarithm of a distance LD between a reference point and at least one point of the object and, on the other hand, the module pj of distance between two barycenters BarZj and BarZj + , said unit determining at least one value Cp 'of substantially constant projection and corresponding to an angle ⁇ of rotation relative to the reference frame p and LD,
- the distance control means are implemented in the size invariance unit allowing the choice of the use of an external distance measurement or the internal determination of the distance,
- the object is observed at different distances by physical displacement or by zoom effect, - the object can be observed at different angles and that at least one rotation correction unit is used, said rotation correction unit making it possible to correct the angle value ⁇ of the reference axis with respect to a pair of values ⁇ , p of angle and module previously determined,
- the rotation correction unit also makes it possible to determine the angle of rotation ⁇ ,
- the angle ⁇ or ⁇ is chosen so that at least the projection of one of the parameters on the corresponding axis p ⁇ p ⁇ has a reduced rate of variation after rotation
- the parameters are two in number and are chosen from the pairs X, Y of pixel coordinates or LogD, p of logarithm of distance of a pixel with respect to a point of reference and angle with respect to said point and to a reference line,
- an analysis tree is created linking the different barycenters according to their order of appearance, - to determine whether a new object observed corresponds or not to an object memorized and labeled previously, the recognition data of the new object are determined and they are compared with label recognition data previously stored, - the label recognition data is also associated with a rotation angle ⁇ and an average distance LD 'by means making it possible to determine said rotation angle ⁇ and said average distance LD ',
- the label recognition data is associated with a dominant color C by means making it possible to determine said color
- - recognition data from a first label are associated with those from at least a second label in order to form a new label corresponding to a higher level of recognition
- the label recognition data is analyzed by a histogram calculation and processing module capable of determining and storing a set of class data.
- the recruitment invention also relates to a device intended to operate according to one or more of the preceding functional characteristics relating to the recruitment process.
- This invention therefore relates to a device for dynamic recruitment of modules for calculating and processing histograms for perception and recognition of an object in a space-time space and represented by parameters carried by data in sequences and sub-sequences evolving over time.
- a device for recruiting means so that the modules determine for at least one parameter and under the control of an API application programming interface of the associated zones and barycenters according to a repository of representation, and so that at at least one of the zones thus determined is divided into several angular sectors centered on the corresponding barycenter of the zone and it is sought in which sector among said sectors a new barycenter appears and that said sector is divided into several sub-sectors, the process can continue in order to continue to progressively refine the sectoring.
- the repository of representation is of any dimension, it can correspond to a plan, a volume or a hyperspace.
- the recruitment system can also be implemented with the following characteristics, possibly combined according to all the technically feasible possibilities:
- the means make it possible to determine at least one angle and one module, the angle being given with reference to the straight line joining the two barycenters and the module corresponding to the distance along said straight line between said two barycenters,
- the second module intended to determine at least a second barycenter having determined a second barycenter, to send to all of the sectorization modules said second barycenter and to determine the validated sectorization module whose sector includes the second barycenter,
- the means make it possible to determine a zone and an associated barycenter by at least two modules, a first module operating in the time domain TD, recruiting at least one second module operating in the space domain SD,
- the representation reference system is for example X, Y or LTS, - the representation reference system is polar, the barycenter corresponding to the pair p, LD, respectively of modulus of distance between barycenters and of logarithm of the distance between the object and a sensor,
- the device can be used with any other type of representation repository, for example for a face, color, distance or for a sound: intensity, frequency, harmonic,
- the means comprise at least one STN module with at least two orientation units p ⁇ p ⁇ of input axes for rotation of reference axes of at least two Cartesian coordinates of input parameters, the module (s) determining in in addition to the barycenter for the input parameters, and in that the means make it possible to determine a first space Zj comprising a barycenter BarZj by an association of a monolinear module processing a first parameter and of a second bilinear module processing the coordinates, a second association determining inside said first space Zj the appearance of a second barycenter BarZj + i, said first space being divided into distinct angular sectors regularly distributed Z ⁇ o, Z ⁇ -j, Zr i2 ..., each of the sectors being processed by a bilinear sector module receiving Zj, BarZj and also the signal from the second barycenter BarZj + i, the bilinear sector module corresponding to the second barycenter B arZ i + ⁇ being put in contact with a set of bilinear sector modules of
- the means make it possible to determine the value C ⁇ of the projection of the straight line uniting the two barycenters BarZj and BarZj + i on the reference axis according to the angle ⁇ ,
- the object can be observed at different distances and that it also comprises at least one size invariance unit, said size invariance unit receiving at least as input, on the one hand, a value of the logarithm a distance LD between a reference point and at least one point of the object and, on the other hand, the module pj of distance between two barycenters BarZj and BarZj +1 , said unit determining at least one value Cp 'of projection substantially constant and corresponding to an angle ⁇ of rotation relative to the reference frame p and LD,
- the size invariance unit includes distance control means allowing the choice of the use of an external distance measurement or the internal determination of the distance,
- the object can be observed at different angles and that it also comprises at least one rotation correction unit, said rotation correction unit making it possible to correct the angle value ⁇ of the reference axis with respect to to a couple of values ⁇ , p of angle and modulus previously determined,
- the rotation correction unit also makes it possible to determine the angle of rotation ⁇ ,
- It comprises at least one unit for transforming the reference frame of parameters by rotation of angle ⁇ , the reference frame being at least on two parameter dimensions, the rotation in the case of a two-dimensional coordinate system for parameters of polar coordinates of pixel X, Y, corresponding to the following matrix operation:
- the angle ⁇ or ⁇ is chosen so that at least the projection of one of the parameters on the corresponding axis p ⁇ p ⁇ has a reduced rate of variation after rotation
- the parameters are two in number and are chosen from pairs X, Y of pixel coordinates or LogD, p of logarithm of distance of a pixel with respect to a reference point and of angle with respect to said point and with a reference line,
- - to determine whether a new object observed corresponds or not to an object previously memorized and labeled, it includes means making it possible to determine the recognition data of the new object and to compare them with recognition data of previously memorized labels, - it comprises means making it possible to associate, in addition to the label recognition data, a rotation angle ⁇ and an average distance LD 'by means making it possible to determining said angle of rotation ⁇ and said average distance LD ',
- the label recognition data is associated with a dominant color C by means making it possible to determine said color
- - recognition data of a first label are associated with those of at least a second label in order to form a new label corresponding to a higher level of recognition
- the device includes means making it possible to analyze the labels by a module computation and histogram processing capable of determining and storing a set of data for categorizing said labels.
- the invention of recruitment as a method or device according to one or more of the preceding characteristics, possibly combined, preferably works on video image data.
- the data space in which the object must be located is preferably a space-time space, that is to say that, on the one hand, it evolves over time and that, on the other hand, the parameters of the object can be represented at a given time on one (point in space) or two (surface) or three (volume or hue + saturation + light or other) or more dimensions as appropriate.
- the recruitment which consists in using new STN modules using results of calculations of STN modules carried out during previous sequences, to produce new results is a general method of the invention and that the parameters which are analyzed by the modules recruits can be any, depending only on the purposes of the calculation and the objects of the scene to be analyzed.
- the parameters can, for example, be a speed, a color, the coordinates (having in particular undergone a rotation for change of axes of reference), the speed and the color making it possible to locate objects in movement of a certain color for example.
- Recruitment is a dynamic process and, preferably, recruitment is stopped when a criterion is reached, for example number of new determined zones lower or higher than a threshold according to the application (the second criterion can be implemented with the variation of resolution where the least significant details appear gradually).
- the modules in question can be reused for other tasks.
- the release of the recruitment modules which occurs during the disappearance (criterion of cessation of recruitment) of the elements perceived, makes it possible to be able to reuse the said modules.
- the methods and devices of the invention relating to mono and multidimensional parameters, multiclass as well as recruitment, can be applied to active visual perception to characterize and recognize an object, in particular for the purposes of identification and localization.
- This application will be more particularly detailed in the illustrative part of this description with the details of the invention presented in relation to the structure and operation of the STN blocks.
- the application therefore relates to a system allowing the perception, recognition and localization of an object in its environment from a digital input signal constituted by a succession of sequences of successive views of the object in its environment and therefore falling within the time domain, each of said sequences being constituted by a succession of sub-sequences, each representative of locations arranged one after the other in said sequences and therefore falling within the spatial domain, characterized in that l '' a temporal variation of the spatial resolution of the object in said digital input signal is performed over a period of several sequences, the variation comprising a phase of substantially Gaussian increase in resolution from a reduced value to an optimal base value, a differentiation is also made, with smoothing between two successive sequences of said substantially Gaussian increase in resolution, in order to obtain a derivative digital signal representative of the variability of the difference of gaussians between these two sequences when the difference, in absolute value, for each same spatial location of said derivative signal, exceeds a threshold, and we deduce from said derivative signal, by comparison in the successive sequences
- FIGS. 1 and 2 represent two embodiments of a device for perceiving or knowing an object according to the invention comprising: means for achieving, during a period, an increase, in stages, substantially Gaussian of the spatial resolution, constituted by an optical assembly for the first embodiment and by an electronic filter for the second; a unit for differentiating gaussians by spatiotemporal smoothing; and means of using differentiation.
- FIG. 3 represents by a curve, the desired variation of the spatial resolution, comprising, during a period, an increase in stages between r min and r max.
- Figures 4a, 4b, 4c illustrate three embodiments of the optical assembly of Figure 1 and means for controlling the focusing thereof by varying its focal distance.
- FIG. 5 shows the application of the invention to a synthetic aperture radar system.
- FIG. 6 illustrates an embodiment of the electronic filter of FIG. 2.
- FIG. 7 represents the input signal of the electronic filter of FIGS. 2 and 6.
- FIGS. 8a, 8b, 8c, 8d and 8e represent the output signal of the electronic filter of FIGS. 2 and 6 for different successive values of the order w of the filter, w decreasing, while the resolution increases, from FIG. 8a to Figure 8e.
- Figure 9 illustrates an embodiment of the Gaussian differentiation unit by spatio-temporal smoothing of Figures 1 and 2, with the CO and DP output signals thereof.
- FIG. 10 represents, by a curve, the histogram of the absolute values of the differences, plotted on the abscissa, between the image just before the increase in spatial resolution and the image of progressively increasing spatial resolution, with the indication of the limit Li of the retained useful portion of the histogram.
- Figures 1 1 and 12 respectively illustrate a still life and a human face as a perceived object, with a curve showing for each the variation of the absolute values of the above-mentioned differences (plotted on the abscissa in Figure 10) as a function of the order parameter w of the filter of figure 6.
- Figure 13 shows the assembly arrangement, one above the other, of the partial figures 13a and 13b to form a global figure, designated below figure
- FIG. 15 illustrates, by curves a, b, c, d, e, the cumulative values of the table in FIG. 14 for the different phases.
- Figures 17 and 18 show two embodiments of a device for perceiving or knowing an object, more elaborate than those of Figures 1 and 2, also allowing the recognition of the object and the positioning thereof, because provided with an additional assembly constituted by a number of STN type units for forming histograms, FIG. 17 illustrating an elaboration of the embodiment of FIG. 1 with optical assembly for increasing the spatial resolution in stages , while FIG. 18 illustrates an elaboration of the embodiment of FIG. 2 with an electronic filter for such an increase.
- FIGS. 19 and 20 represent respectively a schematic illustration and a detailed illustration of a one-dimensional or unilinear STN unit for determining histograms, denoted STN (1), of said additional set of FIGS. 17 and 18.
- Figures 21a and 21b show two one-dimensional histograms, respectively with one and two peaks or values maximum values, determined by the unit STN (1) of figures 19 and 20.
- FIG. 22 schematically illustrates a two-dimensional or bilinear unit for forming histograms.
- Figure 23 shows the assembly arrangement, one next to the other along Z-Z ', of the partial figures 23a and 23b to constitute a global figure, hereinafter designated figure 23a-23b, which illustrates in detail such a two-dimensional or bilinear STN unit for determining histograms, denoted STN (2), of said additional set of FIGS. 17 and 18.
- FIG. 23c illustrates a partial variant of the classification sub-unit CH of the block STN (2) of FIG. 23a-23b.
- FIG. 24 represents a two-dimensional histogram with two peaks determined by the unit STN (2) of FIGS. 22 and 23a-23b.
- FIG. 25 is a schematic view representing the essential elements of an STN (2) self-adapting module with anticipation.
- Figures 26 and 27 show two sets capable of successively selecting the classes in the case of several classes corresponding to several peaks according to Figures 21 b and 24, the assembly of Figure 27 constituting an improvement thereof according to Figure 26 to allow faster selection.
- FIGS. 28 and 28a illustrate the flow diagram of the display phase of the results within the framework of the successive determination of several classes and its application in an integrated API.
- FIGS. 29a and 29b show, side by side, sets for carrying out the calculation phase, respectively without sorting the classes and with sorting the classes, in the case of several classes.
- Figure 30 illustrates, in more detail, the memory of Figure 29b.
- Figures 31 and 32 show, in more detail, two portions of the assembly of Figure 30.
- Figure 33 illustrates an arrangement implementing
- FIG. 34 illustrates the mounting of an STN unit (2) and of two orientation units p ⁇ , p ⁇ of plane discharging into this STN unit (2) to determine a mean or barycenter position (BarZj) debited in a bus.
- BarZj barycenter position
- FIG. 34a illustrates the delimitation of an object OB by a zone defined from two axes of orientation p ⁇ 2 and p ⁇ 3 of the plane and the determination of the BarZo barycenter of this zone.
- FIG. 35 schematically represents bands between Terminal A and Terminal B, with different orientations, coming from two-dimensional STN units (2) of the assembly of FIG. 34 (discharging in a common bus) and passing through the BarZ barycenter 0 determined according to Figure 34a.
- Figure 36 illustrates an improved classification arrangement of the assembly 101 of Figure 23a-23b.
- Figures 37 and 38 show the areas delimited respectively by the outlet 101 s of said assembly 101 and by the assembly of Figure 36 replacing this assembly 101.
- Figures 39a, 39b, 39c represent three successive phases of the determination of the polar coordinates p 3 and ⁇ 3 defining the relative position of BarZi, determined according to FIG. 36, relative to the position of BarZ 0 delivered by the unit STN (2) of FIG. 34 and determined according to FIGS. 34a and 35.
- FIG. 40 illustrates the father-son relationship between BarZ 0 upstream and BarZ-i downstream by implementing the polar coordinates p 3 and ⁇ 3 determined according to FIGS. 39a, 39b, 39c.
- FIG. 41 represents an assembly constituted by an STN module (1) and an STN module (2), such an assembly being used in the assemblies of FIGS. 42 and following.
- FIG. 42 illustrates an assembly for determining the polar coordinates by implementing the method illustrated in FIGS. 39a, 39b and 39c.
- FIGS. 43a, 43b, 43c illustrate the successive operations implemented in the electronic assembly of FIG. 42 to determine, in Cartesian coordinates, the position of BarZ * - (the "son") relative to BarZ 0 (the "son")
- FIGS. 44a to 44e, 45 and 46 relate to the case of zones Zo to Z 30 all nested one inside the other; in particular: FIGS. 44a, 44b, 44c, 44d and 44e represent the successive phases of the determination of the successive barycenters from BarZo to BarZ 30 , when these correspond to zones Zo to Z 30 nested one inside the other (as illustrated in figure 45) and therefore to a dynamic tree without connections (that of figure 46).
- FIG. 45 illustrates the relative positions of the barycenters BarZ 0 to Bar 3 o of the zones Zo to Z 30 determined by the assembly of FIG. 42.
- FIG. 46 illustrates the tree, without connection, of the BarZo to BarZ 30 barycenters of FIG. 45.
- FIGS. 47a to 47d, 48 and 49 relate to the case of zones Z10 to Z 30 without any interlocking, these zones Z-io to Z 3 o being all included in the zone Zo, in particular:
- Figures 47a, 47b, 47c and 47d represent the successive phases of the determination of successive barycenters BarZo to BarZso, these corresponding to zones Z 10 to Z 30 without any interlocking (as illustrated in figure 48) and therefore to a dynamic tree with connections (the one illustrated in figure 49).
- FIG. 48 illustrates the relative positions of the barycenters BarZ 0 to BarZ 3 o of the surfaces Zo to Z 0 determined by the arrangement of FIGS. 47a to 47d.
- Figure 49 illustrates the BarZo tree at Z 30 in Figure 48.
- FIG. 50 represents the successive determination of zones Z 0 to Z 2 ⁇ and Z22 in the intermediate case of nested and non-nested zones and with the corresponding tree with connection from BarZo and father-son relationship between BarZ- 12 and Z 2 1 in Figure 50a.
- FIG. 51 represents a portion of said additional set of FIGS. 17 and 18 constituted by one-dimensional units according to FIGS. 19 and 20 treating a parameter representative of the object observed and two-dimensional units according to FIGS. 22 and 23a-23b treating the parameters x and y coordinates.
- FIG. 52 represents another portion of said additional set of FIGS. 17 and 18 constituted by uni-bi- and three-dimensional units relating respectively to the value of CO, to the y and x coordinates of row and column and of parameters L, T and S of luminance, hue and saturation determining a color, the assembly of this figure 52 making it possible to preserve the fugitive information determined by the assembly of figure 51.
- FIG. 53 represents the unit for memorizing the objects perceived, with invariance in translation only.
- FIGS. 54 and 55d illustrate a unit for determining the invariance in size or dimension.
- Figures 55a, 55b and 55c correspond to Figures 43a, 43b and 43c respectively, but with, in coordinates, the module p and the angle ⁇ , previously determined according to Figures 43a to 43c, and the logarithm of the entry distance LD previously determined according to Figure 54.
- FIG. 56 illustrates the result obtained by the processing of Figures 54, 55, 55b and 55c.
- FIG. 57 represents an improvement of the unit for determining the invariance in size of FIG. 54, with looping of the logarithm of the distance, which allows a continuation of the operation in the absence of an input signal representative of the distance.
- FIG. 58 illustrates an improvement of FIG. 47d, additionally comprising units for determining the size invariance according to FIG. 57.
- FIG. 59 illustrates a unit for determining the invariance in rotation.
- Figures 60a and 60b are partial views of the entire object recognition device with invariance in translation, size and rotation, comprising units of Figures 58, 53 and 59.
- Figure 61 illustrates the angular displacement of the vision sensor to move from an object observed to another object to be observed.
- Figure 62 represents a scene with three objects identified from the point of view of their nature and their position.
- a device according to the invention and assemblies and constituent units thereof, allowing the implementation of the method according to the invention.
- the invention is described in its preferred application implementing a video type signal, it is not limited to this type of signal; it also applies, in particular, to a signal emitted by a synthetic aperture radar.
- a video image sensor 2 of CCD or CMOS type (in particular of the type ⁇ (retinal sensor "with high concentration of pixels in the center and decreasing concentration of pixels away from the center), being part of a camera, camcorder or webcam, observing an OB object located substantially in the plane 6 and debiting in response a digital signal 7 having undergone a reduction in resolution during certain periods (as specified below) and constituted by a succession of sequences (images or frames of the video signal) representative of successive views of the object in its environment and therefore falling within the time domain, each of said sequences being constituted by a succession of sub-sequences (lines of the video signal) representative of a succession of locations of pixels (constituting sequenced sub-units) of the video signal and therefore falling within the spatial domain, all the locations (pixels) arranged in rows and columns forming a matrix, for example rectangular.
- a video image sensor 2 of CCD or CMOS type (in particular of the type ⁇ (retinal sensor "with high concentration
- the video image sensor 2 is associated with a lens 5 whose focal length can be varied, therefore the focusing relative to the plane 6 of the object OB observed, under the control of a control unit 1, three modes of the assembly of the unit 1 and of the objective 5 being illustrated in FIGS. 4a, 4b and 4c described below.
- This variation of the focal distance of the objective 5 is controlled by the unit 1 so as to vary the spatial resolution, in the video signal 7, of the object OB to be perceived, which is in the plane 6, advantageously according to the curve of FIG.
- a unit 9 extracts from the video signal 7 after Gaussian transformation (FIG. 1), on the one hand, the Cartesian coordinates x and y of the rectangular matrix of pixels (or the polar coordinates for a circular matrix of pixels defined by a retinal sensor) and, on the other hand, the conventional Sync synchronization signals of a video type signal (frame and line), used as explained below.
- the unit 9 extracts from it the luminance L which is applied to the unit 3. If the video signal is in black and white, the signal 7 consists of the signal L.
- the unit 3 differentiates gaussians between two successive sequences of the video signal 7, a preferred embodiment of this unit 3, which advantageously performs space-time smoothing, being illustrated in FIG. 9 and described below with reference to this one.
- This unit 3 finally outputs two digital signals of DP overshoot and of quantification of CO overshoot, explained below, the successive values of the CO signal, which is an adjustable time constant, a function of the difference in Gaussians, therefore being a signal representative of the object perceived;
- the DP and CO signals can be viewed on a monitor M (that of a television set or a computer for example) and / or processed in a processing assembly M ′, for example (but not exclusively) of the type represented in 8 in Figures 17 and 18, explained in Figures 19 and following and discussed with reference to them;
- the signal DP which is binary, it can, when it has one of its two values 0 or 1, carry out an inhibition of the signal CO, as explained below.
- the signal DP which is binary, it can, when it has one of its two values 0 or 1, carry out an inhibition of the signal CO, as explained below.
- the video sensor 2 there is the plane 6 in which the object OB observed is substantially arranged, the video sensor 2, the unit 9 for extracting the Cartesian coordinates x and y (or polar coordinates) pixels and the Sync synchronization signal, namely, for each frame, the synchronization signal of this frame at the start of it, and the line synchronization signals (if signal 7 is a color video signal, the 'unit 9 also extracts the luminance component L of the signal 7') and the space-time processing unit 3 which outputs the CO and DP signals, the CO output signal being able to be displayed on a monitor M and / or processed in the processing assembly M ′, monitor and assembly similar to those of FIG. 1, in the absence of inhibition by the signal DP.
- the video image sensor of FIG. 2 is provided with a lens 5 with variable focus, that is to say with variable focal distance, controlled by the control unit 1 capable of vary the focus and therefore the resolution
- the sensor 2 in FIG. 1 is provided with a 5 'objective with focal distance kept fixed during the duration of an operation of knowledge and visual recognition, including the at the point is made initially on the object OB which is substantially in the plane 6.
- the sensor 2 therefore outputs a signal 7 'at constant resolution, the variation of the resolution being carried out after extraction of the coordinates x and y and also luminance L '(if the video signal is in color) in the unit 9 which operates on the signal 7' (in black and white) or the luminance L 'by an electronic filter 4 controlled by the control unit 1 'which imposes a variation of the resolution advantageously according to FIG. 3, by application of a parameter w constituting the order of the filter.
- the filter 4 outputs a signal 7 or L, analogous to the signal 7 or L of FIG. 1, which is processed like this in a space-time processing unit 3 similar to that of FIG. 1.
- the Gaussian or quasi-Gaussian increase in the resolution from rmin to rnax is carried out, in the embodiment of FIG. 1 by refocusing of the objective 5 during the period T3 of FIG. 3 (after defocusing during the period T1 of decrease in resolution from rmax to rmin and constancy of resolution at the value r min during period T2), whereas, in the embodiment of FIG. 2, this increase is effected by the electronic filter 4 which electronically performs the equivalent of a refocusing during the period T3 (after the equivalent of a defocusing in the period T1 and maintenance of the resolution in the period T2).
- FIG. 4a we find the plane 6, the object OB, the video image sensor 2, the objective with variable focal distance, denoted 5a, the control unit, denoted 1a, for adjusting the distance focal length of the objective of FIG. 1, this unit 1 a acting on the position of at least one of the lenses (or of the single lens) of the objective 5a; the change in position of the lens by displacement in both directions, which is schematically illustrated by the double arrow f, modifies the focusing of the objective relative to the plane 6, with a substantially Gaussian increase in steps of the resolution during the period T3 of increase thereof from rmin to rmax, after the abrupt decrease in resolution to r min during the period T1 and the maintenance at r min of the resolution during the period T2.
- the control unit denoted 1a
- the assembly 1 b-5b may for example be of the type described in the French patent application having the publication number 2.769.375 (filed on October 8, 1997 under the number 97 12781) .
- FIG. 4c the combination of a 5 ′ objective with constant focal length and a lens hollow 5c with variable focal length, disposed in front of the objective 5 ′, this lens 5c being controlled by a control unit 1 c which regulates the arrival of a transparent fluid inside this lens 5a between two sheets 5 ⁇ and 5 2 which are substantially parallel at rest and which bulge under the action of the arrival of said fluid by modifying the focal distance of this lens 5c.
- FIGS. 4a, 4b and 4c The three means of varying the focal length, therefore of the resolution, of FIGS. 4a, 4b and 4c result, due to the laws of optics applied to objective 5 (of type 5a or 5b) or to the lens 5c, to modify the resolution of the optical system in a Gaussian, or at least quasi-Gaussian manner, and consequently, this results in a signal L or a signal 7 which constitutes the Gaussian transform of the video signal, in particular during the period T3.
- FIG. 5 the application of the invention to the processing of the output signal 7R (analogous to the output signal 7 or L of FIGS. 1, 4a, 4b, 4c) of a radar transceiver 2 is illustrated.
- a control unit 1d controls a unit 5d of synthetic opening of the emission lobe to regulate, on the one hand, the width of the emitted lobe and, on the other hand, its matrix scanning, normally according to a rectangular matrix, corresponding to a matrix of video pixels of this type.
- the signal 7R emitted by the transceiver 2 'examining the object OB in the plane 6 is very similar to the signal 7, or rather L, of Figures 4a, 4b and 4c and can therefore be processed in a way similar to this signal 7, or rather L, in unit 3 of FIG. 1; it will be noted, in fact, that this signal 7R in FIG. 5 has only one digital component, unlike signal 7 in FIGS. 4a, 4b, 4c which, in the case of a color video sensor 2, has three color components, this signal 7R being rather analogous to the signal L of luminance of FIG. 1 which is also a signal with a single component.
- the unit 9 ′ (corresponding to the unit 9 in FIGS. 1 and 2) only performs the extraction of the Sync synchronization signals from the radar signal 7R (in addition to the signals of coordinates x and y not indicated on Figure 5).
- the signal 7R due to its obtaining as an output signal from a synthetic aperture radar, does have the form of a Gaussian or possibly quasi-Gaussian transform.
- a Gaussian or quasi-Gaussian processing is carried out by electronic filtering, in the filter 4, in the case of FIG. 2 (as specified below), as is carried out a Gaussian or substantially Gaussian modification of the optical focusing in the case of FIG. 1.
- the electronic filtering assembly 4 constituting the filter of FIG. 2, which in particular varies the input signal 7 ′ during the period T3 of incremental increase in resolution (FIG. 3), consists of the succession of two successive filtering units for the x and y coordinates, each of these two units being either of the Gaussian type or of the substantially Gaussian type and, in this case, be constituted by a Canny filter, discussed for example in an article by Messrs Didier DEMIGNY and Tawfik KAMLEH entitled "A discrete expression of Canny's criteria for step edge detection performancies evaluation" in I .EEE Pattern Analysis and Machine Intelligence, volume 19, N ° 1 1, pp.
- the electronic filtering assembly 4 is produced to perform a substantially Gaussian, or even Gaussian, filtering according to the x column and y line coordinates of the pixel matrix; advantageously, this filtering is carried out in two stages, namely preferably first a filtering according to y in a unit 20, then a filtering according to x in a unit 21, as illustrated in FIG. 2 inside the filter 4.
- the filter 20 of FIG. 6 finally supplies the quasi-Gaussian transform in y of the input signal L '(or 7'), ce. signal L '(or 7') being the one present at the input of the total filter 4 ( Figure 2).
- the unit 21 of this filter 4 which performs a quasi-Gaussian processing at x, is identical to the unit 20 illustrated in the figure 6 and that we have just described, apart from the fact that the input, instead of being constituted by the signal L '(or 7 ′), is constituted by the output sy en y of the unit 20, while that the output of unit 21 is constituted by this transform in y (debited by unit 20) having in turn undergone a quasi-Gaussian transformation in x; moreover, the unitary shift of the registers is this time not of a sub-sequence or line of pixels, but of a pixel position for the Re registers, while the shift is of w pixel positions for the registers A and B.
- the output of the unit 21, and therefore the output of the total filter 4 of FIG. 2 is a signal L (or 7) analogous to the signal L (or 7) of FIG. 1.
- + w (w + 3) ⁇ , with Cp given by an aforementioned formula, along the two axes y and x of the image, while w, which constitutes the order of the filter 4, is given by the relation ⁇ 0.3217w + 0.481 (w being therefore substantially equal to 3 ⁇ for relatively high values of ⁇ ); it is recalled that ⁇ is the standard deviation of the Gaussian and that the number of coefficients of the impulse response of the filter, linked to the resolution, is equal to 2w + 1, the value chosen for Cp having as its object to make equal to 1 the sum of said coefficients.
- the signal 7 ′ has been illustrated, namely the output signal from the video sensor 2 of FIG. 2, possibly reduced to the luminance signal L ′ in the case of color video, for a particular line of given frame of the video signal; in this figure, the x coordinate representing the successive pixels of the line has been plotted on the abscissa, with indication of the column number in the rectangular matrix formed by all the pixels of a frame arranged in columns and in lines, while 'on the ordinate, the level ni of the signal 7' has been increased, in particular of the luminance L ', which, as we recall, is at resolution maximum rmax ( Figure 3) or basic resolution, before having undergone a Gaussian transformation in the filtering assembly 4 of Figures 2 and 6.
- figure 7 represents the signal 7 '( in particular
- Figure 7 corresponds to the period T0 of Figure 3
- Figures 8a to 8e correspond to the phases ( or stages) t0 to t4 of the periods T2 and T3 of FIG. 3, w gradually decreasing until finding the zero value corresponding to rmax, that is to say at the resolution before filtering or defocusing, in the phase t5 of period T3.
- the output, denoted by simplification L or 7, of the filtering assembly 4 is constituted by the succession of signals 7a, 7b, 7c, 7d and 7e (of FIGS. 8a, 8b, 8c, 8d, 8e respectively) during phases tO (period T2), t1, t2, t3, t4 (of period T3) in Figure 3.
- the signal L (or 7), or 7R, that is to say the Gaussian or quasi-Gaussian transform either of FIG. 1 or of FIG. 2, or of FIG. 5, is processed in a smoothing unit 3 spatio-temporal illustrated schematically in FIG. 9.
- This unit 3 is of the type described and illustrated for the first time in the international publication WO-98/05002 (or rather its priority document) having the same inventor as the present application, in particular in the passage on page 14, line 24 to page 21, line 19 and to which one can possibly refer.
- the unit 3 in FIG. 9 firstly comprises two sub-units 10 and 11, the sub-unit 10 being a memory, while the sub-unit 1 1 (corresponding to the sub-unit 15 in FIG. 3 of the publication cited last) performs a spatio-temporal smoothing treatment.
- the LO and CO parameters circulate in a loop being reinjected, starting from the outputs of the subunit 1 1, at the inputs of this one, after having undergone, in memory 10, a delay equal to a sequence, namely, a image or frame in the case of a video signal.
- the unit 11 compares the values of LO and CO earlier of a sequence with their current values, for each pixel, the letters t, t-1, x and y respectively representing the instant t, the instant t-1 (t-1 corresponds to the instant preceding the instant t of the duration of a sequence), the abscissa x and the ordinate y of the pixels, t therefore corresponding to the current sequence, while t-1 corresponds to the sequence just preceding.
- LO t , ⁇ , y LO t . 1
- LO (t- * ⁇ ) represents the anterior substantially Gaussian image, smoothed in the subunit 11 and delayed by a sequence in memory 10, while Pixt is the current Gaussian image represented by the input signal L (or 7), or 7R, LO t therefore constituting a space-time Gaussian difference (called DOG);
- LO represents the successive smoothing values, while Pix represents the pixels of the signal L or 7, or the values of the radar signal 7R, at different positions in the pixel distribution matrix or of the point radar signals;
- CO and DP are the output signals from the spatiotemporal smoothing assembly 3.
- the input signal consisted of a conventional video signal output by a video sensor, here we have the signal L (or 7) of figures 1 and 2, or the signal 7R of figure 5, namely, the input signal Pix of unit 3 of figure 9, more particularly on one of the inputs of the subunit 1 1 of this unit 3 which calculates a difference of gaussians.
- the output signals CO and DP of the unit 3 result from the comparison of a characteristic parameter of the video signal 7, for example the luminance L of the pixels thereof, for the same pixel location, between the value for the just previous frame and the value for the current frame of the video signal, the adaptive time constant CO tending to reduce this difference as shown by the group of formulas cited last.
- Unit 3 is associated with a sub-unit 14 constituted by an STN module (1), of the type described below with reference in FIG. 20, which forms the histogram of the difference in absolute value (explained below with reference to FIGS. 19, 20, 22 and 23 and denoted Dif) between Pix t , x, y and LO t - ⁇ , x , y , calculated in the smoothing sub-unit 1 1, which will be used as explained below with reference to FIGS. 10, 1 1 and 12.
- STN module (1) of the type described below with reference in FIG. 20, which forms the histogram of the difference in absolute value (explained below with reference to FIGS. 19, 20, 22 and 23 and denoted Dif) between Pix t , x, y and LO t - ⁇ , x , y , calculated in the smoothing sub-unit 1 1, which will be used as explained below with reference to FIGS. 10, 1 1 and 12.
- FIG. 5 with variation of the resolution by modification of the radar lobe, it is the increasing value of the resolution in stages, by refocusing or reduction of the opening of the radar lobe respectively, which is substituted for the decreasing value by stages of the order w of filter 4 of figure 2.
- the space-time smoothing unit 3 is associated with the control unit 1 (FIG. 1) or 1 ′ (FIG. 2), detailed on FIG. 9 and which firstly comprises a module STN (1) 14, of the one-dimensional type with registers Reg, for example that described with reference to FIG.
- the signal Dif I Pixt, x , y -L0t- ⁇ , x , y l, this absolute value of the difference between two successive images (Img) at times t and t-1 (the unit value 1 corresponding to the duration of a signal sequence L, 7 or 7R) also being represented by
- the processing of the signal Dif in the module 14 provides the histogram of this signal, as illustrated in FIG. 10.
- the control unit 1 or 1 ′ also includes a control box 15 for the STN module (1) 14, the start of which is controlled by the initialization signal Dep and which, under the control of the synchronization signal Sync common to the entire device of FIG. 1 or 2, outputs the period selection signals representative of the successive periods of FIG. 3, namely the periods T1, T2, T3, T4, for the module STN (1) 14.
- control box 15 Once the control box 15 has been triggered by the signal Dep and it outputs the successive signals of the phases t1, t2, t3, t4, t5, t6, t7, etc. which determine the successive values of w in the embodiment of FIGS. 2 and 6 (t1 to t5), or the successive resolutions of objective 5 of FIG. 1 (in particular the objective 5a, 5b, 5c of FIGS. 4a , 4b, 4c respectively), or the synthetic opening unit 5d of a radar determining the successive openings of the radar lobe Lob (FIG. 5), these period signals T1, T2, T3, T4 with their phase signals t1, t2, etc.
- PiXwo, ⁇ , y " P-Xwi, x, yl or simply Dif l lmg w o-lnngwi I> l nr ⁇ 9 being an abbreviated notation of the word image, while wO and wi denote the initial and current values of w
- N plotted on the ordinate.
- the difference corresponding to the limit Li represents Terror between the same points of two images, the first original, unfiltered of rank 0, delayed by the subunit 10, representative of the sequence T0, the second, filtered L (7) of rank corresponding to Li in FIG. 1 0. This allows you to finish working on the most stable elements of the image.
- the limit Li having been determined from the histogram of distribution of the differences Dif (on FIG. 10), we consider this limit as being the maximum variation brought about by the filtering at w ma ⁇ with a number of intervals of I such as Li / threshold> I.
- the threshold in question here is the sensitivity threshold which appears at 12a in unit 11 of FIG. 9, this threshold being greater than the background noise in order to distinguish two successive stages.
- the value of p is forced to I (number of intervals), namely 5 in the particular case illustrated.
- the unit 3 of Gaussian differentiation is associated a subunit 16 for calculating the number of steps, integer value estimated by default of Li / threshold, Li coming from the module STN (1 ) 14 and being deduced from the histogram of FIG. 10 determined by the STN module (1), while the threshold (of sensitivity) comes from the location 12a of the subunit 11 where it has been previously registered.
- the value of p determined by the sub-unit 16 acts on the command 17 of the unit 4 (FIG. 2) to vary the order w of the filter 4, therefore the spatial resolution determined by this filter (or the spatial resolution of the objective 5 of FIG. 1, or the unit 5d for controlling the opening of the radar lobe in the case of FIG. 5).
- FIG. 9 finally illustrates, on the one hand, TAPI for controlling the sub-unit 14 of the sub-units 16 and 17.
- the input signals of the subunit 1 1 are constituted, during the phase tO (period T2), by the image of resolution rmin conveyed by the signal 7 (or L) or 7R, which enters as Pix (video signal) or analogous signal of the electromagnetic radar type and by l rmax resolution image that fits as LO.
- the calculation of p is carried out in subunit 16 and the value of p thus determined is entered in 12b, then, on the one hand, forced during phase t1 in subunit 1 1 as C0 t - ⁇ namely C0 t - o , via the multiplexer 13 and, on the other hand, transmitted to the control sub-unit 17, which gives, in response, the order of execution of the successive stages of increase in the resolution from rmin to rmax in five steps (figure 3), either to objective 5 to vary its focal length (figure 1), or to filter 4 in y for progressive reduction of w (figure 2), or to unit 5d opening synthetic to vary the aperture of the Lob radar lobe (Figure 5).
- FIGs 1 1 and 12 there are shown the histograms of the values of the difference Dif in relation to corresponding images which illustrate, on the one hand, a still life ( Figure 1 1) and, on the other hand, a face human ( Figure 12).
- Horizontal tax on the abscissas of the histograms represents the values of w, therefore the level of reduction in resolution which changes in the opposite direction
- Vertical tax on the ordinates represents the difference in absolute value, denoted Dif, resumption of the abscissas in Figure 10.
- FIG. 13a The first image at the top left of FIG. 13a, subtitled IMG, is the image of FIG. 9 corresponding to the signal 7 ′ at the maximum resolution rmax before the reduction in resolution; it is a complete, clearly legible image.
- the image at the top right of FIG. 13a it corresponds to the image on the left with maximum defocusing (or resolution r min) during the phase t0, after application of the value 20 for the order w of the filter 4.
- This image which is the image I ⁇ in phase t0, or the in phase t1 in FIG. 9, eliminates a large part of the details of the previous IMG image.
- the value a represents a filtering at very low spatial frequency and the values b, ç, d and e correspond to increasingly higher spatial frequency filtering.
- t5 that is to say of to + p
- additional information has been represented with respect to the preceding table, in particular, the indication of the periods, with indication of the initial period T1, with X for the previous values of n or CO, and the numerical values representing the cumulative pixel variations per frame for a, b, c, d, e, increasing from the bottom and from the left to the top and to the right.
- the modification of the resolution is carried out in the case of FIG. 2 by means of a filtering assembly 4 (detailed in FIG. 6) controlled by a parameter w constituting the order of the filter; such a Gaussian or substantially Gaussian transformation, in the filtering assembly 4, is equivalent to the optical defocusing carried out in the embodiment of FIG. 1 and therefore the signals 7, essentially of luminance L, and the digital signals CO and DP are the same in the case of FIG. 1 and in the case of FIG. 2 and we therefore obtain substantially the same images at the output of the space-time smoothing unit 3 of FIGS. 1 and 2, that is that is to say that FIGS.
- the signal L, 7 or 7R undergoes a very rapid reduction in its resolution from rmax to rmin, while w goes from 0 to 20 for example (figure 3), and the space-time smoothing unit 3 (FIGS. 1 or 2 and 9) has the storage of the unfiltered image of the period T0.
- the following period T2 is constituted by the only phase t0, of duration equal to a sequence for example, like the phases t1 to t9 of the periods T3 and T4 (FIG. 3).
- Unit 9 records the minimum resolution signal and, among other things, outputs a signal LO t o which, for its different values at the locations x, y of the pixel matrix (or similar positions in the case of an electromagnetic signal of the scanning radar type) is represented by the unfiltered image of rank 0 in, correspondence a signal L, 7 or 7R (image L (7) o of FIG. 13a) is introduced in Unit 1 1.
- the unit 1 1 outputs a signal Dif used by the module STN 14 to establish the histogram of FIG.
- p 5 and this value is forced into the subunit 1 1 during the next phase t1.
- the resolution of the signal L, 7 or 7R is increased by one step and begins the normal routine operation of the space smoothing unit 3 -temporal with calculations of LO, CO, DP and Dif debited by the subunit 1 1; during this phase t1, the image entering this subunit is the image I ⁇ (FIG. 9); this image is calculated from the incoming image of the previous phase corresponding to the image at minimum resolution, namely the image 1a (FIG. 9).
- FIG. 17 and FIG. 1 The difference between FIG. 17 and FIG. 1, on the one hand, and between FIG. 18 and FIG. 2, on the other hand, is constituted by the fact that the CO and DP signals from the above-mentioned unit 3 are processed in a set 8 which performs the training and classification of histograms using STN type modules, explained below.
- the space-time smoothing unit 3 does indeed include a unit of STN type, but this unit processes the signal Dif, to obtain the histogram of FIG. 10, and not the signals CD and DP, of the subunit 1 1.
- FIGS. 4a, 4b and 4c apply as well to the assembly of FIG. 17 as to that of FIG. 1, while the embodiment of FIG. 6 also applies well in the assembly of FIG. 18 than in that of FIG. 2.
- the other parameters treated in each of the units 8 being, on the one hand, two position parameters, denoted, x, y, and, on the other hand, three parameters representing a color, with namely the luminance L, the hue T and the saturation S, these five parameters having been extracted by Tunity 9 'similar to Tunity 9 (which in the embodiments described above only extracted the parameters x and y) from FIGS. 1 and 2; of course, instead of the parameters L, T and S, a color can be represented by other triads of parameters, for example parameters representing the fundamental colors red R, green V and blue B or else, keeping the brightness L, in place of T and S, the two parameters CR, which corresponds to LR, and CB, which corresponds to LB.
- the parameters L, T and S are preferably chosen, it is to take advantage of the fact that, in the event of a modification of lighting, the parameters T and S are practically not modified, only the luminance L being a direct function of Clearly, so that only one parameter is modified and, as explained, it is this parameter of luminance L which is favored compared to the other two, while, if Ton used triads of parameters of the red, green and blue or of type L, LR and LB, the three parameters are modified in the event of modification of the lighting.
- Dis An additional parameter as an input in the set 8 denoted Dis, equal to the distance measurement between the object OB and the sensor 2, resulting from an external unit 25 (not described), assists in the perception of the object by a size invariance calculation by STN modules described later.
- the output of the assembly 8 constituted by STN type modules, outputs two groups of signals allowing identify the nature of the object (WHAT or LABEL signal) and the position of the object (OR or Zo signal).
- the set 8, whether it belongs to FIG. 17 or to FIG. 18, as explained below, is the same since, as explained above, the variation of the resolution, whether it is made by optical means ( Figures 1 and 17) or by electronic filtering ( Figures 2 and 18), in both cases leads to the same type of increase in resolution, namely Gaussian or substantially Gaussian, during the period T3, Tunity 3 of FIGS. 17 and 18 ultimately receiving the same type of signal 7, essentially of luminance L, which is processed spatio-temporally in this unit, and delivering CO and DP signals of the same nature in the case of FIGS. 17 and 1 and in the case of FIGS.
- the signal CO (which is an adaptive time constant appearing in the monomial 2 C0 ) is in fact a digital signal n constituted by a small integer, while DP is a binary signal whose value 1, by definition, illustrates an overshoot (of sensitivity threshold) which is accumulated in the CO signal.
- the set 8 of FIGS. 17 and 18 is made up of STN modules of unilinear or unidimensional type, of bi-linear or bi-dimensional type and optionally of trilinear or three-dimensional type, unilinear modules treating CO parameter tunic, bilinear modules treating two parameters, such as the x and y coordinates of the pixels following the columns and the rows of the matrix of the pixels or radar signals, and finally the tri-linear modules processing three parameters, for example three parameters defining a color, and in the latter case, these three parameters are advantageously the brightness L, the hue T and the saturation S.
- unilinear, bilinear or trilinear modules it is possible to use such types of unilinear, bilinear or trilinear modules to constitute the unit 8 of FIGS. 17 and 18; however, it is preferable to implement preferred embodiments described below with reference to FIGS. 20 and 23a-23b relating respectively to a unilinear module (denoted STN (1) and symbolized in FIG. 19) and a bilinear module ( denoted STN (2) and symbolized in FIG. 22), it being understood that a tri-linear module (denoted STN (3) and not illustrated) derives (as explained below) very simply, by a few additions, from a bi-linear module. Referring to FIG.
- an STN (1) module of unilinear or one-dimensional type essentially comprises three units, namely a unit for calculating histograms CH, a unit CL classification and RA retro-annotation unit.
- the histogram calculation unit CH comprises several sub-units and firstly a digital analysis memory 100, for example a DRAM or an SDRAM having a number of addresses equal to the number of possible levels chosen for the values of the input parameter DATA (A) to namely CO and a word width representative of the number of pixels per frame (or image), namely for example 18 bits for an image of 256,000 pixels.
- This memory 100 comprises three inputs, namely, an input 100a DATA IN for the data, an input 100b WR for writing and an input 100c ADRESS for the addresses.
- the data input 100a is supplied from a data input multiplexer MUX 106 which receives, on the one hand, at 106a, the output signal 107s from an adder or incrementor S 107 for validation of incrementation and, on the other hand, at 106b, a signal "0", if the initialization signal INIT applied to the input 106c of the multiplexer 106 is equal to "1".
- the write input 100b of the memory 100 receives the output of an OR gate 13 whose two inputs respectively receive the initialization signal INIT (this signal also acting as indicated above on the multiplexer 106) and the signal WRITE write order.
- the ADRESS input 100c of the memory 100 receives the output signal 105s from an address multiplexer MUX 105 which has three inputs, namely an input 105a which receives the input parameter CO from the space-time smoothing unit 3 of FIGS.
- an input 105b which receives the COUNTER signal from a counter (not shown) and an input 105c which receives the output of a powered adder / subtractor 108, of a firstly, in 108a, by the counter signal COUNTER and, secondly, in 108b, by the output signal 1045s from the register POSRMAX 1045 (which will be specified below) through an AND gate 109 with two inputs, Tune receiving the aforementioned 1045s signal and the other the ARMED signal, this gate transmitting the 1045s signal only if the ARMED signal is actually present, but forcing the value “0” otherwise; in addition, the adder / subtractor 108 is controlled, on its input 108c, by a sense signal, denoted SENS, constituted either by the signal “0”, or by the signal “1”, namely for example "0" to order an addition and "1" to order a subtraction.
- SENS sense signal
- the aforementioned adder S 107 receives, on its first input 107a, a signal 100s coming from the output OUT 100d of the memory 100 and, on its second input 107b, a validation signal 102s coming from the retro-annotation unit RA as specified below; the output 107c of the adder S 107 outputs a signal 107s which is equal to the output of the analysis memory 100 (signal 100s) if the validation signal 102s is equal to "0", but equal to said output increased by 1 if the validation signal 102s is equal to "1"; the output 107c of the adder 107 is connected to the input 106a of the multiplexer 106, as indicated above, and to the input P, namely P4, of a comparator 140, which i compare this signal 107s to the signal of output 1044s of register RMAX 1044, received on its input Q, to deduce from this comparison (if P> Q) a signal 140s allowing to insert POSRMAX (the position of RMA
- the histogram calculation unit CH of the module STN (1) ultimately outputs three output signals, namely 100s, output by the memory 100, 105s, output by the multiplexer 105, and 107s, debited by the adder 107, to the classification unit CL and receives the signal 1045s therefrom, from register 1045 of this unit; in addition, the CH unit receives the signal 102s coming from the retro-annotation unit RA.
- the classification unit CL which constitutes a passive classifier, this essentially comprises an operating sub-unit 101, a classification updating sub-unit 103 and a storage sub-unit 104 comprising several registers.
- the operating sub-unit 101 playing the role of classifier comprises two comparators 1 10 and 1 1 1 which receive, on their input P (Pi, P2), the signal CO which constitutes the input data and, on their input Q (Q 1 ( Q 2 ), respectively the value of the CO classification terminal A and that of the CO classification terminal B, from the respective registers 1042 and 1043, Each of the comparators 1 10 and 11 1 compares the values received in P with the values received in Q.
- each comparator sends a signal to an input a or b (after inversion for the latter) of an AND gate 1 12 which also receives, on its input 1 12c, from the time smoothing unit 3 FIGS. 17 and 18, the signal DP indicating an overshoot by its value 1, or non-overshoot by its value 0, of a threshold in this unit 3.
- the classification updating sub-unit 103 of the classification unit CL comprises a selector 1122, of which three inputs a, b and c respectively receive the outputs of the registers 1044, 1041 and 1046 of the sub-unit 104, namely RMAX, NBPTS and THRESHOLD, which will be mentioned below.
- the signal from the output 1 152s from the selector 1 152 is applied Q, denoted Q 3 , of a comparator 1 151, which therefore receives either RMAX, or NBPTS, or THRESHOLD, according to a Choice signal received on the input e of 1,152.
- the other input of comparator 1,151, denoted P 3 is connected to the output 100d of the memory 100 to receive the signal 100s therefrom.
- the comparator 1151 checks whether the value of the signal on the input P 3 is greater than or equal to the value of the signal on its input Q3 (P> Q). If this is the case, an output signal 1 151 s is applied to the inputs a of two logic controllers 1063 and 1064 which also receive, at their input b, the signal END of the end of operation and, on their input c (after inversion for the second), the signal SENS (that also received by the adder / subtractor 108 on its input 108c); a fourth input d from each of the logic controllers 1063 and 1064 is capable of receiving an ARM arming signal which makes it possible to perform the task as long as the function represented by the signal 1115 s received on the input a is true (P > Q), i.e. P 3 > Q 3 .
- the outputs e of the logic controllers 1063 and 1064 output signals 1063s and 1064s respectively which validate in En (enable) the registers 1042 and 1043 for the terminals Terminal A and Terminal
- the third subunit 104 of the classification unit CL comprises several registers, namely first of all a register 1044 for the maximum value RMAX, a register 1045 indicating the relative position POSRMAX of this maximum value, a register 1041 for the number of NBPTS points and finally a register 1046 of THRESHOLD, registers 1044, 1041 and 1046 debiting in the multiplexer 1 152, as mentioned above, while register 1045 debits its output signal 1045s on input 108b of the adder / subtractor 108 of Tunity of CH histogram calculation, when AND gate 109 of the CH histogram calculation unit receives the ARMED signal.
- the subunit 104 also includes two other registers mentioned above, namely 1042 for terminal A and 1043 for terminal B; each of these two registers receives, on its input En (Enable or validation or authorization), the validation signal 1063s, 1064s debited by the output of the logic controller 1063, 1064 respectively, and, on its input a, the signal 105s output from multiplexer 105 of the CH histogram calculation unit.
- En Enable or validation or authorization
- the retro-annotation unit RA the inputs of which are connected to the bus 1 14 common to several STN type units, essentially comprises a battery 102 of registers 102r, denoted 1, 2, 3 ..., m, which receive respectively the values in1, in2, in3 ... inm of the 101 s output signals of the sub-units 101 of the various STN units connected to the bus 1 14.
- the one-dimensional unit STN (1) of FIG. 20 can also include registers 1047 and 1048 for storing the position of the average value representative of the input data CO; this mean position POSMOY corresponds in a way to the barycenter of the values of this datum CO.
- the comparator 1,153 if P> Q (in fact P5> Q5), outputs an output signal 1153s which is applied, after inversion, to the input a of the first register POSMOY 0 1047.
- the output signal 1047s of the register POSMOYo 1047 is received on data entry a from a second additional register POSMOY ! 1048 until the arrival of an END signal on another input b of this register 1048.
- the register POSMOYo 1047 determines the current average position (POSMOY), also designated the barycenter, of NBPTS resulting from the comparison in the comparator 1,153 of NBPTS / 2 and of the cumulative value in the signal 1,155 s , while the register POSMOY-i 1048 stores the mean position (or the barycenter) just anterior which is refreshed by the signal 1047s debited by the register POSMOY 0 1047.
- this one as well as the operation of the bilinear module STN (2) illustrated in FIG. 23a-23b and also that of the module of trilinear type not illustrated, comprises, for each of the consecutive sequences, three successive phases or cycles, namely an initialization cycle, a calculation cycle and a cycle for updating the registers storing the results.
- the STN module (1) receives an INIT signal from an external sequencer not shown.
- This INIT signal a) force the registers POSRMAX 1045, RMAX 1044, NBPTS 1041 and reg 1 155 to 0; b) starts a COUNTER counter which scrolls the successive values 0 to M (namely the maximum size of memory 100); c) force at “0”, via the multiplexer MUX 106 (which receives “0” on its input 106b and INIT on its input 106c), the input 100a DATA IN of the memory 100; d) places the memory 100 in write mode via the OR gate 1 13 through the input 100b WR of this memory 100; and, e) activates the output signal 105s of the MUX multiplexer 105 to impart, to the address of the memory 100, by its input 100c, the successive values of the counter
- the WRITE signal starts the calculation cycle which includes for each pixel cycle (CLOCK-PIXEL ): a) the cumulative value of CO in memory 100 each time that the VALIDATION signal 102s is true, this being transmitted, as explained in relation to the structure of the module STN (1), by the output OUT of unit 102, the cumulative values of CO in the adder 107 being transmitted to the memory 1 00 (on its input 100a) through the multiplexer 106; b) the comparison in the comparator 140, if this signal 102 is true, of the value RMAX of the comparator 1044, with the value of the signal 107s leaving the adder 107; if the output of comparator 140 is true, the value of the signal 107s, which constitutes a new value of RMAX, (because greater than that previously entered in register 1044 as demonstrated by the comparison in comparator 140), is entered in register R
- RMAX is registered in the register POSRMAX 1045, the entry of this register 1 044 being also connected, like the entry of the register POSRMAX at the exit of the gate AND 141; at the same time the value entered in the NBPTS 1041 register is incremented by "1"; c) the calculation of the classification output 101 s, of the CO value, in true value, each time this CO value is between the terminals A and B of the unit 101. 3) Result registers update cycle This last cycle begins when the WRITE signal becomes zero (equal to 0) and the END signal becomes active on the MUX 105 multiplexer, while the ARME signal is equal to "0"; this register update cycle consists of two successive half-cycles.
- the first half-cycle, which calculates POSMOY comprises the following successive operations: a) the value of POSMOYo from register 1047 is transferred to the register POSMOY *! 1048; b) the Choice command, applied to selector 1052, forces the output d of this register to the value NBPTS / 2; c) the value of the COUNTER counter is transmitted by the multiplexer 105, due to the END command also arriving on this multiplexer and thereby addresses the memory 100, by its input 100c, to this value; d) the sweep by the signal COUNTER brings out all the stored values 100s which are accumulated in the register reg 1 155, thanks to its initial value equal to 0 and to the summation made by the adder 1 156 which receives, in addition to the signal 100s, output 1 155s from the register
- the second half-cycle calculates the classification limits, namely Terminal A and Terminal B, from the maximum position POSRMAX.
- This second half-cycle implements the following operations: a) the COUNTER counter is reset to 0; b) the ARMED signal is validated and therefore the POSRMAX value, constituting the output signal 1045s from the register 1045, is transmitted to the subtractor-adder 108 by the AND gate 109; c) the signal SENS commands, in the adder / subtractor 108, the subtraction of POSRMAX-COU NTER (the first value arriving at 108b and the second at 108a), the result of this subtraction finally constituting the output signal
- the counter increments the COUNTER signal by one unit and this signal is valid as long as the output in 100d of memory 100 outputs a signal 100s greater than the output signal of the multiplexer 1 152 (output d), the comparison being carried out in the comparator 1,151 which outputs an output signal 1,151 s to the logic controllers 1063 and 1064; e) the validation logic function 1063 is true and validates the writing of the value of the output signal 105s in the register Terminal A 1042 in which it arrives at input a; f) as soon as the output signal 1 151 s is no longer valid, the process of the abovementioned operations c, d, e is interrupted and the value of Terminal A remains frozen at the last value recorded in the register 1042; g) the counter (not shown) delivering the COU NTER signal is reset to 0; h) the signal SENS forces the addition of POSRMAX + COUNTER in the adder / subtractor 108, the result
- FIG. 21 a a histogram of the values of the CO parameter with a single peak P
- FIG. 21 b represents a histogram with two peaks P1 and P2.
- RMAX / 2 defines the terminals A and B shown in FIG. 21 a, these two terminals being stored respectively in registers 1042 and 1043; these terminals are also applied to comparators 1 10 and 1 1 1 to be compared with the CO input parameter.
- the output signal 101 s of the assembly 101 of the comparators 1 10 and 1 1 1 above and of the AND gate 1 12 is illustrated at the bottom of FIG. 21 a and it can be seen that it has a slot (positive) between the Terminal A and Terminal B.
- FIG. 21 b the case is shown where the histogram of the CO values comprises two peaks P1 and P2 (instead of the single peak P in FIG. 21 a).
- the peak P2 corresponds, as in the case of FIG. 21 a, RMAX, on the one hand, and POSRMAX, on the other hand.
- RMAX / 2 we define RMAX / 2 as in the case of Figure 21.
- the peak P1 also defines a maximum value which is referenced RMAX '.
- the value of RMAX / 2 instead of defining only two terminals, namely Terminal A and Terminal B, defines, in relation to FIG.
- this signal which has the same form as the signal 101 s of FIG. 21 a, being defined by terminals A and B corresponding to RMAX of the highest peak P2.
- the STN module (2) detailed in FIG. 23a-23b comprises, like the STN module (1), essentially three units, namely a histogram calculation unit CH, a classification unit CL and a retro-annotation unit RA, and debits in a bus 1 14 common to several STN modules; in addition, advantageously, as in the case of the STN module (1), the STN module (2) can comprise in addition to the registers provided in the classification unit CL and discussed below for storing the position of the average value representative of the data input (in the case of the STN (2) module two input parameters), units 1 156, 1 155 and 1 153.
- the unit for calculating histograms CH of the register STN (2) of FIG. 23a differs from the corresponding unit CH of the module STN (1) of FIG. 20 by the fact that provision is made, as regards the input 105a of the multiplexer 105, not the input of a single input signal DATA (A), namely CO, but of two associated input signals DATA (A y ) and DATA (A x ) , for example the two Cartesian coordinates y and x extracted, by unit 9 of FIG. 1 or of FIG. 2, from the input signal respectively 7 (after defocusing) or 7 '(before processing in the filter 4 performing an operation analogous to a defocusing).
- 23a comprises, in addition to the STN module (1) of FIG. 20, a unit 130 of shift shift to the left ( multiplication) which receives the signal END ( -i ) on the one hand, and the signal MOY y on the other hand, the association of these two signals applying an offset equal to the number of bits of the maximum DATA (Ax) signal to the signal input COUNTER to output a signal 130s on input 108a of the adder / subtractor 108, whereas, in the case of an STN module (1) (figure 20), the signal COUNTER is applied directly without offset on the input 108a of the adder / subtractor 108.
- the classification unit CL comprises a certain number of duplicates compared to the corresponding unit of the module STN (1) of figure 20.
- the sub-unit of storage 104 comprises, not a pair of registers 1042 and 1043 for Terminal A and terminal B respectively, but two pairs of registers, namely, 1042 (1) and 1043 (1) for Terminal A x (relating to the parameter input x) and Terminal B x (also relating to parameter x), on the one hand, and a pair of registers 1042 (2) and 1043 (2) for Terminal A y (relating to input parameter y ) and for Terminal B y (also relating to the input parameter y), these two pairs of registers receiving, as input signal, the signal 105s debited by the multiplexer 105.
- the two automata 1063 and 1064 of the module STN (1) of figure 20 are also duplicated in the case of a module STN (2 ) and in FIG. 23b, two pairs of such automata have been illustrated, namely, 1063 (1) and 1064 (1) corresponding to the input parameter x and 1063 (2) and 1064 (2) corresponding to the input parameters y.
- the operating sub-unit 101 of FIG. 20 is split in the case of an STN module (2) of an assembly 101 b, into sub-units 101 (1) and 101 (2), playing a role. analogous to that of subunit 101, each of these subunits 101 (1) and 101 (2) outputting a signal, respectively 101 (1) s and 101 (2) s, analogous to the single signal 101 (s) debited by the STN module (1) of Figure 20; the two signals 101 (1) s and 101 (2) s are output on the inputs a and b of the AND gate 131 which delivers an output signal 101 s only in the case where the two signals 101 (1) s and 101 (2) arrive simultaneously on the two entrances of this door 131; it is this signal 101 s which is debited on the bus 1 14.
- FIG. 23a-23b comprises, in addition to FIG. 20 concerning a unilinear module STN (1) anticipation means which are constituted by an anticipation block 1049 ( Figure 23b) which receives, on its two inputs, the outputs of the registers POSMOYo 1047 equal to POSMOY X y relating to the two input parameters x and y, and POSMOY-j 1048 and which determines, in response, the differentials ⁇ x and ⁇ y representative of the value of the two registers 1047 and 1048.
- an anticipation block 1049 Figure 23b
- the variation of POSMOY for the parameter y is denoted ⁇ y while that that for the parameter x is noted ⁇ x;
- ⁇ y and ⁇ x calculated in the calculation module 1049 are debited respectively on two subtractors 145 and 146, which receive these values on their entry b while their entry a receives respectively DATA (A y ) and DATA (A X ).
- the outputs of subtractors 145 and 146 which respectively calculate DATA (A y ) - ⁇ y and DATA (A X ) - ⁇ x , hence anticipation based on a linear variation of DATA (A y ) and DATA (A x ) , are debited concomitantly in unit 101 b in order to debit a classification signal 101 s.
- the POSMOY xy output of register 1047 is applied to the input b of a comparator 1050 (FIG.
- the cycle 1) of initialization of the STN module (1) is kept as is for the STN module (2); on the other hand, the cycle 2) calculation, the classification phase is split; finally, in the cycle 3) of updating the results systems, the DATA (A X ) and DATA (A y ) are processed successively.
- the module STN (2) receives the input parameters DATA (A y ) and DATA (A X ) on the input 105a of the multiplexer 105; it processes these data DATA (A y ) and DATA (A X ) in the same way as the single input data DATA (A), namely CO, of the STN (1) of FIG. 20; in fact, in the two embodiments (modules STN (1) and STN (2)), the input parameter CO or the input parameters DATA (A y ) and DATA (A ⁇ ) are sent by the multiplexer 105, as a signal 105s, to the input 100c ADRESS of the memory 100.
- the AND gate 131 combines the output signals 1 01 (1) s, from the subunit 101 (1), and 101 (2) s, from the subunit 101 (2), to finally debit , in the event of the simultaneous presence of the two signals, the global output signal 101 s applied to the bus 114. 3) Update cycle of the result registers
- Data processing, DATA (A y ) implements, among the duplicated blocks, those having the index (1), namely registers 1042 (1) and 1043 (1) for the terminals, Terminal A x and Terminal B x , the logic controllers 1063 (1) and 1064 (1), and the operating subunit 101 (1).
- the unit 108a triggers the transition from the processing of this data x to that of the second input data y, namely DATA (A y ) , this being treated in an analogous manner, but using among the duplicated blocks those comprising the index (2), namely the registers 1042 (2) and 1043 (2), the logic automata 1063 (2) and 1064 ( 2) and the operating sub-unit 101 (2).
- This last cycle begins when the WRITE signal becomes zero (equal to 0) and the END signal (1, 0) becomes active on the MUX multiplexer 105, while the ARMED signal is equal to "0"; this register update cycle consists of:
- the second third of the cycle calculates the classification limits Terminal A (x) and Terminal B (x) from the DATA portion (A x ) with the operations a to I previously described,
- the third third of the cycle calculates the classification limits Terminal A ( y ) and Terminal B ( y ) from the DATA portion (A y ).
- Unit 130 is activated by the association of the commands MOY y and END (1) and the operations a to I described above are carried out.
- FIG. 24 represents, in two dimensions along x and y, namely DATA (A X ) and DATA (A y ), two peak surfaces V1 and V2, in a manner analogous to FIG. 21 b illustrating, in a single dimension along x, namely CO, a curve with two peaks P1 and P2; in this figure 24, an example of the results obtained with a two-dimensional module STN (2) of the type illustrated in figure 23a-23b has been illustrated.
- the three-dimensional Cartesian coordinate system we find along the coordinates (x) and (y), DATA (A X ) and DATA (A y ), and according to (z) the number or quantity Q of points having the coordinates (x ) and (y).
- Ton performs a test on the sign of the anticipation value and therefore on the direction of anticipation and that Ton can increment or decrement the rewrite address of the result
- Ton can increment or decrement the rewrite address of the result
- said transfer function in block 120 obliges to increase the length of the "words" stored in the memory 100, therefore to oversize the latter, compared to the memory 100 of FIG. 23a; indeed if, for example, the memory 100 of FIG. 23a was intended to store 10-bit words and if the coefficient Km (applied to block 120 of FIG. 23c), can vary from 1 to 2 m , the capacity of the memory 100 of FIG. 23c must be dimensioned to store words of (10 + m) bits.
- the signal Km is debited by a register 121 controlled by TAPI of the module STN (2), according to the importance of the learning desired for this module.
- Unit 8 of FIGS. 17 and 18 can comprise tri-linear STN modules, for example for the three parameters representing a color, such as the LTS (or RGB) parameters.
- a tri-linear module is analogous to the bilinear module of figure 23a-23b (possibly with the improvement of figure 23c), except that:
- the block 130 of the histogram calculation unit CH of an STN register (3) is capable of controlling two offsets , namely for the second and third DATA (the three DATA being for example DATA, DATA T and DATA S ; • on the other hand, the units referenced (1) and (2) of FIG. 23a-23b are accompanied by a third unit similar to units 101 (1) and 101 (2), 1063 (1) and 1063 (2), 1064 (1) and 1064 (2) 1041 (1) and 1041 (2), 1042 (1) and 1042 (2);
- a tri-linear STN module is similar to that of the bi-linear module illustrated in FIG. 23a-23b except for the fact that Ton successively processes the three parameters in three sets referenced (1), (2) and ( 3) instead of only two sets (1) and (2), the three parameters such as L, T and S being treated successively as have been treated successively, in the linear module, the parameters x and y .
- Ton successively processes the three parameters in three sets referenced (1), (2) and ( 3) instead of only two sets (1) and (2), the three parameters such as L, T and S being treated successively as have been treated successively, in the linear module, the parameters x and y .
- Figure 25 illustrates a solution to improve the classification of DATA parameters with multiple bounds (in any number) with anticipation.
- most of the STN devices (2) have been taken from Figures 23a-23b or 23a-23c.
- Figure 25 is identical to Figure 23a-23b except
- Tunity 101 which is replaced by Tunity 101 c consisting of a memory 1 1 8, equal in number of words to memory 100 but of word width of 1 bit, and by the addition of a multiplexer 144 with 2 inputs.
- This multiplexer 144 is controlled by an end of operation signal END, which when it is equal to zero, outputs the result of the units 145 and 146 and when it is equal to one, outputs the signal 105s.
- the multiplexer 144 outputs an output signal 144s on the ADRESS input of a memory 118 whose input DATA IN consists of the output of a comparator 11151 which receives on its Q input half of RMAX from its register and on its input P the output 105s of memory 100; this comparator 1 151 compares the input signal on its input P and that of its input Q and if P> Q, the comparator 1 151 outputs a signal, equal to one, on the DATA IN input of the memory 1 18, and equal to zero otherwise. It is this signal on the output OUT of the memory 1 18 which then constitutes the signal 101 s of output, with anticipation, applied to the bus 1 14.
- the memory 1 18 of this module 101 c is written during the setting cycle.
- END signal equal to one.
- a comparison is made between the value of the memory 100 and a threshold defined by the CHOICE command, and defined by the value of the bit to be written to memory 1 18; equal to 1 if the value of memory 100 is greater than or equal to the zero threshold otherwise, the command to write memory 1 18 being defined by the signal END.
- the classification with anticipation is carried out during the calculation cycle, by means of the action of the multiplexer 144, which transmits the result of the parameter DATA, modulated by anticipation, as the address of the memory 1 18, the OUT output of this memory defines the classification value.
- the implementation of the improvement in FIG. 25 leads to a result similar, although improved, to that obtained by the implementation of the means illustrated in FIGS. 23a-23b, or 23c-23b.
- FIG. 21 b (already considered) in the case of a one-dimensional histogram (peaks Pi and P2) and in FIG. 24 (also already considered) in the case of a two-dimensional histogram (peaks Vi and V 2 ).
- peaks correspond to two classes in the classification of the histogram of CO values (figure 21 b) or DATA (A X ) and DATA (Ay) (figure 24).
- the one-dimensional module STN (1) of figure 20 or the two-dimensional module STN (2) of figure 23a-23b makes it possible to extract the dominant class represented by the peak P 2 or V1 respectively.
- peaks or classes occurs, for example, when analyzing an observed scene, in general, with a movement parameter MVT, since there can be a global or overall movement, for example traveling or zooming effect of the lens relative to the scene associated with particular movements, or an expansion from an expansion point (in particular in the case of a camcorder on board a moving car on a road).
- MVT movement parameter
- a simplified solution consists in determining the dominant class (peak P 2 or Vj) in a first STN module during a first sequence, in eliminating this class by inhibition, in determining the new dominant class, and (after said inhibition) on a second STN module during the following sequence, to inhibit this new dominant class, to determine, on a third STN module during a third sequence, a new dominant class, and so on until the last class.
- plate 12 comprising three modules STN, of the one-dimensional type STN (1), for example according to FIG. 20, processing the movement MVT and denoted ST1, ST2 and ST3; these modules all receive on their input a, the parameter Z; the output signal MVT1 of ST1, representative of the first dominant class of MVT, is sent at time ti, as inhibition signal, on a second input b, of inhibition, of ST2 established at time t 2 , while that the output signal MVT2 of the module ST2, representative of the second dominant class of MVT1, and the output signal MVT1 of the module ST1, representative of the first dominant class, are transmitted, as inhibition signals, on the second and third inhibition inputs b and c of ST3 which outputs the output signal MVT3 representative of the third dominant class of MVT.
- FIG. 27 A solution to increase the speed is illustrated in FIG. 27, in which the one-dimensional module STN, noted ST'O, operating during the to phase extracts all the classes from the MVT signal during the CALCULATION cycle and, during the RESULTAT cycle, extracts one after the other, the different classes, in decreasing quantitative order, and rewrites them in the associated STN modules.
- the first dominant class found is transferred to a second one-dimensional module STN, denoted ST'1.
- the classified values of the histogram memory 100 are then counted to create a new transferred NBPTS value and set to zero, which makes it possible to determine a new dominant class in this second block ST'1, which debits the signal MVT1, representative of the new dominant class of MVT.
- Another means of speeding up the determination of several classes appearing in the same sequence consists in locating these classes in order to determine levels of the quantity of pixels from the maximum value.
- This illustrative comparison makes it possible to better understand the process for extracting successive classes which will be explained below.
- the histogram comprising several peaks is stored in memory 100, while the classifier 1 18 constitutes a memory with a bit (0 or 1), the value 1 corresponding to an overrun by the value of the parameter, such as A, determined by TERMINAL A x , TERMINAL B x , TERMINAL A y , TERMINAL B y , according to the diagram:
- the classifier 1 18 outputs “1” as a signal 101 s when the value A or B of the parameter exceeds the expected threshold, such as RMAX / 2, at the output of the selection unit 1 152 (FIGS. 20, 23b or 25).
- the sorting and sequencing by decreasing values of RMAX of the content of the memory 100 to obtain the storage of the addresses ordered in decreasing RMAX is achievable by a processing software which comprises three successive cycles of initialization, calculation and updating results registers.
- the initialization cycle consists in resetting all the memories M 0 , Mi, M 2 , M 3 and the register Rc to zero.
- the calculation cycle essentially consists of determining and storing in the memory M 0 the values of the histogram determined by the module STN.
- the update cycle of the results registers comprises three successive sub-cycles A, B and C. It is during this cycle of updating the registers of results (illustrated in FIG. 28) that the extraction of the classes is carried out in the case where several classes appear during the same sequence (constituted for example by one or more frames or video images).
- the second sub-cycle B searches for and defines the different classes of the histogram contained in the memory MO. It comprises several stages, namely successively i.
- M A second step in which a loop successively unrolls all the values of p, from 0 to n inclusive, to execute, for each of these values, the three successive operations of block 501 of FIG.
- A (M0) pt; and c) retrieving the values noted a, b and c (or a, b %) adjacent to that of the class selected in the memory of the classes M2 at the level or address pt, namely pt + 1 (a, b ) for a unilinear histogram (two positions), pt x + 1 and pt y + 1 a, b ...
- test 502 revealed that there are no ("OR I") adjacent values, such as a, b ...; in this case an automaton 504 is activated to execute the following successive operations: a) the register Rc of the number of classes is incremented by "1", which is conventionally noted Rc ++ (operation 504a, FIG. 28); b) the associated threshold A / 2 is stored in the memory
- test 502 revealed that there is at least one adjacent value: in this case a PLC 503 is activated to execute the following successive operations: a) the logic operations 503a, 503b and 503c select the lowest value (non-zero) of class among the different values of the classes (a or b) adjacent a, b; b) this value is written in the class M2 memory at the address defined by the value pt (501 a, figure 28), distinguishing the case of writing a non-zero "a" from that of a "B" not zero
- the parameter p of the pointer is incremented by one unit (p ++ of operation 510) whatever the result of tests 502, 503a, 503b, 503c.
- the third sub-cycle C performs the updating of the STN modules required for the detection of at least two classes found during the same sequence, as illustrated in FIG. 28, this sub-cycle comprising several stages.
- the first step includes an operation 505 which sets the value of the register of the current classes Cl-Actu to “1”.
- the second step comprises the first operation of the loop 508, namely the sequence of all the Cl-Actu classes found, from 1 to the final class defined by the register Rc of the number of classes.
- the third step is performed by an automaton 507 which assigns to a new STN module a serial number equal to the value of the current class Cl-Actu, - determines the decision threshold corresponding to the content of the memory M3 at the address defined by the current class Cl-Actu, and initializes the pointer p to “0”.
- the fourth step is carried out by the loop 509, the first operation of which consists in carrying out a test
- operation 509a to determine whether the content which has just been transcribed in the memory M2, (operations 503d, 503e, 504c, FIG. 28) at the address p of the pointer, is equal or not to the value of Cl-Actu .
- the automaton resets the content of memory 1 18 of the STN module previously selected to zero (operation 509b), but if the content is equal to the value of Cl- Actu, the test 509c determines if the content of the histogram reading MO at the address p is greater than the value of the threshold and, in this case, the content of the memory 1 18 1 bit of the allocated STN module is put a "1" at address p; otherwise, this content is set to “0” (operations 509d and 509b respectively), iv
- the final step consists of incrementing the address of the pointer p by operation of “1” and, when this address exceeds n (operation 509f), incrementing of "1" Cl-
- an electronic arrangement can be implemented allowing updating of the sorting of the classes during the calculation cycle.
- the memories MO and M 1 are ready at the end of this calculation phase and the software for the display cycle of the results now only includes the sub-cycles B and C and only requires a duration which is very advantageous.
- the minimum number of treatments becomes equal to: n. (Rc + 1).
- Figure 28a shows the flowchart of Figure 28 applied as an integrated API for maximum optimization in execution speed.
- Sub-cycle A is integrated into the calculation phase and will be explained later.
- the flowchart starts at the start of the update cycle of the result registers, directly in the sub-cycle B, by searching for and defining the different classes of the histogram MO, associated with its sorting M 1, contained in Unit 600 and obtained by the previous calculation phase.
- This sub-cycle B has the same steps defined in Figure 28 except the following details:
- the third sub-cycle C updates the memory M2, serving as a classifier in the calculation cycle.
- This sub-cycle C comprises the same steps defined in FIG. 28 except for step 509b which is canceled, and steps 509c and 509d in FIG. 28 which are replaced by steps 509c 'and 509d' in FIG. 28a.
- the test 509c ' determines whether the RMAX content of the reading of the histogram 600 via the multiplexer 105 at the address p is greater than the threshold value and in this case, an operation (509d', FIG. 28a ) M2 memory is read through the multiplexer 105, a Val flag is hung and everything is rewritten at the same address.
- the classes Rc classes transcribed, the referral command Sel. ⁇ p is disabled (512, figure 28a) and the BUSY flag canceled (513, figure 28a). It may also be advantageous to know the number of pixels belonging to each of the classes found, in this case a memory M4 is dedicated to the NBPTS values of each of the classes described.
- the NBPTS value corresponding to the memory position M4 of address Cl-Actu is updated by adding the following two steps:
- NPTS barycenter
- This sub-cycle D begins with an initialization at zero of the memories MT and MS, and of the pointer p. -
- - follows a loop of calculation of POSMOY values for all the values of p from 0 to n inclusive.
- the following automaton verifies that the content of the temporary memory MT at the address Cl, is less than the content divided by two of the memory M4 at the address Cl. [(MT) c ⁇ ⁇ (M4 ) c ⁇ / 2] If the previous test is verified, then there is an accumulation of the histogram value RMAX at address p in the memory MT at address Cl.
- the pointer p represents the different fields of the DATA data.
- the previous calculation loop defines the part of POSMOY fields by the leftmost part of fields of the value p. It is therefore necessary to repeat the loop processing as many times as the number of fields defining the data DATA (A), and by permuting the fields of the value p.
- FIGS. 29a and 29b we have arranged side by side, on the one hand (FIG. 29a, which summarizes FIG. 23a-23b), the embodiment without sorting of the classes during the calculation cycle according to the method described above. above and, on the other hand (FIG. 29b), the embodiment with sorting of the classes during the calculation cycle.
- FIG. 29a which summarizes FIG. 23a-23b
- FIG. 29b the embodiment with sorting of the classes during the calculation cycle.
- 29b with integrated sorting corresponds to the assembly, in a single unit 600, of the memories MO and M1, this unit MO-1 containing opposite the RMAX and the POSRMAX for the values from 0 to p (included) of the pointer, the scheduling being carried out in descending order of RMAX from position 0 to position n (as indicated in the two preceding tables).
- RMAX content is also accessible by its position
- the output signal OUT of said unit is then directed to the input LECT- p from Tunity 602 via a multiplexer 654 controlled by the signal Sel. ⁇ p from Tunity 602. This operation is carried out during the test (operation 509c ', figure 28a) of the RMAX value at position p with a predefined threshold value.
- the memory M2 acts as a multiclass classifier, the signal Sel- ⁇ p then being disabled, the multiplexer 105 transmits the signal DATA (A) as an address to the memory M2 which outputs a signal 651 s accompanied by 'a Val signal, active in the case of a defined class.
- This signal 651 s passes through a demultiplexer 653 which is validated on its input En by the signal Val from the memory M2.
- This unit 653 outputs signals Cl-i to Cl k representative of the membership class of the input signal DATA (A).
- FIG. 30 The assembly 603 of the units 600 and 601 is illustrated in more detail in FIG. 30.
- Figure 29b is shown in more detail in Figure 30 which develops this Figure 29b, while Figures 31 and 32 illustrate (in more detail) portions of Figure 30, namely respectively the unit B0 which does not include RMAX and POSRMAX input, and any of the units B1, B2 ... Bn which have an RMAX input and a POSRMAX input.
- FIG. 32 it is possible to make memory block 600 understand only assemblies of the type illustrated in FIG. 32, by using for Unit B0 only a part only of the assembly of FIG. 32 (without the multiplexer blocks MUX input).
- the input of an IN signal 107s, corresponding to the new accumulation, is validated by a comparator 706 with P and Q inputs, the first receiving the new IN signal and the second the content of the RMAX register 704, if and only if P> Q, that is to say if the value of the new signal IN is greater than the RMAX already memorized, because, in this case, a new, higher RMAX value is entered.
- This registration of the new RMAX is carried out through the door ET 707 which produces, during the calculation cycle (CALCULATION signal), a WRITE registration signal applied to the recording entry En of register 704 of RMAX.
- the block B0 of FIG. 31 also includes an AND gate 710 with three inputs receiving the signal 706s from the comparator 706s output, the signal 709s from the comparator 709s output, after inversion, and the calculation signal CALCULATION of the flip-flop 71 1 after the first CALCULATION operation after the RESET signal has stopped.
- the four output signals RMAXout, POSRMAX or t- a 0 and TR 0 of the MO unit are applied as input to the next unit B1.
- FIG. 32 a unit Bi representative of one of the units B 1 to Bn has been illustrated.
- the unit Bi receives, on the one hand as the MO unit, the control signals IN, ADR-in, CALCULATION and INIT (from which RESET derives) and, on the other hand, RMAX in , (constituted by RMAX or ut from the unit previous), aj.i .TRj.-i and POSRMAX in , (consisting of POSRMAX or t of previous Tu nity), coming from anterior B unit (M 1 receiving the RMAX output signals 0U t- a 0 , TR 0 and POSRMAX or t of MO, while Unit Mi receives the corresponding output signals from Unit Mj.-i).
- the Bi unit first comprises two multiplexers 712j and 713i which make it possible to choose between two inputs for each of the registers RMAX 704i and POSRMAX 705j (analogous respectively to the registers RMAX 704 and POSRMAX 705 of FIG. 31).
- the multiplexer 712i chooses between the input IN and the value of the register RMAXj n of the upstream unit B (B0 or more generally Bi-1), while the multiplexer 713i chooses between the input ADR-in and the value of the content from the register POSRMAXjn of Unit B upstream.
- the SelMux selection signal for the multiplexers are validated by SelMux if the signal IN has a value greater than that contained in the RMAX register 704i (the comparison being carried out in the comparator 706i) and if the input signal a ⁇ - ⁇ , coming from unit B upstream, is equal to "0", the AND gate 714i outputting a signal 714s if the comparator 706i outputs a signal and at the same time a ⁇ -1, before inversion, is zero.
- the multiplexers 712j and 713 transmit RMAX in and POSRMAX in from unit B (namely B 0 if B
- Bi).
- the RMAX register 704i, the POSRMAX register 705i and the comparator 706i, on the one hand, the comparator 709i and the gate 707i; (whose roles will be specified below), on the other hand, are analogous to the corresponding units in Figure 31 without the index i.
- the OR gates 715i and ET 707i carry out the validation of the write signal WR for the registers RMAX 712j and POSRMAX 713i when SelMux or TRj.i is valid and at the same time the signal CALCULATION commands the writing.
- the AND gate 710 (analogous to the AND gate 710 in FIG. 31) outputs a signal TRj if simultaneously the output signal from the gate 715 is valid, the address signal ADR- in is not equal to the output signal from the register POSRMAX 705j and the flip-flop 71 1 j (analogous to the flip-flop 71 1 in FIG. 31) is triggered by the first signal WR after the RESET signal resulting from the INIT signal has stopped.
- the unit 708j with AND gate (analogous to unit 708 in FIG. 31) outputs an signal OUT constituted by the content of register RMAX 704i to the adder 107 (FIGS. 29b and 30) when the comparator 709 has found equality.
- Tunity Bj debits, in addition to the signal OUT, four signals in Tunity Bj + i downstream, namely RMAX 0U t, ai, TRj and PORSMAXout analogous to the corresponding signals debited by Tunity B 0 in Figure 31.
- the signals RMAX 0U t and POSRMAX or t (simply denoted RMAX, with an index of 1 to n, and POSRMAX, with an index from 1 to n) are sent on a block 617 "(namely 617--, 617 2 ...
- STN modules allow the calculation of MIN, MAX, NBPTS, RMAX, POSRMAX values (in particular) during the calculation phase.
- numerous logic gates must be implemented specifically for each of these calculations, hence a consumption and a chip surface occupation which can be relatively high. It is therefore proposed, as an alternative, that the calculation of all or part of these values be carried out by a more generic sequential calculation machine, of the microprocessor type for example.
- the memory 100 is essentially updated and it is only at the end of the CALCULATION phase (for example at the very start of the RESULT phase) that the automatic calculator for calculating said values, these once calculated being retrieved as above.
- Such an implementation if it can slightly lengthen the time taken to produce the results, however allows a significant gain on the number of logic gates and consumption.
- FIG. 33 a schematic illustration has been made of the implementation of STN modules, incorporating the method of extracting classes previously described in FIGS. 29 to 32.
- An STN 660 module of this type is controlled by a zone signal Z defining the analysis zone in the scene, and receives, during the sequence to, the movement signal MVT, this bi-linear type signal incorporates the information detection and speed of movement of the scene pixels.
- the demultiplexer 653 debits the classes CMVTO, CMVT ⁇ I CMv ⁇ 2 ⁇ t CMVT3 during the following sequence t1.
- Each classification signal controls an STN module of multiple extraction type of class also called multi-class STN module each receiving a bilinear signal of position x / y.
- the class signal CMVTO controls the module STN 661, the signal C M v ⁇ 2, the module STN 662, the signal C M v ⁇ 2, the module STN 663, and so on, taking into account the number of classes found. There are as many STN modules recruited as there are classes found.
- the signals of class ZjMVT j of the STN modules receiving the signal x / y define all the zones associated by categories of movement.
- This implementation in two sequences, reveals, in the first sequence, a processing of a signal in the time domain TD, then in the second sequence, a processing in the spatial domain SD from the results of the previous classification resulting from previous processing in the TD time domain.
- FIGS. 21 a, 21 b and 24 which illustrate the obtaining of the information necessary concerning the nature of the object OB observed and the position thereof in the context of the use of a single representative signal, namely the luminance L
- FIGS. 34 et seq. improvements making it possible to determining the relative location of two objects, implementing trees representing the relative positions of the barycenters of the different zones, implementing three color components and finally memorizing and recognizing the objects perceived with invariance in translation, in size and in rotation.
- the average positions are essentially implemented, of the parameters or dimensions applied to the inputs of the units STN (1), STN (2) and STN (3) constituting the unit 8 of FIGS. 17 and 18.
- an STN module (2) of the type illustrated in Figures 23a-23b (possibly with the modifications of Figures 23c and / or 25) , and two orientation units p ⁇ and p ⁇ these two orientation units receiving the coordinates x and y and an orientation or rotation angle, namely ⁇ for orientation unit 150 and ⁇ for orientation unit 151 , so as to rotate the coordinate axes, from the initial position determined by the x and y axes to a derived position defined by the angles ⁇ and ⁇ , or respectively the slopes p ⁇ and p ⁇ , these orientations and slopes being represented in Figure 34a discussed below.
- the module STN (2) of FIG. 34 therefore receives, as input data, not DATA (A X ) and DATA (A y ), but rather DATA (A p ⁇ ) and DATA (A pP ).
- the processing carried out in the STN module (2) is controlled by a program register 152.
- the signals collected on the bus 1 14 ultimately represent not only the Zj but especially the barycenters , namely, BarZj therefore dependent on the slopes p ⁇ and P ⁇ , that is to say the angles ⁇ and ⁇ respectively.
- FIG. 34a two particular values of the angle ⁇ have been illustrated, namely ⁇ * ⁇ and ⁇ 2 , and two straight lines of slopes p ⁇ -i and p ⁇ 2 relative to the direction of Tax of x, representing the processing in The unit illustrated in FIG. 34.
- the slopes p ⁇ 9 and p ⁇ 10 are perpendicular respectively to the slopes p ⁇ 1 and p ⁇ 2 so as to produce Cartesian coordinates of slopes p ⁇ 1 and p ⁇ 9 for the angle ⁇ 1 and p ⁇ 2 and p ⁇ 2 for the angle ⁇ 2.
- T object OB is thus defined by its barycenter BarZo (the index “0” indicates that it is the first barycenter or barycenter of origin of the tree discussed below) in the diamond referenced 160 (constituting the initial zone Z 0 ) defined by the lines of coordinates a, b, c and d delimiting this object.
- FIG. 36 which illustrates a variant of a portion of FIG. 23a, namely the unit 101 thereof, there are the comparators 1 10a and 1 1 1 a, on the one hand, and 1 10b and 1 1 1 b, on the other hand, as well as AND gate 1 12a (which receives the direct outputs of comparator 1 10a and inverted from comparator 1 1 1 a) and AND gate 1 12b (which receives direct outputs of comparator 1 10b and inverted of comparator 1 1 1 b). Also found in Figure 36 the AND gate 131 receiving the outputs of AND gates 1 12a and 1 12b.
- FIG. 36 like the corresponding part of FIG. 23a, relates to a classification subset (1 01 in FIG.
- FIG. 36 which illustrates a variant of FIG. 23a, we see that the assembly illustrated in this FIG. 36, realizes, at the output of the gate 131, the same Boolean operation AND as the assembly of FIG. 23a, the output signal 131 s being equal to that indicated above. But, if we also consider the OR gate 132, which the assembly includes from FIG. 36, a wider field of application of the output signal from gate 132 is obtained compared to that of gate 131, the output signal 132s also extends to (ay.by) + (ax. This difference is illustrated by comparing the fields of FIG. 37, which corresponds to the assembly according to FIG.
- the signal 101 s is constituted either by the signal 131 s, or by the signal 132s; in the first case, the domain of 101 s, namely that of 131 is that illustrated in FIG. 37; in the second case, the domain 101 s, namely that of 132s, is that illustrated in gray Z or in FIG. 38, this domain being comprised either between Terminal A x and Terminal B x or between Terminal A y and Terminal B y , by simply eliminating the rectangles left blank in FIG. 37, in which the signal is simultaneously less than Borne Ax, as regards its x coordinate, to Borne Ay, as regards its y coordinate, or even greater than both at Borne Bx, with regard to its x coordinate and at Borne By with regard to the y coordinate.
- the multiplexer 133 of FIG. 36 makes it possible to choose between the two solutions, namely between the output 131 s of the AND gate 131 (domain in gray in FIG. 37) and the output 132s of the OR gate 132 (domain in gray on FIG. 38) depending on the case that you wish to process and this under the control of the signal 134 for controlling this multiplexer (coming from TAPI).
- FIG. 35 which corresponds to the illustration of the single AND gate 131, numerous orientations of narrow bands or "lines" were used, delimited by Terminal A and Terminal B, for different orientations or slopes, by example p ⁇ 30 for the Zr30 band, the BarZo barycenter of the Zr zone being defined by the crossing point of the ZrO bands (slope 0 °) to Zr 170 (slope 170 °).
- the device After a first sequence for determining a first Z 0 and its BarZo barycenter, the device starts a second sequence according to FIG. 39, in which the shaded portion corresponds to the shaded portion of FIG. 38 when the axes instead to be orthogonal, form an angle (acute) defined by ⁇ 1 and ⁇ 2, while the terminals, instead of being defined by Borne Ax, Borne Bx and Borne Ay, Borne By are representative of the position of the BarZo barycenter (included in Zo) by the slopes p ⁇ 1 and p ⁇ 2.
- terminals Terminal Ap ⁇ 1 and Terminal Bp ⁇ 1 are represented for the slope p ⁇ 1 and the terminals Ap ⁇ 2 and Bp ⁇ 2 for the slope p ⁇ 2, the zone Zr1 being that seen by the module STN (2) of FIG. 34 as signal 101 s derived from the output signal 132s from OR gate 132.
- the sectoring is refined by dividing the sectoral zone Z 21 , determined during the third sequence into several sectoral sub-zones Zra, Zrb, Zrc Zrd by the addition of an STN module ( 2) by sub-zone operating in the same way as the STN module (2) implemented according to FIG. 39a; therefore the slopes p ⁇ 1 and p ⁇ 3 are replaced by intermediate slopes limiting the aforementioned sub-areas.
- one of the sub-zone STN modules (2) has recovered BarZ-i, namely the Zrc sub-zone (FIG. 39c); the bisector of this particular subzone Zrc, defined by two very similar slopes, accurately determines the slope p ⁇ 3 illustrated in Figure 39c.
- the distance between the axes of two bands perpendicular to p ⁇ 4 passing through BarZ 0 and BarZ * - and perpendicular to the band BarZ 0 -BarZ ⁇ represents the distance p3 between these two barycenters (that corresponding to the angle ⁇ 3); the angle ⁇ 3 of the slope p ⁇ 3 and the distance p3 are the two polar coordinates of this barycenter, such as BarZi relative to the barycenter BarZ 0 taken as the origin of the coordinates, and an axis (not illustrated), for example parallel to the lower edge of the Zo zone ( Figures 39a and 39b).
- BarZ-i which is the barycenter of zone Zi included in zone Zo
- BarZ 0 representing the "father” from which the "son” BarZi comes
- ⁇ 3 and p3 being the polar coordinates of the son in relation to the father.
- This description in four sequences shows the method for determining the p ⁇ link between two barycenters from a dynamic recruitment of STN (2) modules during sequences two to four.
- a unit M (0) is illustrated, constituted by a pair of STN modules, the first module 296 being a single-line module STN (1) with an input parameter DATA (A), simply denoted A, in 296a, which is processed by a desired function FoG 'to supply a group of analysis output registers, noted reg 296b, in which accumulate values representing, in the form of a histogram, the statistical distribution of the parameter DATA (A ); the output signal, at 296c, from the STN module (1) 296 is a CA 350 classification signal.
- the second module 297 is a bilinear module STN (2) with two input parameters, namely the x and y coordinates applied in 297a, which are processed by a desired analysis output function FoG ", feeds a group of analysis output registers noted reg 297b classifying in histograms the two parameters x and y; an output signal from the module STN (2 ) 297 is constituted by zone Z 0 and it is fed back, as signal 297s, to an auxiliary input 296d of the first module 296, while another output signal from register 297b is constituted by BarZ 0 .
- the STN (1) 296 and STN (2) 297 modules are advantageously constituted as illustrated respectively in FIG. 20 and in FIG. 23a-23b (possibly with the variants of FIGS. 23c, 25 and / or 36), in order to obtain finally, as explained with reference to these figures, the signals d e output Z 0 and BarZo.
- FIG. 42 which represents a set of STN modules capable of executing the successive operations described above with reference to FIGS. 39a, 39b and 39c
- Unit M (0) of FIG. 41 constituted by a couple of blocks STN (1) 296 and STN (2) 297, the input signals denoted A, x and y and the output signals Z 0 and BarZ 0
- this first unit M (0) of two modules STN (1) 296, and STN (2) 297 determining Z 0 and its barycenter BarZo are associated,
- Next unit (not shown) with two STNs determines Z and BarZ 2 from Zi and parameters C, x and y; the following units, also not shown, determine Z 2 and BarZ, etc .; and
- a set M (2) of STN modules (2), representing dynamic recruitment noted 300, 301, 302, 303 ... 307 receiving as input a pair of slopes pO and p1, p1 and p2, p2 and p3, p3 and p4, ... p6 and p7 to determine respectively ZrO, Zr1, Zr2, Zr3 ...
- Zr7 (illustrated in Figure 43a) from Z 0 and BarZ 0 from Unit M (0), and BarZ-i, coming from unit M (1), the three values of which are received as input by each of the units 300 to 307;
- the set M (2) is accompanied by similar sets, not shown, each receiving Zo and BarZo from the set M (2) and in addition BarZ 2 , for the set of type M (2) of rank just after the set M (2) illustrated, BarZ 3 for the type M (2) unit of rank just after and so on, each of the STN (2) of each of these sets M (2) not illustrated receive a pair of slopes at the STN (2) 300 to 307 illustrated: these successive M (2) type units determine at the output of their modules similar to modules 300 to 307 illustrated, signals defining sectors of type ZrO to Zr7, but relating to Zi, Z 2 etc ... and no longer at Z 0 .
- the classification signal 350 developed in module 296, arrives in module 297 operating in two-dimensional mode with the x and y coordinates (defining the pixel of the video image) as input parameters (in addition to the signal 350).
- the result of the histogram calculation in module 297 controls the FoG "automatic classification function with anticipation when the signal 350 arrives, that is to say during the END phase at the end of the sequence.
- the classification in the register reg 297b of the module 297 makes it possible (as explained with reference to FIG. 34) to determine, on the one hand, Zo by means of the update of
- the first sequence therefore makes it possible to determine Z 0 and BarZ 0 .
- the modules STN (2) 300 to 307 (of which only certain modules have been illustrated), programmed to receive as input a pair of slopes oriented from 22 ° 30 'to 22 ° 30' between 0 ° (slope pO) and 157 ° 30 '(slope p7) - each slope of index j representing an angle of 22.5d - perceive BarZo inside the Zo zone according to their degree of orientation ( Figure 43a).
- the histogram calculations in the STN (2) modules of type 300 to 307 make it possible to update the classification terminals Terminal A, Terminal B and therefore to define oriented zones called search zones ZrO to Zr7, as illustrated in FIG. 43a for the zones Zr1, Zr2, Zr3 ... Zr7 determined by the blocks STN (2) 300, 301, 302, 303 .. 307, respectively.
- FIG. 43a in fact represents the cumulation of the oriented zones Zr defined by the modules 300 to 307 during the second sequence.
- the second sequence therefore makes it possible to position the segmentation of the plan around BarZo.
- a characteristic appears linked to the first signal of classification 350, namely a subzone characteristic (not shown) generating a BarZ-i barycenter (illustrated in FIGS. 43b and 43c which correspond to FIGS. 39b and 39c) which is used to determine the modulus p and the angle ⁇ orientation.
- the STN module module (2) 300 to 307 receives, in addition to Zo and BarZo, a third signal BarZi while continuing to receive the slopes p 0 , pi. P7; pi equivalent to the difference between the two classification bounds, namely, from these inputs, one of the STN modules of M (2) - for example 303 - mark (figure 43b) an additional barycenter BarZi and becomes at the end of the sequence , the STN module selected to operate in the following sequence, namely the fourth sequence.
- FIGS. 44a to 44c and 47a to 47d represent the sets M of STN modules
- FIGS. 45 and 48 the corresponding relative arrangements of the zones
- FIGS. 46 and 49 the trees of the BarZo to BarZ 3 o barycenters these areas.
- the Data parameter (A), denoted A determines, thanks to the module M (0) a zone Zo and its barycenter BarZ 0 .
- the signal Zo produced by M (0) in the phase Co, acts, on the one hand, on a module M (1) for locating the subzone Z- io triggered by a Data item (B), denoted B, in order to determine Z 0 and its barycenter BarZio and, on the other hand, by a module M (2) operating as described above with reference to FIGS. 42, 43a, 43b and 44c and grouping the different orientations p 0 , Pi ... Pm, in order to position these orientations around the BarZo barycenter.
- the module M (2) receives BarZio (in addition BarZ 0 ) which has just been determined by M (1) and searches for the sector which contains BarZio.
- a module STN (2) denoted SBi (figure 44e) among the STN (2) modules of the set M (2) is selected, which determines the value of i corresponding to the angles of the bisector of the subsector selected by BarZio.
- the dynamic grouping of M (2) disappears in favor of the selected bilinear STN (2) module SBi.
- the registers 1070, 1071 and 1072 respectively containing p, ⁇ and C ⁇ are updated.
- FIG. 46 illustrates the parentage of the barycenters BarZ 0 - »BarZio -» BarZ o -> BarZ 3 o, with the polar coordinates p, ⁇ of a downstream barycenter (wires) compared to an upstream barycenter ( father); for example BarZ 2 o is the son of BarZio.
- the tree in FIG. 46 illustrates the arrangement with successive interlocking of the corresponding zones Zo, Z10, Z 2 o, Z 30 of FIG. 45, Z 30 being housed in Z o which is housed in Z10, itself housed in the inside Z 0 .
- FIGS. 47a and 47b are identical respectively to FIGS. 44a and 44b so as to also first obtain Zo and then BarZo, Z10 and BarZio at the first and second times to and t1 respectively, since Z 10 is contained in Z 0 , as in the case of FIG. 45, on the other hand, the rest of the processing is different, since the zone Z 2 o in this second example of FIG. 48 is not contained in the zone Z 10 , but is distinct from this, however, being contained in the zone Zo.
- the module M (1 a) similar to the module M (1), but with C as input instead of B no longer receives Z10 as in the case of FIG. 44c ( since now Z 2 o is included in Z 0 , but not in Z10) and determines Z20 and BarZ2o, while the STN module M (2a) determines the p corresponding to an area
- Z 0 , Z 10 , Z o and Z 30 are thus successively determined and a tree with branches or branches is obtained from BarZo.
- a tree with branches or branches is obtained from BarZo.
- the letters A, B, C correspond to the different curves a ', b', c 'of figure 15.
- FIG. 50 illustrates such an intermediate case, in which there is both nesting of at least one zone in another, for example from Z21 in Z12 (Z12 - »Z21), and positions separated from certain zones, such that ⁇ , Z 12 as well as Z 22 and Z 21 , all these zones being contained in the zone Zo, as can also be seen on the tree in FIG. 50a.
- the first processing steps have been shown during refocusing, or more generally the increase in resolution, with the abscissa, from left to right, the durations of several successive sequences or groups of sequences to, ti, t. 2 and t 3 , and on the ordinate, from top to bottom, the succession of treatments carried out on each sequence or groups of sequences.
- the first row of FIG. 50 illustrates, from left to right, the successive appearance of zones due to the increase in resolution as a function of the parameters: first Dif (difference between the initial sequence at maximum resolution r max during To and the subsequent sequence at minimum resolution r min during T 2 or to (figure 3), then a, b .... (figures 14 and 15).
- the second row in FIG. 50 illustrates the first three of the successive treatments carried out on the first perceived area, namely Zo.
- the third row of FIG. 50 illustrates the first two of the successive treatments carried out on the two sub-zones determined after the first zone Zo, namely the two sub-zones Zn and Z ⁇ 2 .
- the fourth row of FIG. 50 illustrates the first of the successive treatments carried out on the two sub-zones determined subsequently, namely Z 2 ⁇ and Z 22 , Z 2 ⁇ being included in
- the first column corresponding to the initialization at time to, represents the appearance of the first zone Z 0 following the first increase in resolution (after its abrupt reduction), during the phase Ti of FIG. 3) , and the determination of the BarZo barycenter of this Zo zone.
- the second column represents the two simultaneous treatments on the following sequence at time ti and illustrates the continuation of the treatments of the zone Zo, namely: - on the one hand (on its first row), the appearance of the subzones Zn and Z ⁇ 2 , inside the Zo zone, with their respective barycenters BarZn and BarZ ⁇ , and
- the third column represents the three simultaneous treatments carried out on the same sequence, that at time t 2 , namely:
- the fourth column illustrates the continuation of the same type of treatment, namely:
- FIG. 50a the tree constructed from the treatments carried out in FIG. 50 is shown.
- FIG. 50a the tree constructed from the treatments carried out in FIG. 50 is shown.
- FIG. 50a the tree constructed from the treatments carried out in FIG. 50 is shown.
- the second parameter has enabled the appearance of the two sub-zones Zn and Z 12 (included in the zone Zo), with their respective barycenter BarZn and BarZi 2 , the father / son relations between the father BarZo and the sons BarZn and BarZ ⁇ 2 being materialized by vector rays 50 and 51 respectively, with the angles an and ai 2 and the modules pu and P12, and
- the third parameter ba allows two new subzones Z 2 ⁇ and Z 22 to appear, the first included in Z 12 (and therefore in Zo) and the second in Zo only, with their respective barycenter BarZ 2 ⁇ and BarZ 22 and the father ⁇ son relationships 53 and 52 from fathers BarZ ⁇ 2 and BarZo respectively, as well as the corresponding ⁇ and p, namely ⁇ 2 ⁇ , ⁇ 22 and p 2 ⁇ and p 22 .
- FIG. 51 we partially illustrate an application of the selection of multiple zones developed in FIGS. 42 to 50, from spatio-temporal parameters obtained during the refocusing, or more generally from the restoration of the resolution. from its minimum value to its maximum value, as indicated above with reference to Figures 1 to 16; Figure 51 corresponds to periods T2 and T3 of Figure 3.
- This difference signal is introduced on the input Dif of the first one-dimensional module STN (1) 90 of the module M ( 0) which also includes a second two-dimensional module STN (2) 91.
- the reference zone Zo has been identified, as well as its barycenter BarZo, by the module STN (2) 91 of l 'set M (0).
- t1, t2 appear in turn the series of spatio-temporal signals a, b, c ... of Figure 15.
- FIG. 52 illustrates an improvement made to the assembly of FIG. 51 to ensure the locking of the information to remedy the fact that the parameters a, b, c ... are temporary.
- each of the sub-zones is locked; the modification consists in adding, in the embodiment of FIG.
- additional three-dimensional modules STN (3) processing the color components L, T, S and referenced 93a, 94a, 95a, 96a and 97a whose input is connected to the classification D output of the corresponding STN (2) modules referenced 93, 94, 95, 96 and 97.
- Locking on the dominant color is achieved by analyzing the dominant color of each of the sub-zones by the module STN (3) 93a to STN (3) 97a having as inputs L, T, S, that is to say the three color components, and as command the zone Zi for each of these sub-zones; at the end of a sequence, the histogram of components L, T, S makes it possible to define classification limits in L, T, S and therefore to output a classification signal 101 s (FIG. 23a-23b, or possibly 23c, 25) corresponding to the dominant color; locking is obtained by connecting this classification signal, corresponding to the dominant color, to the input of the corresponding STN module which calculates the associated area.
- the pairs of modules 93-93a, 94-94a, 95-95a, 96-96a and 97-97a are symbolically represented on the right with position (color x, y coordinates) and color (component) entries.
- FIG. 53 explain how, still within the framework of the invention, it is possible to characterize the shape of the localized object, in particular with a view to being able to check whether a new shape is analogous to a form already determined, the characterization of the perceived object with regard to its form being carried out in two stages: it is first of all a question of coding the information on Tobject concerning its characteristics, then of checking whether a new object corresponds to the characteristics of a previous object, already perceived and coded or if it is new.
- FIG. 53 we have illustrated in the left column the bilinear modules m (0), m (1), m (2) ... m (j) ... homologous to the bilinear module SBi of FIG. 44a which expresses the result of dynamic recruitment in four stages according to this figure; the inputs and outputs of the modules m (0) ... are the same as those of the module Sbi; in particular the outputs of these modules m (0) ... debit all of the aforementioned couples ⁇ , p perceived in the zone Zo (with an index 0, 1, 2 ... j ...) which are applied to a double bus BB which serves as an input to a set MM which constitutes the essential of the means for determining a representation of the characteristics of the perceived object.
- a double bus BB which serves as an input to a set MM which constitutes the essential of the means for determining a representation of the characteristics of the perceived object.
- the set MM consists of several sub-sets MM (0), MM (1), MM (2) ... MM (i) ..., each of these sets comprising two STN modules, the first of two-dimensional type referenced 801, 802, ... 80i ..., which receives as input, from the double bus BB ⁇ , p, and the second of the one-dimensional type referenced 810, 81 1, 812, ... 81 î ... , which receives a signal from LABEL whose role will be explained below.
- the first STN (2) module 800, 801, 802 ... 80i debits in the second STN (2) 810, 81 1, 812 ...
- the indication of the order (0, 1, 2 ... i), framed by a square, defined by the first STN module (2) of each subset constitutes the entry of the second STN module (1) of all sub-assemblies by bus L from LABEL.
- any classifier which is valid apart from that defined by the pointer indicating a "last acquisition" it is the learning phase which is activated and which consists in that the MM subset of last acquisition, which has its classifiers still valid, forces the validation of its associated LABEL on the bus L of LABEL and the histogram of the first module (from 800 to 80i) of the MM subset of last acquisition 820-82i stores the pair ⁇ , p, value 820 to 82i associated.
- Such an operation is repeated for any new pair ⁇ , p during the period T3 of passage from the minimum resolution to the maximum resolution (for example during the entire refocusing phase);
- the RMAX and POSRMAX values of each of the affected STN (1) unilinear modules are updated and a general test on all the unilinear modules (STN 1) already used makes it possible to find the largest RMAX value and its associated POSRMAX register which is read and therefore corresponds to the most probable LABEL value, as explained above with regard to the TAPI calculation software.
- the content of the associated bilinear histogram is then read and serves as anticipation of decision support as a dynamic recruiting agent during action, the treatment previously described in four phases being reduced to three (as indicated above) with reference to FIG. 42), which accelerates convergence for the recognition of the object.
- the calculation phase stops and the analysis phase of the results begins.
- This is characterized by reading the histogram memories of the bilinear STN (2) and unilinear STN (1) modules, comparing the values read with a reference threshold, namely RMAX / 2, and transcribing into the memory 1 associated bit.
- the result of the 1-bit memory of the classifier of the bi-linear module is extended in order to accept adjacent ⁇ , p pairs during future tests.
- the final value of LABEL corresponds to the name of Tobject included in the Zo zone. This LABEL value corresponds to the so-called episodic memorization.
- FIGS. 54, 55a, 55b, 55c and 55d The realization of the resolution of the invariance in size is illustrated in FIGS. 54, 55a, 55b, 55c and 55d.
- FIG. 54 represents an assembly constituted by the same electronic elements as the assembly illustrated in FIG. 42, but the processing is carried out on different input parameters. The basic assembly has been described with reference to FIG.
- the input parameters p and logarithm LD of the derived distance are substituted by Unit 400 of the initial parameter Dis distance, coming from sequencer 450, itself coming from external Unit 25 (not explained).
- the assembly in FIG. 54 uses a unit 450 corresponding to a sequencer, which outputs the various potential values LD and p during the time of the sequence and includes three units, namely 400, 451 and 440.
- the unit 400 consists of a logarithm calculator receiving the distance Dis at the input and calculating LD, the logarithm of Dis which constitutes the output LD;
- Tunity 451 is a classification unit debiting a binary Barpi signal, equivalent to the BarZi signal used previously, which is validated when the signals LD and p from the sequencer 450 are equal and the LDI and pi values presented;
- Tdoc 440 which corresponds to the M2 dynamic recruitment module defined in Figure 42, with instead:.
- This unit 420 outputs two signals 420 0s and 420 ⁇ s corresponding to the definition of a sector ZrO by the transformations of angle axes ⁇ 0 and ⁇ , the following units 421 to 427 ensure the sectoring of the plane defined by the variables
- FIG. 55d illustrates the result after processing the STN dynamic recruitment modules during three consecutive sequences illustrated in FIG. 55a, 55b and 55c
- Unit 401 which corresponds to Unit 150 of FIG. 34, receives the sequencing LD and p coming from Unit 450 and, on its input of command, ⁇ i corresponding to pr, finally Unit 402 is a one-dimensional module STN (1) which receives as input p
- the input value Dis which is equal to the distance from the object to sensor 2 (such as a camcorder), or 2 'such as a radar ( Figures 1, 2, 4a, 4b, 4c, 5, 17, 18), measured externally 25 for example by means of a laser associated with said sensor, is converted, as indicated previously, into its logarithmic value, the two parameters Dis and p being the essential input parameters of the assembly of FIG. 54.
- the distance Dis can be measured for example by binocular convergence, by use of parallax, using knowledge of the actual size of the object perceived.
- the dynamic recruitment module 440 there are, as illustrated in FIG. 42, groups of STN modules coming into play successively; as in the case illustrated in figure 43a, corresponding to what one obtains with the assembly of figure 42, one obtains with the assembly of figure 54 the sectorization illustrated in figure 55a, on which one finds the zones or sectors such as Zro, Zr, Zr 2 , Zr3 ... Zr 7 .
- the BarZo barycenter instead of using the BarZo barycenter, it is around the point referenced po, LDo defined by Barp 0 that the sectors Zr 0 to Zr 7 are arranged.
- Figures 55b and 55c correspond to Figures 43b and 43c respectively, but this time also, the barycenters such as BarZo and BarZi are replaced by the points referenced po, LD 0 and pi, LDi respectively Barp 0 and Barpi, always in zone Zr 3 .
- Ton compares Figures 43b and 43c with Figures 55b and 55c, we see that the axes of coordinates x and y have been replaced by the axes of coordinates p and LD.
- the calculated values ⁇ LD, ⁇ , Cp ' are written in the registers 1070, 1071 and 1072 respectively, and only a single-line STN module (1) remains, represented in FIG. 55d.
- the sectoring, in the case of FIGS. 54 to 55c is therefore made from a first measurement po, LDo and from a second measurement made at a distance different from the first and providing pi, LDi represented, as poLD 0 , in FIG. 55b, an orientation being defined by a straight line p ⁇ perpendicular to the direction of the pair of points po, LDo and pi, LDi, as illustrated in FIG.
- Figure 57 illustrates an improvement in the assembly of Figure 55d.
- the unit 400 of FIG. 55d sequencing by the signals LD and p with its input Dis and its output LDj, namely the logarithm of the distance Dis.
- This value LD constitutes with pi the parameters of the Unit 410 which comprises all the units 401 and 402 of FIG. 55d and, moreover, the improvements constituted by the units 403, 404, 405, 406, 407 and 408.
- the output of the assembly 410 is constituted by the values Cp 'and LD' endowed with the index i, while the assembly of the units 401 and 402 in FIG. 55d only included the output Cp '.
- a multiplexer 403 receives, on the one hand LDj, coming from the unit 400 and, on the other hand, the output of the summator 406 which outputs, as explained below, the calculated signal LD 'and chooses one of these two input signals. If the value of the logarithm of the distance, namely LDj, is valid, the multiplexer 403 transmits this value, otherwise it transmits the calculated value LD ', this output signal being applied to a unit 401 analogous to Unit 401, of FIG. 55d ; the choice between these two values is controlled by a control signal 403c, activated at least during the learning phase, coming from the unit 408 which receives the input signal LDj, and which analyzes its validity.
- the calculation of the value LD ' is carried out when the external value LDj is valid, Unit 407 comparing the value of incoming LDj with that of LD' calculated previously; this unit 407 controls the gain of two amplifiers K1 404 and K2 405 which is equal to -tg ⁇ for the first and 1 / cos ⁇ for the second.
- a summator 406 adds the two values -pi x tg ⁇ j and Cp '/ cos ⁇ j, the sum of these two values being equal to LD' (in Figure 57, we added the index i for the parameters p, ⁇ and LD ' , on the one hand, and Cp ', on the other hand).
- the unit 407 corrects the coefficients Ki and K 2 of the amplifiers in order to reduce the difference between the calculated value LD'j and the input value LDj.
- the main interest of the device of FIG. 57 compared to the device of FIG. 55d, is that it can possibly dispense with a measurement of the distance Dis and therefore a determination of LDj from this distance, once that Tobject has been spotted and learned and also to be able to size a new object among the known objects without having to measure the distance. There are therefore as many Cp 'calculations as there are stored modules.
- FIG. 58 corresponds to FIG. 47d, but with the integration of the modules 410 of FIG. 57 which ensure the calculation of invariance in size. All the modules M delivering p or ⁇ in FIG. 47d are followed by modules 410 associates debiting Cp 'and LD'.
- the LD 'values allow to appreciate a volume aspect by their relative variations, as well as a distance by a calculation of their average values.
- the assembly of this figure 59 comprises first of all a bilinear module STN (2) 800 receiving, on the one hand, Cp 'and, on the other hand ⁇ ' and a set 900, itself constituted by another module unilinear STN (1) 901, a counter 902, a multiplexer 903 and a summator 904.
- the role of the assembly of figure 59 is to apply a double correction to the couple p, ⁇ : p is transformed into Cp ', as indicated previously , while ⁇ is transformed into ⁇ 'by Unit 900.
- the counter 902 starts at ⁇ and increments on 2 ⁇ radians, while the multiplexer 903 controls, by a signal initiating a search phase, the transfer of the value of the counter 902 on a summator 904 modulo 2 ⁇ radians, which calculates a value ⁇ '.
- the classification in the STN 800 module commands the validation of the STN 901 module and the current value of the counter is stored in the histogram of this last module.
- the analysis of the histogram of the STN 901 module gives, as a result, the content of its POSRMAX corresponding to the mean value ⁇ '.
- the multiplexer 903 is switched and this new calculated value is transmitted to the summator 904. This makes it possible to recognize Tobject, the value ⁇ which leaves the assembly 900 indicating the rotation of T object.
- Figure 60a a portion of the previous figure 58 debiting the triplets Cp ', ⁇ , LD' in the block MM of Figure 53 improved by incorporating the 'integration of the invariance calculation in size and rotation; indeed (Figure 60b), in the sub-module M (0) of Figure 53, we integrated a unilinear module STN (1) 920 which calculates the rotation ⁇ and a unilinear module STN (1) 921, which calculates the distance LD 'average, the module 800 corresponding to the STN module 800 of Figures 53 and 59, while the module 810 of Figure 60b corresponds to the STN module 810 of Figure 53.
- FIG. 61 illustrates, in a schematic manner, the recognition of objects by increasing the resolution and rotation of the sensor 2-5 or 2-5 ′, by virtue of the generation of an egocentric reference frame relating to and this observer, by memorization the organization of the shapes of objects OB1 and OB2, the angles D1 and D2 being equal.
- the rotation of the sensor in fact first of all decreases the spatial resolution (period Ti in FIG. 3); at the appearance during rotation of a relatively high value of the signal Dif, an order to stop the rotation is issued by means (not shown) and the process of Gaussian increase in spatial resolution described above begins (period T 3 ) of Figure 3); moreover, the angle of rotation of the sensor is determined and stored.
- This rotation can be carried out along two axes and this in principle requires the memorization of two angles, as well as the order of the rotations along the two axes.
- the senor perceives a part of the scene in the environment which depends on the position and the focal length thereof, which means that the notion of "object” also depends on these two parameters; as well as the sensor can transform into frames or successive sequences either a whole person, or his face, or a part of his face, T “object” being each time different.
- the method and the device according to the invention make it possible to obtain, at the end account, the results indicated below, from which arise a number of applications.
- one or more representative histograms the weight that is to say the quantities, of the different levels of the data contained in said derivative signal M 'and therefore in said initial digital signal (in particular the initial digital video signal), which makes it possible, first of all, to determine the successive zones corresponding to the details appearing in decreasing order of importance in said desired signal and calculate the positions of these zones, with their barycenter determined from these weights in their order of appearance, therefore of decreasing importance, and then advantageously to make a hierarchical representation in the form of a tree ( Figures 44c-44e to 49) with the father - »son relationships.
- an angular sectoring (FIGS. 34 to 43c), by passing from the Cartesian coordinates x and y to the polar coordinates p and ⁇ (or p) makes it possible to determine relatively wide angular zones centered on the barycenter of the first zone appeared (the one with the most notable feature), then narrower angular subzones covering the relatively wide area, inside which a new barycenter has been found, these angular subzones also being centered on the barycenter of said first zone, and to determine a first relation of son with respect to the father, defined by p in an absolute manner, independently of any rotation.
- the description of an object is given by its characteristics appearing in decreasing order of importance, by means of four successive sequences making it possible to calculate, by changing the coordinates of x and y into p and ⁇ , elements relating to this object in the direct sense, that is to say by defining said object by its characteristics first the most important, then the least important; at the end of this four-step process, we determined not only the tree with the aforementioned father ⁇ son filiations, but also a label or label defining the subject examined.
- the sensor stops in its rotational movement, the details appear, due to a refocusing, in decreasing order of importance and simultaneously this information concerning the details is compared with the information already in memory in the form of a label relating to examined objects. previously, which makes it possible to accelerate the examination of the new object by possibly skipping steps in the recognition of the new object and its comparison with an already known object.
- the method according to the invention it is possible to start from an area Z 0 representative of the scene and of its BarZo barycenter to determine the localization of the various objects of the scene, with their ⁇ , p associated relatively to BarZo, and to memorize a scene with its objects, in the form of a label which will then make it possible to determine if a new observed scene is constituted or not by the labeled initial scene.
- Such a procedure can be used in particular in navigation, the principles of which will be explained below in the context of the applications of the invention.
- the recognition of a face, and therefore of a person is one of the preferred applications of the invention.
- the perception of a face and the recovery of its associated labels make it possible to compare the labels of a new face with the labels of the faces already examined.
- the memorization of the face labels as and when they are examined can be carried out in a calculator, a smart card, a magnetic identity card or a bar code, etc. From a stock of such information of face labels, it is possible to check whether a new face, which has just been examined by the method and the device according to the invention, corresponds to already memorized labels.
- the label or labels of a person can also be introduced into an identity card, a driving license, a registration card, etc. ;
- a similar application can be applied to mobile telephones comprising a visual sensor; during its first use by an authentic purchaser, the telephone will be able to determine the labels of this one and possibly of one or other authorized persons; he can then verify, by comparison of the labels stored with the labels of the user, whether the current user is an unauthorized person having in his possession the telephone following theft or loss; in this case, the telephone may transmit the label of the unauthorized person to the network operator on which the telephone depends for all practical purposes and possibly block the telephone.
- the invention can also be applied to the indexing of a photo library or image bank comprising a large number of photos.
- one or more labels are associated with each photo or image, for example a label corresponding to "cow” or "sunset".
- each label is associated with an alphanumeric reference which is stored in a "dictionary" next to the corresponding label and the set of label and reference number of the photo or image is stored in a memory.
- a user When a user wishes to find the photos or images, among those stored, comprising a cow or a sunset, he introduces a keyword indicating cow or sunset.
- the dictionary of the photo library translates this keyword in alphanumeric reference, which makes it possible to find, in the memory, the number of the photo (s) containing the keyword.
- a user wishes to find a photo or an image including several characteristics, for example a sunset and a cow, it will be enough for him to introduce the two corresponding keywords.
- a user of the method and of the device according to the invention in order to be able to locate himself, first examines with the sensor the environment of the place where he is, in particular a port or a jagged coast , and this environment is labeled. Then, the sensor is moved along a nautical chart and, in the event of identity, between the already memorized labels of the environment and the labels of a zone of the nautical chart at the observed place, it can determine where his boat is.
- Another application to navigation consists in locating a route, for example to find it when we want to return to the starting point.
- a device according to the invention makes it possible to label successive positions, with the corresponding environment; on return, the "navigator” tries to find, in the environment he observes with the sensor of the device, if he recognizes a place already visited on the outward journey.
- object encompasses not only the objects mentioned proper, but also persons or portions of persons, in particular faces, and animals, as well as possibly images or scenes (in particular for navigation applications).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP03756531A EP1529265A2 (fr) | 2002-08-07 | 2003-08-06 | Procede et dispositif de perception visuelle active pour caracterisation et reconnaissance par analyse de parametres mono/multidimentionnels dans des unites multiclasses de calcul et traitement d histogramme, recrutement dynamique d unites |
| JP2004526984A JP2006500643A (ja) | 2002-08-07 | 2003-08-06 | 装置のダイナミック補充による、マルチクラスヒストグラム計算および処理装置での一次元/多次元パラメータの分析による特徴抽出及び認識のためのアクティヴ視覚的認知方法及び装置 |
| AU2003282816A AU2003282816A1 (en) | 2002-08-07 | 2003-08-06 | Active visual perception method and device for characterisation and recognition through the analysis of mono/multidimensional parameters in multiclass computing units and histogram processing, dynamic unit recruitment |
Applications Claiming Priority (8)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| FR0210067 | 2002-08-07 | ||
| FR0210066A FR2843470B1 (fr) | 2002-08-07 | 2002-08-07 | Procede de fonctionnement et dispositif d'analyse de parametres mono et multidimensionnels par calcul et traitement d'histogramme, application a la perception active |
| FR0210065 | 2002-08-07 | ||
| FR0210064A FR2843467B1 (fr) | 2002-08-07 | 2002-08-07 | Procede et dispositif a recrutement dynamique de modules de calcul et traitement d'histogrammes pour caracterisation et reconnaissance d'objets |
| FR0210067A FR2843471B1 (fr) | 2002-08-07 | 2002-08-07 | Procede et dispositif de perception visuelle active pour caracteriser et reconnaitre un objet, notamment aux fins d'identification et de localisation |
| FR0210064 | 2002-08-07 | ||
| FR0210065A FR2843469B1 (fr) | 2002-08-07 | 2002-08-07 | Procede de fonctionnement et unite fonctionnelle multiclasse pour calcul et traitement d'histogramme, application a la perception active |
| FR0210066 | 2002-08-07 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2004015630A2 true WO2004015630A2 (fr) | 2004-02-19 |
| WO2004015630A3 WO2004015630A3 (fr) | 2004-10-07 |
Family
ID=31721630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/FR2003/002478 Ceased WO2004015630A2 (fr) | 2002-08-07 | 2003-08-06 | Procede et dispositif de perception visuelle active pour caracterisation et reconnaissance par analyse de parametres mono/multidimentionnels dans des unites multiclasses de calcul et traitement d'histogramme, recrutement dynamique d'unites |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP1529265A2 (fr) |
| JP (1) | JP2006500643A (fr) |
| AU (1) | AU2003282816A1 (fr) |
| WO (1) | WO2004015630A2 (fr) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2884625A1 (fr) * | 2005-04-15 | 2006-10-20 | Holding Bev Sa Sa Luxembourgeo | Procede de guidage automatique de vehicule dans une voie de circulation, dispositif correspondant |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2611063B1 (fr) * | 1987-02-13 | 1989-06-16 | Imapply | Procede et dispositif de traitement en temps reel d'un flot de donnees sequence, et application au traitement de signaux video numeriques representatifs d'une image video |
| JPH0668763B2 (ja) * | 1988-04-11 | 1994-08-31 | 株式会社イーゼル | 画像処理方法 |
| FR2751772B1 (fr) * | 1996-07-26 | 1998-10-16 | Bev Bureau Etude Vision Soc | Procede et dispositif fonctionnant en temps reel, pour le reperage et la localisation d'une zone en mouvement relatif dans une scene, ainsi que pour la determination de la vitesse et la direction du deplacement |
| US6249602B1 (en) * | 1998-05-28 | 2001-06-19 | Cognex Corporation | Method and apparatus for determining a reference point of an object profile within an image |
| FR2821459B1 (fr) * | 2001-02-23 | 2004-11-12 | Holding Bev Sa | Procede et dispositif de reperage d'un objet par sa forme, sa dimension et/ou son orientation |
| FR2805629B1 (fr) * | 2000-02-24 | 2002-08-30 | Holding Bev Sa | Procede et dispositif de perception automatique |
-
2003
- 2003-08-06 JP JP2004526984A patent/JP2006500643A/ja not_active Withdrawn
- 2003-08-06 EP EP03756531A patent/EP1529265A2/fr not_active Withdrawn
- 2003-08-06 AU AU2003282816A patent/AU2003282816A1/en not_active Abandoned
- 2003-08-06 WO PCT/FR2003/002478 patent/WO2004015630A2/fr not_active Ceased
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2884625A1 (fr) * | 2005-04-15 | 2006-10-20 | Holding Bev Sa Sa Luxembourgeo | Procede de guidage automatique de vehicule dans une voie de circulation, dispositif correspondant |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2006500643A (ja) | 2006-01-05 |
| AU2003282816A8 (en) | 2004-02-25 |
| AU2003282816A1 (en) | 2004-02-25 |
| EP1529265A2 (fr) | 2005-05-11 |
| WO2004015630A3 (fr) | 2004-10-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3707676B1 (fr) | Procédé d'estimation de pose d'une caméra dans le référentiel d'une scène tridimensionnelle, dispositif, système de réalite augmentée et programme d'ordinateur associé | |
| Sun et al. | Unsupervised domain adaptation through self-supervision | |
| Li et al. | Deep dehazing network with latent ensembling architecture and adversarial learning | |
| EP1866834A2 (fr) | Système et procédé de localisation de points d'intérêt dans une image d'objet mettant en uvre un réseau de neurones | |
| EP1259939B1 (fr) | Procede et dispositif de perception automatique | |
| EP0492724B1 (fr) | Procédé d'extraction de mouvement comprenant la formation d'images de différences et un filtrage tridimensionnel | |
| FR2858447A1 (fr) | Procede et dispositif automatise de perception avec determination et caracterisation de bords et de frontieres d'objets d'un espace, construction de contours et applications | |
| Swami et al. | Candy: Conditional adversarial networks based fully end-to-end system for single image haze removal | |
| Ghosh et al. | Pb3c-cnn: An integrated pb3c and cnn based approach for plant leaf classification | |
| EP4396789A1 (fr) | Procédé de mise en relation d'une image candidate avec une image de référence | |
| Tao et al. | Rethinking motion representation: Residual frames with 3d convnets for better action recognition | |
| Sabaghi et al. | Deep learning meets liveness detection: recent advancements and challenges | |
| WO2004015630A2 (fr) | Procede et dispositif de perception visuelle active pour caracterisation et reconnaissance par analyse de parametres mono/multidimentionnels dans des unites multiclasses de calcul et traitement d'histogramme, recrutement dynamique d'unites | |
| EP0447306B1 (fr) | Dispositif de reconnaissance de séquences dans un signal multidimensionnel | |
| FR2843469A1 (fr) | Procede de fonctionnement et unite fonctionnelle multiclasse pour calcul et traitement d'histogramme, application a la perception active | |
| FR2843467A1 (fr) | Procede et dispositif a recrutement dynamique de modules de calcul et traitement d'histogrammes pour caracterisation et reconnaissance d'objets | |
| FR2843471A1 (fr) | Procede et dispositif de perception visuelle active pour caracteriser et reconnaitre un objet, notamment aux fins d'identification et de localisation | |
| FR2843470A1 (fr) | Procede de fonctionnement et dispositif d'analyse de parametres mono et multidimensionnels par calcul et traitement d'histogramme, application a la perception active | |
| WO2002069265A1 (fr) | Procede et dispositif de reperage d'un objet par sa forme, sa dimension et/ou son orientation | |
| Rowntree et al. | Real-Time Human Gaze Estimation | |
| Balaji et al. | Designing and Developing a Dependable and High-Throughput Model with Enhanced DNN Using ABC Algorithm | |
| EP2062196A1 (fr) | Procede de cadrage d'un objet dans une image et dispositif correspondant | |
| Wieschollek | Learning From Multi-Frame Data | |
| WO2024194538A1 (fr) | Procede et dispositif d'enrichissement d'une image d'une scene, au sein d'un appareil utilisateur | |
| CA2439180A1 (fr) | Procede et dispositif d'adaption de la sensibilite de detection d'un parametre porte par un signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
| AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
| WWE | Wipo information: entry into national phase |
Ref document number: 2004526984 Country of ref document: JP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2003756531 Country of ref document: EP |
|
| WWP | Wipo information: published in national office |
Ref document number: 2003756531 Country of ref document: EP |
|
| WWW | Wipo information: withdrawn in national office |
Ref document number: 2003756531 Country of ref document: EP |


















