Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
  • G06V40/161—Detection; Localisation; Normalisation
  • G06V40/167—Detection; Localisation; Normalisation using comparisons between temporally consecutive images
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/20—Analysis of motion
  • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
  • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10016—Video; Image sequence
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30196—Human being; Person

Definitions

• the present invention generally relates to the processing of video images generated by a camera, and more particularly to real-time tracking of a person's bust position on camera-generated images.
• the present invention also relates to the measurement of the movements of an individual, in particular in the so-called context of "virtual reality", in which the movements of the individual are interpreted by computer means to be used, for example, for subsequent applications.
• Head tracking systems or “head tracking” in Anglo-Saxon terms
• head tracking from a stream of images taken from the head of the individual generally use:
• one or two cameras such as cameras, for capturing the images
• Computer resources such as a computer to process images through specific algorithmic processing.
• GR Bradski, in Computer Vision Face Tracking For Use in a Perceptual User Interface, Intel Technology Journal, 1998 also uses the principle of color to identify the position of a face in the image of the camera. The image is filtered to obtain the probability distribution of the color that corresponds to the color of the skin (through a histogram). The center of gravity of this distribution gives the center of the face. The roll of the head is given by the calculation of the moments of the distribution.
• US-6,580,810 discloses a three-dimensional face reconstruction method from a single camera. On the first image, three significant points are identified (for example the two eyes and the mouth), these points forming a triangle in 3D whose position of the points in the space is known. Then, for each of the captured images, the three characteristic points are identified and their 2D coordinates are calculated. The 3D model then undergoes rotations and translations to find the configuration that minimizes the error between the projection of the 3D model of the triangle and the 2D measurement.
• a 3D model of the face obtained for example with two cameras, is compared to the image obtained with a camera.
• a principal component analysis determines the orientation and position of the face.
• the invention proposes a method of tracking the position of the bust of a user from a stream of video images, the bust comprising the torso and the head of the user, the method comprising determining the position of the torso on a first image, in which process
• a virtual repository is associated with the torso on the first image, and in that, for a second image:
• a new position of the virtual repository is determined on the second image
• the invention is usable without calibration or without a 3D model of the user previously acquired or drawn for example from a library. These measurements are available in real time from a single video stream.
• the invention consists in adding and following a virtual repository (positioned on the image of the torso of the user) so as to differentiate the rotations of the translations of the head. Moreover, the invention does not require an algorithm using an apprenticeship or a calibration.
• the head of the user comprises the face of the user, and to deduce the relative position of the head, at least one position of the face is measured with respect to the new repository position on the current image.
• the face can be easily recognized by a particular point such as the nose, eyes or mouth, neck or an external object such as a cap on the head, using techniques like the one mentioned before.
• the comparison of the movement of the face with respect to the torso makes it possible in particular to determine the position of the head, a position that can be characterized for example by the azimuth, the elevation, as well as the horizontal and vertical translations of the latter.
• one or more of the following may also be used: determining the position of the user's face on the first image, and deducing the position of the torso on said first image in order to associate a virtual repository with the first image,
• the position of the torso is determined vertically to the position of the face
• the head includes the neck, and the relative position of the head is characterized by:
• T x O and Tyo the coordinates representative of the position of the torso on the first image
• T x and T y the coordinates representative of the position of the torso on the second image
• V ⁇ o and V y o the coordinates representative of the position of the face on the first image
• V x and Vy the coordinates representative of the position of the face on the second image, these coordinates being measured in a common reference linked to the horizontal and vertical axes of the first and second images
• C1 and C2 constants such as:
• the coordinates representative of the position of the face and the torso on the first image are the coordinates of the center of gravity of a first cloud of points defined on the face and the center of gravity of a second cloud of points defined on the torso , the coordinates representative of the position of the face and the torso on the second image being the coordinates of the centers of gravity of the first and second point clouds considered in their new position on said second image.
• the first image is a calibrati image .
• the second image is a current image of the video image stream.
• the invention also provides a system for tracking a user's bust position from a video image stream, the bust including the user's head and torso, the system comprising:
• a video image capture system for generating at least a first image and a second image of the user's torso
• the invention also relates to a computer program product, intended to be executed in a memory of the processing unit of a computer system, the program comprising instructions for implementing implementation of the method according to the invention during its execution in the processing unit.
• FIG. 1 is a diagram of steps of one embodiment of the method according to the invention.
• FIG. 2 is an illustration of a video image of the implementation of the method according to the invention
• FIG. 3 is an illustration of the head of a user to define the azimuth and the elevation of said head.
• the method according to the invention can be implemented from video image capture means, for example a webcam. It also requires calculation means such as a processor, and can thus be implemented by a computer.
• the term "bust of the user” refers to the assembly comprising the head and the torso.
• the head further includes the face of the user.
• the position of the head can be described by its azimuth Az, its elevation E1, and its displacements or horizontal translations P x and vertical P y .
• the azimuth corresponds to the rotation of the head about the axis constituted by the neck of the user; the zero rotation corresponding to the neutral position of the neck.
• the elevation also defined in Figure 3, corresponds to the rotation of the head in the plane of symmetry of the face, the zero rotation also corresponding to the neutral position of the neck.
• translational means the rectilinear displacement of the head, it has a horizontal component and a vertical component in directions parallel respectively to the horizontal and vertical edges of the video image.
• FIG. 1 represents a diagram of steps of an implementation mode of the method according to the invention.
• a camera for example a webcam, generates a stream of video images, and a first video image 10 is captured by the camera.
• a first step 20 it is checked whether the process has been initialized.
• a virtual repository is associated with the torso on the first image 10 in a step 30.
• This step 30 includes determining the position of the user's torso. This position may for example be characterized by the coordinates T x0 and T y0 , coordinates representative of the position of the torso and measured on the first image 10. These coordinates correspond for example to the coordinates of a specific point of the torso identified in the plane of The first picture.
• coordinates or translations can be measured in pixels or in distance (in meters for example), while rotations are expressed in degrees or radians.
• the position of the user's face in the first image is first determined, for example by means of V x0 and V y0 , the coordinates representative of the position of the face on the first image. image of the face, and measured in the plane of the first image. This determination of V x0 and V y0 , the coordinates representative of the position of the face on the first image. image of the face, and measured in the plane of the first image. This determination of V x0 and
• Vy 0 Vy 0
• the user is then positioned in the center of the image and actuates the initialization command such as a key on the keyboard.
• the initialization command such as a key on the keyboard.
• a face recognition algorithm such as for example the algorithm CAMSHIFT developed by G. R. Bradski, and detailed in "Computer Vision Face Tracking for Use in a Perceptual face recognition algorithm
• the torso is then considered vertically in the face and in the lower zone of the image capture. It is identified by the coordinates T x0 and T y0 .
• the position of the head is then initialized. If it is characterized by its azimuth, elevation, and two translations, this data is initialized to a null value. Once the position of the torso determined, a virtual repository is associated with the torso on the first image 10. It thus allows to position the head relative to the torso.
• step 30 the position of the head and the torso is determined for each subsequent current image of the video stream as follows.
• a second image is generated by the camera.
• steps 40 and 50 are identical and independent. They may consist in obtaining respectively:
• Tracker Description of the algorithm ", Intel Corporation, 2000, can for example track the movement of pixels in the facial area and in the torso area compared to the previous image or the calibration image.
• the new position of chest x T and T allows them to determine a new position of the virtual repository.
• a relative position of the head relative to the new position of the virtual repository is computed, by comparison with the position of the virtual repository on the first image, to independently determine the movements of the head and the torso.
• At least one position of the face is measured relative to the new position of the frame on the second frame, to deduce the relative position of the head.
• the calculation is done as well.
• the movements of the head are characterized by the calculation of the 4 degrees of freedom mentioned above, that is to say the azimuth Az, the elevation El, and its horizontal translations P x and vertical P y .
• the calculations take into account the projection of the rotation of the head on a vertical plane and a horizontal plane.
• FIG. 2 represents a current image of the video stream to which are associated a horizontal axis (axis XX 'of abscissas) and a vertical axis (axis YY' of ordinates), and which are respectively parallel to the sides of the image.
• axis XX 'of abscissas a horizontal axis
• axis YY' of ordinates a vertical axis
• the relative position of the head 120 is measured from a comparison between the positions of the torso 130 and the face 115 on the first image, and the new positions of the torso 130 and the face 115 of the user on the second image.
• the translation of the torso 130 is defined by its horizontal and vertical components with the following relations:
• This definition defines the translation of the head to the identical of the translation of the torso. This definition makes it possible to overcome the displacement of the torso 130 and thus facilitate the calculation of rotations of the head.
• C- 2 is a constant such as C 2 with - ⁇ El ⁇ -
• the constants Ci and C 2 also have as unity the pixel.
• the constant Ci is equal to the distance (in pixels) between the initial position of the center of the head with respect to the bust (V y0 - T y0 ) and the position of the head with respect to the bust (V y - T y ) when the latter has an elevation of 90 ° and an azimuth of 0 °.
• the constant C 2 is equal to the distance (in pixels) between the initial position of the center of the head with respect to the bust (V x o - T x o) and the position of the head with respect to the bust (V x - T x ) when the latter has a zero elevation and an azimuth of 90 °.
• the relative position of the head relative to the torso of the individual is characterized from its elevation El 1 and its azimuth Az, and translations P x and P y between the first and the second axis. second image.
• This characterization is independent of the selected object tracking algorithm (s).
• it is 4 degrees of freedom that define the relative position of the head relative to the torso, unlike known methods which are limited to two degrees of freedom.
• the relative position of the head relative to the torso of the individual is characterized from at least one element among its elevation E1, its azimuth Az, and translations P x and P y between the first and the second image.
• the first image may be replaced during the processing of the video image stream with a new image on which the positions of the face V x0 are determined and
• This change of first image can be voluntary or automated, for example when the system implementing the method determines a positional deviation of the bust that is too significant between the first and the second image.
• the first image is a calibration image while the second image is a current image of the video stream.
• calibration image is meant a reference image identical to each processing of a new current image of the video stream.
• the movements of the head on a current image are calculated by comparison with the same calibration image.
• the change of the first image of the previous embodiment amounts to a change of calibration image.
• a first step the user places his face in the center of a rectangle and for example launches the initialization command on the keyboard.
• the camera captures a first image, the calibration image, shown in Figure 2.
• a plurality of virtual points is located on the portion delimited by the rectangle 100 visible in Figure 2 and corresponding to an area around the nose 125. Their positions can be determined by the algorithm developed by J. Shi, and C. Tomasi, and presented in "Good features to track," IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 94), Seattle, 1994.
• a equivalent number of virtual points is positioned in a second rectangle 110 vertically to the face 115, as shown in Figure 2. This number of points corresponds to the torso of the user.
• the width of the second rectangle and its horizontal position are the same as those of the first rectangle. Its height is equal to a fraction, here 1/6 th , of the height of the image.
• the second rectangle is positioned at the bottom of the image. The centers of gravity of the two scatter plots contained in each rectangle, with respective coordinates
• V x0 , V yO j and (T XO , T y o) are then calculated, corresponding respectively to the position of the face and the torso on the first image.
• the new position of the points is calculated for a current image using a pyramidal implementation of the Lucas-Kanade algorithm mentioned above.
• the stability of the method is verified in an additional step performed after the determination of the new position of the virtual points.
• the new center of gravity is calculated from non-replaced points.
• the coordinates of the center of gravity related to the face are V x and V y
• the coordinates of the center of gravity linked to the torso are T x and T y .
• the translations P x and P y as well as the azimuth and the elevation are calculated respectively according to the equations (1.1), (1.2), (2) and (3).
• the constants Ci and C 2 are for example equal to 70 pixels for an image width of 340 pixels. These values make it possible to obtain the exact values of azimuth and elevation for a person positioned at 50 cm from the camera. For other distances between the user and the camera, the measured values are then linearly proportional to the azimuth and the elevation.
• 1D Kalman filters can be used to stabilize the calculated values.
• the input values for the filters are then directly the azimuth Az, the elevation E1 and the two translations P x and P y .
• the invention aims at a method of monitoring the bust of a user, fast and effective, and which provides the orientation of the head in azimuth and elevation and its lateral and vertical translations from a only video stream (for example a webcam) and in real time (30 images or more per second).
• the method according to the invention makes it possible to position a virtual repository on the torso of the user which will make it possible to differentiate the rotations of the translations from the head.
• the applications concerned by this invention are in the fields of telecommunications and leisure.
• the efficiency of the method makes it possible to use the measurements made in virtual reality and in particular during applications using sound spatialization techniques and in particular binaural techniques.
• the invention can be used as an additional control when using video games, especially for first person games (in English, "First Person Shooter”: these are games in which a player "is” a character of the game. play, hear and see what this character hears and sees).
• the method can also be used to control the mouse with the head and can thus be a valuable aid to people with disabilities.
• One can also consider using the results of the method according to the invention to seek to optimize the orientation of the webcam, the position of the head to control a steerable webcam through a motorized system according to the new position of the head of the user.
• Another advantage of the invention is that the measured rotations are independent of the translations.
• the implementation of the method according to the invention shows that the resultant performance improvement allows the use of inexpensive video capture equipment such as standard webcams.
• the method allows a real-time measurement (at least 30 images per second) without processor saturation (5% to 20% of a 3GHz Pentium IV with a resolution of 160x120 to 320x240 pixels).
• the invention also provides a system for tracking a user's bust position from a video image stream, the bust including the user's head and torso, the system comprising:
• a video image capture system for generating at least a first image and a second image of the user's torso, memories, and
• the invention also relates to a computer program product, intended to be executed in a memory of the processing unit of a computer system, the program comprising instructions for implementing the method according to the invention when its execution in the processing unit.

Landscapes

• Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Multimedia (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Oral & Maxillofacial Surgery (AREA)
• General Health & Medical Sciences (AREA)
• Human Computer Interaction (AREA)
• Health & Medical Sciences (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)
• Closures For Containers (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=36691470

Family Applications (1)

Country Status (4)

Families Citing this family (22)

Family Cites Families (11)

• 2007
  • 2007-01-31 US US12/223,536 patent/US8571258B2/en not_active Expired - Fee Related
  • 2007-01-31 EP EP07730893A patent/EP1982306A1/de not_active Withdrawn
  • 2007-01-31 WO PCT/FR2007/000176 patent/WO2007090945A1/fr not_active Ceased
  • 2007-02-07 BR BRPI0701982-3A patent/BRPI0701982A/pt not_active Application Discontinuation

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events