JP4670010B2 - Mobile object distribution estimation device, mobile object distribution estimation method, and mobile object distribution estimation program - Google Patents

Description

  The present invention relates to a mobile body distribution estimation apparatus, a mobile body distribution estimation method, and a mobile body distribution estimation program that estimate the number and distribution of mobile bodies using a plurality of photographing means.

  The method of detecting a person's movement using an image can reduce the burden on the user because the user does not need to wear a detection device and can detect the movement without contact. However, there is a problem that the detection process becomes unstable due to factors such as changes in environmental conditions such as clothes and lighting conditions and scene dependent factors such as occlusion between pedestrians. is there.

On the other hand, in the conventional multi-viewpoint system using multi-view observations taken with multiple cameras, a wide range of observations are possible while reliably reducing occlusion, so much research has been conducted ( For example, refer nonpatent literature 1).
Hirokazu Kato et al., “Real-time human tracking using an ellipsoid model”, 1999, Information theory, 40 (11), 4087-4096.

  However, in the multi-viewpoint system described above, the number of viewpoints required increases according to the size of the detection area and the distribution of people (distance between people). It becomes difficult to completely suppress the occurrence of occlusion due to the increase in the number of the occlusions. Also, in the above multi-viewpoint system, when the position of each person is detected independently using the background difference or motion vector, if the occlusion occurs between the persons, each person cannot be separated, so the position detection process fails. .

  An object of the present invention is to provide a mobile object distribution estimation device, a mobile object distribution estimation method, and a mobile object distribution estimation program capable of estimating the number and distribution of mobile objects with high accuracy even in a complex scene having a large number of occlusion areas. Is to provide.

The moving body distribution estimation apparatus according to the present invention moves in a moving area divided into M (M is an integer of 2 or more) blocks having a size that allows only one moving body to be located. A plurality of photographing means for photographing the moving body, a creating means for creating a plurality of observation silhouette images representing silhouettes of the moving body from a plurality of photographed images photographed by the plurality of photographing means, and created by the creating means Model selection that is projected onto the image coordinate system of each imaging means when the moving silhouette located within each block is assumed to be captured from the camera viewpoint of each imaging means to the observation silhouette image group a estimating means for estimating the most suitable number and distribution of the moving object in a mobile region observed silhouette image group based on the reference, the estimating means increases the number of mobile one from zero The description length of the observed silhouette image group is calculated by fitting the model images of each number of moving bodies to the observed silhouette image group while changing the value representing the block in which the moving object exists from 1 to M, and calculating If the described description length is reduced, the block data representing all blocks in which the moving object is present is updated based on the model image, and the number of moving objects and the block data when the description length is no longer reduced are changed to the most suitable movement. a it shall be estimated as the number and distribution of the moving object region.

In the moving body distribution estimation apparatus according to the present invention, the movement that moves in the moving area divided into M (M is an integer of 2 or more) blocks having a size that allows only one moving body to be located. The body is photographed by a plurality of photographing means, a plurality of observation silhouette images representing the silhouette of the moving body are created from the plurality of photographed images, and the camera viewpoint of each photographing means is created with respect to the created observation silhouette image group The model image projected on the image coordinate system of each imaging means when it is assumed that the moving body located in each block is captured from is applied, and the most suitable movement for the observation silhouette image group based on the model selection criteria Since the number and distribution of moving objects on the area are estimated, the number and distribution of moving objects can be estimated with high accuracy even in complex scenes with many occlusion areas. .

Further, since the moving area is divided into M (M is an integer of 2 or more) blocks having a size that allows only one moving body to be located, one moving body is included in one block. Since the moving region can be divided into blocks of a size suitable for estimation, the number and distribution of moving objects can be estimated with higher accuracy.

In addition, the estimation means increases the number of moving objects by one from zero, and changes the value representing the block in which the moving object is present from 1 to M while changing the number of moving object models for the observed silhouette image group. The description length of the observation silhouette image group is calculated by fitting the image, and if the calculated description length is reduced, the block data representing all blocks in which the moving object is present is updated based on the model image, and the description length is reduced. The number of mobile objects and the block data at the time of disappearance are estimated as the most suitable number and distribution of mobile objects on the moving area.

Therefore, the number of the mobile is increased from zero one by one, by fitting the model image of the moving object in each number a value representing the block which the mobile body is present for the observed silhouette images while changing from 1 to M movement when observed silhouette images of description length is calculated, if the reduced calculated description length updates the block data representing all blocks moving object based on the model image is present, no longer reduced description length Since the number of bodies and block data are estimated as the most suitable number and distribution of moving objects on the moving region, it is possible to avoid an increase in the number of moving objects and the calculation amount of the distribution estimation process. Numbers and distributions can be estimated faster.

  The creation means binarizes the observation silhouette image, and the estimation means fits the binarized model image to the binarized observation silhouette image group, and is based on a model selection criterion. It is preferable to estimate the number and distribution of suitable mobiles.

  In this case, the most suitable number and distribution of moving objects are estimated based on the model selection criteria using the binarized observation silhouette image and model image, so the number and distribution of moving objects are estimated even faster. The number and distribution of moving objects can be obtained in real time.

  The estimation means calculates a description length of the observation silhouette image group using an observation error as a portion where the pixel value of the binarized observation silhouette image is inverted with respect to the pixel value of the binarized model image Then, it is preferable to estimate the number and distribution of the moving objects having the shortest description length as the number and distribution of the moving objects on the most suitable moving region.

  In this case, considering the observation error, the number and distribution of moving objects with the shortest description length are estimated as the number and distribution of moving objects on the most suitable moving region. It can be estimated with high accuracy.

  The apparatus further comprises storage means for storing the plurality of model images in advance, and the estimation means sequentially reads the model images stored in the storage means, and sequentially applies the read model images to the observation silhouette image group. Preferably, the most suitable number and distribution of moving objects are estimated based on the model selection criteria.

  In this case, it is not necessary to create a plurality of model images, and the model image creation process can be omitted, so that the number and distribution of moving objects can be estimated at a higher speed.

  The plurality of model images are preferably created by morphing processing from a plurality of images obtained by photographing the moving body from different camera viewpoints. In this case, since an image taken from an arbitrary camera viewpoint can be created from a small number of images, a model image can be easily created.

Mobile distribution estimation method according to the present invention, multiple imaging means, a mobile distribution estimation method using a mobile distribution estimation apparatus comprising creating means and estimating means, said plurality of imaging means, one M only moving body has the possible size of the position (M is an integer of 2 or more) comprising the steps of photographing a moving body that moves a moving region divided into blocks, the creation means, said plurality Creating a plurality of observation silhouette images representing silhouettes of moving objects from a plurality of photographed images photographed by the photographing means, and the estimation means for each of the observation silhouette image groups created by the creation means, Based on the model selection criteria, fitting the model image projected on the image coordinate system of each photographing means when assuming that the moving body located in each block is photographed from the camera viewpoint of the photographing means Comprising the steps of estimating the most suitable number and distribution of the moving body on the movement area, the estimating means, the number of the mobile is increased from zero one by one, a value representing the block which the mobile is present from 1 The model length of the observation silhouette image group is calculated by fitting model images of each number of moving bodies to the observation silhouette image group while changing to M, and if the calculated description length is reduced, the model image is moved. Updating block data representing all blocks in which a body exists, estimating the number of mobile objects and block data when the description length no longer decreases as the most suitable number and distribution of mobile objects on the moving area; and Is included.

The moving object distribution estimation program according to the present invention is a moving object that moves in a moving area divided into M (M is an integer of 2 or more) blocks having a size that allows only one moving object to be located. and generating means for generating a plurality of observation silhouette image representing the silhouette of the moving body from a plurality of captured images captured by the multiple imaging means for observing the silhouette images created by the creation means, each imaging means The model image projected on the image coordinate system of each imaging means when it is assumed that the moving body located in each block is captured from the camera viewpoint of the camera, and on the most suitable moving area based on the model selection criteria a estimating means for estimating the number and distribution of the moving body, observing the number of the mobile is increased from zero one by one, while changing a value representing the block which the mobile is present from 1 to M Fit the model image of each moving object to the Luet image group to calculate the description length of the observation silhouette image group, and if the calculated description length is reduced, all blocks where the moving object exists based on the model image Update the block data to represent the number of moving objects and the block data when the description length no longer decreases, and let the computer function as estimation means for estimating the number and distribution of moving objects on the most suitable moving area It is.

According to the present invention, a plurality of observation silhouette image is created representing the silhouette of the moving body from a plurality of images captured for the observed silhouette images created, in each block from the camera viewpoint of the respective photographing means The model image projected on the image coordinate system of each imaging means when it is assumed that the moving body located at is captured is fitted, and the movement on the moving area most suitable for the observation silhouette image group based on the model selection criteria Since the number and distribution of bodies are estimated, the number and distribution of moving objects can be estimated with high accuracy even in a complex scene having a large number of occlusion regions.

  Hereinafter, a mobile body distribution estimation apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a moving object distribution estimation apparatus according to an embodiment of the present invention. In the following description, the case of estimating the distribution and number of pedestrians (humans) as a moving body will be described as an example, but the moving body to which the present invention is applied is not particularly limited to this example, and its shape Can be similarly applied to various mobile objects.

  The moving body distribution estimation apparatus shown in FIG. 1 includes K cameras 11 to 1K (K is an integer of 2 or more) and an image processing apparatus 20. The image processing apparatus 20 includes an image acquisition unit 21, a person region extraction unit 22, an observation vector creation unit 23, a description length calculation unit 24, a distribution estimation unit 25, and a shape projection model storage unit 26.

  The cameras 11 to 1K are attached to a predetermined area such as a ceiling or a wall such as a room or a hallway that is a movement area divided into a plurality of blocks, and color one or more pedestrians moving in the movement area. An image is captured, and the captured image data is output to the image acquisition unit 21 via a predetermined network such as a LAN. Note that the internal parameters, positions, and orientations of the cameras 11 to 1K are manually calibrated in advance. The cameras 11 to 1 </ b> K are not particularly limited to the above examples, and various imaging devices that capture black and white images and infrared images can be used. In consideration of the estimation accuracy, the number of cameras 11 to 1K is preferably 3 or more, more preferably 4 or more, but it is also possible to estimate using only one camera. is there.

  The image processing device 20 includes a normal computer including a ROM (Read Only Memory), a CPU (Central Processing Unit), a RAM (Random Access Memory), an external storage device, an input device, an image interface device, a display device, and the like. The functions of the respective blocks are realized by executing a moving body distribution processing program for executing a moving body distribution estimation process, which will be described later, by a CPU or the like. Note that the configuration of the image processing apparatus 20 is not particularly limited to this example, and the functions of each block shown in the figure are configured from dedicated hardware circuits, or a part or all of each of the above functions is one or more. You may make it perform using the computer of.

  The shape projection model storage unit 26 projects an image on the image coordinate system of each of the cameras 11 to 1K when it is assumed that a single pedestrian located in each block is photographed from the camera viewpoint of each of the cameras 11 to 1K. A shape projection model which is a model image is stored in advance.

  Here, the shape projection model will be described in detail. First, the camera projection model will be described. FIG. 2 is a schematic diagram for explaining a camera projection model.

  In a general pinhole camera model, the following relationship holds between a three-dimensional point (X, Y, Z) in space and a corresponding image (x, y) on the image plane. Here, S is an internal parameter representing the projection onto the image plane, and R and t represent the three-dimensional posture and position of the camera, respectively.

As shown in FIG. 2, the observed image is considered to be one or a plurality of three-dimensional human shapes projected on the image plane according to the above formula. Here, the floor which is the moving area is divided into M blocks, and x _j (1 ≦ j ≦ M) indicates whether or not there is a person in the block j as in the following equation.

However, one block is set to a sufficiently small area so that two or more people cannot enter at the same time. At this time, the distribution of pedestrians in the moving region can be expressed as the following expression as an M-dimensional vector X having x _j as an element.

When the silhouette image of a pedestrian obtained by observing this scene with the camera k is A ^k and N is the number of pixels of the silhouette image, the following equation is established.

FIG. 3 is a schematic diagram showing the correspondence between silhouette images and the distribution of pedestrians on the floor. As shown in FIG. 3, in the following description, consider the relationship between X and A ^k, if there is a pedestrian only certain blocks l, when the distribution and X _l, the following equation is established.

The above distribution with respect to X _l, whether belonging to the pixel a _{l, i} ^k is the human region, not only the distribution X _l, because also on the pedestrian's size and position, with respect to distribution X _l The probability that the i-th pixel a _{l, i} ^k is 1 is p _{l, i} ^k .

In the silhouette image actually observed, it is considered that a plurality of person images are further superimposed. For example, when there are pedestrians at L locations (x ₁₁ ,..., X _1L ), the probability p _i ^k that the pixel a _i ^k of the pedestrian silhouette image A ^k obtained by observing with the camera ^k is 1 is , Pixel a _i ^k is expressed by the following equation as the probability of belonging to the person area of any pedestrian.

  Assuming that the change of each pixel is independent from each other, the distribution of silhouette images observed by the camera k can be expressed as the following probability image.

  In order to generate a silhouette image for an arbitrary camera arrangement and pedestrian distribution, use of a model including three-dimensional geometric information such as a polygon model can be considered, but in this embodiment, a finite number of observations is used as a simpler method. A model generation image in which silhouette images from directions are averaged is created, and these are combined to create a silhouette image in an arbitrary direction in advance by morphing processing.

  FIG. 4 is a schematic diagram for explaining a method for acquiring a model generation image. As shown in FIG. 4, with a blue screen BS in the background, while changing the position and orientation of one human (male, height 170 cm) in an upright position, three types of horizontal, diagonally 45 degrees above and directly above Images are taken from the observation direction, and 70 to 100 silhouette images are acquired using the chroma key. At this time, the position of the human is within an area of about 30 cm × 30 cm (one block) on the floor, and the reference height for calculating the observation direction is set to the floor while rotating the human so that observation images from multiple directions are included. An image is taken with a surface of 80 cm.

FIG. 5 is a diagram illustrating an example of an average image in each observation direction. 5A shows an average image P ₀ taken from the horizontal direction (0 degree), FIG. 5B shows an average image P ₄₅ taken from above 45 degrees obliquely, and FIG. 5C shows an average taken from directly above. An image _P90 is shown. In order to generate a projection image at an arbitrary observation angle from these average images, the person position XX = [X Y Z] ′ and the camera position XX _C = [X _C Y _C Z _C ] ′ are calculated by the following equations. The observation angle θ is used.

It should be noted that Z = 80 and the camera is always positioned above the horizontal direction. If 0 ° <θ <45 ° according to the obtained observation angle, the average images P ₀ and P ₄₅ are obtained. If 45 ° <θ <90 °, a predetermined projected silhouette image is generated by morphing using the average images P ₄₅ and P ₉₀ , respectively.

  FIG. 6 is a schematic diagram for explaining a method for generating a projected silhouette image, and FIG. 7 is a diagram illustrating an example of a projected silhouette image. As shown in FIG. 6, an image MI generated by morphing is projected onto the camera image plane to obtain a projected silhouette image. Thereby, a projected silhouette image P for an arbitrary camera arrangement and pedestrian distribution can be generated. For example, as shown in FIG. 7, θ = 0, 15, 30, 45, 60, 75, 90 (degrees). Each projected silhouette image can be generated.

Next, as shown in the following equation, the projected silhouette image for X _l in the camera k is binarized to obtain a binarized silhouette image B _l ^k .

  A set B of binarized silhouette images for K cameras is defined as follows.

In the present embodiment, a binarized silhouette image set B1 is calculated in advance for all blocks l in advance, and binarized as a shape projection model that models the relationship between the position of the pedestrian and the silhouette image. A silhouette image is stored in advance in the shape projection model storage unit 26, and the binarized silhouette image B in the case where a person is present at L locations (x ₁₁ ,..., X _11L ) is obtained by simplifying the expression (6). It is calculated by the following formula.

  In the present embodiment, the created shape projection model is stored in advance in the shape projection model storage unit 26, but is not particularly limited to this example, and the above average image is stored in the shape projection model storage unit 26 in advance. Then, a shape projection model used for the moving object distribution estimation process is created from the stored average image by morphing process, or a three-dimensional model of the moving object is stored in advance in the shape projection model storage unit 26 and stored. Various modifications such as creating a shape projection model used for the mobile body distribution estimation process from the three-dimensional model are possible.

  Referring again to FIG. 1, the image acquisition unit 21 adds shooting time information to a plurality of image data shot by the cameras 11 to 1 </ b> K and outputs the image data to the person region extraction unit 22. The person area extraction unit 22 extracts a person area representing a human silhouette from image data as a silhouette area by executing a known person area extraction process such as inter-frame difference, and captures a silhouette image specifying the silhouette area as shooting time information. At the same time, the result is output to the observation vector creation unit 23. The observation vector creation unit 23 binarizes the silhouette image into a person region and a background region based on the silhouette region, and outputs the binarized silhouette image to the description length calculation unit 24 as an observation vector (observation silhouette image). To do.

Here, the observation vector will be described in detail. First, assuming that color images are input from the cameras 11 to 1K, one input image C _t ^k composed of N pixels obtained by the camera k at time t is expressed as follows.

Each pixel c _{i, t} ^k is composed of R, G, B pixel values r _{i, t} ^k , g _{i, t} ^k , b _{i, t} ^{k as in} _{the following equation} .

In order to simplify the calculation, in the present embodiment, the person area extraction unit 22 extracts pixels that greatly change within a certain time interval as a person area. Input vector Z ^k for the camera k is expressed as follows.

Therefore, a set Z _t of observation vectors for K cameras is defined as follows.

  The description length calculation unit 24 reads the shape projection model from the shape projection model storage unit 26 while sequentially increasing the number of pedestrians, and applies the shape projection model to the observation vector (observation silhouette image group) output from the observation vector creation unit 23. Is used to calculate the description length, and the shape projection model with the shortest description length of the observation vector is determined as the shape projection model most suitable for the observation vector (Minimum Description Length (MDL)).

  Here, the minimum description length criterion is one of the model selection criteria proposed to determine an appropriate model for describing observed data, and takes into account both the complexity of the model and the fitting error related to the input data. Model is selected. This minimum description length standard has already been widely used in the field of image processing and computer vision in area division and curve fitting. In this embodiment, the plane (floor surface) that becomes the moving area is made into small blocks. Using the shape projection model, the number and distribution of pedestrians are directly estimated using the minimum description length criterion for pedestrian image analysis.

  In addition, since ideal observation data cannot actually be obtained due to observation errors, in this embodiment, the observation error is modeled as a phenomenon in which the observation value is inverted, and a description that takes the observation error into account as follows. It is carried out.

First, assuming that the probability of Z _j = 0 for the human region is q and r is the probability of Z _j = 1 for the background region, the probability P (Z _t ^k | B) that the observation vector Z _t ^k is observed in the camera k. ^k ) can be calculated as follows.

At this time, the probability that the image set Z _t is observed under the model distribution B is expressed by the following equation.

Here, n _a , n _b , n _c , and n _d are (b = 1, z = 1), (b = 1, z = 0), (b = 0, z = 1), (b = 0, z = 0), and is expressed as follows.

As a result, the log likelihood of the input vector Z _t is expressed as follows:

  On the other hand, in general, the minimum description length criterion is formulated as follows, where F is the degree of freedom of the model and n is the number of data.

  The description length D (B, Zt) of the shape projection model corresponding to the above equation can be calculated from the equation (20) by the following equation, where h is the number of pedestrians (h << M).

  Note that the statistical model selection criteria applied to the present invention are not particularly limited to the above example, and Akaike's information criterion (AIC: Akaike Information Criterion), a criterion based on Bayesian theory (BIC: Baysian Information Criterion). Other model selection criteria such as may be used.

  The distribution estimation unit 25 determines that there is a human in each block corresponding to the shape projection model determined by the description length calculation unit 24, and estimates the number of blocks in which the human exists as the number of pedestrians on the moving region. The position of a block where a human is present is estimated as the distribution of pedestrians on the moving area.

  In the present embodiment, the cameras 11 to 1K correspond to an example of a photographing unit, the image acquisition unit 21, the person region extraction unit 22, and the observation vector creation unit 23 correspond to an example of a creation unit, and a description length calculation unit 24 and The distribution estimation unit 25 corresponds to an example of an estimation unit, and the shape projection model storage unit 26 corresponds to an example of a storage unit.

  Next, the mobile body distribution estimation process by the mobile body distribution estimation apparatus configured as described above will be described. FIG. 8 is a flowchart for explaining the mobile body distribution estimation process by the mobile body distribution estimation apparatus shown in FIG.

  First, in step S1, the cameras 11 to 1K photograph pedestrians who move in the moving area, and output the photographed image data to the image acquisition unit 21. The image acquisition unit 21 captures the image data together with the photographing time information. The data is output to the area extraction unit 22.

  Next, in step S2, the person area extraction unit 22 extracts a person area representing a human silhouette from the image data as a silhouette area, and outputs the silhouette image specifying the silhouette area to the observation vector creation unit 23 together with the shooting time information. To do.

  Next, in step S3, the observation vector creation unit 23 binarizes the silhouette image into a person area and a background area using the above equations (14) to (15), and converts the binarized silhouette image into an observation vector. To the description length calculation unit 24.

Next, in step S4, the description length calculation unit 24 sets h representing the number of pedestrians to 0 and D _min representing the minimum description length to D (0, Z) as initial values.

Next, in step S5, the description length calculation unit 24 sets the flag to 0, and executes the following processing while changing l _{h + 1} representing the block where the pedestrian is present from 1 to M. That is, the description length calculation unit 24 reads out the corresponding shape projection model from the shape projection model storage unit 26, and sets a set B = B _l1 ∪B _l2 ∪... ∪B _{lh + 1} of the shape projection models of the K cameras _{11 to} 1K. Using the calculated shape projection model set B and the observed vectors Z of the K cameras 11 to 1K, the inequality D (B, Z) + ((h + 1) / 2) (logM) <D _min is set. If this inequality is satisfied, that is, if the description length is reduced, D _min is updated to D _min = D (B, Z) + ((h + 1) / 2) (log B) At the same time, the flag is updated to flag = 1, and the distribution estimation unit 25 updates ans representing all blocks in which humans exist to ans = [l ₁ l ₂ ... L _{h + 1} ] ′.

  Next, in step S6, the description length calculation unit 24 determines whether or not flag = 1. If flag = 1, that is, if the description length is reduced, the number of pedestrians is determined in step S7. Update the represented h to h = h + flag to increase the number of pedestrians and repeat the processes in and after step S5. If flag = 1 is not satisfied, that is, if the description length is not reduced, the process proceeds to step S8. To do.

When the description length is not reduced, in step S8, the distribution estimation unit 25 walks to the blocks l ₁ , l ₂ ,..., L _{h + 1} based on the current ans = [l ₁ l ₂ ... L _{h + 1} ] ′. The number and distribution of pedestrians are estimated. Thereafter, returning to step S1, image data of the next frame is acquired, and the above processing is continued in units of frames to search for a person position in the scene that matches the input image group.

  Through the above processing, in the present embodiment, a plurality of binarized silhouette images representing pedestrian silhouettes are created as observation vectors from a plurality of captured image data, and binary values are obtained while sequentially increasing the number of pedestrians. Description length minimum criterion is calculated using a plurality of digitized observation vectors and a binarized set of shape projection models, and a search is performed until the description length minimum criterion is not reduced. Since the number and distribution of pedestrians on the moving area is estimated by determining that there are pedestrians in the block corresponding to the model, based on the shape projection model that models the relationship between the position of pedestrians and silhouette images, It is possible to estimate the pedestrian distribution that maximizes the likelihood of the observed image group, and the number and distribution of pedestrians can be accurately and reconstructed even in complex scenes with many occlusion areas. It can be estimated in-time.

  In the above processing, the position of each pedestrian is fixed while gradually increasing one by one from the case of zero. In this case, there is a possibility that the processing falls into a local solution, for example, a model of one pedestrian close to the cameras 11 to 1K is applied to the silhouettes of a plurality of pedestrians located far from the cameras 11 to 1K. Since the image groups obtained from multiple viewpoints using 11 to 1K are simultaneously evaluated, it is possible to suppress falling into a local solution as described above.

  Next, the estimation accuracy of the above mobile body distribution estimation apparatus will be described. First, the pedestrian position distribution was estimated using a simulation image obtained based on the shape projection model. The existence range of the pedestrian was a rectangular area of 330 cm × 330 cm on the floor, the area was divided into 11 × 11 blocks, and the size of one block was 30 cm × 30 cm. Pedestrians were assumed to be in one of these blocks.

  FIG. 9 is a diagram illustrating an arrangement example of cameras. As shown in FIG. 9, three virtual cameras CA were installed around the existence range of the pedestrian HM to generate an observation image. In consideration of the environmental constraints in actual image observation, which will be described later, this camera arrangement was selected so that observation was possible from various directions as much as possible.

  FIG. 10 is a diagram illustrating an example of a generated image for three pedestrians. In the example shown in FIG. 10, three pedestrians have coordinates (x, y) = (0, 60), (x, y) = (210, 210), (x, y) = (240, 180). It is a generated image when it exists in three places. In this example, occlusion occurs between the second and third pedestrians in all camera observations. Using this image as an input, the position distribution of the pedestrian was estimated by the moving body distribution estimation apparatus shown in FIG.

  FIG. 11 is a diagram illustrating a result of estimating the position distribution of pedestrians. In the figure, ◇ indicates the placement of pedestrians, and + indicates the position estimation result. FIG. 11 shows that the estimated number of pedestrians at this time is three, which is consistent with the estimation result, and that the correct estimation result is obtained for all pedestrians in the pedestrian distribution.

  Next, the stability of the mobile body distribution estimation apparatus shown in FIG. 1 was examined by virtually placing pedestrians at random positions and performing repeated estimation. In this example, while changing the number of pedestrians to 1, 2, 3, 4, and 5, pedestrians are arranged within the range of 330 cm × 330 cm as in the above example, and three virtual Ten sets of projected images for each camera were generated. The position of the pedestrian is on a grid point where the floor surface is divided into 30 cm × 30 cm, and in consideration of physical interference between pedestrians in the actual image, a plurality of pedestrians are not positioned at adjacent grid points. Random selection was made to estimate the pedestrian distribution.

  12 is a diagram showing estimation results of the number of pedestrians, FIG. 13 is a diagram showing averages and standard deviations of the estimation results shown in FIG. 12, and FIG. 14 is a true value closest to each estimated position. FIG. 15 is a diagram showing the pedestrian position distribution estimation error when calculating the distance to the position, and FIG. 15 is a diagram showing the average and standard deviation of the X and Y positions of the position distribution estimation error shown in FIG. is there. From FIG. 12 to FIG. 15, it can be seen that an estimated value close to the true value is obtained for both the number of people and the position, although there is an error in the estimated number of people except for the case of one person.

  Next, the number of pedestrians is two, and the distance between the two is fixed to any one of 1 to 5 blocks (30 to 150 cm), and the position of the pedestrian is selected at random and estimated 10 times each. Went. 16 is a diagram showing a pedestrian position distribution estimation error, and FIG. 17 is a diagram showing the average and standard deviation of the X and Y positions of the position distribution estimation error shown in FIG. When the distance is 2 blocks or less, occlusion occurs for many observations. From FIG. 16 and FIG. 17, the mobile object distribution estimation apparatus shown in FIG. I understand.

  Next, in order to show the relationship between the number of cameras and the estimation accuracy, the number of cameras used for observation was changed from 1 to 3 for the case of 5 pedestrians, and estimation was performed 10 times in the same manner as described above. 18 is a diagram showing an estimation result of the number of pedestrians with respect to the number of cameras, FIG. 19 is a diagram showing an average and standard deviation of the estimation results shown in FIG. 18, and FIG. 20 is a diagram showing pedestrians with respect to the number of cameras. FIG. 21 is a diagram showing the average and standard deviation of the X and Y positions of the position distribution estimation error shown in FIG.

  From FIG. 18 to FIG. 21, it can be seen that in this example, there is no difference in the estimation results between the two cameras and the three cameras, but when the number of cameras is 1, the estimation accuracy is deteriorated. This result shows the effect of multi-viewpoint observation when there are many people, and this tendency becomes more remarkable as the number of pedestrians increases.

  Next, it estimated using the observed image which image | photographed the pedestrian using the three cameras 11-13 actually. In this example, while changing the number of people to 1, 2, 3, 4, and 5, pedestrians are arranged within a range of 330 cm × 330 cm as in the above examples, and three cameras 11 Images of 10 pairs of each person were taken by ˜13, and silhouette images were obtained by differential processing with images taken in an unattended state in advance.

  FIG. 22 is a diagram illustrating an example of an observation image. The camera arrangement and the pedestrian arrangement at this time were the same as in the examples of FIGS. Here, the breakdown of pedestrians is four men (height 183 cm, 178 cm, 176 cm, 170 cm) and one woman (height 165 cm).

  FIG. 23 is a diagram showing an estimation result of the number of pedestrians, FIG. 24 is a diagram showing an average and standard deviation of the estimation results shown in FIG. 23, and FIG. 25 is a true value closest to each estimated position. FIG. 26 is a diagram illustrating a position distribution estimation error of a pedestrian when the distance to the position is calculated, and FIG. 26 is a diagram illustrating an average and a standard deviation of the X position and the Y position of the position distribution estimation error illustrated in FIG. is there.

  From FIG. 23 to FIG. 26, it can be seen that estimation is performed with accuracy of less than ± 80 cm in position and ± 1 or less in the number of people for a real image. On the other hand, in addition to the increase in error with the increase in the number of people, the estimation accuracy is also deteriorated compared to the results of FIGS. This may be due to the noise of silhouette observation itself, occlusion between pedestrians, differences in posture and physique, and differences in clothing shapes. Therefore, in this embodiment, only the pedestrian's upright posture is modeled, but after creating a shape projection model for each of a plurality of different postures and detecting the pedestrian's posture, the shape projection model of the detected posture is It may be used to estimate the number and distribution of pedestrians.

  Finally, the position of the pedestrian moving in the scene was continuously estimated using the moving body distribution estimation apparatus shown in FIG. FIG. 27 is a diagram illustrating a result of continuously estimating the position of a pedestrian moving in the scene, and FIG. 28 is a diagram illustrating an example of an observation image used in the example illustrated in FIG.

  In this example, the pedestrian was instructed to follow the locus on the broken line shown in FIG. 27, and a silhouette image was obtained by the difference process shown in Expression (14) for the observed image. The + mark in FIG. 27 indicates the estimated position at each time. As shown in FIG. 27, the average distance between the position estimation result and the taught course was 8.59 cm (standard deviation of the distance distribution was 11.6 cm). As a result, in the present embodiment, it is understood that the position can be estimated even for continuous motion.

  In this example, the teaching course does not coincide with the center of the block except for eight places (X = 30, 60,..., 240) on the upper side (Y = 300) of the teaching course. In the present embodiment, since a pedestrian distribution that is most suitable for the input image is selected in units of blocks assumed at the time of model generation, it is impossible to estimate position fluctuations that are finer than the block size. For this reason, it is estimated that the pedestrian who exists in the block boundary vicinity exists in one of the surrounding blocks. In the tracking result of FIG. 27, it can be said that the estimated position is obtained in the peripheral blocks of the teaching course, indicating the above situation.

  The above examples confirmed the effectiveness of the present embodiment in estimating the number and distribution of pedestrians. In each of the above examples, the image size is 80 × 60 pixels, and the processing speed when parallel processing is performed by four general personal computers is 3 to 10 frames / second.

It is a block diagram which shows the structure of the mobile body distribution estimation apparatus by one embodiment of this invention. It is a schematic diagram for demonstrating a camera projection model. It is a schematic diagram which shows the correspondence of a silhouette image and the distribution of the pedestrian on a floor surface. It is a schematic diagram for demonstrating the acquisition method of the image for model generation. It is a figure which shows an example of the average image in each observation direction. It is a schematic diagram for demonstrating the production | generation method of a projection silhouette image. It is a figure which shows an example of a projection silhouette image. It is a flowchart for demonstrating the mobile body distribution estimation process by the mobile body distribution estimation apparatus shown in FIG. It is a figure which shows the example of arrangement | positioning of a camera. It is a figure which shows the example of the production | generation image with respect to three pedestrians. It is a figure which shows the result of having estimated the position distribution of the pedestrian. It is a figure which shows the estimation result of the number of pedestrians. It is a figure which shows the average and standard deviation of the estimation result shown in FIG. It is a figure which shows the position distribution estimation error of a pedestrian when the distance to the position of the true value nearest to each estimated position is calculated. It is a figure which shows the average and standard deviation of X position and Y position of the position distribution estimation error shown in FIG. It is a figure which shows the position distribution estimation error of a pedestrian. It is a figure which shows the average and standard deviation of X position and Y position of the position distribution estimation error shown in FIG. It is a figure which shows the estimation result of the number of pedestrians with respect to the number of cameras. It is a figure which shows the average and standard deviation of the estimation result shown in FIG. It is a figure which shows the position distribution estimation error of a pedestrian with respect to the number of cameras. It is a figure which shows the average and standard deviation of X position and Y position of the position distribution estimation error shown in FIG. It is a figure which shows an example of an observation image. It is a figure which shows the estimation result of the number of pedestrians. It is a figure which shows the average and standard deviation of the estimation result shown in FIG. It is a figure which shows the position distribution estimation error of a pedestrian when the distance to the position of the true value nearest to each estimated position is calculated. It is a figure which shows the average and standard deviation of X position and Y position of the position distribution estimation error shown in FIG. It is a figure which shows the result of having continuously estimated the position of the pedestrian who moves within the scene. It is a figure which shows an example of the observation image used for the example shown in FIG.

Explanation of symbols

11 to 1K camera 20 image processing device 21 image acquisition unit 22 human region extraction unit 23 observation vector creation unit 24 description length calculation unit 25 distribution estimation unit 26 shape projection model storage unit

Claims (7)

A plurality of photographing means for photographing a moving body that moves in a moving area divided into M (M is an integer of 2 or more) blocks having a size that allows only one moving body to be located ;
Creating means for creating a plurality of observation silhouette images representing silhouettes of moving objects from a plurality of photographed images photographed by the plurality of photographing means;
A model projected on the image coordinate system of each photographing means when it is assumed that the moving body located in each block is photographed from the camera viewpoint of each photographing means with respect to the observation silhouette image group created by the creating means An estimation unit that fits an image and estimates the number and distribution of moving objects on a moving region most suitable for an observation silhouette image group based on a model selection criterion ;
The estimation means increases the number of moving objects by one from zero, and changes the value representing the block in which the moving object is present from 1 to M, and displays a model image of each number of moving objects for the observation silhouette image group. The description length of the observation silhouette image group is calculated by fitting, and if the calculated description length is reduced, the block data representing all blocks in which the moving object exists is updated based on the model image, and the description length no longer decreases. mobile distribution estimation apparatus the number and the block data of the moving body, characterized that you estimated as the number and distribution of the most suitable moving object in a mobile region when. The creating means binarizes the observation silhouette image,
The estimation means fits a binarized model image to the binarized observation silhouette image group, and estimates the most suitable number and distribution of moving objects based on a model selection criterion. The moving body distribution estimation apparatus according to claim 1 . The estimation means calculates a description length of the observation silhouette image group using an observation error as a portion where the pixel value of the binarized observation silhouette image is inverted with respect to the pixel value of the binarized model image 3. The mobile body distribution estimation apparatus according to claim 2, wherein the number and distribution of mobile bodies having the shortest description length are estimated as the most suitable number and distribution of mobile bodies on the moving area. A storage unit for storing the plurality of model images in advance;
The estimation means sequentially reads out the model images stored in the storage means, sequentially applies the read model images to the observation silhouette image group, and determines the most suitable number of moving objects based on the model selection criteria and The mobile body distribution estimation apparatus according to claim 1 , wherein the distribution is estimated. 5. The moving body distribution estimation apparatus according to claim 1 , wherein the plurality of model images are created by morphing processing from a plurality of images obtained by shooting the moving body from different camera viewpoints. Multiple capturing means, a mobile distribution estimation method using a mobile distribution estimation apparatus comprising creating means and estimating means,
A step in which the plurality of photographing means shoots a moving body that moves in a moving area divided into M (M is an integer of 2 or more) blocks having a size that allows only one moving body to be located ; When,
The creating means creating a plurality of observation silhouette images representing silhouettes of a moving object from a plurality of photographed images photographed by the plurality of photographing means;
When it is assumed that the estimation means has photographed a moving body located in each block from the camera viewpoint of each photographing means for the observation silhouette image group created by the creating means, the image coordinate system of each photographing means And estimating the number and distribution of moving objects on the most suitable moving region based on the model selection criteria , wherein the estimating means reduces the number of moving objects from zero to one. By increasing the value representing the block in which the moving object exists from 1 to M, the model length of each moving object is fitted to the observed silhouette image group to calculate the description length of the observed silhouette image group. If the calculated description length is reduced, the block data representing all blocks in which the moving object is present is updated based on the model image, and the number of moving objects when the description length no longer decreases Fine block data, the most suitable mobile distribution estimation method characterized by comprising the steps of estimating the number and distribution of the moving object in a mobile region. Multiple taken by multiple imaging means mobile M (M is an integer greater than or equal to 2) to move the moving area divided into blocks having the level allowing only one mobile is located Creating means for creating a plurality of observation silhouette images representing silhouettes of moving objects from captured images;
A model projected on the image coordinate system of each photographing means when it is assumed that the moving body located in each block is photographed from the camera viewpoint of each photographing means with respect to the observation silhouette image group created by the creating means An estimation unit that fits an image and estimates the number and distribution of moving objects on the most suitable moving area based on a model selection criterion, and increases the number of moving objects by one from zero, and there are moving objects. The description length of the observation silhouette image group is calculated by fitting the model images of each moving object to the observation silhouette image group while changing the value representing the block from 1 to M, and the calculated description length is reduced. Update the block data representing all blocks in which the moving object exists based on the model image, and the most suitable number of moving objects and block data when the description length no longer decreases. Mobile distribution estimation program for causing a computer to function as estimating means for estimating the number and distribution of the moving body on the moving region.