JPH11316823A - Image synthesizing method and apparatus, and recording medium recording this method - Google Patents

Abstract

(57) [Problem] To provide an image synthesizing method and apparatus for synthesizing a plurality of images having depth information without inconsistency. SOLUTION: When reading an image file into a memory of a computer, a depth information input unit 110 reads depth information described in the image file for each pixel. The depth information providing unit 120 calculates, for the input depth information of each pixel,
In the virtual space set in the computer, depth information is newly added under unified conditions, and depth information is newly added to image data in the memory. Image combining means 130
Synthesizes a plurality of image data in accordance with the newly assigned depth information in the memory. The image display means 140 displays the synthesis result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image synthesizing method for synthesizing a plurality of different image data in the same application program.

[0002]

2. Description of the Related Art Conventionally, when combining images recorded in a bit map format on a computer (computer), that is, when a plurality of image data are superimposed on the same area, an image resulting from the superimposition is an original image. Color information or non-transmissivity (α value) possessed by each pixel of the image.

[0003]

For this reason, for example, when a plurality of images having a sense of depth are superimposed and synthesized,
In some cases, unnaturalness may occur in the anteroposterior relationship between the image A and the image B. This is because the general image file format records only the color information of each pixel, and the recording format of the depth information is not unified.

[0004] On the other hand, there is a file format for recording a three-dimensional data structure, but the generated image greatly depends on a program for interpreting a file recording the data structure and generating an image.

Further, even if there is a two-dimensional image in which depth information or a priority order for each pixel is recorded, there is a problem that the relative positional relationship between the depth information is not known.

An object of the present invention is to solve the above problems,
It is an object of the present invention to provide an image synthesizing method and apparatus for synthesizing a plurality of images having depth information without inconsistency.

[0007]

The present invention solves the above-mentioned problems by the inventions listed below.

The invention of (1) is a method for synthesizing a plurality of images having depth information using a computer, wherein a common depth range is constructed in the computer and different depth ranges are recorded in different storage capacities. Read a plurality of images in association with the common depth range and the storage capacity constructed in the computer, and newly read the read image in accordance with the common depth range constructed in the computer. An image synthesizing method, wherein depth information is added, a plurality of images are synthesized according to the newly added depth information, and the synthesized image is displayed.

[0009] The invention of (2) is based on the invention of (1).
When the new depth information is added, the number of possible values of the depth information to be constructed in the computer is determined according to the number of possible values of the depth information in the read original image. Is an image synthesizing method.

The invention of (3) is characterized in that, in the invention of (1) or (2), when the new depth information is added, the newly added depth information is obtained from the information of the original image read. An image synthesizing method characterized by performing one or more processes of changing, modifying, and editing.

The invention according to (4) is the invention according to (3),
When the new depth information is added, the new depth information is newly added to the same depth information possessed by the read original image in accordance with the color information of the pixel having the depth information. An image synthesizing method characterized in that:

The invention of (5) is an apparatus for synthesizing a plurality of images having depth information using a computer, wherein a plurality of images in which different depth ranges are recorded in different storage capacities are constructed in the computer. Means for reading in association with the common depth range and the storage capacity, means for newly adding depth information to the read image in accordance with the common depth range constructed in the computer, An image synthesizing apparatus, comprising: means for synthesizing a plurality of images in accordance with the depth information given to the image, and means for displaying the synthesized image.

The invention according to (6) is the invention according to (5), wherein
The means for adding the new depth information determines the number of possible values of the depth information to be constructed in the computer according to the number of possible values of the depth information in the read original image. An image synthesizing apparatus characterized by the following.

(7) The invention according to (5) or (6), wherein the means for adding the new depth information changes the newly added depth information from the information of the original image read. And at least one of a correction and an editing process.

The invention according to (8) is the invention according to (7), wherein
The means for newly adding depth information is a means for newly adding depth information to the same depth information possessed by the read original image in accordance with the color information of the pixel having the depth information. An image synthesizing apparatus characterized in that:

The invention of (9) is a recording medium which records a method of synthesizing a plurality of images having depth information by using a computer, wherein a procedure for constructing a common depth range in the computer and a different depth are provided. A procedure for reading a plurality of images whose ranges are recorded in different storage capacities in association with a common depth range and the storage capacity built in the computer, and for the read image, a procedure built in the calculator. A procedure for adding new depth information according to a common depth range, a procedure for combining a plurality of images according to the newly assigned depth information, and a procedure for displaying the combined image. A recording medium on which an image synthesizing method characterized by the above is recorded.

According to the invention of (10), in the invention of (9), in the step of adding the new depth information, the procedure in the computer according to the number of possible values of the depth information in the read original image is performed. And a process for determining the number of possible values of the depth information to be constructed.

The invention of (11) is characterized in that (9) or (10)
In the invention, the procedure for adding new depth information includes one or more processes of changing, correcting, and editing the newly added depth information from information of the read original image. This is a recording medium on which an image synthesizing method is recorded.

According to the invention of (12), in the invention of (11), in the step of adding the new depth information, the same depth information possessed by the original image which has been read is stored in the depth information. Is a recording medium on which is recorded an image synthesizing method, characterized by having a process of newly adding depth information in accordance with color information of a pixel having.

In the present invention, when the image file is read into the memory of the computer, the depth information described in the image file is read, and the depth information is newly added to the image data in the memory according to the common depth range. By synthesizing a plurality of image data in accordance with the newly assigned depth information in the memory and displaying the synthesis result, it is possible to synthesize a plurality of images having depth information without inconsistency in depth.

[0021]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing one embodiment of the basic type of the present invention, which is the first embodiment of the present invention, and shows one embodiment of the inventions (1) and (5). FIG.

In FIG. 1, reference numeral 110 denotes a depth information input unit, 120 denotes a depth information adding unit, 130 denotes an image synthesizing unit, and 140 denotes an image display unit.

The depth information input means 110 is means for reading the depth information recorded in the input image file into a memory in the computer. Alternatively, depth information possessed by image data stored in the memory of the same or another computer may be directly read into the memory.

At this time, it is assumed that the depth information is recorded or stored in pixel units of the image data, and how many gradations the depth information is represented and recorded depends on whether the depth information is recorded in the image itself in advance. Or the structure of the image itself. For example, the depth information of one pixel is recorded as an 8-bit unsigned integer value.

The depth information adding means 120 newly adds depth information to the input depth information of each pixel in the virtual space W set in the computer under unified conditions.

[0027] The depth of each pixel in the virtual space W shall be represented by K ₀ gradation, and max the maximum value of K _0, the minimum value, respectively _{(K 0), min (K} 0), maximum depth, respectively Value (close to the viewpoint) and minimum value (far from the viewpoint).

Further, the depth information is represented by min (K ₀ ) = z ⁰ ₀ ,
z ⁰ ₁ ,. . . , Represented by ^{_{z 0 K0-1 = max (K 0}} ).

The number of images to be synthesized is N, and the image I
_{i (i = 1,2, ...,} N) the gradation of depth information of the K _i, the depth information ^{_{min (K i) = z i}} 0, z i 1,. . . ,
It is represented by z ⁱ _Ki-1 = max (K _i ).

Thereafter, the following processing is performed. FIG. 2 is a schematic diagram of an event appearing in the following processing, and the depth is represented by eight gradations. The depth information providing means is up to the third step, and the fourth step is processing in the image synthesizing means.

Under the appropriate origin, the depth information of the pixel at the position (x, y) of the synthesized image is defined as p (x, y).
Initialize all the depth information of the memory or frame buffer in which the image is stored. That is, it is assumed that all the pixels are at the farthest point from the viewpoint, and this value is set to p ₀ (x, y). Thereafter, the following processing is performed from i = 1 to i = N.

1. If K _i = K _0, and converts the data type of the value that indicates the depth information in the I _i in the data type in the virtual space W, applied to the I _i as the value of it is depth information. At this time, as shown in FIG. 2A, min (K ₀ )
It is most rational that the values of max (K ₀ ), min (K _i ), and max (K _i ) are the same.

If not aligned, see FIGS. 2 (b-1) and (b-
As shown in 2), if the _value does not fall within the range of min (K ₀ ) and max (K ₀ ), the pixel of that portion may be deleted, or the depth information of that portion may be replaced by min (K _0). )
Alternatively, it may be max (K ₀ ). At this time, the depth information newly added to I _i is newly added from the reference value as shown in FIGS. 2 (b-1) and 2 (b-2).

2. If K _i <K _0, converts the data type of the value that indicates the depth information in the I _i in the data type in the virtual space W, as shown in FIG. 2 (c) min and (K ₀₎ m
min (K _i ) or max (K _i ) is z ⁰ _j so that min (K _i ) and max (K _i ) are included in ax (K ₀ ).
(J = 0, 1,..., K ₀ -1) is determined, and depth information in the virtual space W is added based on the value.

[0035] min (K ₀₎ and max (K ₀₎ min in the
If (K _i ) and max (K _i ) are not included, then 1. The same processing is performed.

3. If K _i > K ₀ , the data type of the value indicating the depth information in I _i is converted to the data type in the virtual space W, and min (K _i ) and m are converted as shown in FIG.
min (K ₀ ) or max from between ax (K _i )
A reference point corresponding to (K ₀ ) is set (corresponding to min (K ₀ ) in FIG. 2D, but either may be used).
Depth information is newly added from the reference point.

At this time, the part that does not fall into the K ₀ gradation is
The pixels in that part may be erased, or the depth information of that part may be min (K ₀ ) or max (K ₀ ).

4. The image synthesizing means 130 performs a synthesizing process. That is, the depth information p ₀ (x, y) stored in the memory or the frame buffer of the computer is compared with the depth information p _i (x, y) possessed by each pixel of I _i , and the viewpoint is closer to the viewpoint. The color and the depth information of the person having the depth information are newly stored in a memory or a frame buffer in the computer as p ₀ (x, y).

The processing such as the time, the size of each image I _i, that is, when the number of pixels in the vertical and horizontal directions are different, to the present invention the user may appropriately align the respective images, for example, align the upper left corner May be performed.

The above process is repeated, and after the N images have been processed, the image is displayed by the image display means 140 based on the color information stored in the memory or the frame buffer in the computer.

It is also possible to newly add pixel depth information after reading information of all N images, and compare the values to determine the combined color. In this case, a larger storage capacity is required than in the above processing procedure.

FIG. 3 shows the flow of the processing described above.

According to the present embodiment, there is an effect that a plurality of images having depth information can be synthesized so as to prevent unnaturalness in the depth order.

FIG. 4 is a block diagram showing a second embodiment of the present invention, and is a view for explaining one embodiment of the inventions (2) and (6).

The configuration and processing of this embodiment are basically the same as those of the first embodiment, but the depth information adding means 120 is provided with a depth information storage capacity determining means 121 at the preceding stage.

[0046] In the first embodiment, the depth of each pixel in the virtual space W computing machine shall be represented by K ₀ gradation, had been fixed, the dynamic is K ₀ in this embodiment It changes to.

That is, as in the first embodiment, the number of images to be synthesized is N, and the images I _i (i = 1, 2,.
Gradation depth information N) upon the K _i, for example, may be a new K ₀ is the highest value among K _i of previously entered information of all images, reads an image one by one in when writing computing machine memory or frame buffer f, the time is compared with K _i and K _0, the larger value K
_{Assuming 0} , the same processing as in the first embodiment is performed.

FIG. 5 shows the flow of processing of the second embodiment described above.

According to the present embodiment, it is possible to combine a plurality of images having depth information so that the anteroposterior relation of the depth does not occur and all the depth information is used. Has the effect of becoming

FIG. 6 is a diagram illustrating a third embodiment of the present invention, which is an embodiment of the invention (3), (7) or (4), (8).

The structure and processing of this embodiment are basically the same as those of the first embodiment or the second embodiment. However, in the depth information adding means 120, a depth information correcting means 122 is provided at the preceding stage. Have joined.

In this embodiment, in addition to the first embodiment or the second embodiment, the main feature is that the image is actually synthesized using the distance represented by the depth information of the image and the color information of the image. Characteristics.

That is, as in the first embodiment, the gradation representing the depth information in the virtual space W is K _0, and the maximum value and the minimum value indicated by K ₀ are max (K ₀ ) and min (K ₀ , respectively). )
And the depth information is represented by min (K ₀ ) = z ⁰ ₀ ,
z ⁰ ₁ ,. . . , Represented by ^{_{z 0 K0-1 = max (K 0}} ).

When the number of images to be synthesized is N, the image I _i (i
= 1, 2,. . . , N), the gradation of the depth information is K _i ,
Depth information is represented by min (K _i ) = z ⁱ ₀ , z ⁱ ₁ ,. . . , Z ⁱ
_Ki-1 = represented by max (K _i).

Further, depth information max (K _i ), min
The values of the depth indicated by (K _i ) (i = 0, 1, 2,..., N) are Z (max _i ) and Z (min _i ), respectively.

The depth distance (Z (max _i ) −Z (min)) / K _{i of} each image including the virtual space W per gradation.
Is Δ _i . For example, in a virtual space in which a certain image I _k is generated, when a depth of 2.4 m is represented by eight gradations, Δ _k = 2.
4/8 = 0.3 m.

At this time, in order to input the depth information of all the images into the virtual space W, the largest one of Z (max _i ) is Z (max ₀ ) and the smallest one of Z (min _i ) is Z (max _i ). things and Z (min _{0), Δ when i} the smallest of the set to delta _0, between the Z (min ₀₎ and Z (max _0), only the floor 1 gradation becomes delta ₀ It must be expressed in tones. That is, K ₀ =
(Z (min ₀ ) −Z (max ₀ )) / Δ ₀

FIG. 7 schematically shows this state.

The depth value and Δ ₀ represented by Z (min ₀ ) and Z (max ₀ ) may be appropriately selected from those of an arbitrary image, and K ₀ may be a value derived therefrom. in any case, the depth information that was in possession of each image I _i will be subject to some kind of impact.

For example, in FIG. 7 (a), the depth of I ₁ and I ₃ is further subdivided, and in FIG. 7 (b), the interval of the depth information of I ₂ is wide, and the depth of I ₃ is I _3. It will be subdivided.

At this time, If the interval between the depth becomes wide, if that is, Δ _i <Δ ₀ denotes a single multiple gradation in I _i, evaluated at the value. For example, the depth information z ⁱ _j, collectively ^{_{z i j + 1, z i}} j + 2 represented by one gradation, and the like. After that, gradation teaching K
Re-attach to the depth information at ₀ .

[0062] 2. When the depth interval is further narrowed and subdivided, a process of changing, correcting, and editing from the information of the original image is performed as follows. For example, between z ⁱ _j and z ⁱ _{j + 1,} when interpolating information such as new a z ⁱ _{j + 1/2} has a z ⁱ _j, for example, as shown in FIG. 8 (a) When the pixel region and the pixel region having z ⁱ _{j + 1} are adjacent to each other, only an appropriate region from the adjacent boundary may be set to z ⁱ _{j + 1/2} , or FIG. As shown in b), when there is a partial region having the same or similar color information in the pixel region having z ⁱ _j and the pixel region having z ⁱ _{j + 1} ,
The depth information of the portion may newly be z ⁱ _{j + 1/2.} The latter case is an embodiment of the invention of (4) and (8).
Then, re-assigned to the depth information in the floor Torture K _0.

Based on the above processing, the first embodiment,
The same combination processing as in the second embodiment is performed.

The processing flow of this embodiment is similar to that of the first embodiment or the second embodiment.
The difference is that the above-described processing is added at the stage of adding new depth information.

According to the present embodiment, a plurality of images having depth information and a depth range are used without causing unnaturalness in the depth order and the depth distance, and all depth information is used. It becomes possible to synthesize
This has the effect.

Note that some or all of the means shown in the configuration diagrams of FIGS. 1, 4 and 6 can be made to function using a computer (computer), or shown in the flow charts of FIGS. 3 and 5. Needless to say, a computer can execute a processing procedure that is executed by a computer, or a program that causes the computer to function as the means or a program that causes the computer to execute the processing procedure. Medium, for example, FD (floppy disk), MO, ROM, memory card, C
D, DVD, removable disk, etc., and can be provided and distributed.

[0067]

As described above, the present invention has an effect that a plurality of images having depth information can be synthesized without inconsistency in depth.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a first embodiment of the present invention.

FIGS. 2 (a), (b-1), (b-2), (c),
(D) is a figure explaining the example of addition of depth information in the said 1st Embodiment.

FIG. 3 is a flowchart of a process in the first embodiment.

FIG. 4 is a configuration diagram showing a second embodiment of the present invention.

FIG. 5 is a flowchart of a process in the second embodiment.

FIG. 6 is a configuration diagram showing a third embodiment of the present invention.

FIGS. 7A and 7B are diagrams illustrating an example of a depth range and a gradation determination in the third embodiment.

FIGS. 8A and 8B are diagrams illustrating an example of subdivision of depth in the third embodiment.

[Explanation of symbols]

 110 depth information input means 120 depth information providing means 121 depth information storage capacity determination means 122 depth information correction means 130 image synthesis means 140 image display means

Claims (12)

[Claims] 1. A method for combining a plurality of images having depth information using a computer, comprising: constructing a common depth range in the computer; and storing a plurality of images in which different depth ranges are recorded in different storage capacities. Reading in association with the common depth range constructed in the computer and the storage capacity, and newly adding depth information to the read image in accordance with the common depth range constructed in the computer. And combining the plurality of images according to the newly provided depth information, and displaying the combined image. 2. The method according to claim 1, wherein when adding the new depth information, the number of possible values of the depth information to be constructed in the computer is determined according to the number of possible values of the depth information in the read original image. The method according to claim 1, wherein the method is determined. 3. The method according to claim 1, wherein, when the new depth information is added, at least one of a process of changing, correcting, and editing the newly added depth information from information of the read original image. The image synthesizing method according to claim 1 or 2, wherein 4. When the new depth information is added, the same depth information held by the read original image is newly added in accordance with the color information of the pixel having the depth information. The image synthesis method according to claim 3, wherein depth information is added. 5. An apparatus for synthesizing a plurality of images having depth information using a computer, wherein the plurality of images having different depth ranges recorded in different storage capacities are stored in a common depth range built in the computer. Means for reading in association with the storage capacity; means for newly adding depth information to the read image according to a common depth range constructed in the computer; and the newly added depth. An image combining apparatus, comprising: means for combining a plurality of images according to information; and means for displaying the combined image. 6. The means for adding new depth information determines the number of possible values of the depth information to be constructed in the computer according to the number of possible values of the depth information in the read original image. The image synthesizing device according to claim 5, wherein 7. The means for adding new depth information includes at least one of processing for changing, correcting, and editing newly added depth information from information of a read original image. The image synthesizing apparatus according to claim 5, wherein the image synthesizing apparatus performs the processing. 8. The means for adding new depth information, wherein the new depth information is newly added to the same depth information possessed by the read original image in accordance with the color information of the pixel possessing the depth information. The image synthesizing apparatus according to claim 7, wherein information is added. 9. A recording medium recording a method of synthesizing a plurality of images having depth information using a computer, comprising: a procedure for constructing a common depth range in the computer; A plurality of images recorded in the procedure to read in association with the common depth range and the storage capacity built in the computer, and for the read image, in the common depth range built in the computer A step of newly adding depth information in accordance with the information, a step of combining a plurality of images in accordance with the newly added depth information, and a step of displaying the combined image. A recording medium on which an image composition method is recorded. 10. In the procedure for adding new depth information, the number of possible values of the depth information to be constructed in the computer is determined according to the number of possible values of the depth information in the read original image. A recording medium on which the image synthesizing method according to claim 9 is recorded, comprising a process of determining. 11. The procedure for adding new depth information includes one or more of a process of changing, correcting, and editing newly added depth information from information of a read original image. A recording medium on which the image synthesizing method according to claim 9 or 10 is recorded. 12. The procedure for adding new depth information includes the step of adding new depth information to the same depth information possessed by the read original image in accordance with the color information of the pixel possessing the depth information. A recording medium recording the image synthesizing method according to claim 11, further comprising a process of giving depth information.