JPH09214920A - Image communication device - Google Patents

Abstract

(57) Abstract: To provide an image communication device which can perform high quality image communication and has excellent operability. SOLUTION: When an image of a specific color or shape is detected by the pointer detection unit 109, position information of the image is detected by the pointer detection unit 109, and output display by a display VRAM 110 is performed based on the detected position information. The pointing cursor information is overlay-displayed by the image composition processing unit 105 in the entire image of the document original, and the position information is notified to the communication partner side by the system control unit 13, and the display VR of the own side is notified to the notified communication partner.
The pointing cursor information is overlay-displayed by the image composition processing unit 105 in the entire document original image output and displayed by the AM 110.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a videophone device capable of multiplexing multimedia information such as voice information, image information and data and performing mutual communication in real time.
AV (Audio Visual) such as a video conference system
l) relates to communication devices.

[0002]

2. Description of the Related Art In recent years, communication services by ISDN (Integrated Services Digital Network) lines have been put into practical use, and multimedia information communication of image information, voice information, data, etc. using such digital lines has been realized. . ITU-
IT at T (International Telecommunication Union Electric Standardization Division)
As the U-T recommendation H series, service specifications, image encoding methods, multimedia multiplexing structure specifications, protocol specifications, etc. are recommended. In addition, a high-speed processing processor capable of sufficiently performing image processing in real time, compression / compression
Coding chips and the like have come to be supplied at low cost, and image communication is being further activated.

As a typical image communication apparatus, there is a video conference system from the past. However, promotion of the standardization, speeding up of the processing processor, cost reduction of various image processing chips, and personal computer (personal computer) workstation. Combined with the higher functionality and lower cost of PCs, personal computers (hereinafter referred to as PCs) and workstations (hereinafter referred to as W)
Realization of video conferencing and video phone functions over S) is being promoted.

In the video conference system, in addition to voice information communication, a person's image of the other party is displayed according to ITU-T Recommendation H.264. 26
By performing compression coding according to a coding algorithm such as 1 and communicating with each other, the conference proceeds while watching each other's person moving images, and the document manuscript is generally captured by a camera for capturing a document manuscript called a document camera. It is a general communication mode that mutual communication is performed as images and a common document original is displayed on a screen to advance a conference.

In actual meetings, it is often the case that not only the person's image is seen but the common document original is seen, so that clear mutual communication of document originals can be easily performed during the conference. Improvement is required.

[0006] As a mutual communication / display system of document manuscripts,
In a PC / WS-based video conference system, a method of transmitting / receiving and displaying a document created on the PC / WS as it is, or a method of transmitting / receiving and displaying a still image captured by a still video camera in advance is also tried. However, problems such as PC / WS differences, complexity of operations, complexity of pointing to the actually displayed document and replacement of documents, etc. Furthermore, it is expensive to prepare a dedicated camera / scanner separately. And there is a problem that the operation becomes more complicated,
In general, it is not popular at present.

The transmission and reception by a document camera is most widely used as a method in which anyone can easily transmit and receive a document original and can also transmit and receive a three-dimensional object. As the document camera, there are a camera dedicated to the document camera and a camera also used as a person camera.

Conventionally, the document original is set on the document camera or under the document camera, and the focus of the camera is adjusted while observing the screen.
Adjust the zoom level and document original setting position. Normal NT
In the SC / PAL image signal output camera, the character center A
Since the characters cannot be decoded even if the entire document original of 4 size or the like is displayed, the zoom control is performed so that a part of the set document original can be displayed in a decipherable manner.
An image is captured by a normal image signal such as NTSC / PAL, and a method of encoding and transmitting it as a moving image similar to a person image is used. On the receiving side, after decryption,
It is displayed like a normal person image.

[0009] In CCD cameras currently on the market, which are generally equivalent to 380,000 pixels, the number of effective read pixels is generally 450 horizontal resolutions and 350 vertical resolutions. Facsimile (FAX) for simplification of explanation
In terms of resolution, when the image of an A4 size document is picked up, it corresponds to about 2 pel / mm (about 50 dpi). In addition, recent OA monitors such as PC / WS,
A high-definition displayable one has been provided at a low price, and a display such as 1280 × 1024/1024 × 768 is possible. Fax A4 size document
When reading at 200 dpi at 1728 × 2339
The number of dots is about half of this, and if there is a resolution of about 800 × 1000, which is equivalent to about 100 dpi, it is possible to display one document of approximately A4 size on an OA monitor that supports high resolution display. It is known that when a read document is displayed, it can be displayed in a decipherable manner. The image input signal of NTSC / PAL, etc., has an aspect ratio of 4: 3, and an A4 size document original is 210 mm.
The size is 297 mm, and there is no problem with the horizontal document original at about 4: 3, but on the contrary, the vertical document original becomes 3: 4, and the directions do not match.

In the case of an A4 size document original calculated from the above resolution, an attempt to actually input data in four divisions has been made using the fact that a substantially satisfactory image can be obtained by inputting in four divisions. Since the camera is mechanically moved for inputting, there is an error in the mechanical movement, so that the finally synthesized whole image has distortion at the boundary portion.

Further, even for the same A4 size document original, it is necessary to change the image reading method depending on the horizontal and vertical directions, and much less the camera control for different size documents such as B4 and A5 documents.
The movable position of the camera also differs depending on the position on the platen where the document is actually set, but it is left to the user's operation to set, and there is no image communication device that fully considers this point. .

Further, when actually used in a meeting, it is usual to explain by pointing a document original set on a document camera stand with a hand or a pointer. With the document original transmission means
Although the resolution of the document manuscript itself becomes high, pointing is performed by using a mouse, a digitizer or the like while separately referring to the display screen.

[0013]

However, in the above-mentioned conventional apparatus, in an apparatus that transmits and receives a document image as a moving image, an input image is captured by a normal image signal such as NTSC / PAL, and therefore, in an office. In order to display the entire A4 size document manuscript created by a manual or word processor (word processor) that is usually handled, the resolution is too low except for documents with large characters, and it is impossible to capture a readable document manuscript image. Met.
In addition, a document original image is divided and input, and an attempt is made to transmit and receive a high-resolution still document image by synthesizing the input images. Therefore, a large-scale mechanism for moving the entire camera is required, which is expensive. turn into. Furthermore, for the document originals that are actually set, B4, A5, B5 sizes, etc.
There is a problem in that document manuscripts of various sizes and the respective directional differences such as a horizontal direction / a vertical direction depend on the user's operation, and a very complicated operation is required.

The present invention has been made in view of the above problems of the above-mentioned conventional technique, and an object of the present invention is to perform high quality image communication and image communication excellent in operability. It is intended to provide a device.

[0015]

In order to achieve the above object, the image communication apparatus of the first invention of the present invention is an image information input means for inputting image information, and input image information input by the image information input means. Encoding means for encoding, communication means for communicating the image information encoded by the encoding means via a communication line, and decoding means for decoding the received image information received from the communication means, In an image communication apparatus having output display means for outputting and displaying the input image information and the received image information, a camera is used as the image information input means, and when a document original is imaged and input by the camera, the size and direction of the document original And a size / direction detecting unit for automatically detecting the size of the document original based on the size information and direction information of the document original obtained by the size / direction detecting unit. Movable means for sequentially moving the camera so as to be divided into a plurality of times, storage means for sequentially storing the images divided into a plurality of times, and the image divided and taken into a plurality of times as a whole document original. An image synthesizing means for synthesizing one document original image as an image, a transmitting means for encoding the document original image as a still image to the receiving side image communication apparatus as a communication partner through the communication line, and the document. Setting means for automatically setting the focus and zoom adjustment so that the entire document area image is transmitted to the camera while the entire document area image is captured by the camera, and a specific color or shape of the image of the entire document area Monitoring means for monitoring whether or not an image appears, and position information for detecting position information of the image when the monitoring means detects an image of a specific color or shape. Means leaving a first overlay display means for overlays the pointing cursor information as the pointing information to the document original whole in the image output displayed by said output display means based on the position information detected by said position information detection means,
A transmission notification means for notifying the reception side image communication apparatus of the communication destination of the position information, and a document output and displayed by the output display means of the own side on the reception side image communication apparatus notified of the transmission by the transmission notification means. The present invention is characterized by further comprising a second overlay display means for overlay-displaying pointing cursor information as pointing information in the image of the entire original.

In order to achieve the same object, the image communication apparatus of the second invention of the present invention is an image information input means for inputting image information, and an encoding means for encoding the image information input by the image information input means. And an image communication device having a communication means for communicating the coded image information via a communication line, and a determination means for determining the presence or absence of a pointer having a predetermined characteristic in the input image, Pointer information communication means for communicating position information on the image composed of the image information of the pointer determined by the determination means, and presence / absence of movement of the pointer are detected. A transmission unit that transmits the position information on the image.

In order to achieve the same object, the image communication device of the third invention of the present invention is the image communication device of the second invention, wherein the predetermined characteristic is a predetermined color or a predetermined shape. It is characterized by showing.

Further, in order to achieve the same object, an image communication apparatus of a fourth invention of the present invention is the image communication apparatus of the second invention, wherein the image information input means divides the image information into a plurality of areas. In addition, image information is input for each divided area, and the input images are combined to form one frame image.

In order to achieve the same object, the image communication apparatus of the fifth invention of the present invention is the image communication apparatus of the fourth invention, wherein the image input means is the size and orientation of the image including the image information. It is characterized by having a driving means for detecting the position and controlling the position according to the detection result.

In order to achieve the same object, an image communication apparatus according to a sixth aspect of the present invention comprises a first input means for dividing an image into a plurality of areas and inputting the areas, and the entire image. Second input means for inputting, determining means for determining the presence or absence of a pointer having a predetermined characteristic in the image input by the second inputting means, and the inputting means for combining the plurality of input areas And transmitting as a 1-frame image, and when the determination means determines that there is a pointer, the first transmission means for transmitting the position information of the pointer, and the pointer after the transmission by the first transmission means Second transmitting means for detecting whether or not the pointer has moved, and for transmitting the pointer position information after the movement when the pointer has moved.

In order to achieve the same object, an image communication apparatus according to a seventh aspect of the present invention is the image communication apparatus according to the sixth aspect, wherein the first transmitting means has an encoding section,
Moreover, it is characterized in that the encoded image is transmitted.

In order to achieve the same object, an image communication apparatus of an eighth invention of the present invention is the image communication apparatus of the sixth invention, wherein the first input means and the second input means are common. Is an input means of.

In order to achieve the same object, the image communication device of the ninth invention of the present invention is the image communication device of the sixth invention, wherein the predetermined characteristic indicates a predetermined color or a predetermined shape. It is characterized by

In order to achieve the same object, the image communication device of the tenth invention of the present invention is the image communication device of the sixth invention, wherein the first transmission means uses the position information of the pointer for the one frame. Is converted into position coordinates on the image and transmitted.

Further, in order to achieve the same object, an image communication apparatus of an eleventh invention of the present invention is the image communication apparatus of the sixth invention, wherein 1 is transmitted by the first transmitting means.
It is characterized by further comprising display means for displaying an image in which pointer information is superimposed on a frame image.

In order to achieve the same object, the image communication device of the twelfth invention of the present invention is the image communication device of the sixth invention, wherein the one frame image is input by the second input means. The image is characterized by higher resolution than the image.

[0027]

Embodiments of the present invention will be described below with reference to the drawings.

(One Embodiment) First, one embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a video conference apparatus as an image communication apparatus according to an embodiment of the present invention. In the figure, 1 is a handset which is one of the voice input / output means of the present device, 2 is a microphone (microphone) which is one of the voice input means of the present device, 3
Is a speaker which is one of the audio output means of this apparatus, and 4 is an audio interface section, which is instructed by a system control section 13 which will be described later.
Switching process for switching between the microphone 2 and the speaker 3, an on / off hook detection process for detecting whether the handset 1 is in an on-hook state or an off-hook state, and an echo is erased when the microphone 2 and the speaker 3 are used as voice input / output means. For example, an echo canceling process for generating a tone, a tone generating process such as a dial tone, a ringing tone, a busy tone, and a ringing tone are performed.

Reference numeral 5 is a voice encoding / decoding unit, which is instructed by the system control unit 13 and is a 64 kbps PCM A-1.
aw / μ-law, 64 kbps / 56 kbps / 48
kbps SB-ADPCM, 16 kbps LD-C
According to voice encoding and decoding algorithms such as ELP,
The transmission voice signal is encoded and the reception voice signal is decoded. Reference numeral 6 is a person camera for inputting a self-portrait and the like, which is one of the image input means of the apparatus, and 7 is a document camera for inputting a document original image and the like, which is another image input means of the apparatus. 8 is a display unit, which is a portrait camera 6
Alternatively, an input image from the document camera 7, an image received from the other party, an operation screen, a shared screen, etc. are displayed. A video interface unit 9 is a switching process for switching between the person camera 6 and the document camera 7, which are image input means, according to an instruction from the system control unit 13, and a display switching for switching display between an input image, a received image, an operation screen, and a shared screen. Processing is performed.

Reference numeral 10 denotes a video editing & processing section, which is a mirror image processing of an input image from the person camera 6, various synthesizing processing of divided input images from the document camera 7, detection processing of a specific image in the input image, input with a received image. It is for performing a combining process with an image, an overlay process of an image with an operation screen instructed by the system control unit 13, graphic data such as pointing information, and the like. Reference numeral 11 denotes a video encoding / decoding unit, which is ITU-T recommendation H.264. The transmission image is encoded and the reception image is decoded according to an encoding algorithm such as H.261. An operation unit 12 includes a keyboard used for inputting control information and various data for controlling the entire apparatus, a mouse for controlling pointing, a touch panel, a digitizer, a trackball, and the like. A system control unit 13 includes a CPU (central processing unit), a ROM (read only memory), a RAM (readable / writable memory), etc., monitors the state of each unit, controls the entire apparatus, and operates / operates according to the state. The display screen is created and the application program is executed.

Reference numeral 14 is a multiplexing / demultiplexing unit, which sends a control command from the system control unit 13 to BA when the ITU-T recommendation is complied with.
S code is converted to ITU-T Recommendation H.264. 221
The BAS code, the audio signal from the audio encoding / decoding unit 5, the image signal from the video encoding / decoding unit 11,
The control signal from the system control unit 13 is multiplexed for each transmission frame, the reception frame is separated into the above-mentioned constituent elements, and each of them is provided with a voice encoding / decoding unit 5 and a video encoding / decoding unit 11. , And to the system control unit 13, respectively. A line interface unit 15 controls a line according to the ISDN user / network interface / PSTN various modem interfaces by exchanging commands with the system control unit 13. Reference numeral 16 denotes a storage unit such as an HD (hard disk) for storing various control information of the entire apparatus, reception / transmission voice information / image information / pre-input document, and the like.
17 is ISDN / PSTN / to which this device is actually connected
A line such as a LAN.

FIG. 2 is a functional block diagram showing a video input / video editing portion from the document camera 7 in the video conference apparatus of FIG. In the figure, reference numeral 101 denotes a camera movement control unit, which controls execution of pan / tilt / zoom operations of the document camera 7 based on an instruction from the system control unit 13. Reference numeral 102 is a document size / direction detection unit that detects the size and direction of a document document placed on a document camera stand (document table), 103 is a video switch that switches the input between the portrait camera 6 and the document camera 7, and 104 is an NTSC. Analog NTS from the person camera 6 and the document camera 7 at the decoder
The C image signal is sampled and digitized so that it can be processed. 105 is an image composition processing unit, which is an NT
After the image signal digitized by the SC decoder 104 is stored in the memory 107 described later via the memorial access control unit 108 described below, overlay synthesis of divided input images and the position extracted by the pointer detection unit 109 described below are performed. After converting the coordinates to the resolution of the display still image for the coordinates or for the pointer information received from the partner device via the communication line, overlay the pre-registered graphic pointing image, etc. Various types of image composition processing are performed.

Reference numeral 106 denotes an image area boundary detection unit which detects a boundary guide indicating an image area from the divided images input by division.
107 is a memory for storing the digitized image, 10
Reference numeral 8 is a memory access control unit that controls access to the memory 107 such as writing and reading, and 109 is a pointer detection unit, which is the color information or shape that should be registered in advance to detect the entire image of the document original captured in the memory 107. It is searched for whether or not the one that matches the information appears, and when it appears, the position coordinate is detected and the system control unit 13 is notified. Reference numeral 110 denotes a display VRAM (video RAM) for displaying an image on the display unit 8.

Although the case where the person camera 6 and the document camera 7 are separately provided is shown for simplification of description, one camera may be used to switch between the person camera mode and the document camera mode. At this time, the video switch 103 operates as an input mode changeover switch. Further, the image composition processing unit 105, the image area boundary detection unit 106, the pointer detection unit 109, etc. may be commonly processed by a high-speed video processor or the like.

FIGS. 3 to 5 are schematic views showing the structure of the upper surface of the document table. In each drawing, 201 is a document camera stand as a document table, and 202 is the document for detecting the size and direction of the document document. A document guide for fixing the document, 203
Reference numerals -1, 203-2, 203-3, 203-4 are slides for moving the document guide 202, and 301 is a center point of the document camera stand 201. 301-P, 301-
L, 301-A4P, 301-A4L, 301-A3
P, 301-A3L is the center point of each document original when various document originals are set on the document camera stand 201.

FIG. 6 to FIG. 10 are conceptual diagrams of the operation / direction of the document camera 7 and the divided input image when the A4 size document original is actually divided into four images.

FIG. 11 is a diagram showing an image of a document original document stored in the memory 107 after division input and combination of an A4 size document original, and in FIG. 11, A1 is a high-resolution A4 landscape image combined after division imaging. Directional document original still image area, A2 is A4 Lateral entire document original still image area to be imaged and stored in the memory 107, a1, b1, c1, d1
Is the divided image area actually divided and input, 501
Indicates color information / shape information registered in advance to be detected by the pointer detection unit 109.

FIG. 12 is a conceptual diagram of a pointing image overlay display procedure when the color information / shape information 501 is actually searched and detected in the A4 size document original whole image A2. A3-1 is a display image on the self side, A
3-2 is a communication partner display image.

13 to 16 are flowcharts showing the operation of the video conference apparatus of FIG.

Next, the operation of the video conference apparatus according to the present embodiment will be described with reference to the drawings.

First, in step S1301 of FIG. 13, a document original is set on the document camera stand 201, and the video switch 103 is operated to switch the image input to the document camera mode. Next, in step S1302, the document guide 202 is slid (203-1, 203-2, 203-3, 20).
3-4) Move along and align with the document original. Next, in step S1303, the document size & direction detection unit 102 detects the document size & direction of the document document based on the position of the document guide 202. Next, step S1304.
In step S1332 of FIG. 16, which will be described later, it is determined whether the document original is an A4 horizontal document.
If the document is an A4 horizontal document, the process proceeds to the next step S130.
Go to 5. In step S1305, the orientation of the document camera 7 is switched to a direction suitable for capturing an A4 landscape document by the control of the camera movement control unit 101.

Next, in step S1306, the camera focus & zoom image pickup area for A4 lateral document original image division input is set. Then, first in step S1307,
As shown in FIGS. 6A and 7A, the regions a1 and b
Based on the information stored in the storage unit 16, the system control unit 13 determines to sequentially shoot 1, c1, and d1. Then, first, in order to capture an image of the first read frame region a1, the document camera 7 is pan-tilted to adjust the camera focus to 401-1 in FIG. 7A, and the zoom is performed so that the entire image capturing region matches the region a1. To match. Next, in step S1308, the image of the area a1 is loaded into the memory 107, and the division boundary area guide 403-in FIG.
The image boundary detection unit 1 detects the images 1, 403-5, and 403-8.
It is detected by 06.

The detected image position is notified to the image composition processing unit 105, and based on the document size and direction information of the document document and the memory write address information of the first frame from the system control unit 13, the image composition processing unit 105. Rewrites in the memory 107 in step S1309 so that each boundary guide is set to a fixed memory address. For example, the center of the boundary guide 403-1 is set to the address (X1
1, Y11), the center of the boundary guide 403-5 is the address (X12, Y11), and the center of the boundary guide 403-8 is the address (X11, Y12). Rewriting is performed on the memory 107 via the memory access control unit 108.

Next, in step S1310, the document camera 7 is adjusted to focus the camera on 401-2 in FIG. 7A in order to capture an image of the region b1 of the second read frame.
Is pan-tilted and the zoom is adjusted so that the entire imaging region matches the region b1. Next, in step S1311, the image of the area b1 is loaded into the memory 107, and
Division boundary area guides 403-2 and 403-in (a)
The image boundary detection unit 106 detects images 5 and 403-6. The position of the detected image is set to the image synthesis processing unit 10
5, the image composition processing unit 105 sets each boundary guide to a fixed memory address in step S1312 based on the document size & direction information of the document document and the memory write address information of the second frame from the system control unit 13. So that it is rewritten on the memory 107. For example, the center of the boundary guide 403-2 is set to the address (X13,
Y11), the center of the boundary guide 403-5 at the address (X
12, Y11) and the center of the boundary guide 403-6 are rotated as needed so that the center of the boundary guide 403-6 becomes the address (X13, Y12), and then rewritten on the memory 107 via the memory access control unit 108. I do.

Further, at this time, as shown in FIG. 7B, rectangular regions (X12-2, X12) sandwiching the boundary guide 403-5 are sandwiched.
12-1), (X11-1 / Y11-2, Y12-2 /
Since Y12-1) is captured in the memory 107 in an overlapping manner with the image region captured in the region a1, when the region a1 is written in the memory 107 by the image composition processing unit 105, the image of the rectangular region is Overwrite and write.

Next, in step S1313 of FIG. 14, in order to image the area c1 of the third read frame,
The document camera 7 is pan-tilted to adjust the camera focus to 401-3 in FIG. 7A, and the entire imaging area is the area c1.
Adjust the zoom to match. Next, step S
At 1314, the image of the area c1 is loaded into the memory 107, and at the same time, the division boundary area guide 403-3 in FIG.
The image boundary detection unit 106 converts the images 403-6 and 403-7.
Is detected by The position of the detected image is notified to the image composition processing unit 105, and based on the document size & direction information of the document document and the memory write address information of the third frame from the system control unit 13, the image composition processing unit 105 performs step S1315. Then, rewriting is performed on the memory 107 so that each boundary guide is set to a fixed memory address. For example, the center of the boundary guide 403-3 is the address (X13, Y13), the center of the boundary guide 403-6 is the address (X13, Y12), and the center of the boundary guide 403-7 is the address (X12, Y13). The memory access control unit 10 is also subjected to rotation processing if necessary.
Rewriting is performed on the memory 107 via No. 8.

Further, at this time, as shown in FIG. 7B, a rectangular area (X12-2 / X) sandwiching the boundary guide 403-6 is provided.
12-3, X13-2 / X13-3), (Y12-3,
Y12-2) is the image area captured in the area b1, and rectangular areas (X12-3, X12-1) and (Y12-3, Y).
Since the image of the portion 12-1) is captured in the memory 107 in an overlapping manner with the image region captured in the region a1, the image synthesis processing unit 105 stores the region c1 in the memory 107.
When writing to, the rectangular area is subjected to image overlay processing and written.

Next, in step S1316, the region d1 of the fourth read frame, which is the last frame of the A4 horizontal document original read, is imaged. Therefore, the camera focus should be set to 401-4 in FIG. 7A. The document camera 7 is pan-tilted and the zoom is adjusted so that the entire imaging area matches the area d1. Next, in step S1317, the image of the area d1 is loaded into the memory 107, and
Division boundary area guides 403-4 and 403-in (a)
The image boundary detection unit 106 detects images 7 and 403-8. The position of the detected image is set to the image synthesis processing unit 10
5, the image composition processing unit 105 sets each boundary guide to a fixed memory address in step S1318 based on the document size and direction information of the document document and the memory write address information of the fourth frame from the system control unit 13. So that it is rewritten on the memory 107. For example, the center of the boundary guide 403-4 is set to the address (X11,
Y13), the center of the boundary guide 403-7 is the address (X
12, Y13) and the center of the boundary guide 403-8 is rotated as necessary so that the center of the boundary guide 403-8 becomes the address (X11, Y12), and then rewritten on the memory 107 via the memory access control unit 108. I do.

Further, at this time, as shown in FIG. 7B, a rectangular area (X11-4 / X) sandwiching the boundary guide 403-8 is provided.
11-1, X12-1 / X12-4), (Y12-4,
Y12-1) is an image area captured in the area a1 and a rectangular area (X12-3, X12) sandwiching the boundary guide 403-7.
12-4), (Y13-4 / Y13-3, Y12-3 /
The image of the portion (Y12-4) is the image area captured in the area c1, the rectangular areas (X12-2, X12-4), and (Y
12-2, Y12-4) portion of the image is captured in the memory 107 in an overlapping manner with the image region captured in the region b1, so the image synthesis processing unit 105 stores the region c1 in the memory 107. At the time of writing, the rectangular area is subjected to image overlay processing and then written.

Next, in step S1319, the above 4
The image is divided and imaged, and the image synthesis processing is performed.
The rectangular area (X11, X1 shown in FIG. 11 surrounded by the divided image boundary guides in FIG.
X13), (Y11, Y13), which are registered in advance in the storage unit 16 and from the system control unit 13 to the image synthesis processing unit 10.
Image areas (X1s, X1e) and (Y1
s, Y1e) is displayed as a still image of an A4 horizontal document original in which the entire portion matches the actual A4 horizontal document original VRAM
The data is transferred to 110 and displayed on the display unit 8, treated as an input still image to the video encoding unit 11, and transmitted to the communication partner.

Next, proceeding to step S1320 of FIG. 15, the system control unit 13 causes the camera movement control unit 10 to focus the document camera 7 on FIG. 5301-A4L.
1, and the entire A4 horizontal document original, that is, FIG.
The zoom is set so that the rectangular area portion (X1s → X1e, Y1s → Y1e) shown in FIG.
Next, proceeding to step S1321, it is determined whether or not the document camera mode has ended. If completed, this processing operation ends, and if not completed, processing proceeds to step S1322. In step S1322, the entire A4 document original area image is written in the memory 107 as the captured image area A2 in the camera mode set in step S1320.

Next, in step S1323, it is searched whether or not the color information / shape information registered in advance appears in the captured image area A2. When a monochrome document is set and a conference is held, it is possible to search very efficiently by using a pointer that is colored in advance.
The same procedure can be used for color document manuscripts,
At this time, it is effective that the pointer detection unit 109 detects not only the specific color but also the motion of the specific color. Further, a still image in which only the document original is set is separately captured as an A2 ′ image, and this A2 ′ is captured.
It is also effective to always compare the image with the newly captured A2 image and detect by the difference occurrence of the specific color. Also,
It is also possible to consider means for searching by taking into consideration not only color information but also shape.

Next, in step S1324, it is determined whether or not there is a corresponding pointer by the above search. If not, the process returns to step S1321. If there is, the process returns to step S13.
In step 25, the detected pointer position coordinates are detected. Next, in step S1326, it is determined whether or not the detected position coordinates are the same as the previously detected position coordinates in the same / error range, and if they are the same, the process returns to step S1321 and the same. If not, step S13
Proceed to 27. In step S1327, the position coordinates of the currently displayed high resolution document original still image are converted. Next, in step S1328, the previously registered graphic pointing image 502 is overlay-displayed on the position coordinates on the document original still image calculated above. Next, in step S1329, the above-mentioned step S132.
The pointer position coordinates detected in 5 are transmitted to the communication partner device as a command.

Next, in step S1330, the receiving side device converts the position coordinates of the received pointing position command into the position coordinates of the still image of the high resolution document original, and then overlays the graphic pointing image. Next, proceeding to step S1331, the entire image of the A4 document original is loaded into the memory 107,
After searching for the corresponding pointer, the process returns to step S1321.

When there is no corresponding pointer in step S1324, or when the pointer is detected in the previous search, the overlay display is stopped on the display side of the own device and the pointing position disappears to the other device. Send a command.

If the document original detected in step S1304 of FIG. 13 is not the A4 lateral document original, the process proceeds to step S1332 of FIG. 16 and the document original whose orientation is detected by the document camera 7 is detected. Switching to a direction suitable for the size & direction, and in the next step S1333, the division imaging method corresponding to each document size & direction is selected to sequentially perform division imaging & creation of composite image. Next, step S1
After the high-resolution document original image completed in 334 is displayed and encoded and transmitted, the focus and zoom setting is performed so that the entire relevant document original matches the imaging area. Next, in step S1335, the entire image of the document original is fetched, and the next step S1336.
In step S1321 to step S1331 of the pointer detection method for the A4 horizontal document shown in FIG. 15, the parameters are replaced with parameters for each document size and direction, and the process is repeated.

(Other Embodiments) In the above embodiment, as a camera for inputting image information, a document original for outputting a normal television image signal such as NTSC / PAL which can be currently obtained at a low price. Although it was described as a high-resolution image acquisition means using a low-resolution camera for imaging, it is still expensive, but it can be applied to a higher-resolution camera such as a digital camera having a resolution of 1.6 million pixels. . For example, if the camera itself is applied to a camera having four times the resolution of the current one, the resolution of A4 document original reading is 200 dpi if it is replaced with a facsimile.
It is possible to a considerable extent, and it is also possible to easily realize an image communication device capable of reading document originals and performing real-time communication with a simple operation and a reading resolution closer to that of a scanner.

[0058]

According to the first aspect of the present invention, the presenter can point to the document original actually placed on the original table, and the high-resolution still image is not affected at all. In addition, it is possible to transmit and receive pointing information and display them in an overlay to see the same image, and it is possible to perform very high quality image communication, improve operability, and provide at low cost. Produce an effect.

According to the second aspect of the present invention, it is possible to directly specify points on the image, and it is possible to improve the operability of specifying points on the input image with a mouse or the like. In addition, after the image is transmitted, only the information of the pointer is transmitted only when the pointer is moved, so that it is possible to perform more efficient communication as compared with the case where the image including the pointer is continuously transmitted.

According to the third aspect of the invention, it is possible to detect a good pointer.

According to the fourth aspect of the invention, the image is divided into a plurality of regions and the input image is one frame, so that it is possible to capture a high resolution image.

According to the invention of claim 5, since the image input means detects the size and orientation of the input image and controls the position of the image input means according to the detection result, the user does not need to perform a complicated operation. Good image input is possible.

According to the invention of claim 6, an image is divided into a plurality of regions and input, and the plurality of regions are combined to transmit a one-frame image. Therefore, high-resolution image communication is possible. . Furthermore, the amount of communication information can be reduced because the position information of the pointer is transmitted when the pointer moves.

According to the invention of claim 7, it becomes possible to transmit an image with a small amount of information.

According to the invention of claim 8, the device configuration can be simplified.

According to the invention of claim 9, it is possible to detect a good pointer.

According to the tenth aspect of the invention, the pointer processing can be easily combined on the side of the device that receives the transmitted image.

According to the eleventh aspect of the invention, the superimposed image of the transmitted image and the pointer can be confirmed on the transmitting side.

According to the twelfth aspect of the present invention, the transmitted image can be a high-resolution image, and the pointer detection that does not require such high-resolution detection is detected by the rough image.
It is possible to prevent the pointers from being transmitted more than necessary.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a video conference apparatus as an image communication apparatus according to an embodiment of the present invention.

FIG. 2 is a functional block diagram of an image input and processing unit in the video conference device of FIG.

FIG. 3 is a diagram showing a configuration of a document table upper surface on which a document of a document camera is set.

FIG. 4 is a diagram showing a configuration of a document table upper surface on which a document of a document camera is set.

FIG. 5 is a diagram showing a configuration of a document table upper surface on which a document of a document camera is set.

FIG. 6 is a conceptual diagram of an operation / direction of a document camera and a split input image when split-capturing an A4 document original.

FIG. 7 is a conceptual diagram of a divided input image when an A4 document original is divided and imaged.

FIG. 8 is a conceptual diagram of a divided input image when an A4 document original is divided and imaged.

FIG. 9 is a conceptual diagram of divided input images when an A4 document original is divided and imaged.

FIG. 10 is a conceptual diagram of a divided input image when an A4 document original is divided and imaged.

FIG. 11 is a memory storage configuration diagram of a divided captured image, a composite document original image, a difference / motion detection image, and the like.

FIG. 12 is a conceptual diagram of pointing detection on a document document placed on a document table.

13 is a flowchart showing an operation of the video conference device of FIG.

FIG. 14 is a flowchart showing an operation of the video conference apparatus of FIG.

15 is a flowchart showing an operation of the video conference apparatus of FIG.

16 is a flowchart showing an operation of the video conference apparatus of FIG.

[Explanation of symbols]

6 People Camera (Image Information Input Means) 7 Document Camera (Image Information Input Means) 8 Display Section (Output Display Means) 11 Video Encoding / Decoding Section (Encoding Means, Decoding Means) 13 System Control Section (Transmission Means) , Transmission notification unit, setting unit) 15 line interface unit (communication unit) 16 storage unit (storage unit) 102 document size / direction detection unit (size / direction detection unit) 105 image combination processing unit (image combination unit, overlay display unit) ) 109 pointer detection unit (monitoring means, position information detection means) 110 display VRAM (output display means)

Claims (12)

[Claims] 1. An image information input unit for inputting image information, an encoding unit for encoding the input image information input by the image information input unit, and a communication line for transmitting the image information encoded by the encoding unit. Communication means for communicating via
An image communication apparatus comprising: a decoding unit that decodes received image information received from the communication unit; and an output display unit that outputs and displays the input image information and the received image information,
A camera is used as the image information input means, and a size / direction detecting means for detecting the size and direction of the document original when the document original is picked up and input by the camera;
Movable means for sequentially moving the camera so as to automatically perform imaging of the document original in a plurality of times based on the size information and the direction information of the document original obtained by the direction detection means; Storage means for sequentially storing the divided and captured images, image synthesizing means for synthesizing the plurality of divided and captured images into one document original image, and the document original image for the communication line A transmitting unit that encodes and transmits as a still image to the receiving-side image communication apparatus that is a communication partner via the camera, and transmits the entire image of the document original and focuses the camera so that the entire area of the document original becomes an image capturing area. Setting means for automatically setting zoom adjustment, and monitoring means for monitoring whether or not an image of a specific color or shape appears in the image of the entire document original area Position information detection means for detecting position information of an image of a specific color or shape detected by the monitoring means, and output by the output display means based on the position information detected by the position information detection means First overlay display means for overlaying pointing cursor information as pointing information in the displayed entire image of the document original; transmission notifying means for notifying the receiving side image communication apparatus of the position information of the position information; The receiving side image communication device notified by the notifying means is provided with a second overlay displaying means for displaying pointing cursor information as pointing information in the whole image of the document original output and displayed by the output displaying means of the own side. An image communication device characterized by the above. 2. An image information input means for inputting image information, an encoding means for encoding the image information input by the image information input means, and the encoded image information are communicated via a communication line. An image communication apparatus having a communication means, wherein a determination means for determining the presence or absence of a pointer having a predetermined characteristic in the input image, and an image formed by the image information of the pointer determined by the determination means It has a pointer information communication unit for communicating position information, and a transmission unit for detecting the movement of the pointer and transmitting the position information of the pointer on the image when the pointer moves. Image communication device. 3. The image communication apparatus according to claim 2, wherein the predetermined characteristic indicates a predetermined color or a predetermined shape. 4. The image information input means divides the image information into a plurality of areas, inputs the image information for each divided area, and combines the input images into one frame image. Item 2. The image communication device according to item 2. 5. The image input means comprises drive means for detecting the size and orientation of an image composed of the image information, and for performing position control according to the detection result. Image communication device. 6. A first input means for separating an image into a plurality of areas and inputting each area, a second input means for inputting the entire image, and an input by the second input means. Judgment means for judging the presence or absence of a pointer having a predetermined characteristic in the image, and the input means for synthesizing a plurality of input areas and transmitting them as one frame image, and the judgment means judges that there is a pointer. In the case where the pointer is moved, the first transmission means for transmitting the position information of the pointer, and whether or not the pointer has moved after the transmission by the first transmission means, and the pointer has moved And an image communication device having second transmitting means for transmitting the pointer position information after the movement. 7. The image communication apparatus according to claim 6, wherein the first transmitting unit has an encoding unit and transmits an encoded image. 8. The first input means and the second input means are common input means.
An image communication device according to claim 1. 9. The image communication apparatus according to claim 6, wherein the predetermined characteristic indicates a predetermined color or a predetermined shape. 10. The image communication apparatus according to claim 6, wherein the first transmission means converts the position information of the pointer into position coordinates on the image of the one frame and then transmits the position coordinates. 11. The image communication apparatus according to claim 6, further comprising display means for displaying an image in which pointer information is superimposed on one frame image transmitted by the first transmitting means. 12. The image communication apparatus according to claim 6, wherein the image of one frame is an image having a higher resolution than the image input by the second input means.