JPH08279999A - Video conference multimedia system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform zoom-up and panning, etc., without physically moving a camera system by wide-band transmitting image data obtained by a camera provided with a wide angle lens and performing an image conversion processing on a transmission side or a reception side. SOLUTION: The camera 2 is provided with the wide angle lens (fisheye lens, preferably) and digitizes and outputs the image data obtained by a CCD sensor. An image processing circuit 3 compression-processes the image data and a transmitter 4 transmits the entire signals including voice interleaved by a controller 5 through a wide band width network to a reception part 20. A receiver 21 receives the signals from plural video sources and the controller 24 supports the entire video/audio system. The image processing circuit 22 corrects the distortion of a spherical lens from received wide angle images and converts them to plane images. Also, corresponding to an instruction from an input device 25, a part of the images is selected and the images are re- mapped as if a still camera is moving.

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION The present invention relates generally to home or office videoconferencing, and more particularly to a camera having a wide-angle lens used at a videoconferencing site to "capture" (observe) a scene at the videoconferencing site. , Home or office video conferencing systems. The scene can be altered by electromechanically zooming up or panning the electrotechnically captured scene without physically moving the camera on the transmit side. The decision to select the scene is
Can be selectively changed according to operator constraints and requirements.

[0002]

BACKGROUND OF THE INVENTION Teleconferences provide two methods of sight and sound, usually graphic communication between individuals or groups. Video teleconferencing facilities range from large conference rooms designed for small theaters with high quality lighting, audio equipment, and commercial broadcasting facilities to personal computers with video equipment.

Modern devices send video over fiber optic cables to video modems where it is converted to electrical signals for coaxial cable for display on a desktop. Local switching of video signals around buildings, office parking lots, or campuses is provided by broadband switches that switch the light pulses carried by fiber optic cables. This operation is typically performed by converting light pulses into electronic pulses. This conversion is performed electro-technically and converted back into light pulses for transmission in the optical fiber.

Video conferencing generally uses a fixed camera with a limited field of view, or a complex and expensive electronic device, and a bulky camera mounted on top of the display screen of a video system. Held in a video conference room.

However, some of these systems allow the operator to control the camera at other positions, thus "looking around" and observing other things of interest to the operator's needs and desires. . Large mechanical gears and control systems are needed to move the camera from one position to another. This makes the system difficult to assemble and configure. Moreover, the cost of manufacturing such a system is very high.

In the first conventional system, the still camera has a fish-eye lens to capture the image to be processed and panned, while in the second conventional system, the still camera captures an image as if it were actually moving. It has a remapping system for processing. Other systems have a spherical lens system that includes a processor to correct spherical lens distortion,
You can see the magnified image as well as part of the image obtained with the spherical lens.

Still other systems have multiple cameras used to develop images that are combined together. Some of the combined images can be viewed independently of the entire combined image.

However, as explained in detail above, the problem with conventional systems is that there is no method and apparatus by which video conferencing can be performed easily, as well as the ability to improve the use of computing systems and the ability to retrofit existing offices. There is no.

In addition, current systems lack the ability to transmit an overall wide-angle image, and the ability to process a wide-angle image as a planar image at the end receive. Currently, conventional systems are capable of transmitting and processing only planar images. Therefore, a significant drawback of the conventional system is that it uses a camera with a fisheye (wide-angle) lens, but sends degraded image signal information to the receiver.

The invention up to now has a high bandwidth (for example, 20M).
A network with a bandwidth of B / s or more) is available and all information gathered by the camera and wide-angle lens (eg, an image of the entire office) can be sent to the receiver.

Another problem with conventional systems is that there is only a mechanical way to scan the sender. In the inventions up to now, there has been no method (or means) for electrically scanning the transmitting side by adding a logic circuit to the camera chip.

[0012]

SUMMARY OF THE INVENTION It is an object of the present invention to provide an improved video conferencing device and multimedia system over current video conferencing systems in terms of cost, complexity, ease of use and functionality. It is to be. The present invention is less expensive than conventional systems and can be easily retrofitted to existing systems.

Another object of the present invention is a video conferencing device and system using a high bandwidth network that is capable of transmitting to a receiver all information gathered by a camera and a wide angle lens (eg an image of an entire office). Is to provide. At the receiving end, the image is processed by receiving end hardware and control signal inputs from the receiving end party.

[0014]

According to a first aspect of the present invention, there is provided a camera with a wide-angle lens for use in a videoconference scene for capturing an entire scene of the videoconference scene. The field of view can be selectively changed by electromechanically zooming up or panning an electrotechnically captured scene without physically moving the camera.

Therefore, the conventional system does not require the complicated mechanical procedure and troublesome setting which are required to scan the relevant parts in the room. Further, the decision to select a field of view can be selectively varied according to operator constraints and needs.

As described above, the video conference apparatus and the multimedia system of the present invention are inexpensive, simple, easy to use, and have high functions as compared with the conventional system. The present invention can be easily modified along with existing systems. The present invention uses a high bandwidth network capable of transmitting to the receiver the images of the entire office collected by a camera (with a wide angle lens) at the transmission end. At the receiving end, the image is processed by receiving end hardware and control signal inputs from the receiving end party.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Referring now to FIG. 1, a video communication conference system used by an operator is shown. The system is connected to a network that supports multimedia operations. In general,
It is conceivable that two such systems will be provided. One at each receiver end and the other at the transmitter end. Of course, it is conceivable that the transmitter and receiver at each end are a single united unit. Generally, the transmitting end and the receiving end are physically separated from each other. The connection of the two locations is a two-way interactive voice and video communication.

The television communication conference system according to the present invention comprises:
As shown in FIG. 1, it has a transmitter 10 and a receiver 20.

The transmitter 10 includes a lens 1, an image processing circuit 3,
It has a transmitter 4, a controller 5 and an input device 6 used by the operator, which will be described later.

The receiving unit 20 has a receiving device 21 for receiving an input from the transmitting device 4, an image processing circuit 22, a display device 23, a control device 24 coupled to the image processing circuit, and an input device 25 used by an operator.

Looking at details of the transmitter 10, a lens (eg, a wide angle of view, preferably 180 near the eyes of the fish)
A fisheye lens with a degree of view is attached to the case of the camera 2.

Preferably, it is made of plastic so that it is easily portable and has a lightweight but permanent construction that can be mounted on the transmitter side. Another suitable material for the case has a frame around the video screen. Preferably, the lens of the video screen (camera and screen integration) is formed on the side with a plastic case or with a part of plastic. This is because it can be easily integrated.

The lens of camera 2 is for producing an incoming image. Lenses are available from Nicon, Inc. The imaging device of the camera 2 is formed by a lens, and especially the image sensor is formed by a CCD (charge-coupled device) attached to a case as an image detection device. CCD is a known device,
oration, or available from Polaroid Corporation,
Works in a known way.

A CCD, for example, generally has a temporarily stored charge (eg, parasitic capacitance between the gate of a metal oxide semiconductor field effect transistor (MOSFET) and the substrate).
Works by passing along. Thus, as is known, photocharges are generated by radiative incidence on a register and the charge is displaced (eg parallel to the storage register, farthest from the output register, etc.). These devices are typically interconnected in strings so that the pulse is applied to a particular input (eg, clock or strobe) that causes the charge to be transferred from the stage to the next. Multiple CCD chips are provided. The preferred format of the image signal is National Television Systems Commi
ttee (NTSC) method, for example Texas Instrument
Obtained from the chip set available from s.

As mentioned above, the image sensor has a light receiving surface, preferably the focal plane of the lens. Camera 2 is Vid
Convert any optical image available from ek, Inc. into electronic memory data for digital cameras etc.
A device that captures sources and images.

The camera 2 can be easily attached to a display screen or an appropriate position in an office or home by using an appropriate fixing material. For example, "Velcro" can be used because it can be easily attached and detached from the screen.

The image processing circuit 3 is connected to the camera 2, and CC
Integrated with D camera 2 to become part of it, or C
It is connected next to the CD camera 2. The image processing circuit 3 is Texa
TMS34061 and T available from s Instruments
Extra large scale integrated circuit (VL) such as MS44C251
SI) Placed on the chip. It is a separate card that is either incorporated into one of the additional cards to connect to the workstation or inserted into the controller 5. The controller 5 is plugged into a slot of a suitable personal computer system or laptop or terminal with sufficient memory to support video / audio applications. It also has appropriate software, including device drivers. The software provides the ability to receive multiple sources of video information and transmit the video information to multiple remote computers. The characteristics of the video information are tailored to the requirements of each remote user.

After being processed, the wide-angle video image signal is transmitted in its entirety by the transmitter 4. The transmitter is a combined picture, available from, for example, C-cube, Inc.
It has a known compression scheme such as Expert Group (JPEG) or Moving Picture Expert Group (MPEG).

The incoming video image signal is received by the receiver 20, in particular by the receiver 21. A high bandwidth network is thus adopted and has the ability to send all the information gathered by the camera 2 and the wide angle lens 1 to the receiver 20 at the receiving position. In the receiver 20, the image is processed by the input from the receiving end hardware and the receiving end party.

In particular, the received video image is an image processing circuit 22 that converts the received wide-angle picture into a planar image.
Processed by suitable electronics having
The above circuit is preferably a Power PCRI based on a microcontroller, appropriate application specific electronics and display driver electronics.
Has a microcontroller like SC. An example of the above circuit is the CLGD 547 available from Cirrus Logic.
0, 5471 and 5472.

The optical equations for "planarizing" a wide-angle image electrically are known and disclosed by US Pat. No. 5,185,667 and can be readily used by those skilled in the image processing circuitry of the present invention.

The image processing circuit 22 is operated by the operator.
Allows selection of a portion of the incoming image so that the image is selectively "rectified" (eg magnified, focused, etc.). These features are also known and disclosed by US Pat. No. 5,185,667. The image processing circuit 22 is a microprocessor or a control device 2 of the same type.
Under 4. The image processing circuit is preferably a Power PC microcontroller available from IBM.

This selection is made with the help of the input device 25. The input device 25 is a "joystick", a "mouse", or under the control of an operator,
(Eg microprocessor) or other suitable input device connected to a similar device at the receiving end. The structure and capabilities described above allow "scanning" around the location of another party (eg, a room) without any mechanical movement of the camera required. All of these conversions are done at a real-time video rate of 30 frames per second, comparable to commercial video standards.

The image signal is reproduced on a suitable display device 23 (eg NTSC monitor, cathode ray tube (CRT) or similar) having video (and audio) capabilities.

In addition, a portion of the image is observed and a signal is sent CC so that the high resolution image is transmitted on the transmission channel CC.
A circuit for transmitting to the D camera is provided. This feature is shown in Figure 2.
As shown in, it requires a logic circuit change in the CCD of the camera.

In FIG. 2, the CCD camera chip is a CCD sensing element organized by a secondary array of sensing elements, so only a portion of the image is transmitted at high resolution. This is easily done if the video communication is between more than two parties. The only requirement is that the electronics take multiple inputs at the same time and select multiple parts of the image.

Generally, a CCD camera chip is a Sharp Co.
Available immediately from rporation or Polaroid Corporation stock, and easily customized by the hardware designer.

Alternatively, the chip and its logic can be easily customized into a single chip by those skilled in the art within the scope of this application. Electronics can be easily developed into a high level logic language and implemented in a gate array.

In operation, the wide-angle lens 1 is added to the camera 2 (eg, physically added or formed as part or side of the camera housing side). The lens 1 converts the optical image into electrical signals, which are electronically digitized (for example by an A / D converter or the like) and stored. For example, it is stored in an image buffer connected to the image processing circuit 22 of the image processor provided with an X map and Y map processor for performing a two-dimensional transformation. The image processing circuit (e.g. comprising an image conversion processor) is preferably controlled by the control unit 5 (e.g. a microcomputer and control interface) to provide system initialization and convert parameter calculations.

The control interface also determines the desired conversion factor based on azimuth, magnification, rotation, and light sensitivity input from input means such as a joystick or computer input means. The image is transmitted by the transmitter 4 from the image buffer of the image processing circuit 3. The image is received by the receiving device 21, processed by an image processing circuit 22 under the control of a control device 24 and an input device 25 operable by an operator and output to display electronics having a video display device for reproduction.

With the video system described above, video teleconferencing also introduces audio transmission known in the art.

A voice receiving system (eg, a speaker or the like) is typically provided at each terminal to receive and process voice signals from other remote stations. These audio signals are synchronized and interleaved with the video image signals in a manner known in the art. The audio part of the video conferencing device according to the present invention is not the subject of the present invention, so the audio part is briefly described below. Standard voice chips are available "in stock" from VLSI Logic, Inc. and can be used to perform all voice functions.

Another feature of the invention is the image resolution enhancement provided by the additional circuitry shown in FIG. High resolution is obtained by selecting only certain parts of the camera image (for example required for document transmission).

Note that multiple CCDs are provided, especially if color images are required. Red in the image,
Three CCDs are provided corresponding to the green and blue parts.

If optical fibers are used to transmit the image signal, then the appropriate electronics are provided for the conversion and transmission of the image.

Video conferencing with the structure of the present invention is simple to conduct and conventional multimedia PCs and workstations can be easily retrofitted, greatly improving computing system usage.

Furthermore, the system according to the invention transmits a wide-angle image of the whole office image produced by a camera with a wide-angle lens, at the receiving end the image is processed into a planar image. The electronics at the ends of the transmitter and receiver allows the observer at the end of the receiver to "look around" to other positions without mechanical movement of the transmitting camera.

Thus, in accordance with the present invention, a video conferencing system has a camera with a wide-angle lens used at a video conferencing site to capture the entire scene of a video conference.
It can be changed by zooming in or panning electronically in an electrotechnically captured scene without moving the camera. The resolution of the selected scene can be changed.

In summary, the following matters will be disclosed regarding the configuration of the present invention.

(1) A video conference multimedia system for communication between a transmitting position and a receiving position, the camera system having a wide-angle lens, and a transmitter for receiving a wide-angle image from the camera system. A receiving device that receives the wide-angle image from the transmitting device, and at least one of the transmitting device and the receiving device that executes image conversion of the wide-angle image at the transmission position without mechanical operation and performs image processing. And a video processing means for processing the wide-angle image described above. (2) The video conference multimedia according to (1), wherein the image processing means has means for selecting a field of view on the transmitting side by using an input electronic signal provided by at least one of the transmitting device and the receiving device. ·system. (3) The video conference unit according to (1), wherein the image processing means gives high resolution to a part of the video image signal other than the part of the video image signal so that the characteristics of the wide-angle lens match. Media system. (4) The image processing means has means for executing at least one of searching, scanning and zooming in and zooming out the wide-angle image on the transmitting side without the operation of the camera system. ) Described video conferencing multimedia system. (5) The video conference multi according to (1), wherein the transmitting device has means for transmitting to a plurality of remote computers, and the receiving device has means for receiving an input from a plurality of remote video sources. Media system. (6) The video conference multi-conference according to (2), wherein the image processing means gives high resolution to a part of the video image signal other than the part of the video image signal so that the characteristics of the wide-angle lens match. Media system. (7) The image processing means has means for executing at least one of searching, scanning and zooming in and zooming out the wide-angle image on the transmitting side without the operation of the camera system. ) Described video conferencing multimedia system. (8) The video conference multi according to (7), wherein the transmitting device has means for transmitting to a plurality of remote computers, and the receiving device has means for receiving an input from a plurality of remote video sources. Media system. (9) A wide-angle lens having a plurality of side surfaces, a first side surface of the plurality of side surfaces having a plastic housing having a wide-angle lens and a charge-coupled device (CCD) chip attached to the plastic housing. The position of the transmitting camera that generates the image and the position of the receiving camera that receives the wide-angle image from the transmitting camera can be seen from the side of the transmitting camera without mechanical operation of the transmitting camera. And a receiving camera having a means for processing the wide-angle image. (10) The transmission camera has means for transmitting to a plurality of remote computers, and the reception camera has means for receiving inputs from a plurality of remote video sources. system. (11) The system according to (9), wherein the image processing means includes means for selecting a field of view on the transmission side using an input electronic signal provided by at least one of the transmission camera and the reception camera. (12) The system according to (9), wherein the image processing means gives high resolution to a part of the video image signal other than the part of the video image signal so that the characteristics of the wide-angle lens match. (13) The image processing means has means for executing at least one of search, scan and zoom in, and zoom out of the wide-angle image on the transmission side without physical operation of the transmission camera. (9) The system described. (14) The system according to (10), wherein the image processing means includes means for selecting a field of view on the transmission side by using an input electronic signal provided by at least one of the transmission camera and the reception camera. (15) The image processing means gives high resolution to a part of the video image signal other than the part of the video image signal so that the characteristics of the wide-angle lens match.
The system described. (16) The image processing means has means for executing at least one of search, scan and zoom in, and zoom out of the wide-angle image on the transmission side without physical operation of the transmission camera. (15) The system according to.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a first embodiment system according to the present invention.

FIG. 2 is a schematic diagram of a camera CCD chip having logic circuitry for selecting a sub-array of CCDs for transmitting a portion of an image.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 lens 2 camera 3, 22 image processing circuit 4 transmitter 5, 24 control device 6, 25 input device 10 transmitter 20 receiver 21 receiver 23 display device 31 logic circuit 32 CCD array

Claims (16)

[Claims] 1. A video conference multimedia system for communication between a transmitting position and a receiving position, comprising: a camera system having a wide-angle lens; and a transmitter for receiving a wide-angle image from the camera system. A receiving device that receives the wide-angle image from the transmitting device, and at least one of the transmitting device and the receiving device that executes image conversion of the wide-angle image at the transmission position without mechanical operation and performs image processing. An image processing means for processing the wide-angle image, and a video conference multimedia system comprising: 2. The image processing means comprises means for selecting a field of view on the transmitting side using an input electronic signal provided by at least one of the transmitting device and the receiving device.
The video conference multimedia system according to claim 1. 3. The image processing means gives a high resolution to a part of the video image signal other than the part of the video image signal so that the characteristics of the wide-angle lens coincide with each other.
Video conferencing multimedia system described. 4. The image processing means comprises at least one of searching, scanning and zooming in and zooming out the wide-angle image on the transmitting side without the operation of the camera system.
Video conferencing multimedia system according to claim 1, comprising means for performing one. 5. The video conference of claim 1, wherein the transmitter comprises means for transmitting to a plurality of remote computers, and the receiver comprises means for receiving input from a plurality of remote video sources. Multimedia system. 6. The image processing means gives a high resolution to a part of the video image signal other than the part of the video image signal so that the characteristics of the wide-angle lens coincide with each other.
Video conferencing multimedia system described. 7. The image processing means includes at least one of searching, scanning and zooming in and zooming out the wide-angle image on the transmitting side without the operation of the camera system.
7. The video conferencing multimedia system of claim 6, including means for performing one. 8. The videoconferencing system of claim 7, wherein said transmitting device comprises means for transmitting to a plurality of remote computers, and said receiving device comprises means for receiving input from a plurality of remote video sources. Multimedia system. 9. A plurality of side surfaces, the first of the plurality of side surfaces.
A side surface of the transmission camera has a plastic housing having a wide-angle lens, and a charge-coupled device (CCD) chip attached to the plastic housing, and a transmission camera for generating a wide-angle image. A receiving camera having means for processing the wide-angle image such that the position of the receiving camera receiving the image can be seen from the side of the transmitting camera without mechanical movement of the transmitting camera; , Video conferencing system. 10. The transmitting camera comprises means for transmitting to a plurality of remote computers, and the receiving camera comprises means for receiving input from a plurality of remote video sources. System. 11. The system of claim 9 wherein said image processing means comprises means for selecting a field of view on the transmit side using an input electronic signal provided by at least one of said transmit camera and said receive camera. . 12. The system according to claim 9, wherein said image processing means provides high resolution to a part of said video image signal other than a part of said video image signal so that the characteristics of said wide-angle lens are matched. 13. The image processing means retrieves the wide-angle image on the transmitting side without physically operating the transmitting camera,
10. The system according to claim 9, comprising means for performing at least one of scanning and zooming in and zooming out. 14. The system of claim 10 wherein said image processing means comprises means for selecting a field of view on the transmitting side using an input electronic signal provided by at least one of said transmitting camera and said receiving camera. . 15. The system according to claim 14, wherein said image processing means provides high resolution to a part of said video image signal other than a part of said video image signal so that the characteristics of said wide-angle lens match. 16. The image processing means retrieves the wide-angle image on the transmitting side without physically operating the transmitting camera,
16. The system of claim 15, comprising means for performing scanning and at least one of zooming in and zooming out.