JPH1093942A - Multipoint control device - Google Patents

Abstract

(57) [Problem] To provide a multipoint control device capable of easily determining a center person of a multipoint video conference and improving the quality of a screen with the center person by effectively using a limited transmission line. I do. SOLUTION: When a video signal from each video conference terminal device is compressed, a control unit for controlling a quantization step size for each macro block constituting a screen of each video conference terminal device is provided, and a television having a central person is provided. A function is provided in which the quantization step size of the video signal from the conference terminal device is set small and the quantization step size of the video signals from the other video conference terminal devices is set large.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a plurality of video conference terminals connected via a communication line such as ISDN.
Multipoint control device (MCU; Multi) for controlling a multipoint videoconference between a plurality of connected videoconference terminal devices
ipoint Control Unit)
In particular, it relates to control of a multi-screen in a multipoint video conference.

[0002]

2. Description of the Related Art FIG. 15 shows an entire configuration of a multipoint video conference. Each video conference terminal device 1
Multipoint control unit (MCU) 3 via SDN network 2
It is connected to the.

In general, a multipoint control unit (MCU) 3
Based on terminal information such as an audio signal from each video conference terminal device 1, a central person (center person) such as a moderator or a speaker of the multipoint video conference is recognized, and the video conference terminal device where the center person is located is recognized. A function of recognizing a video signal and an audio signal from the video conference terminal 1, a function of combining images from a plurality of video conference terminals 1 including a video signal from the video conference terminal device 1 having the center person on a multi-screen, It has a function of quantizing the encoded multi-screen, encoding and compressing it, and transmitting it to each video conference terminal device 1.

[0004] A multi-screen in a conventional multipoint video conference, for example, when video conference terminals 1 at nine locations are connected, combines images from the respective video conference terminals 1 and combines them as shown in FIG. A nine-segment multi-screen is configured and transmitted to each video conference terminal device 1. The number of horizontal macroblocks (described in detail later) of the multi-screen in FIG.
Individual. When nine screens are used as shown in the figure, 22 macroblocks and screens cannot be divided by three screens, and each screen and macroblock cannot be equally applied. Therefore, adjustment is made so that the screen and the macroblock are adapted at the time of the screen synthesis. In FIG. 16, 22 is obtained by 7 + 8 + 7.

[0005] In performing the encoding, adaptive control is performed according to the bandwidth allocated to the moving image transmission and the characteristics of the source video.

For example, when the amount of codes to be transmitted increases when the motion is intense as it is, the generated code amount is suppressed by roughening the quantization characteristic. By making the quantization characteristics finer (to reduce the quantization error), a video as fine as possible is obtained.

ITU-T Recommendation H. As shown at 261, the quantization step size is a parameter having an even value from 2 to 62. Conceptually, the original image to be encoded is a discrete cosine transform (DCT).
e Transform (discrete cosine transform) is calculated, and then divided by this value to limit the dynamic range. Therefore, if this value is reduced for the same image, the quantization error is reduced, and the image quality is improved, but the code data is increased. If the amount of code per unit time exceeds a certain amount for the band allocated to the moving image on the transmission path, the video codec performs frame skipping (dropping frames) of the image to avoid overflow of the transmission buffer. Thus, the operation is performed to suppress the increase of the code data. When the frame skip occurs, the ability to follow the motion of the image is reduced. That is, when the quantization step size is set to be small, the image quality is improved, but the motion tracking performance is reduced. Conversely, when the value is set to a large value, the image quality is reduced, but the motion tracking performance is improved.

[0008]

Generally, in a conventional multi-screen in a multipoint video conference, the display mode of the image on each terminal screen is the same regardless of who is the center person. Therefore, the central person such as a moderator or a speaker is difficult to recognize at a glance, and the amount of information of each multi-screen terminal screen transmitted to each video conference terminal device is the same. However, it has not been possible to improve both the image quality and the followability of the screen on which the center person wants to pay attention.

Note that Japanese Patent Application Laid-Open No. 5-68248 discloses that
At the time of multi-screen, the screens other than the speaker are darkened so that the other party (center person) who is talking can be easily identified. Since the amount of information transmitted to the terminal device does not change, it cannot be expected to fundamentally solve the above-mentioned problems.

Therefore, the present invention has been made to solve such a problem, and a center person can be easily identified, and a limited transmission line can be effectively used to display a screen with the center person. It is an object of the present invention to provide a multipoint control device capable of improving quality.

[0011]

In order to achieve the above object, according to the first aspect of the present invention, a plurality of video conference terminals are connected via a communication line, and a plurality of connected video conference terminals are connected. Controls multipoint videoconferencing between terminal devices, recognizes the center of the multipoint videoconference based on terminal information from each videoconference terminal device, and recognizes the video signal from the videoconference terminal device where this center person is located Then, the images from the plurality of video conference terminals including the video signal from the video conference terminal having the central person are synthesized on a multi-screen,
In a multipoint control device that compresses a synthesized multi-screen and transmits it to each video conference terminal device, when a video signal from each video conference terminal device is compressed, each macro block constituting a screen of each video conference terminal device is compressed. Control means for controlling the quantization step size, the quantization step size of the video signal from the video conference terminal device where the center person is located is set small, and the quantization step size of the video signal from the other video conference terminal device is set. Has a function of setting a large value.

According to a second aspect of the present invention, in the multipoint control apparatus according to the first aspect, a quantization step size is arbitrarily set for each macro block constituting a screen of each of the video conference terminals. It is provided with setting means.

Further, according to a third aspect of the present invention, in the multipoint control device according to the second aspect, a storage means for storing the quantization step size set by the setting means is provided.

On the other hand, according to the invention of claim 4, a plurality of video conference terminals are connected via a communication line, and a multipoint video conference performed between the plurality of connected video conference terminals is controlled. Recognize the center person of the multipoint video conference based on the terminal information from the video conference terminal device, recognize the video signal from the video conference terminal device where the center person is located, and recognize the video signal from the video conference terminal device where the center person is located. In a multipoint control device that combines images from a plurality of video conference terminals including video signals into a multi-screen, compresses the synthesized multi-screen, and transmits the compressed multi-screen to each video conference terminal, When the video signal is compressed, the video conference terminal device includes control means for controlling color information for each macro block constituting the screen of each video conference terminal device. The color information of the video signal is enabled, the color information of the video signal from the other video conference terminals is obtained by such a function to invalidate.

According to a fifth aspect of the present invention, a plurality of video conference terminals are connected via a communication line, and a multipoint video conference performed between the plurality of connected video conference terminals is controlled. Recognize the center person of the multipoint video conference based on the terminal information from the video conference terminal device, recognize the video signal from the video conference terminal device where the center person is located, and recognize the video signal from the video conference terminal device where the center person is located. In a multipoint control device that combines images from a plurality of video conference terminal devices including video signals into a multi-screen and transmits the synthesized multi-screen to each video conference terminal device, the video signal from each video conference terminal device is A control unit for controlling whether or not to update the video signal sent from each video conference terminal device when synthesizing a multi-screen is provided. Video signal from the device is continuously updated, the video signal from the other video conference terminals is obtained by so as to have a function of updating at regular intervals.

According to a sixth aspect of the present invention, in the multipoint control device according to the fifth aspect, a video signal from a video conference terminal other than the video conference terminal at which the center person is located is transmitted at regular time intervals. They are updated in order.

According to a seventh aspect of the present invention, a plurality of video conference terminals are connected via a communication line, and a multipoint video conference performed between the plurality of connected video conference terminals is controlled. Recognize the center person of the multipoint video conference based on the terminal information from the video conference terminal device, recognize the video signal from the video conference terminal device where the center person is located, and recognize the video signal from the video conference terminal device where the center person is located. In a multipoint control device that combines images from a plurality of video conference terminals including video signals into a multi-screen, compresses the synthesized multi-screen, and transmits the compressed multi-screen to each video conference terminal, A control unit for controlling a quantization step size for each macro block constituting a screen of each video conference terminal device when compressing a video signal; The function of setting the quantization step size of the video signal from the conference terminal device to be small, and setting the quantization step size of the video signal from other video conference terminal devices to be large, and the video signal from each video conference terminal device. At the time of compression, control means for controlling color information for each macro block constituting a screen of each video conference terminal device is provided, and color information of a video signal from the video conference terminal device where the center person is located is validated, and other Function to invalidate the color information of the video signal from the conference terminal device, and whether to update the video signal sent from each terminal device when synthesizing the video signal from each video conference terminal device into a multi-screen Control means for controlling whether or not the video conference terminal device with the central person continuously updates the video signal. With the video signal and a function of updating at regular intervals, said those with selectable selection means one of the respective functions.

According to an eighth aspect of the present invention, in the multipoint control device according to the seventh aspect, the selection means selects a corresponding function based on a function selection operation input from the video conference terminal device. It was made.

[0019]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram of a multipoint control unit (MCU) 3 according to the present embodiment. In the figure,
The CPU 3a includes a ROM 3b, a RAM 3c, a DM
Along with the A controller 3d and the like, it realizes an operation control function and a video conference communication control function of each unit of the multipoint control unit (MCU) 3, and the means described in each claim of the present application is also realized by the CPU 3a. This is realized by executing a program shown in a flowchart described later.

The ROM 3b stores in advance a control program to be executed by the CPU 3a and various data necessary for executing the control program. A work area when the program is executed by the CPU 3a is formed on the RAM 3c. The RAM 3c is backed up by a battery so that various setting data in the present invention can be registered. This setting data is stored in the flash ROM
When the data is stored in a rewritable nonvolatile storage device such as a hard disk or a hard disk, the battery backup of the RAM 3c is unnecessary. On the other hand, the DMA controller 3d
It controls DMA (Direct Memory Access) for performing high-speed data transfer without using U3a.
During the MA transfer, the CPU 3a can perform other processing. CPU 3a, ROM 3b, RAM 3c, DMA
The control unit 30 is constituted by the controller 3d and the like.

The timer 3e outputs time information and measures time.
f interrupts the CPU 3a based on, for example, time information and time information of the timer 3e. Also,
The operation display unit 3g is for performing various operations for maintenance work and the like of the multipoint control unit (MCU) 3.

The communication control section 3h is connected to the above-mentioned video conference terminal device 1 via an ISDN line, and controls predetermined call control procedure processing, video conference communication control procedure processing, demultiplexing processing, and the like. . The audio CODEC (codec) 3i performs encoding / decoding and decoding / decompression of audio data, and the audio mixer 3j mixes audio signals from the respective video conference terminal devices 1 to perform a multipoint video conference. Things.

Also, a moving image CODEC (codec) 3k
Performs encoding compression and encoding expansion of moving image information, and the image synthesizing unit 31 synthesizes image information from each video conference terminal device 1 in order to perform a multipoint video conference.

The communication control unit 3h, the audio CODE
The C3i and the moving image CODEC 3k are mounted for the number of channels serviced by the multipoint control unit (MCU) 3.

These CPU 3a, ROM 3b, RAM
3c, DMA controller 3d, timer 3e, interrupt controller 3f, operation display unit 3g, communication control unit 3h, audio CODEC 3i, audio mixer 3j, moving image CODEC 3
k, the image synthesizing unit 31 and the like are connected to each other via a system bus 3m, and data exchange between these elements is performed via the system bus 3m.

In the video conference communication function of the present embodiment, an ITU-
Recommendation T. H. The moving picture coding method specified in H.261 is applied. This ITU-T Recommendation H. Next, the moving image coding system of H.261 will be described. Here, two types of formats, CIF and QCIF, are defined as the moving image information, but the following description mainly describes CIF.

CIF (Common Intermed)
iate Format (common intermediate format) is a basic format of moving image information.
Displays a screen of 29.97 frames per second. As for the luminance component (Y), one frame (image frame) is composed of 288 lines, one line is composed of 352 pixels, and a color difference component (Y For Cb, Cr), one frame is composed of 144 lines, and one line is composed of 176 pixels.

Then, the frame is divided into 12 GOBs (Group Of Bloes) as shown in FIG.
ck), and each GOB is divided into 33 macroblocks of 11.times.3, as shown in FIG.

Further, as shown in FIG. 2C, one macro block includes four 2 × 2 luminance blocks,
Each of them is composed of one color difference block.
Therefore, the luminance component (Y) and the color difference components (Cb, Cb
One block of r) is composed of 8 pixels × 8 lines.

QCIF (Quarter CIF)
Is a format in which the number of pixels is set to 1/4 of CIF. In addition, ITU-T Recommendation H. In 261, movie CO
The DEC specifies that it is necessary to handle at least QCIF, and further specifies that when the CIF and the QCIF face each other, communication is performed using the QCIF.

This moving picture information corresponds to the hierarchical structure of frame / GOB / macroblock (MB) / block as described above, and its multiplexing processing system diagram is shown in FIG. The following description is based on ITU-T Recommendation H.264. H.261, which is a domestic standard corresponding to H.261.

First, as shown in FIG. 3A, one frame of moving picture information indicates a frame start code PSC indicating the start of a frame, a frame number TR indicating a frame sequence number, and information on the entire frame. Type information PTYPE, extension data insertion information PEI indicating presence / absence of the next extension data area (spare information), spare information PSPA
RE and GOB data (GOB layer). Here, the extension data insertion information PEI may include at least one, and may not include the preliminary information PSPARE.
Further, the frame start code PSC, the frame number TR, the type information PTYPE, the extension data insertion information PEI, and the preliminary information PSPARE are called a frame header.

One GOB data is arranged for each GOB. One GOB data is a GOB indicating the start of the GOB data as shown in FIG.
GOB number G indicating start code GBSC, GOB position
N, quantization characteristic information GQUAN indicating quantization characteristic information
T, extension data insertion information GEI indicating presence / absence of the next extension data area (preliminary information), preliminary information GSPARE, and macroblock data (MB layer). Here, the extension data insertion information GEI may include at least one, and may not include the preliminary information GSPARE.
Also, GOB start code GBSC, GOB number GN, quantization characteristic information GQUANT, extension data insertion information GE
I and the preliminary information GSPARE are called a GOB header.

One macro block data (MB layer) is arranged for each macro block. However, if there is no information in that part of the frame, it is not transmitted.

As shown in FIG. 2C, one macroblock data includes a macroblock address MBA for indicating the position of the macroblock, a type of the macroblock, and information for indicating which data element appears. It is composed of type information MTYPE, quantization characteristic information MQUANT indicating quantization characteristics, motion vector information MVD, a significant block pattern CBP indicating that at least one transform coefficient is a macroblock to be transmitted, and block data (block layer). . Here, the quantization characteristic MQ
UANT, motion vector information MVD and significant block pattern CBP appear when indicated by type information MTYPE. Also, the macro block address MBA,
Type information MTYPE, quantization characteristic information MQUANT,
The motion vector information MVD and the significant block pattern CBP are called a macroblock header.

As shown in FIG. 3D, the block data includes, for example, a transform coefficient TCOEFF indicating transform coefficient data obtained by performing DCT (discrete cosine transform) processing on the image data of the block, and an end of the block. Is formed of a block end code EOB.

Now, according to the ITU-T recommendation H.264. In the moving image coding method of H.261, as described above, the quality of the moving image to be exchanged can be determined to some extent by appropriately changing the quantization step size of the moving image within a settable range (an even value of 2 to 62). it can. Then, the quantization characteristic information GQUANT included in the GOB data is replaced with the subsequent GOB data.
And the quantization characteristic information MQUANT included in the macroblock data is set to the content corresponding to the quantization stepsize applied to the subsequent macroblocks. Is done.

That is, conceptually, this ITU-
Recommendation T. H. In the moving image coding method of H.261, the dynamic coefficient of a code can be limited by dividing a transform coefficient obtained by performing DCT processing on image data by a value of a quantization step size.

Therefore, when the quantization step size is set to a small value, the quantization error is reduced, and the image quality when reproducing the encoded moving picture information is improved (ie,
Higher definition), but on the other hand, the data amount increases. In this way, the amount of data per frame increases,
When the amount of data per second becomes larger than the data transmission rate assigned to the moving picture information, the moving picture CODEC is used to prevent the transmission buffer from overflowing.
A frame skip (frame drop) of a moving image is performed to suppress the data amount of the moving image information. That is, in this case, the ability to follow the movement of the image is reduced.

Conversely, when the quantization step size is set to a large value, the quantization error increases, and the image quality when reproducing the encoded moving picture information deteriorates (that is, the definition decreases). Since the data amount is reduced, the followability of the screen is improved.

Further, in the moving image CODEC, the quantization step size is set adaptively, and the applied factor referred to at that time is, for example, the amount of code data. Therefore, the moving image CODEC normally operates so as to adaptively change the quantization step size in the direction of reducing the code data amount. In addition, the maximum value of the quantization step size is defined by the maximum quantization step size, and this maximum quantization step size can be externally designated for the moving image CODEC.

Therefore, if the maximum quantization step size specified for the moving picture CODEC is set to a large value,
The DEC adaptively changes the quantization step size up to the maximum quantization step size.
The amount of transmission data per frame can be reduced.

Conversely, when the maximum quantization step size specified for the moving image CODEC is set to a small value, the moving image COD
Since the EC limits the quantization step size to a small value, the quantization error is reduced, and the image quality when reproducing the encoded moving image information is improved.

The operation of this embodiment will now be described with reference to FIGS.
This will be described with reference to the flowchart shown in FIG.

First, each video conference terminal device 1 is connected to the multipoint control unit (MCU) 3 (process 101 in FIG. 4).
Then, the ISD is transmitted from each of the connected video conference terminals 1.
Information transmitted via the N line is taken in by each communication control unit 3h.

In this embodiment, three new functions 1 to 3
Is added, but in the initial state, it is a mode in which no functions are used. If the new function is not used (No in decision 102), the coded data from each video conference terminal device 1 is decoded (process 103), and each decoded data is combined into a multi-screen as in the conventional case (process 103). (Process 104), a DCT operation is performed on the combined image (Process 105), and the image is further quantized and encoded (Process 106), and transmitted to each video conference terminal device 1 (Process 106). This is repeated.

On the other hand, when any one of the functions is used, there are two types of selection methods, a method of directly setting to the multipoint control unit (MCU) 3 and a method of setting externally by telephone or the like.

First, in the case of setting from outside, first, the determined telephone number of the multipoint control unit (MCU) 3 is input (dial). When input, it is connected to the multipoint control unit (MCU) 3 through the line, and the multipoint control unit (MC
U) The communication control unit 3h of 3 outputs a message "Please input a password" to the partner terminal. According to the message, the terminal operator inputs a predetermined password (for example, * 212 * 3). If the input password is incorrect, the multipoint control unit (MCU) 3 outputs a message prompting the user to input the password again to the partner terminal. The communication control unit 3h disconnects the line. If the password is correct, a message "Enter the mode number" is sent to the terminal operator, and the number of the desired function is entered. The input number is reflected in the control unit 30 (Yes of the judgment 102 → judgment 1)
08).

When setting is made directly by the multipoint control unit (MCU) 3, the function selection is notified to the control unit 30 by an input device such as a switch in the operation display unit 3g. The notified control unit 30 displays "What function do you want to select?" On the display device of the operation display unit 3g. Enter the desired number from the displayed functions using the input device. The input information is reflected in the control unit 30 (Yes in decision 102 → decision 10
8).

By allowing the function to be selected in this manner, the degree of freedom in selecting the function is increased. In particular, since the function can be selected from outside, any video conference terminal device 1 can be selected.
You can freely select functions even when you are in the place.

Now, for example, when a new function 1 is selected by any of the above-mentioned methods, first, a center person such as a moderator or a speaker is determined or determined based on conference setting information, voice information from each terminal, or the like. It is detected (step 109 in FIG. 5). Next, the user selects whether to use or change the normal registration data (decision 110). When the normal registration data is used, it becomes as shown in FIGS. In this case, the position of the center person of the screen is specified, the quantization step size for each screen corresponding to the position is specified, and the number of horizontal macroblocks for each screen is specified. FIG. 8 shows the designation of the position of the center of the screen and the number of macroblocks on the side of the screen, and FIG. 9 shows the designation of the corresponding quantization step size for each screen. In the setting based on the normal registration data, since the center person such as the moderator and the speaker is the screen 4, the step size is reduced to “2” in order to improve the image on the screen 4, but the coding information is thereby increased. . Therefore, the setting is made such that the step size of all the macroblocks of the other screens is increased to “30”, and the entire encoded information is reduced.

First, the quantization step data and the like set as described above are detected (No in decision 110 → process 11).
1). Next, in order to synthesize the encoded data, the data is decoded by the moving image CODEC 3k (process 112), and the decoded data is sent to the image synthesizing unit 3l. Are adjusted to synthesize a multi-screen (process 113). First, the synthesized information is video C
The DCT operation is performed and quantized by the ODEC 3k, and the quantization step size at the time of the quantization is converted into the size specified above and quantized. Then, it is encoded and transmitted from the communication control unit 3h to each video conference terminal device 1 (process 114 → process 115 → process 116 → process 117).

As described above, by refining only the image of the central person on the multi-screen, the central person can be easily distinguished, and the screen of the central person most closely watched can be clearly seen. In addition, by reducing the image quality of other images, a limited transmission path can be used effectively.

On the other hand, when the change is made without using the normal registration data, the change mode is entered by an external input (decision 11).
0 Yes). When entering the change mode, there are a new registration mode and a selection mode. When the new registration mode is selected (Yes in decision 118), the value of the quantization step size specified when encoding the video signal of each video conference terminal device 1 ( Two
(Even value of 62) and the position of the image information of the central person in the multi-screen and the number of macroblocks of the horizontal width for each screen.
As an example of registration of the quantization step data, a name is given and registered in a storage device such as the RAM 3c (process 119 → process 120). Select or reflect the registered value immediately,
In the case of reflection, the position of the screen and the horizontal width of the screen are adjusted at the time of synthesis, and the quantization step data is designated and reflected (process 120 → No in determination 110 → process 111 and thereafter).

As described above, by adjusting the definition of the image of the central person, it is possible to specify the desired definition. Further, by storing the specified definition, the same definition can be immediately reproduced without adjusting again as shown below.

In the case of the selection mode, the pattern of the registered data registered in advance in the storage device such as the RAM 3c is read out as described above (No in decision 118 → process 1).
21), select a pattern such as a step size (Process 1)
22), the selected registration data is designated and reflected (process 123).

Subsequent processing is left to the codec application control described above. Also, the value of the quantization step size is
This is the maximum value setting.

Next, the case where the function 2 is selected will be described. Also in this case, first, a center person such as a moderator or a speaker is determined or detected based on the conference setting information, voice information from each terminal, and the like (process 124 in FIG. 6). Next, the user selects whether to use or change the normal registration data (decision 12).
5). When using the normal registration data, refer to FIGS.
As shown in FIG. In this case, the position of the center of the screen is specified, the screens to be colorized and the screens to be black and white corresponding thereto are specified, and the number of macroblocks of the horizontal width for each screen is specified. FIG. 10 shows the position of the center of the screen and the number of macroblocks on the side of the screen, and FIG. 11 shows the specification of the corresponding color screen and the black and white screen.

First, a macro block other than the color screen set as described above, that is, a macroblock of a screen to be converted to black and white, is detected (No in determination 125 → processing 126). Next, in order to combine the encoded data, the moving image CO
The data is decoded by the DEC 3k (process 127), and the decoded data is sent to the image synthesizing unit 31 to adjust the position of each screen and the width of the screen at the time of synthesis to synthesize a multi-screen (process 1).
28), DCT operation is performed on the synthesized image information (Process 12)
9). Then, in order to prevent transmission of the color difference information of the macroblock of the screen to be converted to black and white detected at the time of encoding, the significant block pattern of the macroblock is changed (process 130), quantized and encoded. (Process 13
1). And it transmits to each video conference terminal device 1 (process 132).

As described above, the color information of the screens other than the central person is not transmitted and the image is converted to black and white, so that the central person can be recognized at a glance and the screen of the central person most closely watched can be clearly seen. In addition, by reducing the image quality of other images, the limited transmission path can be used effectively.

On the other hand, when changing without using the normal registration data, the position of the center person's screen and the number of macroblocks having the width of the screen are input (Yes in decision 125 → process 13).
3) Further, a designation of which terminal screen is to be colored is input (process 134). The input data is stored and reflected in the storage device (step 135 → step 126 and thereafter).

Next, the case where function 3 is selected will be described. Also in this case, first, a center person such as a moderator or a speaker is determined or detected based on the conference setting information, voice information from each terminal, and the like (process 136 in FIG. 7). Next, the user selects whether to use or change the normal registration data (decision 13).
7). When using the normal registration data, refer to FIGS.
3, as shown in FIG. In this case, specify the position of the center person of the screen, specify the corresponding continuously updated screen (write all received frames) and the screen to be updated at regular intervals, and specify the number of macroblocks of the horizontal width for each screen I have. FIG. 12 shows the designation of the position of the center person of the screen and the number of macroblocks on the side of the screen. FIG. 13 shows the designation of the screen to be continuously updated and the screen to be updated at regular intervals.
FIG. 14 shows the time intervals when updating is performed at regular intervals and the order of the screen to be updated. In this case, since the central person is the screen 4, the other image data is designated so as to sequentially send information at regular time intervals. Since information is sent sequentially at regular time intervals, the amount of information is significantly reduced compared to sending image information in real time, so the amount of information on the screen of the designated central person is increased and the specified screen image is much better can do.

First, the update interval and the like set as described above are detected, and a screen to be updated is instructed to the image synthesizing unit 3l (No in decision 137 → process 138 → process 139). Next, in order to combine the encoded data, the moving image CO
The encoded data is decoded by the DEC 3k (process 140),
The decoded data is sent to the image synthesizing unit 3l, the position of the synthesized screen and the horizontal width of the screen are adjusted at the time of synthesis, and the order of updating the specified image (screen 4 in this case) at a fixed time interval has come. Only the image information of the screen is updated to compose a multi-screen (process 141). The synthesized image is a movie CODEC
DCT operation at 3k, further quantization and code compression,
The data is transmitted to each video conference terminal device 1 (process 142 → process 143 → process 144).

As described above, the image information other than the screen of the central person is not transmitted continuously in the multi-screen,
It is possible to send a lot of screen information of the center person, to easily understand the center person, to see the screen of the center person who is watching the most clearly, and to reduce the image quality of other images by that amount, thereby limiting The used transmission path can be used effectively. In addition, by transmitting screen information other than the center person at a certain time interval without sending the screen information all at once, the efficiency of the transmission path can be further improved.

On the other hand, if the change is to be made without using the normal registration data, the number of macroblocks of the center person's screen position and screen width is input (Yes in decision 137 → process 14).
5) Further, a screen for writing all the frames received at the time of synthesizing the terminal image is specified, and the number of seconds to update the screen to be transmitted at a constant interval is input (process 146). The input data is stored and reflected in the storage device (process 147 → process 1).
38).

It is to be noted that there is a method in which information is not transmitted at regular intervals by designating each macro block. That is, (c) of FIG.
In the macro block (MB) loop L shown in FIG. In this method, I
TU-T Recommendation H. It is assumed that the forced screen update specified in H.261 is automatically updated by the codec.

[0068]

As described above, according to the first aspect of the present invention, a plurality of video conference terminal devices are connected via a communication line, and a plurality of video conference terminal devices are connected between the connected video conference terminal devices. Controlling the point-to-point video conference, recognizing the center of the multipoint video conference based on the terminal information from each video conference terminal, recognizing the video signal from the video conference terminal where the center is located, and In a multi-point control device that combines images from a plurality of video conference terminals including video signals from a video conference terminal having a multi-screen, compresses the synthesized multi-screen, and transmits the compressed multi-screen to each video conference terminal, Control means for controlling a quantization step size for each macroblock constituting a screen of each video conference terminal device when compressing a video signal from each video conference terminal device Quantization step size of the video signal from the video conference terminal device are centered shall set small,
Since the quantization step size of the video signal from the other video conference terminal device has a function to be set large, only the image of the center person is made fine in the multi-screen, and the image quality of other images is reduced accordingly. , While the center person can easily determine, and effectively use the limited transmission line,
There is an effect that the image of the center person who watches the most can be seen finely and clearly.

According to the second aspect of the present invention, in the multipoint control apparatus according to the first aspect, the quantization step size is arbitrarily set for each macro block constituting the screen of each video conference terminal. Since the setting means is provided, the same effect as that of claim 1 can be obtained, and the definition of the image of the central person can be adjusted, and the user can specify his or her desired definition.

Further, according to the third aspect of the present invention, in the multipoint control device according to the second aspect, there is provided storage means for storing the quantization step size set by the setting means. The same effect as described above can be obtained, and by storing the specified definition, there is an effect that the same definition can be immediately reproduced without adjusting again.

According to the fourth aspect of the present invention, a plurality of video conference terminals are connected via a communication line, and a multipoint video conference to be performed between the connected plurality of video conference terminals is controlled. Based on the terminal information from each video conference terminal device, the center person of the multipoint video conference is recognized, the video signal from the video conference terminal device having the center person is recognized, and the video conference terminal device having the center person is recognized. In a multipoint control device that synthesizes images from a plurality of video conference terminals including video signals into a multi-screen, compresses the synthesized multi-screen, and transmits it to each video conference terminal, Control means for controlling color information for each macro block constituting the screen of each video conference terminal device when compressing the video signal of the video conference terminal device. The color information of the video signal from the TV conference terminal is valid, and the color information of the video signal from the other video conference terminal device has a function to invalidate it. By not transmitting, the center person can be distinguished at a glance, and there is an effect that the screen of the center person most closely watched can be clearly seen by effectively using the limited transmission path.

According to the fifth aspect of the present invention, a plurality of video conference terminals are connected via a communication line, and a multipoint video conference to be performed between the plurality of connected video conference terminals is controlled. Based on the terminal information from each video conference terminal device, the center person of the multipoint video conference is recognized, the video signal from the video conference terminal device having the center person is recognized, and the video conference terminal device having the center person is recognized. In a multipoint control device that combines images from a plurality of video conference terminals including video signals into a multi-screen and transmits the synthesized multi-screen to each video conference terminal, a video signal from each of the video conference terminals is provided. Is provided with control means for controlling whether or not to update the video signal sent from each video conference terminal device when combining video into a multi-screen. Video signal from the terminal device is updated continuously, since the video signal from other television conference terminal device to have a function of updating at regular intervals,
By not transmitting the screen information other than the screen of the center person continuously in a multi-screen, the center person can be easily identified, and the limited transmission line is effectively used to clean the screen of the center person who watches the most. There is an effect that can be seen.

According to the invention of claim 6, in the multipoint control apparatus according to claim 5, the video signal from a video conference terminal device other than the video conference terminal device in which the center person is located is transmitted at regular time intervals. The same effect as in claim 5 can be obtained, and the screen information other than the screen of the central person is not sent at once, but is transmitted in order at regular time intervals. This has the effect of being able to further increase the efficiency.

According to the seventh aspect of the present invention, a plurality of video conference terminals are connected via a communication line, and a multipoint video conference performed between the plurality of connected video conference terminals is controlled. Based on the terminal information from each video conference terminal device, the center person of the multipoint video conference is recognized, the video signal from the video conference terminal device having the center person is recognized, and the video conference terminal device having the center person is recognized. In a multipoint control device that synthesizes images from a plurality of video conference terminals including video signals into a multi-screen, compresses the synthesized multi-screen, and transmits it to each video conference terminal, Control means for controlling the quantization step size for each macroblock constituting the screen of each video conference terminal device when compressing the video signal of The function of setting the quantization step size of the video signal from the video conference terminal device to be small and the quantization step size of the video signal from the other video conference terminal device to be large, and the video signal from each video conference terminal device When compressing a video conference terminal device, a control unit for controlling color information for each macro block constituting the screen of each video conference terminal device is provided. The function of invalidating the color information of the video signal from the video conference terminal device and updating the video signal sent from each terminal device when synthesizing the video signal from each video conference terminal device on a multi-screen. Control means for controlling whether or not the video conference terminal device having the central person is continuously updated, and the other video conference terminal devices are updated. Has a function of updating the video signal at a fixed interval, and has a selection means capable of selecting one of the functions, so that it is similar to the above-mentioned claims 1, 4, and 5. And an effect of increasing the degree of freedom in selecting a function.

Further, according to the invention described in claim 8, in the multipoint control device according to claim 7, the selection means selects a corresponding function based on a function selection operation input from the video conference terminal device. Claim 7
In addition to the same effect as described above, the function can be selected from the outside, so that the function can be freely selected from any video conference terminal device even when not present.

[Brief description of the drawings]

FIG. 1 is a block configuration diagram of a multipoint control device according to an embodiment of each invention of the present application.

FIG. 261 is an explanatory diagram of a basic format of moving image information in the moving image encoding method specified in H.261.

FIG. 3 is a multiplex processing diagram of the moving image information.

FIG. 4 is a flowchart showing a part of the operation of the embodiment.

FIG. 5 is a flowchart showing a part of the operation of the embodiment.

FIG. 6 is a flowchart showing a part of the operation of the embodiment.

FIG. 7 is a flowchart showing a part of the operation of the embodiment.

FIG. 8 is a diagram showing designation of the position of a center person on the screen and the number of macroblocks on the side of the screen according to normal registration data in function 1 of the embodiment.

FIG. 9 is a diagram showing designation of a quantization step size for each screen corresponding thereto.

FIG. 10 is a diagram illustrating designation of the position of a center person on the screen and the number of macroblocks on the side of the screen by normal registration data in function 2 of the embodiment.

FIG. 11 is a diagram showing designation of a color screen and a black and white screen corresponding thereto.

FIG. 12 is a diagram showing designation of the position of the center person of the screen and the number of macroblocks on the side of the screen by the normal registration data in function 3 of the embodiment.

FIG. 13 is a diagram showing designation of a screen to be continuously updated and a screen to be updated at a constant interval corresponding thereto.

FIG. 14 is a diagram showing a time interval when updating is performed at a constant interval and an order of a screen to be updated.

FIG. 15 is a schematic diagram showing the overall configuration of a multipoint video conference.

FIG. 16 is a diagram showing a nine-segment multi-screen.

[Explanation of symbols]

 Reference Signs List 1 video conference terminal device 2 ISDN network 3 multipoint control unit (MCU) 3a CPU 3b ROM 3c RAM 3d DMA controller 3e timer 3f interrupt controller 3g operation display unit 3h communication control unit 3i audio CODEC 3j audio mixer 3k video CODEC 3l image Synthesizing unit 30 Control unit

Claims (8)

[Claims] A plurality of video conference terminals are connected via a communication line, and a multipoint video conference performed between the plurality of connected video conference terminals is controlled, and terminal information from each video conference terminal is transmitted. A plurality of video conference terminals including a video signal from the video conference terminal device having the central person, recognizing a central person of the multipoint video conference based on the video signal from the video conference terminal device having the central person. In a multipoint control device that combines images from the devices into a multi-screen and compresses the combined multi-screen and sends it to each video conference terminal device, when compressing the video signal from each video conference terminal device, Control means for controlling a quantization step size for each macro block constituting a screen of the conference terminal device, and a video signal from a video conference terminal device having a center person. Coca step size is set small, the other video conference multipoint control unit, wherein the quantization step size for a video signal having the function of setting largely from the terminal device. 2. The multipoint control apparatus according to claim 1, further comprising setting means for arbitrarily setting a quantization step size for each macroblock constituting a screen of each of the video conference terminal apparatuses. 3. The multipoint control apparatus according to claim 2, further comprising storage means for storing the quantization step size set by said setting means. 4. A plurality of video conference terminal devices are connected via a communication line, and a multipoint video conference to be performed between the plurality of connected video conference terminal devices is controlled, and terminal information from each video conference terminal device is transmitted. A plurality of video conference terminals including a video signal from the video conference terminal device having the central person, recognizing a central person of the multipoint video conference based on the video signal from the video conference terminal device having the central person. In a multipoint control device that combines images from the devices into a multi-screen and compresses the combined multi-screen and sends it to each video conference terminal device, when compressing the video signal from each video conference terminal device, Control means for controlling color information for each macro block constituting the screen of the conference terminal device is provided, and the color information of the video signal from the video conference terminal device having the center person is effective. Other video conferencing multipoint control unit, wherein the color information of the video signal which has a function for disabling from the terminal device. 5. A plurality of video conference terminals are connected via a communication line, and a multipoint video conference performed between the plurality of connected video conference terminals is controlled, and terminal information from each of the video conference terminals is transmitted. A plurality of video conference terminals including a video signal from the video conference terminal device having the central person, recognizing a central person of the multipoint video conference based on the video signal from the video conference terminal device having the central person. In a multipoint control device that combines images from the devices into a multi-screen and transmits the combined multi-screen to each video conference terminal device, when the video signal from each video conference terminal device is combined into a multi-screen, It has control means for controlling whether or not to update the video signal sent from the video conference terminal device, and the video signal from the video conference terminal device where the center person is located is continuous. Update, others of the video signal from the television conference terminal devices multipoint control unit, characterized in that it has a function of updating at regular intervals. 6. The multipoint control device according to claim 5, wherein the video signal from the other video conference terminal device is updated sequentially at regular time intervals. 7. A plurality of video conference terminal devices are connected via a communication line to control a multipoint video conference between the plurality of connected video conference terminal devices, and to convert terminal information from each video conference terminal device into a multipoint video conference. A plurality of video conference terminals including a video signal from the video conference terminal device having the central person, recognizing a central person of the multipoint video conference based on the video signal from the video conference terminal device having the central person. In a multipoint control device that combines images from the devices into a multi-screen and compresses the combined multi-screen and sends it to each video conference terminal device, when compressing the video signal from each video conference terminal device, Control means for controlling a quantization step size for each macro block constituting a screen of the conference terminal device, and a video signal from a video conference terminal device having a center person. A function to set the quantization step size small and increase the quantization step size of the video signal from other video conference terminal devices, and to compress the video signal from each video conference terminal device. A control means for controlling color information for each macro block constituting a screen of the device is provided, and color information of a video signal from a video conference terminal device having a center person is valid, and a video signal from other video conference terminal devices is validated. Function to invalidate color information and control to control whether or not to update the video signal sent from each video conference terminal when synthesizing video signals from each video conference terminal on a multi-screen Means, the video signal from the teleconferencing terminal with the central person is continuously updated, and the video signals from the other teleconferencing terminals are constant. Which has a function of updating the 隔毎, multipoint control unit, characterized in that it comprises a selectable selection means one of the said respective functions. 8. The multipoint control device according to claim 7, wherein the selection means selects a corresponding function based on a function selection operation input from a video conference terminal device.