JPH0965312A - TV conference system - Google Patents

Abstract

(57) [Summary] [Problem] To provide an efficient TV conference system adapted to a network. SOLUTION: The center station 10 of this TV conference system
Is a bit stream conversion unit 11 that receives and combines the image information transmitted from each of the terminals 101 to 104 through the uplink of the communication line 1, and the combined image information, and encodes them according to the MPEG2 system. The terminal 101 to 104 is provided with a screen synthesizing unit 12 that transmits to each of the terminals 101 to 104 using the downlink, and each of the terminals 101 to 104 acquires the image information and the acquired image information in the H.264 standard. 261
Encoders (ENC) 111 to 114 encoded by the method and transmitted to the center station 10 through the uplink, and a decoder for receiving and decoding image information transmitted from the center station 10 through the downlink of the communication line 1. (DEC) 121
~ 124 and the monitor 14 displaying the combined image information
1-144.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information communication system used when a plurality of terminal stations are provided at different points to hold a video conference.

[0002]

2. Description of the Related Art In recent years, with the advance of communication technology, a remote conference system (hereinafter referred to as a TV conference system) capable of holding a conference while being in a remote place has become widespread.

In such a TV conference system, when moving image communication is performed between two points, for example, FIG.
As shown in FIG. 3, a terminal 701 at a certain point and a terminal 702 at another point distant from the certain point are connected by a communication line 703 having an up line, a down line, etc. Communication is generally performed by so-called symmetric communication control, in which transmission is performed at the same transmission rate using the same encoding method.

In this case, the same coding method A and the same transmission rate (B bps) are used for both the communication from the terminal 701 to the terminal 702 and the communication from the terminal 702 to the terminal 701.

Further, as shown in FIG. 11, when a multipoint conference is conducted by connecting three or more points, the terminals 705, 706, 707 and the center station 704 arranged at three different points.
And the communication lines 708, 709, and 710 are connected to each other, and the center station 704 is configured to relay the image information from the terminals 705, 706, and 707.

In this case, the center station 704 and each terminal station 7
Image communication with 05, 706, and 707 uses the same encoding method, although there may be some difference in transmission rate between the uplink and downlink of each communication line 708, 709, and 710. It is common to communicate.

By the way, in this case, each of the terminals 705, 70
6, 707 need only transmit screen information for one terminal using the uplink, while the center station 70
No. 4 needs to generate screen information for a plurality of terminals using the downlink and transmit it to each communication line.

Therefore, the transmission rate of the downlink (C bp
In some cases, the transmission speed of the uplink (B bps) is higher than that of s). In this case, in a video conference or the like, the image and the sound are transmitted to the terminals 705, 706, and 707 at the respective points in a shifted manner, which may make the conversation unnatural.

Generally, the H.264 standard is an international standard for image encoding systems of TV conference systems. 261 (ITU-T Rec.H.261)
However, since there was no other standard method until now, there was almost no need to consider connection with a different coding method.

However, due to the spread of digital CATV networks in recent years, in such CATV, MPEG2 (ISO / IEC 13818-2) is used as a new international standard system for coding high quality images for downlink coding. | ITU-T Re
c. H.262) has come into use.

When a subscriber of this CATV conducts a video conference using each terminal, the MPE is used in the downlink.
Since it has a G2 decoder, it is most natural to use the MPEG2 system. If you adopt another method,
It is necessary to newly install the decoder of that system.

On the other hand, it is natural that the uplink transmits image information by the same coding system as the downlink, that is, the MPEG2 system.

However, the encoder for MPEG2 is very expensive, and the transmission speed of the up line is generally much slower than that of the down line, so that there is a problem that the MPEG2 is not efficient.

In such a case, if MPEG2 adapted to the downlink is adopted as the encoding method for the uplink,
An expensive and inefficient MPEG2 encoder will be installed on the uplink, and H.264 will be installed on the downlink in line with the uplink. When the H.261 system is adopted, the
A decoder for H.261 is required, which is a double investment with the decoder for MPEG2. In any case, an inefficient encoder or decoder is required, which reduces the efficiency of equipment investment and information transmission.

[0015]

As described above, in the above-described conventional information communication system, when the same method is used for the up-link and the down-link, the load of information transmitted to each line is different. Information arrives at different times, especially T
There is a problem that the conversation becomes unnatural when a V-conference is held.

Even when different encoding methods are used for the up-link and the down-link, conventionally, an inefficient encoding method has to be adopted for either the up-link or the down-link, which results in information transmission. There was a problem of poor efficiency.

The present invention has been made to solve the above problems, and an object thereof is to provide an information communication system capable of improving information transmission efficiency and performing information communication in a more natural manner. I am trying.

[0018]

In order to achieve the above-mentioned object, the invention according to claim 1 connects at least one center station and a plurality of terminal stations on a transmission line having upstream and downstream lines. In the information communication system, the center station receives image information encoded by the terminal stations in the first encoding method and transmitted through the uplink, and decodes the image information. The image coding means for coding each image information decoded by the image decoding means by the second coding method, and the image information coded by the second coding method by the image coding means. The information transmission means for transmitting to each of the terminal stations using a downlink is provided, and each of the terminal stations acquires the image information transmitted from the own station, and the information acquisition means acquired by the image acquisition means. Picture Image coding means for coding information by the first coding method, information transmission means for transmitting the image information coded by the image coding means to the center station through the uplink, and the center station Image decoding means for receiving and decoding image information coded by the second coding method and transmitted through the downlink, and display means for displaying the image information decoded by the image decoding means. It is equipped with.

An information communication system according to claim 2 is
The information communication system according to claim 1, wherein the center station further comprises an image synthesizing unit for synthesizing each image information decoded by its own image decoding unit so as to be integrated on one screen. .

Further, in the information communication system according to claim 3, in the information communication system according to claim 1, the information transmitting means of the center station sends image information to the plurality of terminal stations all at once. It is characterized in that it is equipped with information communication means.

The information communication system according to claim 4 is
2. The information communication system according to claim 1, wherein the image encoding means of the center station performs variable length decoding on the characteristic amount of the data encoded by the first encoding method used in the uplink. It is characterized in that the data is extracted, converted into a feature amount of the second coding method used in the downlink, and variable length coded by this second coding method to generate transmission data. Further, the information communication system according to claim 5 is the information communication system according to claim 4,
The image coding means of the center station corrects the quantized decoded value to an odd number when the quantized characteristic value is an even number in the first coding system, and quantized decoded value in the second coding system. Gives a quantization characteristic value such that is an odd number,
It is characterized in that the quantized value in the second encoding method is determined so that the quantized decoded value becomes equal to the quantized decoded value in the first encoding method.

The information communication system according to claim 6 is
5. The information communication system according to claim 4, wherein the image coding means of the center station uses the second coding method for a macroblock to which an in-loop filter has been applied by the first coding method. The feature is that the value of the motion vector is changed to a value shifted by half a pixel.
Further, in the information communication system according to claim 7, in the information communication system according to claim 6, when the image coding means of the center station shifts the value of the motion vector by half a pixel, it is in units of frame or field. It is characterized by changing the order of upper left, lower right, lower left, upper right.

In the above invention, the optimum encoding system is adopted for each of the up and down lines, and the center station which relays the image information from a plurality of terminal stations converts the encoding system of the image information from each terminal station. Then, the image information combined into one screen on the downlink is transmitted by broadcast communication.

Therefore, the difference in arrival time of the image information delivered to each terminal station by relaying the up and down lines is reduced, and a TV conference or the like can be conducted in a good and natural manner.

That is, by adopting the optimum coding method for each of the up-link and the down-link, it is possible to adapt to the network and efficiently perform the information communication.

[0026]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a schematic diagram of a TV conference system which is one embodiment of an information communication system according to the present invention.

As shown in FIG. 1, this TV conference system is constructed by connecting a plurality of terminals 101, 102, 103, 104 and a center station 10 via a communication line 1 as a transmission line. The communication line 1 has an up line (up line) and a down line (down line). Each terminal 101, 102, 10 from the center station 10
The moving picture data is transmitted to the mobile stations 3 and 104 through the downlink by the MPEG2 system which is the second coding system, while the first coding system is transmitted from the terminals 101, 102, 103 and 104 to the center station 10. H. The H.261 system or the like is used to transmit moving image data through an uplink.

The center station 10 receives the H.264 received from the communication line 1. The bit stream conversion unit 11 converts the 261 bit stream into an MPEG2 bit stream, and the image combining unit 12 combines the converted MPEG2 bit streams so as to be integrated into one screen.

The terminal 101 uses a camera 131 for picking up an image of a conference participant of its own station and the image data of the conference participant captured by the camera 131 as an H.264 image. Encoder (ENC) 111 that encodes a 261 bit stream and sends it to the communication line 1
And a decoder (DEC) 121 for decoding the MPEG2 bit stream received from the communication line 1, and the decoder 1
And a monitor 141 for displaying the image decoded by 21. Note that the terminals 102 other than the terminal 101,
Since 103 and 104 have the same configuration as the terminal 101, description of each configuration will be omitted.

As shown in FIG. 2, the bit stream converter 11 receives the H.264 signal received by the center station 10 through the communication line 1. H.264 that decodes a H.261 bit stream into moving image data. 261 decoder 201 and MPEG for encoding the decoded moving image data into an MPEG2 bit stream.
2 encoder 202.

As shown in FIG. 3, the image synthesizing unit 12 has a plurality of decoders 301, 301, 302, 30 for decoding each MPEG2 bit stream into moving image data.
3, 304 and these decoders 301, 301, 302,
A screen rearrangement unit 305 that rearranges the screen of the moving image data decoded by 303 and 304 onto one screen, and encodes the data of one screen rearranged by this screen rearrangement unit 305 into an MPEG2 bit stream. It is composed of an encoder 306 for sending to the communication line 1. next,
The operation of this TV conference system will be described.

First, in this TV conference system, information from a plurality of terminals 101, 102, 103, 104 is relayed by the center station 10 and the terminals 101, 102, 10 are relayed.
The case of transmitting the data to 3 and 104 will be described.

In this case, each terminal 101, 102, 10
Cameras 131, 132, 13 connected to 3, 104
The images captured at 3 and 134 are encoded by the encoder 1 in each terminal.
11, 112, 113, 114 in H.M. It is encoded into a 261 bit stream. The encoded data is transmitted to the center station 10 through the communication line 1.

Each H.264 transmitted to the center station 10. 26
The data of one bit stream is H.264 in the bit stream converter 11. After being decoded by the H.261 decoder 201, M
The MPEG2 bitstream is encoded by the PEG2 encoder 202 and output to the image synthesizing unit 12.

In the image synthesizing unit 12, the MPEG2 bit stream is transferred to each of the decoders 301, 301, 302, 303,
The moving image data is respectively decoded by 304, and each moving image data is rearranged (combined) as moving image data of one screen by the screen rearrangement unit 305. Then, the rearranged moving image data of one screen is processed by the encoder 306 into MP
EG2 bitstream encoded, communication line 1 again
Each terminal 101, 102, 103, through (downlink)
It is transmitted to 104.

Each terminal 101, 10 through the communication line 1
The data sent to Nos. 2, 103 and 104 are MPEG-decoded by the respective internal decoders 121, 122, 123 and 124.
The 2-bit stream is decoded into moving image data, and one screen is displayed as a moving image divided into four on the display screens of the monitors 141, 142, 143, 144.
Thus, according to the video conference system of this embodiment,
The uplink of the communication line 1 is H.264. Data is transmitted using the H.261 system and the MPEG2 system on the downlink. This enables data communication with an encoding method suitable for the image data of each line. For example, the H.
261 → MPEG2 bit stream converter 11,
By providing the screen synthesizing unit 12 and the like, the converted data can be smoothly transmitted to the upstream and downstream lines without a time difference, the delay of the data transmission time can be reduced, and each coding system can be used in the network. Is applicable, the transmission efficiency of moving image data is improved, and a video conference can be held in a more natural form.

Another embodiment of the TV conference system will be described below.

The bit stream converter 11 shown in FIG.
Although the configuration example of 1 is extremely simple, information may be lost because it is encoded again after decoding, which causes deterioration of image quality. Further, since much time is spent on the decoding / encoding processing, information transmission tends to be delayed.

Therefore, the bit stream converter 11
As shown in FIG. 261 variable length decoder VLD4
01, a data reconstructing unit 402, and a variable length coder VLC403 of MPEG2 to form a bitstream converting unit 41, so that the image is subjected to only variable length decoding so as to prevent information loss as much as possible. It is also possible to convert information.

In this case, the H. The H.261 bit stream is based on H.264. Variable length decoding is performed by the VLD 401 of H.261, and information included in the bitstream is extracted. H. Two
Although the encoding methods of 61 and MPEG2 are different in detail, the basic method is the same, so the data reconstructing unit 402
By H. The information that the H.261 encoding method has is analyzed and M
It is converted into the information of the PEG2 encoding method. The information reconstructed (converted) by the data reconstruction unit 402 is, for example, header information such as a start code, motion vector information, and DCT coefficient component information. The start code and macro block position are specified by MPEG.
Convert to 2 code. Motion vector information is MPEG
Since 2 corresponds to a half pixel, the value is doubled. The DCT coefficient component is converted according to the quantization characteristic value. The converted information is variable-length coded by the MPEG2 variable-length encoder VLC403, and an MPEG2 bit stream is output.

As described above, according to the video conference system of this embodiment, the data conversion is performed in a state where only the variable length decoding (VLD) is performed without performing the inverse DCT process and the motion compensation process. The video conference can be conducted in a more natural form with less delay time.

Further, although the configuration example of the image synthesizing unit 12 shown in FIG. 3 is extremely simple, since it is re-encoded after decoding, image quality deterioration and delay time occur.

Therefore, as shown in FIG. 5, the image synthesizing unit 12 controls the variable length decoders (VLD) 501, 502, 50.
3 and 504, the variable data conversion unit 505, and the variable length encoder (VLC) 506 are used as the image synthesis unit 52 to reconstruct (convert) image information in a state where only variable length decoding is performed. It can be performed.

In this case, each bit stream input to the image synthesizing unit 52 is a variable length decoder (VLD) 501,
Variable length decoding is performed at 502, 503, and 504, and the information included in the bitstream is extracted. Position information of the extracted information is converted by the variable data conversion unit 505 according to the position where new image data is arranged, and the data is rearranged in the encoding order (bitstream reconstruction).

This information is used by the variable length encoder (VLC) 5
Variable length coding is performed at 06, and an MPEG2 bit stream is output.

As described above, according to the video conference system of this embodiment, the image information conversion (bitstream reconstruction) is performed in a state where only the variable length decoding is performed without performing the inverse DCT process and the motion compensation process. The loss of information is small and the delay time can be reduced.

By the way, H. Complete data conversion cannot be performed between the H.261 system and the MPEG2 system.

Techniques for data conversion include an in-loop filter and inverse quantization processing.

H. In the H.261 system, there is a mode in which a filter having a coefficient characteristic as shown in FIG. 6 is applied in order to reduce the block distortion rate due to motion compensation.
No filter can be applied in the two methods. Since this filter is in the loop of the encoding process, the disorder of the decoded image due to this error increases with time.

As a method of reducing this error, MPE
G2 half-pixel prediction is used.

For example, by applying a filter characteristic as shown in FIG. 8 by shifting the value of the motion vector in the MPEG2 system by half a pixel to the upper left as shown by the "x" mark in FIG. The effect can be obtained.

Then, the value of the motion vector is further calculated as shown in FIG.
By sequentially shifting as shown in (b) to FIG. 7 (d), the filter coefficients shown in FIG. 6 can be approximated on average.

In addition, H. In the case of the inverse quantization processing in the H.261 system, the inverse quantization value (REC) is the quantization characteristic value (Q
UANT) and the quantized representative value (LEVEL) are given by the following equation.

When QUANT is an odd number, REC = QUANT × (2 × LEVEL + 1); LEVEL> 0 REC = QUANT × (2 × LEVEL−1); LEVEL <0 When QUANT is an even number, REC = QUANT × (2 × LEVEL + 1) -1; LEVEL> 0 REC = QUANT * (2 * LEVEL-1) +1; LEVEL <0 On the other hand, in the case of the MPEG2 system, the inverse quantization value (F) is the quantization representative value (QF) and the quantization. The characteristic value (QS) and the value (W) based on the quantization matrix are given by the following expressions.

F = ((2 × QF + 1) × W × QS / 32); QF> 0 F = ((2 × QF-1) × W × QS / 32); QF <0 Therefore, when QUANT is an odd number Can make REC and F coincide with each other by setting QF = LEVEL and QS = QUANT × 32 / W, but if QUANT is an even number, they cannot simply coincide.

Therefore, in this case, QF = 16, W = 1, QS = QUANT × (2 × LEVEL + 1) −1; LEVEL> 0 QF = −16, W = 1 by using escape coding of the MPEG2 system or the like. , QS = QUANT × (2 × LEVEL−1) +1; LEVEL <0, so that F can be approximated to REC.

In this way, when a new center station is realized by combining the form of the bit stream converter 41 shown in FIG. 4 and the form of the image synthesizer 52 shown in FIG. A variable length decoder (VLD) and a variable length encoder (VL) are provided between the device 41 and the image synthesizing unit 52.
C) will overlap.

Therefore, a new center station 60 can be constructed by omitting one VLD and one VLC from the bit stream converter 41 and the image synthesizer 52.

As shown in FIG. 9, this center station 60
Is a variable length decoder 601, 602, 603, 604,
Encoding method conversion units 611, 612, 613, 614;
Arrangement conversion unit 605 and variable length coder (VLC) 606
It is composed of

In this case, each H.S. The H.261 bit stream includes H.264 bit streams. 261
The variable length decoders 601, 602, 603, and 604 perform variable length decoding to extract information included in the bitstream. The extracted information is transmitted to each of the encoding method conversion units 611 and 6
12, 613, and 614 convert the information into MPEG2 information, and the arrangement conversion unit 605 further converts and rearranges the position information according to the newly arranged position. This information is variable-length coded by a variable-length coder (VLC) 606 of MPEG2, and an MPEG2 bit stream is output.

According to the video conference system of this embodiment, the center station 10 is configured by omitting one VLD and one VLC from the bit stream converting section 11 and the image synthesizing section 12, and performs the same encoding as above. A method conversion process can be performed.

The present invention is not limited to the above embodiment.

In the above-described embodiment, the H.264 channel is set on the uplink. Two
61, each of the MPEG2 systems is applied to the downlink, but any other suitable coding system may be applied.

Further, in the above-mentioned embodiment, the uplink coding systems are all the same, but there may be a plurality of uplink coding systems. In this case, the center station may be provided with a decoder or a variable length decoder adapted for each uplink.

Further, in the above-described embodiment, the same bit stream is output to all downlink terminals, but different bit streams may be output to each terminal. In this case, output function units for screen layout conversion may be prepared for the number of output bitstreams, and variable length coding may be performed. As a result, different images can be received for each terminal. Also, by inserting a coding method converter in the preceding stage of the variable length coder, it is possible to transmit a bit stream of a different coding method for each terminal.

[0067]

As described above, according to the present invention, in the center station, the information conversion is performed by the efficient and optimum coding method for each of the uplink and the downlink.
Communication between the terminal stations can be efficiently performed.

Further, in the center station, a plurality of image information is combined into one screen and transmitted to the downlink simultaneously, so that the difference in arrival time of the information from the uplink side is reduced, so that a video conference etc. can be performed between the terminal stations. In doing so, there is no delay in the movement of the images, and it is possible to have a more natural conversation.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a TV conference system which is an embodiment according to the present invention.

FIG. 2 is a block diagram showing a bitstream conversion unit of the TV conference system of FIG.

FIG. 3 is a block diagram showing an image composition unit of the TV conference system of FIG.

FIG. 4 is a block diagram showing another embodiment of the bitstream conversion unit of FIG.

5 is a block diagram showing another embodiment of the image synthesizing unit in FIG.

FIG. It is a figure which shows the coefficient of the filter in a loop in 261.

7A to 7D are diagrams showing motion vector values based on MPEG2 half-pixel prediction.

FIG. 8: It is a figure which shows the coefficient by the half-pixel prediction of the MPEG2 system approximated to the in-loop filter in the H.261 system.

9 is a diagram showing a case where a new center station is realized by combining the form of the bit stream converting unit shown in FIG. 4 and the form of the image combining unit shown in FIG.

FIG. 10 is a block diagram showing a configuration of a conventional TV conference system between two points.

FIG. 11 is a block diagram showing the configuration of a conventional three-point video conference system.

[Explanation of symbols]

1 ... communication line, 10, 60 ... center station, 11,41 ...
Bitstream converter, 12, 52 ... Image synthesizer, 1
01, 102, 103, 104 ... Terminals, 111, 12
1, 131, 141 ... H.261 encoder, 121, 12
2, 123, 124 ... MPEG2 decoder, 131, 132,
133, 134 ... Cameras, 141, 142, 143, 1
44 ... Monitor.

Claims (7)

[Claims] 1. An information communication system in which at least one center station and a plurality of terminal stations are connected on a transmission path having upstream and downstream lines, wherein the center station uses a first code by each of the terminal stations. Image decoding means for receiving and decoding each image information encoded by the encoding method and transmitted through the uplink, and encoding each image information decoded by the image decoding means by the second encoding method And image transmission means for transmitting image information encoded by the second encoding method by the image encoding means to each of the terminal stations using the downlink. Each of the terminal stations includes an image acquisition unit that acquires image information transmitted from the terminal station, and the image information acquired by the image acquisition unit.
Image coding means for coding by the coding method of No. 2, information transmission means for transmitting the image information coded by the image coding means to the center station through the uplink, and the second information is transmitted by the center station. An image decoding means for receiving and decoding the image information coded by the coding method and transmitted through the downlink; and a display means for displaying the image information decoded by the image decoding means. Characteristic information communication system. 2. The information communication system according to claim 1, further comprising an image synthesizing means for synthesizing the image information decoded by the image decoding means of the center station so as to be integrated on one screen. An information communication system characterized by the above. 3. The information communication system according to claim 1, wherein the information transmission means of the center station includes a broadcast communication means for transmitting image information to the plurality of terminal stations all at once. Information and communication system. 4. The information communication system according to claim 1, wherein the image encoding means of the center station has a characteristic amount of data encoded by the first encoding method used in the uplink. Is extracted by variable-length decoding, converted into the characteristic amount of the second encoding method used in the downlink, and variable-length encoded by this second encoding method to generate transmission data. Information and communication system. 5. The information communication system according to claim 4, wherein the image encoding means of the center station corrects the quantized decoded value to an odd number when the quantization characteristic value is an even number in the first encoding method. In this case, a quantization characteristic value that gives a quantized decoded value that is an odd number in the second encoding method is given so that the quantized decoded value becomes equal to the quantized decoded value in the first encoding method. An information communication system, characterized in that a quantized value in the second encoding method is determined. 6. The information communication system according to claim 4, wherein the image coding means of the center station performs the second coding on the macroblock to which the in-loop filter is applied by the first coding method. An information communication system characterized in that the value of a motion vector is changed to a value shifted by half a pixel by the encoding method of. 7. The information communication system according to claim 6, wherein the image coding means of the center station shifts the value of the motion vector by a half pixel, and the upper left, lower right, and lower left in a frame unit or a field unit. ,
An information communication system characterized by being changed in the order of upper right.