JPH0222991A - Packet television conference terminal - Google Patents

Abstract

PURPOSE:To prevent time difference between a reproducing image and a reproducing voice from being generated by matching the arrival time of an image packet and a voice packet. CONSTITUTION:A terminal 7 sends the image packet and the voice packet whose sequence numbers coincide encoded and attached at the same time in transmission to an image codec part 9 and a voice codec part 10, respectively simultaneously at the time of receiving a packet from a line, and decoding and reproduction are performed at the same time, Therefore, even when the image packet and the voice packet encoded at the same time arrive at a reception terminal with time difference, the difference can be absorbed and they can be reproduced. In such a way, it is possible to surely dissolve the time lagging between the reproducing image and the reproducing voice.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a packet video conference terminal using a packet network.

[Background Art] In recent years, a video conference system that realizes a conference by connecting conference rooms located at different locations through communication lines has been attracting attention.

Dedicated lines are generally used as communication lines in such video conferencing systems, but since the amount of image information varies greatly due to the movement of the target image, etc.
Packet communication lines may be used.

FIG. 5 is a diagram showing connections including a network of packet video conference terminals. In the figure, (7) is a packet video conference terminal, and (8) is a packet connecting the packet video conference terminals (7). It's a net.

FIG. 6 is a block diagram of the packet video conference terminal (7). In the figure, (9) is an image codec unit that encodes/decodes image information, (10) is an audio codec unit that encodes/decodes audio information, and (1) is a packet of image information and audio information. A transmission control unit that performs assembly/disassembly and multiplexing/separation; (11) a camera that converts captured images into electrical signals and supplies them to an image codec unit (9);
) is a monitor that plays back the image information from the image codec unit, (13) is a microphone that converts audio into an electrical signal and supplies it to the audio codec unit (10), and (14) is the audio from the audio codec unit (10). This is a speaker that plays information.

FIG. 7 is a block diagram of the transmission control unit (1), in which (2) is an image PAD unit that assembles/disassembles packets of image information, and (3) assembles/disassembles packets of audio information. The audio PAD unit (5) is a multiplexing/separating unit that multiplexes and separates image/audio packets, and (6) is an image/audio identifying unit that adds and identifies multiplexing identification flags of image/audio packets.

FIG. 8 is a conventional data flow diagram of image/audio information, showing the flow of image information and audio information sent from the transmission control section (1) to the image codec section (9) and the audio codec section (10). It shows. In this data flow, it means that the left side is faster in time.

Next, the operation will be explained. In FIG. 6, first, the operation in the case of transmission will be explained. Image information from the camera (11) is encoded by an image codec section (9) and sent to an image PAD section (2) in a transmission control section (1). Similarly, the voice information from the microphone (13) is encoded by the voice codec (10), and the voice P in the transmission control section (1) is encoded.
It is sent to the AD section (3).

Image and audio information are provided in the image PAD section (2) and audio P, respectively.
Packets are assembled in the AD section (3) and sent to the multiplexing/demultiplexing section (5)
is multiplexed and transmitted to the line. At this time, the image/audio identifying section (6) adds an identification flag for identifying whether the packet is an image packet or an audio packet.

Next, the reception operation will be explained. Packets arriving from the line are first processed by the multiplexing/demultiplexing unit (5) in the transmission control unit (1).
), the image/audio identification unit (6) checks the identification flag added to the packet, separates it into an image packet and an audio packet, and separates the packet into an image PAD.
The signal is sent to the audio PAD section (2) and the audio PAD section (3).

Image packets and audio packets are decomposed into packets in an image PAD section (2) and an audio PAD section (3), respectively, and output as image information and audio information, which are sent to an image codec section (9) and an audio codec section (10), respectively. Ru.

As shown in FIG. 8, the overall operation of the transmission control unit (1) is to sequentially separate and decompose the arriving packets, and to
), and the image information and audio information are decoded in the image codec section (9) and the audio codec section (10), and are played back on the monitor (12) and speaker (14), respectively.

[Problems to be Solved by the Invention] Conventional packet video conference terminals are configured as described above, so that image/audio packets transmitted at the same time may arrive at the receiving terminal with a time lag due to fluctuations in the packet network. When the images arrive, there is a problem in that there is a time lag between the reproduced image and the reproduced audio.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a packet video conference terminal that can match the arrival times of image packets and audio packets.

[Means for Solving the Problems] The packet video conference terminal according to the present invention adds the same sequence number to image packets and audio packets encoded at the same time during transmission, and transmits them by adding the same sequence number to them. The apparatus is equipped with an image/audio synchronization processing section that matches the arrival times of image packets and audio packets using sequence numbers added at the time of transmission.

[Operation] When a packet video conference terminal according to the present invention receives a packet from a line, it simultaneously processes an image packet and an audio packet whose sequence numbers, which were encoded and added at the same time at the time of transmission, match, by the image codec unit and the audio packet, respectively. By sending the audio to the audio codec unit, decoding and reproducing it at the same time, it is possible to reliably eliminate the time lag between the reproduced image and the reproduced audio at the time of reproduction.

[Example] Hereinafter, an example of the present invention will be described with reference to the drawings. 1st
The figure is a block diagram of the transmission control section of the packet video conference terminal in which the same parts as in FIG. 5 are given the same reference numerals.
In the figure, (4) is an image/audio synchronization processing section.

FIG. 2 is an internal configuration diagram of the processing unit (4). In the figure, the image/audio synchronization (21) is the image transmission buffer, (22)
) is the image reception buffer, (31) is the audio transmission buffer,
(32) is an audio reception buffer, (41) is a sequence number processing unit, (42) is a sequence number comparison unit, (43)
(44) is a transmission starting unit, and (44) is a sequence number generating unit.

FIG. 3 is a data flow diagram of image/audio information, showing the flow of image information and audio information sent from the transmission control unit (1) to the image codec unit (9) and audio codec unit (10). .

FIG. 4 is an explanatory diagram of sequence number addition, in which sequence numbers are added when image information and audio information are not simultaneously sent from the image codec section (9) and the audio codec section (10) to the transmission control section (1). It is a figure explaining a means.

Next, the operation will be explained. Image information camera (11) in FIG. 6 -> image codec section (9) - transmission control section (1) -> line - transmission control section (1) -> image codec section (9) -> monitor (12) and audio information The transmission and reception route of the microphone (13) → audio image codec unit (10) → transmission control unit (1) → line - transmission control unit (1) - audio codec unit (10) - speaker (14) is the same as in the conventional example. be.

Since the difference from the conventional example is the operation of the transmission control section (1), the operation will be explained below with reference to FIGS. 1, 2, 3, and 4.

The encoded image information from the image codec unit (2) and the encoded audio information from the audio codec unit (3) are sent to the image PAD unit (2) and audio PAD unit in the transmission control unit (1), respectively. In (3), the packet is assembled. At that time, the same sequence number is added to the image information and audio information that arrive at approximately the same timing by the sequence number generation section (44) in the image/audio synchronization processing section (4).

The sequence number generation unit (44) includes an image codec unit (
It is recognized by the image information enable signal and the audio information enable signal whether the image information from 9) and the audio information from the audio codec unit (10) arrive at the same time.

In addition, if the image information and audio information are not output at the same time, as shown in Figure 4, the sequence number is rOJ
, and notifies the receiving terminal that the image information and audio information do not need to be synchronized.

Next, in the multiplexing/separating section (5), the image PAD section (2
) and the audio packets from the audio PAD section (3) are multiplexed, an identification flag is added by the image/audio identification section (6), and the images are transmitted to the line.

At the receiving terminal, as in the conventional example, packets arriving from the line side are sent to the demultiplexer (1) in the transmission controller (1).
5), and is separated into an image packet and an audio packet by checking the identification flag by the image/audio identification unit (6), and is sent to the image PAD unit (2) and the audio PA, respectively.
It is sent to section D (3).

Image packets and audio packets are depacketized in the image PAD unit (2) and audio PAD unit (3), respectively, but after the packet decomposition, they are each decomposed into the image codec unit (
9) and the timing of sending to the audio codec unit (10) are controlled by the image/audio synchronization processing unit (4).

The image/audio synchronization processing section (4) sends image information and audio information having the same sequence number to the image codec section (9) and the audio codec section (10) at the same timing. In other words, the sequence number “1” is sent before the line side.
” even if the audio packet arrives, it is not immediately output to the audio codec unit (10) and the sequence number “1” is sent.
After the image packet arrives, image information and audio information will be output at the same timing.

The operation of the image/audio synchronization processing section (4) at this time will be explained with reference to FIG.

The image reception buffer (22) and audio reception buffer (32) in the image PAD unit (2) and audio PAD unit (3) transfer the sequence number added to the received data to the image/audio synchronization processing unit (4). Sequence number processing unit (41)
Send to.

The sequence number processing unit (41) compares the image sequence number and audio sequence number with the sequence number comparison unit (41).
42), and if they are the same, the sending starting section (4
3) Outputs the “same” signal to

The transmission activation unit (43) activates the transmission of received information to the image reception buffer (22) and the audio reception buffer (32) upon reception of the same signal.

However, the sequence number "0" does not synchronize with the image/audio information and immediately sends it to the image codec section (9) or the audio codec section (10).

Image information and audio information are stored in the image codec section (9
) and the operation after being output to the audio codec unit (10) is the same as in the conventional example.

[Effects of the Invention] As described above, according to the present invention, a packet conference terminal can be configured to simultaneously decode image packets and audio packets with matching sequence numbers that were encoded and added at the same time at the time of transmission. Therefore, even if an image packet and an audio packet encoded at the same time arrive at the receiving terminal with a time lag, the lag can be absorbed and played back. It has the effect of reliably eliminating the problem.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a transmission control section of a packet video conference terminal according to an embodiment of the present invention, FIG. 2 is an internal configuration diagram of an image/audio synchronization processing section, and FIG. 3 is a data flow diagram of image/audio information. , Figure 4 is an explanatory diagram of sequence number addition, Figure 5
The figure shows the connection of the packet video conference terminal including the network, Figure 6 is a block diagram of the packet video conference terminal, Figure 7 is a block diagram of the transmission control section of the conventional packet video conference terminal, and Figure 8 is the conventional packet video conference terminal. FIG. 3 is a data flow diagram of image/audio information in the packet video conference terminal of FIG. In the figure, (1) is the transmission control unit, and (2) is the image PAD.
section, (3) is an audio PAD section, (4) is an image/audio synchronization processing section, (5) is a multiplexing/demultiplexing section, (6) is a packet network, (
9) is an image codec section, (10) is an audio codec section, (21) is an image transmission buffer, (22) is an image reception buffer, (31) is an audio transmission buffer, (32) is an audio reception buffer, (41) is the sequence number processing unit, (4
2) is a sequence number comparison section, (43) is a transmission starting section,
(44) is a sequence number generation unit. It is. In addition, in the figures, the same reference numerals indicate the same or equivalent parts. Agent: Patent attorney Masuo Oiwa (and 2 others) I, 5th year of junior high school, detailed description of the invention, brief description of drawings, and drawings in the specification subject to amendment. 6. Contents of amendment Procedural amendment 1 Sagi (spontaneous) 1. Indication of the case Patent application Sho No. 2. Name of the invention Packet video conference terminal 3. Person making the amendment Representative Moriya Shiki 4. Agent A

Claims (1)

[Claims] An image codec unit that encodes and decodes image information; an audio codec unit that encodes and decodes audio information; and encoded image information from the image codec unit and the audio codec. The encoded audio information from the section is packetized and multiplexed and transmitted to the line side, or the packets received from the line side are separated and packet-decomposed into image information and audio information, and the encoded audio information is sent to the image codec section and the said image codec section, respectively. a transmission control unit for sending to an audio codec unit; and a packet video conference terminal having the same sequence number added to image packets and audio packets encoded at the same time when transmitting to the line side. The receiving side includes an image/audio synchronization processing unit that matches the arrival times of the image packet and the audio packet based on the sequence number added at the same time at the time of transmission, A packet video conference terminal characterized by eliminating the time lag between.