JPH0884329A - Image communication terminal - Google Patents

Abstract

(57) [Summary] [Purpose] To enable special playback of stored images. [Structure] The image coding circuit 22a conforms to ITU-T Recommendation H.264.
The communication coding circuit according to H.261 and the storage coding circuit according to MPEG1 are provided. When storing the received image in the image storage device 23, the storage notification control unit 44 of the image receiving side terminal transmits control information for notifying the image transmitting side terminal of the storage of the received image in advance. When the reception control information recognition unit 42 of the image transmission side terminal detects the control information for notifying the accumulation of the received image from the image reception side terminal, it instructs the image encoding circuit 22a to select the accumulation encoding circuit. In the mode in which the image information encoded by the terminal is stored in the image storage device 23, the encoding circuit 22a encodes the image information in the storage encoding circuit according to the control signal from the image storage device 23. Instead of selecting the storage encoding circuit, the communication encoding circuit may insert an intra-frame encoded frame at regular intervals.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image communication terminal device.

[0002]

2. Description of the Related Art In recent years, the development of image compression coding technology and the spread of digital communication lines have been remarkable, and service regulations and protocol regulations for audiovisual services for videophones or videoconferences, and multimedia multiplexing frame configuration regulations. It has been prepared as a recommendation such as, with this,
There has been proposed a communication terminal device for audio and moving images which can be used for a videophone and a video conference system.

As is well known, in order to directly digitally transmit moving image information, a transmission speed of several hundred Mbps is required. Therefore, it is normally compressed and encoded for transmission. Various compression coding schemes have been proposed for this purpose.

As a compression encoding method for moving image communication,
The mainstream is a hybrid coding method that removes temporal redundancy by motion-compensated interframe predictive coding that uses correlation in the temporal direction, and spatial redundancy by orthogonal transform coding that uses correlation in the spatial direction. ing. That is, ITU-T (formerly CCITT) Recommendation H.264 is used as a communication encoding system for video conference systems and video phones. 261
There is MPEG1 as a storage encoding method whose main purpose is storage on a digital storage medium such as a compact disc (CD).

[0005]

[Problems to be Solved by the Invention] In a conventional moving image communication terminal device equipped with a communication codec represented by H.261 and a storage codec represented by MPEG1 (or MPEG2), a communication codec is normally used for real-time communication and image storage is performed. Uses a storage codec.

However, even in such a moving picture communication terminal device, in the conventional example having the answering machine function, the received moving picture information is compressed by the communication codec even though the recorded message is also stored in the storage medium. It is encoded and recorded or stored in a recording medium or a storage medium. ITU-T Recommendation H.264. 261
Then, since inter-frame predictive coding and intra-frame coding are selectively used for each macro block, for example, when accumulating a received image, regarding a macro block whose accumulation is started from encoded data subjected to inter-frame encoding processing, Cannot reproduce the correct image during the period until the next intra-frame coded data is received.

An object of the present invention is to provide an image communication terminal device which does not cause such inconvenience.

It is another object of the present invention to provide an image communication terminal device in which a communication codec and a storage codec are used properly according to the purpose.

[0009]

An image communication terminal device according to the present invention compresses and encodes input image information by one of a communication encoding system and a storage encoding system selected. A transmitting means for transmitting the encoded image data by the encoding means, a receiving means for receiving control information from a terminal on the side that receives the encoded image data by the transmitting means, and a receiving means for receiving by the receiving means. In response to the control information, there is provided encoding control means for instructing the encoding means to select the storage encoding method.

The image communication terminal device according to the present invention further comprises an encoding means for compressing and encoding the input image information by one of the communication encoding system and the storage encoding system, and the encoding system. Storage means for storing the coded image data by the means, and coding control means for instructing the coding means to select the intraframe coding method at a predetermined interval when the coded image data is stored by the storage means. It is characterized by including.

The image communication terminal device according to the present invention also comprises a receiving means for receiving the encoded image information, and an accumulating means for accumulating the encoded image data received by the receiving means.
Prior to the accumulation of the coded image data by the accumulating means, there is provided control information transmitting means for transmitting control information for notifying accumulation of a received image to a transmission side terminal of the received encoded image data. .

The image communication terminal apparatus according to the present invention also compression-codes the input moving image information by selectively using an intra-frame coding method and an inter-frame coding method for each predetermined coding unit. Encoding means, transmitting means for transmitting the encoded image data by the encoding means, and first receiving means for receiving first control information from the receiving side terminal of the encoded image data by the transmitting means, An encoding control means for instructing the encoding means to forcibly select the intraframe encoding method at every predetermined timing in response to the first control information received by the first receiving means. It is characterized by doing.

The image communication terminal apparatus according to the present invention is also a code for compression-coding the input image information by selectively using an intra-frame coding method and an inter-frame coding method for each predetermined coding unit. Encoding means, accumulating means for accumulating the encoded image data by the encoding means, and in accumulating the encoded image data by the accumulating means, the encoding means is forcibly encoded at every predetermined timing. Encoding control means for instructing to select an encoding method.

[0014]

With the above-mentioned means, it becomes possible to forcibly switch to the storage coding method and perform the compression coding in response to the control information from the transmitting side terminal. This also facilitates special reproduction of the stored image.

Further, in response to the control information on the transmitting side, it becomes possible to forcibly insert the intra-coded frames (updated screen) at predetermined intervals, and as a result, the accumulated image can be reproduced halfway. Like

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a schematic block diagram of a moving picture communication terminal according to an embodiment of the present invention.

The configuration of FIG. 1 will be described. Reference numeral 10 is a camera for photographing conference participants, 12 is a document camera for photographing conference materials such as drawings, 14 is an image input interface for selecting outputs of the cameras 10 and 12, and converting the output into a predetermined internal format. 16 is an image display. An image output interface 18 supplies an image signal to the monitor 16. The monitor 16 may be a single image display device or a plurality of image display devices. Further, the monitor 16 may be a single image display device capable of displaying a plurality of images in separate windows by a window system.

Reference numeral 20 denotes a selection / combination circuit that selects and combines the input images and the received images from the cameras 10 and 12 and supplies them to the image output interface 18. For example, the selection / combination circuit 20 has, for example, a picture-in-picture or screen freeze function. Reference numeral 22 is an image coding / decoding circuit including an image coding circuit 22a for coding the image signal to be transmitted and an image decoding circuit 22b for decoding the received coded image signal.

An image storage device 23 stores and stores image information to be transmitted and / or received image information in a compressed state. Details of the function of the image storage device 23 will be described later.

Reference numeral 24 is a handset including a microphone and a speaker, 26 is a microphone, 28 is a speaker, and 30 is a voice input / output interface for the handset 24, the microphone 26 and the speaker 28. The voice input / output interface 30 not only switches the voice input / output of the handset 24, the microphone 26, and the speaker 28, but also performs echo / cancel processing, dial tone, ringing tone,
Tones such as busy tones and ring tones are generated. Reference numeral 32 is a voice encoding / decoding circuit including a voice encoding circuit 32a for encoding the voice signal to be transmitted and a voice decoding circuit 32b for decoding the received encoded voice signal.

34 is a communication line (for example, ISDN line)
Line interface 36 of the image coding circuit 22
a and the coded information to be transmitted from the voice coding circuit 32a are multiplexed and supplied to the line interface 34, and the coded image information and the coded voice information are separated from the received information supplied from the line interface 34, They are demultiplexing circuits which are supplied to the image decoding circuit 22b and the audio decoding circuit 32b, respectively.

Reference numeral 38 denotes the whole, particularly the image input interface 14, the image output interface 18, the selection / combination circuit 20, the image encoding / decoding circuit 22, the image storage device 23,
A system control circuit for controlling the voice input / output interface 30, the voice encoding / decoding circuit 32, and the demultiplexing / multiplexing circuit 36, and 40 is an operating device for inputting a predetermined instruction to the system control circuit 38 by the user (for example, a ten・ Keys, keyboards, etc.).

The system control circuit 38 internally transmits a reception control information recognition section 42 for monitoring and recognizing the control data contained in the reception data, and an accumulation of the reception image data according to an instruction signal from the image accumulation device 23. The storage notification control unit 44 that notifies the terminal is provided. These functional parts 42,
Reference numeral 44 is actually realized as a program for the CPU of the system control circuit 38.

In the present embodiment, the image coding circuit 22a of the image coding / decoding circuit 22 has the functions of both communication coding and storage coding, and the image decoding circuit 22b.
Has both the decoding function corresponding to the communication coding method and the decoding function corresponding to the storage coding method. That is,
The image encoding / decoding circuit 22 has functions of both a communication codec and a storage codec, and encodes image information by a method in accordance with an instruction from the system control circuit 38.
Also, the compressed image is decoded by a method corresponding to the compression method of the received image. The details will also be described later.

The basic operation of the moving picture communication terminal device shown in FIG. 1 will be described. The input images from the camera 10 and the document camera 12 are selected by the image input interface 14,
One of them is input to the selective combining circuit 20. The selective synthesizing circuit 20 normally outputs the input images from the cameras 10 and 12 as they are to the encoding circuit 22a of the image encoding / decoding circuit 22. The image encoding circuit 22a encodes the input image signal in the encoding mode according to the control signal from the system control circuit 38 and the internal determination, and supplies it to the image storage device 23, which will be described in detail later. In the normal case, the image storage device 23 outputs the output of the image encoding circuit 22a as it is to the demultiplexing circuit 3.
6 is output. Further, in the case of image communication, the image encoding circuit 22a compresses image information by the communication encoding method.

On the other hand, the input voice signal from the microphone or the microphone 26 of the handset 24 is input to the voice encoding circuit 32a of the voice encoding / decoding circuit 32 via the voice input / output interface 30 and is encoded and demultiplexed. 36
Is applied to

The demultiplexing circuit 36 is the image coding circuit 2.
Image information encoded by 2a, audio encoding circuit 32
The voice information encoded by a and the control information from the system control circuit 28 are multiplexed and output to the line interface 34. The line interface 34 outputs the signal from the demultiplexing circuit 36 to the connected communication line.

The signal received from the communication line is supplied from the line interface 34 to the demultiplexing / multiplexing circuit 36. The demultiplexing circuit 36 separates the coded image signal, the coded audio signal, and the control information from the received signal, and sends the coded image signal to the image decoding circuit 22b via the image storage device 23 to the coded audio signal. The control information is supplied to the system control circuit 38 to the voice decoding circuit 32b. Image decoding circuit 22b
Is for decoding the encoded image signal from the demultiplexing circuit 36 input via the image storage device 23, and
Apply to. In the case of image communication, the image information is coded according to the communication coding method, so the image decoding circuit 2
2b expands the compressed image information by a decoding method corresponding to the communication encoding method.

The selective combining circuit 20 is a system control circuit 38.
Image input interface 14 according to the control signal from
The input image from and the received image from the image decoding circuit 22b are selectively combined and output to the image output interface 18. The selective combining circuit 20 performs, for example, picture-in-picture or fitting into a corresponding window in the window display system as a combining process. The image monitor 16 displays the image signal from the image output interface 18 as an image. As a result, the input image and / or the received image is displayed on the screen of the monitor 16.

The received voice signal decoded by the voice encoding circuit 32b is applied to the speaker and / or the speaker 28 of the handset 24 via the voice input / output interface 30. As a result, the voice from the communication partner can be heard.

The reception control information recognizing section 42 in the system control circuit 38 notifies various control information multiplexed in the reception data (or notified by the control line), particularly, notification of accumulation of the reception image from the image receiving terminal. When the control information to be recognized is recognized, the image coding circuit 22a is instructed to select the image compression by the storage coding method. Details of the encoding operation of the image encoding circuit 22a for this will be described later.

When the terminal shown in FIG. 1 is an image receiving terminal and the image storage device 23 stores the received image data, the storage notification control unit 44 starts the storage of the image prior to the storage. The control information to be notified is notified to the image transmission side terminal. Details of the storage operation of the image storage device 23 at this time will be described later.

FIG. 2 is a schematic block diagram showing a circuit configuration example of the image coding circuit 22a. ITU-T Recommendation H.50. Reference numeral 261 is a communication encoding circuit for compressing and encoding a moving image, and reference numeral 52 is a storage encoding circuit for compressing and encoding a moving image according to MPEG1. Reference numeral 54 denotes a storage code in the image encoding control circuit 56 in response to control signals from the reception control information recognition unit 42 of the system control circuit 38 and the self-encoded image storage monitoring circuit 66 (FIG. 4) of the image storage device 23. 2 is a storage-corresponding control circuit for instructing the selection of a coding method. The image coding control circuit 56 uses either the communication coding circuit 50 or the storage coding circuit 52 according to the control signal from the storage corresponding control circuit 54, the control signal from the system control circuit 38, the generated code amount, and the like. Or control the operating parameters and the like in the encoding circuits 50 and 52 to be used.

During normal operation, the system control circuit 38
However, the image coding control circuit 56 is switched between the communication coding circuit 50 and the storage coding circuit 52 according to the user setting.
Instruct.

H.264 executed by the communication encoding circuit 50. Two
An outline of the compression encoding processing of moving image information according to 61 will be described. H. In H.261, an inter-frame predictive coding mode for coding the difference from the prediction value in the reference frame and an intra-frame coding mode for coding the difference without taking the difference can be selected for each macro block as a coding unit. .
In an image with little movement or change, the current frame and the previous frame are very similar. Therefore, by using the inter-frame predictive coding (INTER) method that encodes the difference between the current frame and the previous frame, its temporal redundancy is reduced. Reduce. By adding motion compensation to this, a moving image can be efficiently coded.

In the case of an image with a significantly large motion or a scene change, since the correlation between frames is small, the coding within the same frame is effective, and the coding within the frame is performed as it is without taking the difference from the previous frame. Encoding (INTR
A) method is used.

These three coding modes, namely, the motion-compensated interframe predictive coding mode, the interframe predictive coding mode without motion compensation (that is, the simple interframe predictive coding mode), and the intraframe coding mode are The moving image is coded by adaptively switching in macroblock units.

The image coding circuit for communication 50 uses the block data thus obtained according to each coding mode,
A DCT (discrete cosine transform) process, which is a kind of orthogonal transform, is performed, the DCT coefficient data is quantized, and then variable length coding is performed together with coding mode information and the like.

MPE executed by the storage encoding circuit 52
A compression encoding process of moving image information according to G1 will be described. M
In PEG1, I that encodes the input image data itself
Picture (Intra Coded Picture)
And a P-picture (Predictive Coded Picture) that encodes the difference between the temporally preceding image and
e) and a temporally forward image, backward image, or a B picture (Bidirectional Predecode) that encodes a difference between the forward image and the interpolated image formed from the forward image and the backward image.
active coded picture) for each frame. An example of arrangement of each picture is shown in FIG.

The P picture is coded by adaptively selecting the most efficient coding mode in the INTRA coding mode and the forward prediction coding mode in macro block units.
The B picture is coded by adaptively selecting the most efficient coding mode among macro blocks in the INTRA coding mode, the forward prediction coding mode, the backward prediction coding mode, and the bidirectional predictive coding mode. .

The image coding circuit for storage 52 subjects the block data obtained according to each mode to DCT (discrete cosine transform) processing, which is a kind of orthogonal transformation, and quantizes the DCT coefficient data. Variable length coding is performed together with coding mode information and the like.

The storage coding method generally has lower compression efficiency due to the insertion of the I picture than the communication coding method, but has the advantage of enabling random access and high-speed reproduction. Also, reverse reproduction is possible by inserting B pictures. In the communication encoding method, generally, reverse reproduction cannot be realized unless the image data reproduced in the forward direction is stored in some storage device.

The operation of the storage-corresponding control circuit 54 will be described.
The storage-corresponding control circuit 54 is supplied with a signal from the image storage device 23 to notify execution of storage of the output image data of the image encoding circuit 22a.
A signal for notifying the execution of storage of received images at the partner terminal is supplied. The storage-corresponding control circuit 54 instructs the image coding control circuit 56 to select the storage image coding circuit 52 in response to these notification signals of execution of storage. That is, when the image is stored in the own terminal and the image received by the other terminal is stored, the image encoding circuit 22a causes the storage image encoding circuit 52 to store the image information into the storage image code. It is compression-encoded by the encoding method.

The internal structure and operation of the image storage device 23 will be described.

FIG. 4 is a schematic block diagram showing a circuit configuration example of the image storage device 23. In FIG. 4, reference numeral 60 denotes an image memory that stores compression-encoded image data, and 62.
Is an input / output selection circuit for switching the input / output of the image memory 60. Reference numeral 64 denotes a received image storage monitoring circuit that monitors the storage of received images and notifies the storage notification control unit 44 of the system control circuit 38 of the start of storage of received images. Reference numeral 66 denotes the storage of image information encoded by the terminal itself. Start the image coding circuit 2
2a is a self-encoded image storage monitoring circuit for notifying (the storage corresponding control circuit 54). 68 is a system control circuit 38
Is an image storage control circuit for monitoring and controlling each unit of the image storage device 23.

The operation of the image storage device 23 will be described. The image storage control circuit 68 instructs the input / output selection circuit 62 to perform input selection based on the storage instruction from the system control circuit 38, and controls the image memory 60 to the data write state.
The input / output selection circuit 62 selects one of the two inputs (the output of the image encoding circuit 22a and the output of the demultiplexing / multiplexing circuit 36) according to the selection instruction from the image accumulation control circuit 68, and the image memory 60 The data from the input / output selection circuit 62 is stored.

The reception image accumulation monitoring circuit and the self-encoded image accumulation monitoring circuit 66 constantly monitor the accumulation control operation of the image accumulation control circuit 68 based on the instruction from the system control circuit 38. When the input / output selection circuit 62 selects the output of the demultiplexing / multiplexing circuit 36 (that is, the received encoded image data) as the image information to be stored, the received image storage monitoring circuit 64 detects this and stores the storage notification control. Notify the section 44. On the other hand, the input / output selection circuit 62 is the image coding circuit 22.
When the image data compressed and encoded by a is selected as the image information to be stored, the self-coded image storage monitoring circuit 6
6 detects this and notifies the image encoding circuit 22a of that fact. Details of what the self-coded image storage monitoring circuit 66 notifies the image coding circuit 22a and its operation will be described later.

The image storage in the image storage device 23 includes image storage in the image storage device 23 of the image transmission side terminal and image storage in the image storage device 23 of the image reception side terminal.

First, the operation of storing the latter image will be described. When the image receiving side terminal stores the received image (for example, in the message recording mode), the image receiving side terminal transmits the control information indicating the storage of the received image to the image transmitting side terminal prior to the image communication. That is, when the received image storage monitoring circuit 64 of the image storage device 23 in the image receiving side terminal recognizes that it is in the received image storage mode, it notifies the storage notification control unit 44 of the start of storage of the received image data. . Storage notification control unit 4
In response to this, 4 transmits control information indicating execution of accumulation of received images to the image transmission side terminal.

In the image transmission side terminal, the reception control information recognition section 42 recognizes the control information indicating the execution of accumulation from the image reception side terminal, and the accumulation correspondence control circuit 5 of the image encoding circuit 22a.
4 is informed of execution of accumulation of received images at the image receiving side terminal. The storage correspondence control circuit 54 of the image encoding circuit 22a is
In response to this, the image coding control circuit 56 is instructed to select the storage coding circuit 52. As a result, the image to be transmitted is compression-encoded by the storage encoding circuit 52 by the storage encoding method. The compression-encoded image information passes through the image storage device 23, is transmitted to the partner terminal via the separation / multiplexing circuit 36, the line interface 34, and the communication line, and is the image storage device 23 of the partner terminal or similar thereto. Stored in the storage device. As a result, the receiving terminal can receive and store the image information compressed by the encoding method suitable for storage.

Next, a case will be described in which the image transmitting side terminal stores the image data that has been compression-encoded by its own image encoding circuit 22a in its own image storage device 23.

The self-encoded image storage monitoring circuit 66 of the image storage device 23 stores a self-encoded image in the self-device prior to the start of image encoding from the control signal supplied from the system control circuit 38 to the image storage control circuit 68. Is detected, and the image encoding circuit 22a is notified that the image storage device 23 stores the encoded image. In response to this, the storage correspondence control circuit 54 of the image coding circuit 22a responds to this by the image coding control circuit 56.
To execute the encoding control corresponding to storage. As a result, the image encoding circuit 22a compresses and encodes the image information from the selective synthesizing circuit 20 by the storage encoding circuit 52 by the storage encoding method. The image memory 60 of the image storage device 23 stores image information encoded by the storage encoding method.

FIG. 5 shows a flow chart of the above-mentioned operation for forcibly selecting the encoding for accumulation in correspondence with the accumulation on the image transmitting side and the accumulation on the image receiving side.

When the compression-encoded image is accumulated in the accumulating means by the above operation, the execution of the accumulation is notified to the image encoding circuit prior to the start of the accumulation, so that the image information of the accumulation encoding method is stored. You can choose compression encoding. As a result, the user can reproduce the accumulated image information by special reproduction such as random access, high-speed reproduction, and reverse reproduction without paying attention to the switching of the encoding method.
Further, it is possible to easily realize a function of discarding a part of the accumulated image and accumulating only frames for a necessary period, or accumulating only I pictures.

In the above embodiment, H.264 is used as the communication coding method. 261 is an MPEG as a storage encoding method.
1 has been described by way of example, the present invention is not limited thereto. That is, it can also be applied when any other communication / accumulation coding method is adopted.

Further, in the above embodiment, the case where the storage encoding method is always selected when executing the storage has been described, but the compulsory control is made flexible by providing a function for prohibiting this by the user setting or the like. May be.

In the above embodiment, the case where the communication encoder and the storage encoder are separately provided has been described. However, a part of the circuit is shared and the communication encoding and the storage encoder are selected by mode switching or the like. It is also clear that encoding switching can be realized.

It is obvious that a part of the embodiment described with reference to FIGS. 1 to 6 can be realized by software.

In the above embodiment, the communication coding means and the storage coding means are provided, and the storage coding means is forcibly coded the image information depending on the situation, that is, when the image storage is required. However, the coding method in the communication coding means is changed to the intraframe coding method (that is,
The same effect can be obtained by forcing a coding mode suitable for storage). FIG. 7 is a schematic configuration block diagram of a modified example of the encoding circuit 22a thus configured.

In the coding circuit shown in FIG. 7, interframe coding (I) for coding the difference from the previous frame (predicted value) is carried out.
The NTER) mode and the intra-frame coding (INTRA) mode for coding in the screen without taking the difference can be selected for each macro block in the frame. For example, in a normal image communication state, INTER mode is used for an image or a still image with little movement in the motion or time direction to reduce temporal redundancy, and a scene change can be performed even for an image with large motion or an image with little motion. In this case, the INTRA mode is used. As described above, the INTER mode includes motion compensation and non-motion compensation. In addition, the quantization step size is changed according to the amount of encoded data to be generated, and if necessary, frame skipping (frame skip) is performed. Although details will be described later, at the time of image storage, the coding mode is controlled so that the intra-frame coding mode is selected at appropriate frame intervals in each macroblock of the frame.

In FIG. 7, reference numeral 140 is a selective combining circuit 20.
The input terminal 142 receives pixel data from the input terminal 140, and the input terminal 142 receives an INT control signal according to an energy comparison result between the pixel value from the input terminal 140 and a prediction error of the pixel value and an external control signal.
It is an encoding mode selection circuit for selecting the RA mode or the INTER mode. The encoding mode selection circuit 142 uses the I
In the NTRA mode, the pixel value from the input terminal 140 is output as it is, and in the INTER mode, the difference (prediction error) from the prediction value (previous frame) is output in macroblock units.

Reference numeral 144 denotes a DCT circuit which performs discrete cosine transform on the output of the coding mode selection circuit 142 and outputs DCT coefficient data, and 146 denotes the DCT coefficient data output from the DCT circuit 144 at a specified quantization step.・
A quantizer circuit for quantizing by size, 148 is a variable length coding circuit for variable length coding the output of the quantizing circuit 146,
Reference numeral 50 is a transmission buffer for buffering the output of the variable length coding circuit 148, and 152 is an output terminal for connecting the output of the transmission buffer 150 to the demultiplexing circuit 36.

Reference numeral 154 is an inverse quantization circuit for inversely quantizing the output of the quantization circuit 146, and 156 is an inverse DCT circuit for inverse discrete cosine transform of the output of the inverse quantization circuit 154. 1
58 adds the predicted value to the output of the inverse DCT circuit 156 in the INTER mode and outputs it, and outputs the inverse D in the INTRA mode.
It is an adder that outputs the output of the CT circuit 156 as it is. Reference numeral 160 denotes a frame memory for motion-compensated interframe prediction, which is the output of the adder 158 (local decoded value).
Is stored.

A motion vector detecting circuit 162 detects a motion vector by comparing the image signal input from the input terminal 140 with the image signal of the previous frame stored in the frame memory 160 in macroblock units. In accordance with the motion vector detected by the motion vector detection circuit 162, a motion compensation circuit 166 that moves the data of the previous frame from the frame memory 160 within the screen in macro block units to cancel the motion, and 166 is a motion compensation circuit 164.
Is a low-pass filter that spatially filters the output of the macroblock. The output of the filter 166 becomes a predicted value of inter-frame prediction and is applied to the adder 158 and the subtractor 168. The subtractor 168 has an input terminal 140.
The difference between the pixel data from (1) and the output (predicted value) of the filter 166, that is, the prediction error is calculated and supplied to the coding mode selection circuit 142.

Reference numeral 170 indicates a coding mode selection circuit 142, a quantization circuit 146, and a variable length coding circuit 170 according to a control signal relating to coding from the system control circuit 38 and a buffer accumulation amount signal from the transmission buffer 150 at normal times. An image encoding control circuit for controlling the circuit 148. The encoding control signal from the system control circuit 38 includes:
There are signals for designating image quality (spatial resolution) priority, motion followability (temporal resolution) priority, or an intermediate value thereof, a selection signal for the document camera 12, and the like.

Similar to the storage-corresponding control circuit 54, 172 stores in the image coding control circuit 170 in response to the control signals from the reception control information recognizing unit 42 and the storage notification control unit 44 of the system control circuit 38. It is a storage-corresponding control circuit for instructing encoding control.

The normal operation of the circuit shown in FIG. 7 will be described first.

The input / output terminal 140 is supplied from the selective combining circuit 20, for example, CCITT Recommendation H.264. CIF (Common Interface) which is a common format according to H.261.
e Format) or QCIF (Qua
The image data is input in the rter CIF format. The image data input to the input terminal 140 is applied to the coding mode selection circuit 142, the subtractor 168, and the motion vector detection circuit 162.

The subtractor 168 calculates the difference (prediction error) between the pixel data from the input terminal 140 and the prediction value output from the filter 166, and the coding mode selection circuit 14
2 is applied. The encoding mode selection circuit 142 performs energy comparison between the pixel value from the input terminal 140 and the prediction error from the subtractor 168, and according to the comparison result and the control signal from the image encoding control circuit 170, the encoding mode. Select. The encoding mode selection circuit 142 uses the INTRA
When the mode is selected, the input pixel value from the input terminal 140 is output as it is to the DCT circuit 144, and when the INTER mode is selected, the prediction error from the subtractor 168 is converted to the DCT value.
Output to the circuit 144.

The DCT circuit 144 performs discrete cosine (DCT) conversion of the data from the coding mode selection circuit 142 in block units, and the DCT coefficient data is quantized by the quantization circuit 14.
6 is output. The quantization circuit 146 uses the quantization step size specified by the quantization characteristic control signal from the image coding control circuit 170 to generate the DC from the DCT circuit 144.
Quantize the T coefficient data. Variable length coding circuit 148
Determines a significant block according to the coding control signal from the image coding control circuit 170, and determines the quantized DCT coefficient as CCI.
TT Recommendation H. In accordance with H.261, variable length coding is performed.

The transmission buffer 150 buffers the variable length coded data by the variable length coding circuit 148 and outputs it to the demultiplexing circuit 36 via the output terminal 152, and at the same time, stores the buffer accumulation amount in the image coding control circuit. To 170.

The inverse quantization circuit 154 is the quantization circuit 146.
The output of the quantization circuit 146 is inversely quantized with the same quantization step size as that selected in, and the representative value of the DCT coefficient is output. The inverse DCT circuit 156 is the inverse quantization circuit 15
The output of 4 is inverse discrete cosine transformed. The adder 158 is
In the INTER mode, the predicted value (output of the filter 166) is added to the output of the inverse DCT circuit 156, and in the INTRA mode, the output of the inverse DCT circuit 156 is output as it is. The output of adder 158 is stored in frame memory 160.

The frame memory 160 has a storage capacity of at least two frames and stores the output pixel value (that is, the locally decoded value) of the adder 158. The motion vector detection circuit 162 compares the pixel data of the current frame from the input terminal 140 with the pixel data of the previous frame stored in the frame memory 160, and detects the motion of the image. Specifically, the pixel data of the previous frame (reference frame) is set in the frame memory 60 as a motion vector search window in the vicinity of the macroblock being processed in the current frame.
Then, the block matching calculation is performed to detect the motion vector.

The motion compensation circuit 164 moves the pixel data of the previous frame from the frame memory 160 in the screen direction so as to cancel the motion according to the motion vector detected by the motion vector detection circuit 162. Filter 16
6 performs filter processing on the pixel data of the previous frame that has been motion-compensated by the motion compensation circuit 164 to reduce discontinuity at the block boundary, and supplies the processed data to the subtractor 168 and the adder 158 as a predicted value. .

The image encoding control circuit 170 controls the overall image encoding according to the control signal from the system control circuit 34 and the data storage amount of the transmission buffer 150. Specifically, the quantizing circuit 146 is adaptively adapted to the change in the input image, the scene change, and the image quality setting of the communicator based on the data storage amount of the transmission buffer 150 so that the transmission buffer 150 does not overflow. The quantization step size, the mode selection in the coding mode selection circuit 142, the significant block determination, and the frame skipping (frame skip) are controlled.

During normal image communication, the image coding control circuit 170 controls the coding in the circuit shown in FIG. 7 as follows. That is, in an image with little movement or change, the current frame and the previous frame are very similar to each other, so inter-frame predictive coding (INTER) that encodes the difference from the previous frame.
The method or mode is used to effectively reduce temporal redundancy. Furthermore, by adding motion compensation (MC), efficient coding can be realized even for a moving image. Since the correlation between frames is small when an image has a significantly large motion or when a scene is changed, encoding within the same frame is effective. That is, the intra-frame coding (INTR) is performed in which coding is performed as it is without taking the difference from the previous frame.
A) Method or mode is used. Image coding control circuit 17
0 is a macro of these three coding modes, that is, the motion-compensated inter-frame predictive coding mode, the inter-frame predictive coding mode without motion compensation (that is, the simple inter-frame predictive coding mode), and the intra-frame coding mode. Adaptively switch in block units.

Regarding the quantization characteristic in the quantization circuit 146, the smaller the quantization step size, the higher the image quality, but the more significant data, the more the number of transmission bits. On the other hand, if the quantization step size is increased, the amount of transmitted data decreases but the image quality deteriorates. CCITT Recommendation H. According to H.261, there is an upper limit to the number of bits generated per frame, and there is also a limit to the improvement of quantization characteristics for high definition image quality. Higher image quality means an increase in the number of transmission bits, which leads to a reduction in the frame rate. That is, the image quality (spatial resolution) and the tracking ability (temporal resolution) of the movement are contradictory, and pursuing high image quality inevitably deteriorates the tracking ability of the motion. Therefore, the image coding control circuit 17
A value of 0 constantly monitors the amount of data stored in the transmission buffer 150, and appropriately and efficiently sets the quantization step size. The circuit 170 also performs frame skipping (frame skip) processing according to the number of encoded bits generated, and adjusts the frame rate.

Next, the storage-corresponding encoding control by the storage-corresponding control circuit 172 will be described. The storage-corresponding control circuit 172 receives a storage execution notification signal of the output image data of the image coding circuit 22a from the self-coded image storage monitoring circuit 44 of the image storage device 23, and also receives from the reception control information recognition circuit 42. , The notification signal that the partner terminal (image receiving terminal) is in the received image storage execution mode is input. The storage-corresponding control circuit 172 forces the image coding control circuit 170 to respond to any one of these notification signals to force intra-frame coding frames (update screen) at predetermined intervals.
Instruct to insert. That is, for example, every few coded frames, the comparison result of the coding mode selection circuit 142 is invalidated, and the intraframe coding (INTRA) mode is selected for all blocks in the frame. Here, it is assumed that the forced processing frame interval is set in the system control circuit 38 in advance.

As in the case of the embodiment using the image encoding circuit 22a shown in FIG. 2, the image accumulation in the image accumulation device 23 is performed by the image accumulation in the image accumulation device 23 of the image transmitting side terminal. There is image storage in the image storage device 23 of the receiving side terminal.

First, the operation for storing the latter image will be described. When the image receiving side terminal stores the received image (for example, in the message recording mode), the image receiving side terminal transmits the control information indicating the storage of the received image to the image transmitting side terminal prior to the image communication. That is, when the received image storage monitoring circuit 64 of the image storage device 23 in the image receiving side terminal recognizes that it is in the received image storage mode, it notifies the storage notification control unit 44 of the start of storage of the received image data. . Storage notification control unit 4
In response to this, 4 transmits control information indicating execution of accumulation of received images to the image transmission side terminal.

In the image transmission side terminal, the reception control information recognition section 42 recognizes the control information indicating the accumulation execution from the image reception side terminal, and the image is stored in the accumulation correspondence control circuit 172 of the image encoding circuit 22a shown in FIG. Notify that the receiving terminal has accumulated the received images. In response to this, the storage correspondence control circuit 172 of the image coding circuit 22a responds to this, and the image coding control circuit 17
0 is used to instruct the encoding control corresponding to storage. That is, the image coding control circuit 170 controls each part of the circuit shown in FIG. 7 so as to forcibly perform the intraframe coding processing on all the blocks in the frame at every predetermined frame period. The image to be transmitted is a frame in which all blocks are intra-coded every predetermined frame period.
Is transmitted to the partner terminal through the separation / multiplexing circuit 36, the line interface 34, and the communication line, and is stored in the image storage device 23 of the partner terminal or a storage device similar thereto. As a result, the coded image information received and accumulated by the receiving terminal is the one in which all blocks are intra-frame coded at a predetermined cycle, and a correctly reproduced received image can be obtained within a relatively short time. Playback can be displayed.

Next, a case will be described in which the image transmission side terminal stores the image data that has been compression-encoded by its own image encoding circuit 22a in its own image storage device 23.

The self-encoded image storage monitoring circuit 66 of the image storage device 23 stores a self-encoded image in the self-device prior to the start of image encoding from the control signal supplied from the system control circuit 38 to the image storage control circuit 68. Is detected and the image encoding circuit 22a (the storage correspondence control circuit 172 thereof) is notified that the image storage device 23 stores the encoded image. In response to this, the storage-corresponding control circuit 172 of the image coding circuit 22a instructs the image coding control circuit 170 to execute the storage-corresponding coding control. In response to this, the image coding control circuit 170 controls each unit of the circuit shown in FIG. 7 so as to forcibly perform the intraframe coding processing on all the blocks in the frame at every predetermined frame period. The image information encoded in this way is stored in the image memory 60 of the image storage device 23.

FIG. 8 is a flowchart showing the coding control by the image coding control circuit 170.

By the above operation, when the compression coded image is stored in the storage means, the execution of the storage is notified to the image coding circuit prior to the start of the storage, so that the intra-frame coded image is generated at a predetermined timing. It will be forcibly inserted. As a result, the intermediate reproduction becomes easy, depending on the insertion period of the intra-frame coded image and the like. It is also possible to discard a part of the stored image and store only the frames for a necessary period, or store only the intra-frame coded frames.

In the above embodiment, the ITU-T
Recommendation H. Although the compression encoding method according to H.261 has been described, it goes without saying that it can be applied to other compression encoding methods.

In the above embodiment, the timing of the intra-frame coding process is described as a fixed interval and a fixed unit which are determined in advance. However, the forced processing timing (control interval and control unit) is included in the control information for notifying the image accumulation. Etc.) is added and transmitted, the receiving terminal (terminal that actually stores the image) can set and control the compulsory intra-frame encoding processing timing.

In the above embodiment, the case where the unit for forcibly performing the intra-frame encoding processing is one frame has been described, but the present invention is not limited to this, and a plurality of frames or one obtained by dividing one frame into an area is compulsory. It may be a processing unit. Further, the control interval is not limited to the above embodiment.

It is obvious that some or all of the elements of the above-described embodiments can be realized by software.

[0091]

As can be easily understood from the above description, according to the present invention, when the image information is stored on the image receiving side or the image transmitting side, the compression coding by the storage coding method is selected. Therefore, it becomes possible for the user to reproduce the image stored on the transmitting side or the receiving side by special reproduction such as random access, high-speed reproduction, reverse reproduction, etc., without being particularly aware of the change of the encoding method.

Further, even if only one encoding means of one encoding method is used, the intraframe encoding processing is forcibly inserted at a predetermined cycle during image accumulation, so that encoding suitable for accumulation is possible. Therefore, not only at the start of reproduction, but also at the start of intermediate reproduction, a correct image with reduced image disturbance can be configured and reproduced and displayed. Further, not only the intermediate reproduction of the accumulated image, but also a function of discarding a part of the accumulated image and accumulating only frames for a required period, accumulating only intra-frame encoded frames, and the like can be easily realized.

[Brief description of drawings]

FIG. 1 is an overall schematic block diagram of a moving image communication terminal which is an embodiment of the present invention.

FIG. 2 is a schematic block diagram of an image encoding circuit 22a.

FIG. 3 is an example of a coded frame in MPEG1.

FIG. 4 is a schematic block diagram of an image storage device 23.

FIG. 5 is a flowchart for selecting an encoding method for storage.

FIG. 6 is an explanatory diagram of motion vectors.

FIG. 7 is a schematic configuration block diagram of another configuration of the image encoding circuit 22a.

8 is a flowchart of encoding control in the image encoding circuit 22a shown in FIG.

[Explanation of symbols]

 10: Camera 12: Document camera 14: Image input interface 16: Monitor 18: Image output interface 20: Selective combining circuit 22: Image encoding / decoding circuit 22a: Image encoding circuit 22b: Image decoding circuit 24: Handset 26: Microphone 28: Speaker 30: Voice input / output interface 32: Voice coding / decoding circuit 32a: Voice coding circuit 32b: Voice / voice decoding circuit 34: Line interface 36: Separation / multiplexing circuit 38: System control circuit 40: Operating device 42: reception control information recognition unit 44: accumulation notification control unit 50: communication image encoding circuit 52: accumulation image encoding circuit 54: accumulation correspondence control circuit 56: image encoding control circuit 60: image memory 62: input / output Selection circuit 64: Received image accumulation monitoring circuit 66: Self-coded image accumulation Visual circuit 68: Image storage control circuit 140: Input terminal 142: Coding mode selection circuit 144: DCT circuit 146: Quantization circuit 148: Variable length coding circuit 150: Transmission buffer 152: Output terminal 154: Inverse quantization circuit 156 : Inverse DCT circuit 158: Adder 160: Frame memory for motion compensation 162: Motion vector detection circuit 164: Motion compensation circuit 166: Low pass filter 168: Subtractor 170: Image coding control circuit 172: Storage correspondence control circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location H04N 5/92 7/30 H04N 7/133 Z

Claims (6)

[Claims] 1. An encoding unit that compresses and encodes input image information by one of a communication encoding system and a storage encoding system, and transmits image data encoded by the encoding unit. Transmitting means, receiving means for receiving control information from the terminal on the side that receives the encoded image data by the transmitting means, and storing in the encoding means in response to the control information received by the receiving means An image communication terminal device, comprising: an encoding control unit that gives an instruction to select an encoding method. 2. An encoding means for compressing and encoding the inputted image information by one of a communication encoding method and a storage encoding method, and accumulating image data encoded by the encoding means. An image, comprising: storage means; and encoding control means for instructing the encoding means to select an intraframe encoding method at a predetermined interval when the encoded image data is accumulated by the accumulation means. Communication terminal equipment. 3. Receiving means for receiving encoded image information, accumulating means for accumulating encoded image data received by the receiving means, and prior to accumulating the encoded image data by the accumulating means. An image communication terminal device, comprising: control information transmitting means for transmitting control information for notifying storage of a received image to a transmission side terminal of received encoded image data. 4. An encoding means for compressing and encoding the inputted image information by selectively using an intraframe encoding method and an interframe encoding method for each predetermined encoding unit, and the encoding means. Transmitting means for transmitting encoded image data,
In response to the first control means for receiving first control information from the receiving side terminal of the encoded image data by the transmission means, and the first control information received by the first reception means, An image communication terminal device, comprising: an encoding control unit for instructing the encoding unit to forcibly select an intraframe encoding system at every predetermined timing. 5. A second receiving means for receiving second control information from the receiving side terminal of the encoded image data by the transmitting means, and the second control received by the second receiving means. Timing control means for responsive to the information to determine the forced intraframe coding timing,
The encoding control means, in response to the first control information from the receiving means, instructs the encoding means to forcibly select the intraframe encoding method at the timing determined by the timing control means. Item 4. The image communication terminal device according to item 4. 6. Encoding means for compressing and encoding the input moving image information by selectively using an intraframe encoding method and an interframe encoding method for each predetermined encoding unit, and the encoding means. Storage means for storing the encoded image data by the storage means, and when storing the encoded image data by the storage means, instruct the encoding means to forcibly select the intraframe encoding method at every predetermined timing. An image communication terminal device, comprising: an encoding control unit.