JPH0935419A - Dynamic image and speech compression unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an efficient forced interruption processing by repeating dynamic image compression, speech compression and system processing in parallel with a short cycle. SOLUTION: The dynamic image and speech compression means 5 comprises means 11 for selecting a dynamic image compression means 12, a speech compression means 13 or a system processing means 14. At first, the dynamic image and speech compression means 5 calls the dynamic image compression means 12 and opens an original compression file to start compression of dynamic image. When processing of one compression unit is finished, the dynamic image and speech compression means 5 calls the speech compression means 13 and reads out a speech data for a reproduction time corresponding to one compression unit being processed from the original compression file before starting compression. Subsequently, the dynamic image and speech compression means 5 calls the system processing means 14 and packeting the compressed data. The control is returned back to a selection means 11 so that the processing can be interrupted at any time during the processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture / sound compression apparatus for compressing and coding a moving picture and a sound, packetizing the both and accumulating them in a file.

[0002]

2. Description of the Related Art In recent years, due to the development of information processing technology, digital processing of moving images, voices, etc. is performed, but the digital processing has a problem that the processed data becomes huge.

For example, in the case of audio data, the sampling frequency is 48 kHz, and in the case of a 16-bit stereo signal, 1
The amount of data for one minute is 11.52 Mbytes. The amount of moving image data is further huge, and for a video signal, one pixel is 24 bits per minute, 352 × 240 pixels per frame, and 30 frames per second is about 435 Mbytes.
Therefore, there are many problems in recording or transmitting such a huge amount of information as it is from the viewpoint of the capacity of the storage medium, the processing speed, and the like. For this reason, a band compression technique has been developed and put into practical use in which the degree of redundancy is reduced by paying attention to the correlation and human sensory characteristics of the signal, and the amount of information is greatly reduced without reducing sound quality or image quality.

[0004] Band compression methods include methods such as predictive coding and transform coding. With these compression methods, data can be compressed by a fraction to several hundredths. In general, data is stored and stored in a compression-encoded format.

Therefore, there is a method described in Japanese Patent Application Laid-Open No. 61-131944 as a method of compressing moving image sound.

This captures a moving image and sound at the same time, 1
Image and audio are compressed in frame units, the encoded image and audio are combined, packetized and output to a file. This is completed with audio data and image data in 1 frame units. It is effective only with the compression method used. However, MPEG (Moving Pict), which is a widely used compression method of moving image and sound at present, is used.
The ures Expert Group) uses different encoding methods for moving images and audio, and cannot use the above method because it is a compression method that takes correlation between frames.

Another method for compressing moving image / sound is disclosed in Japanese Patent Laid-Open No. 4-285769.

[0008] This is a method in which a moving image is once captured as a compressed code, further, a voice is captured as a compressed code, and the captured audio data and moving image data are connected in frame units. In this method, MPE
The G system can also be encoded. However, if a forced termination request is issued by the operator during the compression process, for example, if a forced termination request is made while capturing an image, the audio up to the position at which the request is made is compression-encoded and captured. A normal compressed file cannot be created unless the process of connecting the image and audio up to the position where the end request is generated is performed.

[0009]

Therefore, if a request for forced termination by the operator occurs during the compression process,
For example, when a forced termination request is generated while capturing an image, the audio up to the position where the request is generated is compression-encoded and captured, and the process of connecting the image and audio up to the position where the forced termination request is generated is performed. If it is not, a normal compressed file cannot be created, which causes a problem that it takes a considerable time from when a forced termination request is issued until the actual compression ends.

In view of the above, the present invention terminates the compression process in a short time even if the operator issues a forced termination request during the compression process, and compresses the compressed data up to the position where the forced termination request is issued. An object of the present invention is to provide a moving image / sound compression device that can output a normal moving image / sound compressed file. Further, according to the present invention, a plurality of applications other than the moving picture compression processing can be executed in parallel, and further, the moving picture sound compression processing can be efficiently performed by controlling the application switching interval according to the output state of the display. An object of the present invention is to provide a video / audio compression device.

[0011]

SUMMARY OF THE INVENTION The present invention provides the above object,
Device control means for controlling the operation of a plurality of applications, applications controlled by the device control means, input means for inputting instruction data from a keyboard, mouse, etc., compression processing state output means for a display, etc. The video and audio compression means comprises an I / F processing means with the device control means, and a processing selection means for deciding which of the video compression processing, the audio compression processing and the system processing is to be performed. A moving image compression processing unit that performs compression encoding of a moving image, an audio compression processing unit that performs compression encoding of audio, a system processing unit that combines the compressed and encoded moving image data and audio data, This is achieved by a moving picture audio compression device having an original file device and a compressed file device.

According to the present invention configured as described above, even when the operator makes a request for forced termination in the middle,
The compression process is completed in a short time, and the compressed data up to the position where the forced termination is issued can be output as a compressed file of normal moving image audio. In addition, a plurality of applications other than the moving image / audio compression process can be executed in parallel, and the moving image / sound compression process can be efficiently performed by controlling the application switching interval according to the output state of the display. Become.

[0013]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a moving picture audio compression apparatus according to the present invention, which is in the case of conforming to MPEG.

The moving picture audio compression apparatus of the present embodiment comprises a device control means 1 for controlling the operation of a plurality of applications,
Application 2 controlled by device control means 1
And a moving image / sound compression unit 5, an input unit 3 for inputting instruction data from a keyboard, a mouse, etc., and an output unit 4 for outputting a compressed state to a display or the like. The moving image / sound compression unit 5 is a device control unit. I / F processing means 10 for interfacing with, a processing selection means 11 for deciding which processing of moving image compression processing, audio compression processing, and system processing is to be performed, and a moving image for compression coding of moving images. A compression processing unit 12, a voice compression processing unit 13 that performs compression encoding of voice, a system processing unit 14 that combines compression-coded moving image data and voice data, a compression source file device 15, and after compression. And a file device 16.

FIG. 2 is a diagram showing the structure of moving image data which is compression coded in conformity with the MPEG, and shows the structure of output data from the moving image compression processing means 12. The upper layer is the video sequence, and the sequence from the first sequence header to the last sequence and end code is the video sequence.
Sequence, one or more sequence headers and G
It is composed of OP (Group of Picture). The GOP is composed of one or more pictures as shown in the lower layer, and one picture represents one image.

There are I picture types that can be expanded only by that frame, P pictures that expand by referring to the previous frame, and B pictures that expand by referring to the preceding and following frames. One GOP has one or more I pictures and 0 or more P pictures and B pictures.

In the case of MPEG, the reproduction order of pictures and the compression code order are exchanged, and I pictures and P pictures following B pictures are coded before B pictures.

Therefore, according to the present invention, the I picture or P picture to the next I picture or P picture are processed as one compression unit in the reproduction order.

For example, the reproduction order is I1, B2, B3, P4.
In the case of B5, B6, P7 ... (I, P, B represent the picture type, and the following numbers represent the playback order), the first I
1, B2, B3, and P4 are one compression unit, and the following P4, B5, B6, and P7 are one compression unit. Here, P2 is included in the two compression units as B2 · B.
This is because the data of the frames I1 and P4 are required for the compression of No. 3, and the data of the frames of P4 and P7 are required for the compression of B5 and B6. However, it is not necessary to perform the compression process twice for the P4 picture, and if the P4 compression is performed when the first compression unit is performed, it is not necessary to perform the P4 compression in the next compression unit. Also, the compression unit of the present invention is one compression unit from the I picture or P picture to the next I picture or P picture.
The unit of GOP may be one compression unit, and in this case, one compression unit completes the image compression code amount control (compression control for matching the compressed data amount with the target data amount). be able to.

FIG. 3 is a diagram showing the structure of audio data that has been compression-encoded in accordance with the MPEG, and shows the structure of output data from the audio compression processing means 13. The upper layer is an audio sequence, and the audio sequence is composed of a plurality of AAUs (audio decoding units). As shown in the lower layer, the AAU is composed of an AAU header containing information such as a sync word, a bit rate, a sampling frequency and compressed audio data.

FIG. 4 is a diagram showing the structure of moving image / sound data compressed and encoded in accordance with MPEG. The output data of the moving image compression processing unit 12 and the output data of the audio compression processing unit 13 in the system processing unit 14 are shown. Is input, packetized and packed, and then output, and the compressed file device 1
6 shows the data structure to be stored. The video / audio data is composed of a plurality of packs, and the pack is composed of a pack header and one or more packets to show an intermediate layer, and a packet includes a video packet and an audio packet.
There is a packet. A video packet consists of a packet header and video data as shown in the lower layer,
A series of video sequences shown in FIG. 3 is formed by taking out only the video data in the video packet and continuing. The audio packet is composed of a packet header and audio data. When only the audio data in the audio packet is taken out and continuously taken out, a series of audio sequences shown in FIG. 4 is formed.

FIG. 5 is a diagram showing the structure of the moving picture compression processing means 12, which is composed of a moving picture compression processing control means 601 for controlling the whole moving picture compression processing and a compression source file device.
An original image reading unit 602 that reads the original image data for each compression unit, and a color signal conversion unit 603 that converts the original image into a color signal format (YCrCb format) that can be used in MPEG.
And a motion search unit 604 that searches the motion of the image of the previous / rear frame and the current frame for each area of the block (8 pixel × 8 pixel space in MPEG), and the motion of the image searched by the motion search unit / Motion prediction unit 605 that calculates the difference between the areas of the blocks of the subsequent frame and the current frame, DCT unit 606 that performs discrete cosine transform, quantization unit 607 that performs quantization, and VLC that performs high-efficiency variable-length coding.
Means 608, a compression code buffer 609 for storing the compressed image data after VLC, an inverse quantization means 610 for performing an inverse quantization, an IDCT means 611 for performing an inverse discrete cosine transform, a difference value and a front / back The motion compensation unit 612 calculates the new reference frame by adding the pixel values of the block area of the frame, and the reference frame unit 613 that stores the images of the previous / rear frame to be referenced.

For moving image compression, as shown in FIG. 6, the original image reading means 602 reads the original image data from the compression source file device for each compression unit, and the color signal converting means 603 outputs Y.
It is converted to CrCb data and the motion search means 604 forwards /
The compression of one frame is started after searching the motion of the image of the subsequent frame and the image of the current frame for each block area. When the I picture is compressed, the DCT means 606 performs discrete cosine transform on the pixel values of the block area of the current frame, and the quantizing means 607 quantizes the pixel values.
Highly efficient compression into a variable length Huffman code according to 08, and the compressed data is sequentially stored in the compression code buffer 609.

Next, in order to decode the compressed image as a reference frame, the quantized data is dequantized by the inverse quantization means 61.
Inverse quantization by 0, inverse discrete cosine transform by IDCT means 611 to calculate pixel values, and reference frame unit 61
Store in 3. When compressing a P-picture or a B-picture, the motion prediction means 605 stores the values of the pixels in the block area of the current frame and the reference frame unit 613 referred to by the motion searched by the motion search means 604. The difference is calculated by the pixel value of the block area of the previous frame or the previous / next frame, the difference value is discrete cosine transformed by the DCT unit 606, quantized by the quantization unit 607, and the variable length is High-efficiency compression is performed on the Huffman code, and the compressed data is sequentially stored in the compression code buffer 609.

Next, when the P picture is compressed, in order to combine the compressed image as a reference frame, the quantized data is inversely quantized by the inverse quantization means 610 and the IDCT is performed.
The means 611 performs the inverse discrete cosine transform to calculate the difference value, and the motion compensating means 612 adds the difference value to the pixel value of the area of the block of the previous frame stored in the reference frame unit 613 referred to by the motion predicting means. And stores it in the reference frame unit 613.

The moving image compression processing control means 601 controls the entire series of compression processing described above, and controls to suspend and restart the compression processing at a fixed time.

FIG. 6 is a diagram showing the structure of the audio compression processing means 13, which is an audio compression processing control means 701 for controlling the entire audio compression processing, and an original sound reading means 702 for reading the original sound data from the compression source file device. , Original sound data cut-out means 703 for cutting out original sound data of 1 AAU from the read original sound data, and 32 frequency band mapping processing means 704 for performing frequency band mapping processing in units of 1 AAU.
A psychological auditory processing unit 705 for performing psychological auditory processing, a quantization coding unit 706 for performing linear quantization and coding,
It is composed of a frame forming means 707 which adds compressed data to the encoded data and creates compressed data for 1 AAU, and a compression code buffer 708 which stores the compressed data created by the frame forming means.

For audio compression, as shown in FIG. 6, the original sound reading means 702 reads the original sound data for the same time as the time required for reproducing the moving image compressed by the moving image compressing means 702, and extracts the original sound data 703. Data of 1 AAU (1152 samples in the case of MPEG audio layer 2) is cut out from the original sound data read in step 1 and the following processing is performed in units of 1 AAU.

In the 32 frequency band mapping processing means 704, the input signal is divided into 32 band sub-band signals by the sub-band analysis filter, and the scale band for each sub-band signal is scaled.
The factor is calculated, the dynamic range is adjusted, the psychoacoustic processing means 705 performs a fast Fourier transform on the input signal, and the result is used to calculate the psychoacoustic masking.
Calculate the bit allocation for each subband. The quantization coding means 706 quantizes and codes in accordance with the determined bit allocation, and the frame forming means 707 adds a header and auxiliary information to the quantized and coded subband signal, shapes it into a bit stream, and compresses the compression buffer 70.
Store in 8.

The audio compression processing control means 701 controls the entire series of compression processing described above, and performs control to suspend and restart the compression processing at a fixed time or at intervals of 1 AAU processing.

FIG. 8 is a diagram showing the structure of the system processing means 14. The moving image data buffer 801 for storing the compressed data received from the moving image compression processing means 12 and the compressed data received from the audio compression processing means 13. , A multiplex means 804 for packetizing and packing moving image data and sound data, a time code counter 803 for adding time data to the pack and packet, and packed moving image sound And a moving image / sound compression code buffer 805 for storing the above data.

In the system processing, as shown in FIG. 6, the moving picture data is taken from the moving picture compression processing means 12 and stored in the moving picture data buffer 801, and the sound data is taken from the sound compression processing means 13 into the sound data buffer 802. The data is stored, the packet size data is cut out from the moving image data by the multiplexing means 804, the packet header and the pack header are added, and the time data at the time of reproduction is taken from the time code counter 803 to be the pack header and the packet header. Add. Also, data of the packet size is cut out from the audio data, a packet header and a pack header are added, and time data at the time of reproduction is taken from the time code counter 803 and added to the pack header and the packet header. The created packs are sequentially stored in the video / audio compression code buffer 805.

Next, the processing of the moving picture audio compression apparatus will be described with reference to FIG.

First, when the operator gives an instruction to start the compression through the input means 3, the apparatus control means 1 calls the moving image / audio compression means 5.

When the video / audio compression means 5 is called, I
After passing through the / F processing means 10, control is transferred to the processing selection means 11. The processing selection means 11 first calls the moving picture compression processing means 12. The moving image compression processing means 12 opens the compression source file designated by the operator, reads the original image data from the compression source file device 15 for each several frames (one compression unit), and starts the compression work of the moving image.

One compression unit of an image is the GOP unit or the picture range from the I picture or P picture to the next I picture or P picture described above.

However, in this system, if the processing is continued continuously for a long time, the input processing of other applications, the keyboard, the mouse, etc., and the display processing on the display cannot be performed. Therefore, the processing can be performed in the shortest possible time. Need to return to. Therefore, the moving image compression processing means 1
In No. 2, the process is temporarily interrupted during the compression of one compression unit, and when the control is next transferred to the moving image / sound compression unit 5, the processing selection unit 11 is notified to continue the moving image compression process.

The processing selection means 11 stores the next processing to be performed and returns control to the I / F processing means 10. The I / F processing means 10 acquires the current screen display state from the device control means 1. If the screen display is focused on the screen of the moving image / sound compression means, the processing is returned to the device control means 1 and the processing selection means 11 is moved to immediately continue the next processing. If the screen display is focused on another application, the process is returned to the device control means 1 so that another application can be processed.
When the processing of other applications is completed, the device control means 1 calls the moving image / sound compression means 5 again.

When the moving image / sound compression means 5 is called again, the control is transferred to the processing selection means 11 through the I / F processing means 10.

The processing selection means 11 calls the moving picture compression processing means 12 in order to continue the previous processing. The above processing is continuously performed, and when one compression unit is completed, the processing selection unit 11 is notified to perform the next audio processing.
The processing is returned to the / F processing means 10.

Next, when the control shifts to the processing selecting means, the voice compression processing means 13 is called. The audio processing unit 13 reads the audio data for the reproduction time of the number of frames (one compression unit) processed by the moving image compression processing unit 12 from the compression source file device, and starts the audio compression process. Similarly to the case of the moving image compression processing, in the case of the audio compression processing, the processing selection means 11 is returned to the processing selection means 11 in a fine compression unit, and the processing selection means 11 is notified that the system processing will be performed next at the last compression. The process ends.

At this time, it is efficient to perform the audio compression in units of 1 AAU, and the maximum AAU is processed within the time of one compression unit of the moving image compression processing. Also, the remaining 1A
Audio data that does not reach AU is compressed the next time control is transferred to the audio compression processing means.

Next, when the processing shifts to the processing selection means 11, the system processing means 14 is called.

The system processing means 14 receives the compression code data for one compression unit compressed by the moving picture compression processing means 12 and the sound compression processing means 13, and packetizes and packs it so that the moving picture and the sound can be synchronized. Write to file after compression.

In this way, the processing of the moving image, sound and system is performed for each compression unit.

The procedure of the processing described above is shown in FIGS. 8 and 9.

In the moving image / sound compressing apparatus of the present invention, by changing the interval for returning the control to the device controlling section 1 depending on whether the screen of the moving image / sound compressing section is in focus or not, the moving image / sound compressing section is displayed on the screen. When the focus is in focus, the compression work is performed at high speed, and when the focus is out of focus, the process is frequently returned to the device control means 1 so that other applications can be smoothly executed.

FIG. 8 shows a procedure of processing when the display of the moving image compression processing means is in focus on the screen, and the interval for transferring the processing from the I / F processing means to the device control means is long. Therefore, it is possible to delay the processing of other applications and perform the moving image compression processing at high speed. Further, FIG. 9 shows a processing procedure when the display of the display is focused on the screen of another application. By frequently transferring the processing from the I / F processing means to the device control means, the other application can be smoothly operated. Can be operated.

FIG. 10 shows a processing procedure in the case where the operator issues a forced termination request during the compression processing. Even if the operator gives an instruction to interrupt the processing during the compression processing, 1 When the processing for the compression unit is completed, all the processing is controlled so as to be completed, and it is possible to leave it as a compression code file of normal moving image audio until the processing is completed.

[0051]

As described above, according to the present invention, even if the operator makes a request for forced termination in the middle, the compression process is terminated in a short time, and the compressed data up to the position where the forced termination is issued becomes normal. It is possible to output as a clear MPEG file. Further, a plurality of applications other than the moving image compression processing can be executed in parallel, and moreover, by controlling the application switching interval according to the output state of the display, it becomes possible to efficiently perform the moving image sound compression processing. .

[Brief description of drawings]

FIG. 1 is a functional block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of MPEG moving image data.

FIG. 3 is a diagram showing a configuration of MPEG audio data.

FIG. 4 is a diagram showing a structure of moving image / audio data of MPEG.

FIG. 5 is a diagram showing a structure of moving image compression processing means.

FIG. 6 is a diagram showing a structure of audio compression processing means.

FIG. 7 is a diagram showing a structure of system processing means.

FIG. 8 is a diagram showing a processing procedure when the focus is on moving image audio compression.

FIG. 9 is a diagram illustrating a processing procedure when the focus is on another application.

FIG. 10 is a diagram showing a processing procedure when a forced termination request is generated.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Device control means 2 Other application 3 Input means 4 Output means 5 Moving image audio compression means 10 I / F processing means 11 Processing selection means 12 Moving image compression processing means 13 Audio compression processing means 14 System processing means 15 Compression source file device 16 File device after compression 601 Moving image compression processing control means 602 Original image reading means 603 Color signal conversion means 604 Motion retrieval means 605 Motion prediction means 606 DCT means 607 Quantization means 608 VLC means 609 Compression code buffer 610 Inverse quantization means 611 IDCT Means 612 Motion compensation means 613 Reference frame section 701 Audio compression processing control means 702 Original sound reading means 703 Original sound data cutting means 704 32 Frequency band mapping processing means 705 Psychological auditory processing means 706 Quantization coding means 70 Frame forming means 708 compression code buffer 801 video data buffer 802 the sound data buffer 803 the time code counter 804 multiplex means 805 video and audio compression code buffer

Continuation of front page (51) Int.Cl. ⁶ Identification number Office reference number FI Technical display area H04N 7/24 H04N 7/13 Z

Claims (6)

[Claims] 1. A moving picture audio compression apparatus comprising a moving picture compression processing means, a sound compression processing means, and a system processing means, wherein the moving picture compression processing means compresses a moving picture in units of several frames, The audio compression processing means compresses the audio in units of the same time as the compression unit of the moving image and the time taken for reproduction, and the system processing means stores the moving image data compressed and encoded by the moving image compression processing means. A moving image audio compression apparatus, characterized in that the audio data compressed by the audio compression processing means is combined and sequentially output as a compressed file. 2. The moving picture audio compression apparatus according to claim 1, further comprising a device control means, interrupting the compression in fine units during the compression of the moving picture, and returning the processing to the device control means. A moving image / audio compression apparatus characterized in that the CPU is handed over to processing other than compression. 3. The moving picture voice compression apparatus according to claim 1, further comprising a device control means, interrupting the compression in small units during the compression of the sound, and returning the processing to the device control means. A moving image and sound compression apparatus characterized in that the CPU is surrendered to processing other than the above. 4. The moving picture voice compression apparatus according to claim 1, further comprising a device control means, interrupting the compression in fine units during the system processing, and returning the processing to the device control means, except for the moving picture sound compression. A moving image audio compression apparatus characterized in that the CPU is yielded to other processing. 5. The moving image audio compression apparatus according to claim 2, further comprising means for acquiring the display state of the display from the apparatus control means, and the processing is returned to the apparatus control means according to the display state of the display. A moving image audio compression apparatus characterized by controlling the interval. 6. The moving image and sound compression apparatus according to claim 1, wherein data processed by a forced termination request from an operator is input as normal compressed data in response to a forced termination request from the operator. A moving image / sound compression apparatus having a function of outputting.