JP2007250048A - Image processing apparatus, image processing method, image processing program, and program storage medium - Google Patents

Abstract

An object of the present invention is to facilitate an image search operation and an editing operation of a desired scene from a series of videos.
The present invention provides a plurality of video display units W57 _{1 to} W57 _{n in} which a time difference is provided in the display order for a plurality of display video data and a moving image is displayed as a three-dimensional spiral video display unit group 58. Since it will be presented, even in a single screen display, it is possible to present a larger number of display video data because of the 3D spiral structure, and in a moment instant by a moving image in a 3D spiral display form. Since it is possible to visually check the display video data of different scenes for each of the plurality of video display units, the operator can recognize the time-series flow intuitively from among a series of videos based on the display video data. It is possible to easily search the video portion of the desired scene, thus facilitating the operator's image search operation and editing operation.
[Selection] Figure 2

Description

  The present invention relates to an image processing apparatus, an image processing method, an image processing program, and a program storage medium, and is suitable for application to, for example, a non-linear editing apparatus.

  Conventionally, in a content production site such as a television broadcasting station, a new clip is created by cutting a part from video / audio material (hereinafter referred to as a clip) obtained by shooting with a video camera. One content is created by connecting a plurality of similarly created clips (for example, see Non-Patent Document 1).

  FIG. 121 shows an example of a GUI (Graphical User Interface) screen (hereinafter referred to as an editing screen) displayed on the display of an editing apparatus used for such editing work. As is clear from FIG. 121, the editing screen 1 includes a clip list display unit 2, a monitor unit 3, a storyboard unit 4, a timeline unit 5, and effect information display units 6A to 6C.

  In this case, the clip list display unit 2 selects a desired bin or file from various bins and files registered in the editing device, and displays a list of clips stored in the bin or file. It can be displayed.

  Then, the operator selects a desired clip from the clips displayed in the list on the clip list display unit 2 and drags and drops the clip on the monitor unit 3, so that the head image of the clip is displayed in the monitor unit 3. Can be displayed.

  In addition, the operator clicks the corresponding button from the button group 7 displayed at the bottom of the monitor unit 3 in this state, thereby starting the reproduction of the dragged and dropped clip and displaying the reproduced video. 3, the displayed playback video can be fast-forwarded or rewinded, and a scrub cursor 8 indicating the position of the current display image in the entire clip displayed above the button group 7 is displayed. By moving left and right by a mouse operation, an image corresponding to the position of the scrub cursor 8 at that time can be displayed on the monitor unit 3.

Thus, the operator searches for a desired frame by operating the playback button or the like in the button group 7 or the scrub cursor 8 while visually confirming the playback video displayed on the monitor unit 3, and monitors the image of the frame. When the in-point button 9 _IN or the out-point button 9 _OUT provided at the lower part of the monitor unit 3 is clicked while being displayed on the unit 3, the start point (hereinafter referred to as “in-point”) of the video / audio portion to be cut out from the clip. A point) and an end point (hereinafter, “out point”) can be designated.

  Then, with respect to a clip in which the In point and Out point are specified in this way, the video / audio part sandwiched between the In point and Out point can be pasted on the storyboard unit 4 by drag and drop as a new clip. . As a result, on the editing screen 1, the clips used for editing are arranged in advance on the storyboard unit 4 so that the operator can easily imagine the editing result. At this time, the storyboard unit 4 displays thumbnails and detailed information of representative images such as the top image of each pasted clip.

  The operator then sequentially pastes the clips pasted on the storyboard unit 4 so as to be arranged on the video track 10V of the timeline unit 5 by drag and drop. At this time, a band 12V having a length corresponding to the material length of the pasted clip is displayed on the video track 10V using the time scale 11 as an index. At this time, if there is sound in the clip, a band 12A having the same length is displayed at the same position on the time scale 11 in the audio track 10A.

  Here, when the bands 12V and 12A are displayed on the video track 10V and the audio track 10A of the timeline unit 5 as described above, the bands 12V and 12A are displayed at the time represented by the time scale 11 when the edited video / audio is output. It means that the video of the clip corresponding to is displayed or the sound is output. Therefore, it is possible to create an edit list that sequentially defines the video or audio of a clip that is displayed as edited video or output as edited audio by such an operation.

  When creating the edit list in this way, for example, when it is desired to perform video special effect processing at the time of switching from the video of the first clip to the video of the second clip, among the effect information display units 6A to 6C An icon (hereinafter referred to as an effect icon) 13 associated with a desired video special effect among various video special effects that can be executed by the editing apparatus displayed in a list in the effect list display section 6C of FIG. The transition between the first and second clips in the transition track 10T of the timeline unit 5 and the time scale 11 are pasted at the same position on the time scale 11 by drag and drop.

As a result, the video special effect processing associated with the effect icon 13 pasted on the transition track 10T as described above is performed at the connecting portion between the video of the first clip and the video of the second clip in the edited video. An instruction to execute can be input.
JP 2000-251451 A

  By the way, in the editing work using the editing screen 1 as described above, the search work for the frame to be the In point or the Out point is performed by a mouse operation called “Scrub” for moving the scrub cursor 8 to the left and right as described above. The video of the clip is reproduced at high speed to find a desired frame.

  However, a desired frame searching operation by scrubbing (hereinafter referred to as an image searching operation as appropriate) requires a lot of time unless the user is familiar with the operation. In addition, in order to improve compression efficiency, the target video / audio material is compressed in a long GOP (Long GOP) format of a so-called MPEG (Motion Picture Expert Group) method, which has a plurality of frames in one GOP (Group of Pictures). In the case of open GOP format in which compression is performed using the data of the previous and subsequent GOPs, a plurality of frames and GOPs must be decoded in order to decode the frames. In some cases, high-speed random reproduction becomes difficult. As a result, in such a case, the responsiveness of the displayed video is deteriorated, and the image search operation by scrub becomes more difficult.

  Further, the so-called cut editing operation for connecting clips as described above is conventionally performed by pasting the clip on the video of the timeline section 5 in the editing screen 1 and the rack 10V or the audio track 10A. Video confirmation before and after the out point was performed by subsequent playback and scrubbing. For this reason, in order to perform such cut editing, several steps are necessary as described above, and there is a problem that it is difficult to perform the editing easily.

  Further, in the editing work, editing is usually performed while taking video and audio into consideration, but for audio, the sound output from the speaker is relied on while visually checking the video, or the editing screen 1 The audio level waveform displayed on the audio track 10A of the timeline unit 5 (see, for example, the audio track 10A of the “audio 3” of the timeline unit 5 in FIG. 121) needs to be edited and checked. There has been a problem that editing work that links audio and voice cannot be performed easily.

  The present invention has been made in view of the above points, and intends to propose an image processing apparatus, an image processing method, an image processing program, and a program storage medium that can facilitate an image search operation and an editing operation.

In order to solve such a problem, in the image processing apparatus, the image processing method, and the image processing program of the present invention, a plurality of display video data to be displayed on a plurality of video display units on the display screen are respectively displayed from the video data. The image generation means to generate and the plurality of display video data to be displayed on the plurality of video display units are arranged and displayed in a three-dimensional spiral form on the display screen in a moving image state while providing a time difference in the display order. And an image processing means.

  Thus, a time difference is provided in the display order for a plurality of display image data, and a plurality of image display units on which moving images are displayed are presented in a three-dimensional spiral display form. Even in this case, because of the three-dimensional spiral structure, it is possible to present a larger number of display video data and to display different scene display video data for each of the plurality of video display units at a moment by moving images in a three-dimensional spiral display form. Since it can be visually confirmed, the operator can easily search for a video portion desired by the operator from a series of videos based on display video data while intuitively recognizing the time-series flow. Editing work can be facilitated.

  Furthermore, in the program storage medium of the present invention, an image generation step for generating a plurality of display video data for display on a plurality of video display units on the display screen from the video data, and a plurality of video display units The information processing apparatus executes a process including an image processing step of displaying a plurality of display video data to be displayed in a moving image state in a moving image state while providing a time difference in the display order and displaying them in a three-dimensional spiral shape on the display screen. I tried to make it.

  As a result, according to this program storage medium, a time difference is provided in the display order for a plurality of display video data, and a plurality of video display units on which moving images are displayed are presented in a three-dimensional spiral display form. Even within a single screen display, because of the three-dimensional spiral structure, it is possible to present a larger number of display video data, and for each of the plurality of video display units instantaneously by moving images in a three-dimensional spiral display form. Since the display video data of different scenes can be visually confirmed, the operator's desired video part can be easily selected from a series of videos based on the display video data while intuitively recognizing the time-series flow to the operator. Searching can facilitate the image searching and editing operations.

  According to the present invention, a time difference is provided in the display order for a plurality of display video data, and a plurality of video display units on which moving images are displayed are presented in a three-dimensional spiral display form. 3D spiral structure can present a lot more display video data, and display different scenes for each video display unit at the moment by moving image with 3D spiral display form. Since the video data can be checked visually, the operator can easily search for and edit the video portion desired by the operator from a series of videos based on the displayed video data while intuitively recognizing the time-series flow to the operator. Image processing apparatus, image processing method, image processing program, and program storage medium capable of facilitating work and thus facilitating image search work and edit work It can be realized.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) Configuration of Editing System According to this Embodiment As shown in FIG. 1, reference numeral 1 denotes an editing system according to this embodiment as a whole, which is roughly divided into an editing device 2, a storage device 22, and a plurality of video tape recorders. and a 23 ₁ ~23n like.

  In the editing system 1, a part or all of the video and audio recorded on the video tape can be taken as a clip into the large-capacity storage device 22 composed of RAID (Redundant Arrays of Independent Disks) via the editing device 2. Has been made.

The editing system 1 can also create an edit list that defines the editing content for obtaining the desired edited video and audio by joining the clips captured in the storage device 22 to a desired state, and in accordance with the created edit list. The edited video and edited audio obtained as a result can be stored in the storage device 22 or recorded on the video tape via the video tape recorders 23 _{1 to} 23n.

  In this case, in the editing apparatus 1, the GPU 4, XDR (Extreme Data Rate) -RAM 5 and the south bridge 6 are connected to the microprocessor 3, and the hard disk device 7, USB interface 8 and sound input are connected to the south bridge 6. An output codec 9 is connected. A speaker 41 is connected to the sound input / output codec 9.

Further, a mouse 38, a keyboard 39, video tape recorders 23 ₁ to 23 _n , a storage device 22, and an operation controller 37 are connected to the south bridge 6 via the PCI bus 15. A display 40 is connected to the GPU 4.

  By the way, the microprocessor 3 includes a general-purpose main CPU core 3M that executes a basic program such as an OS (Operating System), and a plurality of (in this case, eight) RISCs connected to the main CPU core 3M via the internal bus 12. (Reduced Instruction Set Computer) type signal processor (hereinafter referred to as a sub CPU core) 3SA to 3SH, a memory controller 13 for performing memory control on an XDR-RAM 5 having a capacity of, for example, 256 [MByte], and a south A multi-core configuration in which an I / O (In / Out) controller 14 for managing input / output of data to / from the bridge 6 is integrated on one chip, and enables an operating frequency of 4 [GHz], for example.

  The microprocessor 3 of the editing apparatus 1 mainly plays a role of a codec such as MPEG decoder, JPEG2000, H.264 / AVC (Advanced Video Coding), and transfers the reproduced video obtained as a result of decoding to the GPU 4. A setting such as a reproduction speed v (described later) is changed, or a physical calculation or the like is performed.

  In particular, in the microprocessor 3, the eight sub CPU cores 3SA to 3SH each play the role of a decoder, and the eight sub CPU cores 3SA to 3SH decode HD (High Definition) playback images simultaneously in parallel. Is possible.

  The main CPU core 3M performs processing and management other than those performed by the eight sub CPU cores 3SA to 3SH. The main CPU core 3M is supplied from the mouse 38, the keyboard 39, or the operation controller 37 via the south bridge 6. An instruction is received, and various processes according to the instruction are executed.

  As described above, the microprocessor 3 is configured to be able to decode the reproduced video simultaneously and in parallel by the eight sub CPU cores 3SA to 3SH, and has a large bandwidth between the microprocessor 3 and the GPU 4. For example, data can be transferred at a transfer speed of, for example, a maximum of 30 [Gbyte / sec] by the bus 10, so that a large number of high-definition reproduced images can be decoded and transferred in a short time. Yes.

  Incidentally, in this case, the eight sub CPU cores 3SA to 3SH can decode, for example, two HD playback images at the same time, that is, 16 pictures at a time from the microprocessor 3 to the GPU 4 (in this case). High-definition playback images can be transferred at high speed. However, the number of HDs that can be decoded simultaneously by the eight sub CPU cores 3SA to 3SH is not limited to this, and a larger number of reproduced videos can be decoded.

  On the other hand, the GPU 4 is responsible for functions such as coordinate transformation calculation processing and enlargement / reduction processing of the playback video, in addition to the final rendering processing related to texture embedding when the playback video displayed on the display 40 is moved. The processing load on the processor 3 is reduced.

  In practice, upon startup, the microprocessor 3 reads out necessary software stored in the hard disk device 7 based on a control program stored in the hard disk device 7 and expands it in the XDR-RAM 5. Necessary control processing is executed based on the operator's operation.

  When the mouse 38, the keyboard 39 or the operation controller 37 is operated and the display instruction of the clip capture window for capturing the video / audio recorded on the video tape into the storage device 22 is input, the microprocessor 3 responds to this. Then, the corresponding image data is read out by controlling the hard disk device 7, while the above-described clip capturing window based on this image data is displayed on the display 40 by controlling the GPU 4 together with this.

Further, in this state, when the mouse 38, the keyboard 39 or the operation controller 37 is operated and a reproduction operation command is input to the video tape recorders 23 ₁ to 23 _n , the microprocessor 3 responds to this and the video tape recorder 23 responds accordingly. by controlling the ₁ ~ 23 _n, to perform a reproduction operation of the video and audio signals recorded on the video tape.

The result from the video tape recorder ₂₃ 1 to 23 _n is the video tape recorder ₂₃ 1 to 23 video and audio signals reproduced from the loaded video tape _n is output, of which the video signal is the PCI bus 15, south bridge 6 and the microprocessor 3 are sequentially supplied to the GPU 4.

  The GPU 4 performs predetermined signal processing on the supplied video signal under the control of the microprocessor 3, and sends the resulting video signal to the display 40, so that the video based on the video signal is obtained. Is displayed at a predetermined position in the clip capture window. On the other hand, the microprocessor 3 outputs the audio signal extracted from the video / audio signal to the speaker 41 via the sound input / output codec 9 at this time, thereby outputting the audio based on the audio signal from the speaker 41.

  Thus, based on the video displayed on the display 40 and the audio output from the speaker 41, the operator can specify a desired part of the video and audio using the mouse 38, the keyboard 39 or the operation controller 37. Management information such as the time code, material length, clip ID, clip name, shooting date / time of the video / audio, creation date / time of the clip, etc. can be registered as metadata. . The management information of the clip registered as metadata is registered in the clip management information database in the hard disk device 7 under the control of the microprocessor 3.

The microprocessor 3 controls the video tape recorders 23 ₁ to 23 _n in response to the operation of the mouse 38, the keyboard 39 or the operation controller 37 and the input of the clip fetching command. As a result, the video / audio reproduction operation of the designated clip is executed.

As a result, the video tape recorders 23 ₁ to 23 _n output video / audio signals of the clips reproduced from the video tape, and the video signals sequentially pass through the PCI bus 15, the south bridge 6, the microprocessor 3, and the bus 10. Via the GPU4. In addition, the microprocessor 3 controls the GPU 4 and the storage device 22 together, and stores the video / audio signal of the clip supplied from the video tape recorders 23 ₁ to 23 _{n in} the storage device 22 via the GPU 4. .

  In this way, in the editing system 1, the video / audio of the designated clip can be reproduced from the video tape and taken into the storage device 22.

  On the other hand, the operator can thereafter display an editing screen (described later) for performing an editing operation by a predetermined operation using the mouse 38, the keyboard 39 or the operation controller 37 on the display 40, and using this editing screen. It is possible to create an edit list that defines edit contents such as which clips and how to connect which clips. In addition, the operator can check the edited video and the edited audio based on the edit list after or during the creation of the edit list.

  When the edit list is created and the mouse 38 is operated to input an edit list registration command, the microprocessor 3 saves the edit data set by all operators defined in the edit list as a file. And registered in the edit list database in the hard disk device 7.

  Further, the microprocessor 3 responds when the mouse 38, the keyboard 39, or the operation controller 37 is operated during or during the creation of the edit list and a playback instruction of the edited video / audio based on the edit list is input. Then, by controlling the storage device 22, the storage device 22 is made to read the video / audio signal of a necessary clip.

  Thus, the video signal out of the video / audio signal read from the storage device 22 is given to the GPU 4 via the PCI bus 15, the south bridge 6, the microprocessor 3 and the bus 10 in order, and then the control of the microprocessor 3 is performed. In addition, video special effect processing is performed as necessary in the GPU 4.

  Then, the GPU 4 performs predetermined signal processing on the edited video signal obtained by performing video special effect processing on the supplied video signal under the control of the microprocessor 3, and obtains the result. The received video signal is sent to the display 40.

  On the other hand, the microprocessor 3 performs an audio mixing process on the audio signal among the video and audio signals, and sends the edited audio signal obtained as a result to the speaker 41 via the sound input / output codec 9.

  As a result, the edited video is displayed at a predetermined position in the editing screen, and the edited audio is output from the speaker 41. In this way, in the editing system 1, the operator can perform an editing operation while confirming the edited video and the edited audio based on the editing list.

  Further, after the edit list is created, the microprocessor 3 operates the mouse 38, the keyboard 39, or the operation controller 37 to input an execution command, thereby controlling the storage device 22 in response to the input. A video / audio signal of a clip used for editing processing is read from the storage device 22 and sent to the GPU 4 via the PCI bus 15, the south bridge 6, and the bus 10.

At this time, under the control of the microprocessor 3, the GPU 4 performs special effect processing and audio mixing processing on the supplied video and audio signals of each clip as required in the same manner as in the playback mode described above. On the other hand, the edited video signal obtained as a result is sent to the storage device 22 or the corresponding video tape recorders 23 ₁ to 23 _n via the bus 10, the microprocessor 3, the south bridge 6 and the PCI bus 15 in order.

Thus, the storage device 22 stores the supplied edited video signal at the designated address position under the control of the microprocessor 3. The video tape recorders 23 ₁ to 23 _n record the supplied edited video signal at a designated position on the video tape under the control of the microprocessor 3.

  In this way, in the editing system 1, the video / audio of the specified clip is edited and processed in the specified state in accordance with the editing data created in advance and stored in the storage device 22 or recorded on the video tape. It has been made so that it can.

  Therefore, in the editing system 1, the microprocessor 3 and the GPU 4 of the editing apparatus 2 can execute video display with special effects on the playback video of the editing screen displayed on the display 40, and perform various expansion operations and An editing process can be executed.

  By the way, a plurality of reproduction videos decoded simultaneously in parallel by the eight sub CPU cores 3SA to 3SH in the microprocessor 3 are transferred to the GPU 4 via the bus 10, and the transfer speed at this time is, for example, The maximum is 30 [Gbyte / sec], so that even complex playback images with special effects can be displayed at high speed and smoothly.

(2) Operation procedure on the edit screen (2-1) Display method of playback video on the edit screen Here, the microprocessor 3 inputs a command to display the edit screen by operating the mouse 38, the keyboard 39 or the operation controller 37. Then, the hard disk device 7 and the GPU 4 are controlled to display the editing screen 50 as shown in FIG.

  This edit screen 50 is a clip list display unit 51 having the same functions as the clip list display unit 2, storyboard unit 4, timeline unit 5 and effect information display units 6A to 6C of the edit screen 1 described above with reference to FIG. The story board unit 52, the time line unit 53, the effect information display unit 54, and a display unit 55 having a function unique to the editing screen 50. In the case of the editing screen 50, as will be described later, the editing work is mainly performed using the display unit 55, so that the storyboard unit 52 and the timeline unit 53 are provided in an auxiliary manner.

Then, the microprocessor 3 displays a plurality of video displays suitable for displaying, for example, a clip playback video composed of a moving image having a series of flows on the display unit 55 of the editing screen 50 as shown in FIG. The parts W57 _{1 to} W57 _n are sequentially arranged in a spiral shape from the depth direction of the screen (hereinafter referred to as the back side) toward the front direction of the front side (hereinafter referred to as the front side). The spiral image display unit group 58 can be displayed.

In this case, each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 is the largest with the video display unit W57 _n positioned on the most front side of the screen as a reference video display unit, and the screen spirally from here. In the portion where the image display portion W57 on the near side overlaps with the image display portion W57 located on the front side, the image display portion on the rear side is arranged so that the distance between the image display portion W57 and the image display portion W57 located thereafter is overlapped. W57 is displayed so as to be hidden by the image display unit W57 on the front side.

However, in each image display unit W57 ₁ ~W57 _n of spiral image display area group 58, it has been made as translucent playback video is displayed respectively, hidden for example by the most front side of the image display section W57 _n It is adapted to be visually to confirm to the operator in a state where the transparent even for a portion of the rear of the video display unit W57 _m (portion indicated by a broken line).

Note that the microprocessor 3 in this case acquires display information including the display sizes of the plurality of video display units W57 _{1 to} W57 _n and the display positions on the display unit 55 (FIG. 2) from the hard disk device 7, and this display information. The display video data to be displayed on the plurality of video display units W57 _{1 to} W57 _n is generated by performing image processing by the GPU 4 based on the above. Incidentally, the display position where the plurality of video display units W57 _{1 to} W57 _n are arranged on the screen is fixed in principle, and the position of each of the video display units W57 _{1 to} W57 _{n as} the line of sight does not change. The operator's line of sight is not blurred and the visibility is greatly improved.

In this editing screen 50 (FIG. 2), for example, a clip desired by the operator is moved from the clips displayed in the list in the clip list display unit 51 into the display unit 55 in response to a drag-and-drop operation. When the microprocessor 3 recognizes that the play button 56 displayed in the lower left corner of 55 is clicked, the microprocessor 3 recognizes each video display unit W57 ₁ in the spiral video display unit group 58 as shown in FIG. from the image display unit W57 ₁ positioned on the screen rear side of ～W57 _n, the image display section _{W57 2,} W57 3 adjacent to the front side, _..., with respect to _W57 m, W57 _n, as shown in FIG. 5 , And sequentially playback with a time difference (hereinafter referred to as a playback time difference) ΔT at a predetermined equal interval set in advance, and a playback video of the clip Are displayed in order.

That is, the innermost side of the reproduced image of the clip on the video display unit W57 ₁ is displayed from the top first, then the next image display section W57 ₂ positioned on one front side after a time lapse of the reproduction time difference ΔT min the clip of reproduced video is displayed from the beginning, and further subsequently displayed next the reproduced video on the video display unit W57 ₃ located one front after a time lapse of the reproduction time difference ΔT min from the beginning, that ...... As described above, the playback video of the clip is sequentially displayed as a time-series flow in the direction of the arrow a (FIG. 4) on each video display unit W57 _{1 to} W57 _n with such a regular playback time difference ΔT.

Thus the microprocessor 3, between a plurality of image display section _W57 1 _{~W57 n,} while in the state of a moving image, respectively a time difference of the reproduction time difference ΔT min, a plurality of video display unit _W57 1 _{~W57 n} is coordination in the display order As the spiral video display unit group 58, the playback video of the clip can be sequentially linked and displayed.

Therefore, for the operator who visually confirms the plurality of video display portions W57 _{1 to} W57 _n in the spiral video display portion group 58, the time difference of the reproduction time difference ΔT is provided from the back side to the front side of the screen. As a result of the replayed video, the video display units W57 _{1 to} W57 _n feel as if they are approaching from the center of the spiral, and the progress of the replayed video displayed on the plurality of video display units W57 _{1 to} W57 _n Since the back side is more advanced than the front side of the screen, the operator feels as if he / she is going through the center of the spiral to the back side of the screen.

In this way editing device 2, also a desired scene displayed on the video display unit W57 ₁ among the plurality of image display section W57 ₁ ~W57 _n constituting the spiral display area group 58 there is the operator loses track, screen it means that sequentially appear image display section W57 ₂ later after the front side of the also displaying the desired scene always, that causes rewinding reproduction image of the image display section W57 ₁ ~W57 _n for looking image There is no need to force the operator to perform complicated operations. The operator only has to wait, and image search and editing can be further facilitated.

Further, as shown in FIG. 6, the clips displayed on the respective video display units W57 _{1 to} W57 _n constituting the spiral video display unit group 58 display the playback video at a preset image update speed, that is, the playback speed v. Is done. Therefore, when the set value of the playback speed v is large, the playback video that has been played back at high speed at the same speed is displayed on each of the video display units W57 _{1 to} W57 _n accordingly, and the set playback speed v is set accordingly. When the video is slow, the playback video that has been played back slowly at the same speed is displayed on each of the video display units W57 _{1 to} W57 _n accordingly.

In particular, in the spiral video display unit group 58 (FIG. 3), the playback video is displayed in a state where the playback time difference ΔT is provided between the video display units W57 _{1 to} W57 _n. The time code notification frames TC1 to TC4 are displayed at predetermined intervals of the video display units W57 _{1 to} W57 _n constituting the spiral of the group 58, and the playback progress of the playback video in the video display units W57 _{1 to} W57 _n The time is presented to the operator to assist the image searching and editing operations by the operator. Incidentally, the editing apparatus 2 can display the time code frames TC1 to TC4 on all the video display units W57 _{1 to} W57 _n , and select not to display at all when unnecessary for the operator. It is also made possible.

  Note that the initial values of the playback time difference ΔT and the playback speed v are “playback” from a context menu (not shown) that is displayed when “setting” in the toolbar 59 displayed at the top of the display unit 55 is clicked. The time difference / reproduction speed can be set using a reproduction time difference / reproduction speed setting dialog 60 shown in FIG. 7 displayed when the operator selects “time difference / reproduction speed”.

  Actually, when the operator sets initial values of the reproduction time difference ΔT and the reproduction speed v, each text provided in the time difference / reproduction speed setting dialog 60 corresponding to the reproduction time difference ΔT and the reproduction speed v, respectively. After the numerical values in the boxes 61 and 62 are changed by, for example, clicking the corresponding up / down keys 63A, 63B, 64A and 64B or directly inputting the desired numerical values using the keyboard 39, “OK” is displayed. The button 65 may be clicked, and the numerical values displayed in the text boxes 61 and 62 at this time are set as initial values of the reproduction time difference ΔT and the reproduction speed v, respectively.

Thereby, in the microprocessor 3 of the editing apparatus 2, for example, the reproduction time difference ΔT of the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n is reduced, and the reproduction speed v of the reproduced video is set to be slow. Thus, the video of a narrow range of the clip can be displayed as the spiral video display unit group 58, the playback time difference ΔT of the playback video displayed on each of the video display units W57 _{1 to} W57 _n is large, and these playback video By setting the playback speed v to be slow, an image in a wide range of the clip can be displayed as the spiral image display unit group 58.

In practice, the playback video displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 as shown in FIG. 6 is displayed at a variable speed in accordance with the change of the playback speed v. In addition, since the time difference is displayed at an arbitrary time interval in accordance with the change of the reproduction time difference ΔT, the entire screen can be swirled from the back side of the center of the screen to the front side as shown by the thick arrow ST1. Displayed in conjunction with the shift display.

In this case, in each of the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58, a playback video at a timing that is most advanced in time is displayed with respect to the video display unit W57 ₁ that is present at the innermost side of the screen. As a result, a playback video with the most delayed timing is displayed on the video display unit W57 _n existing on the most front side, so that from the back side of the screen to the front side in time series order. The operator can intuitively recognize that the video display that flows spirally along the traveling direction is performed.

Also, as shown in FIG. 8, the playback video displayed on each video display unit W57 _{1 to} W57 _n is displayed in variable speed in the reverse method when the playback speed is set to −v, and the playback time difference ΔT is Since the time difference is displayed at an arbitrary time interval according to the change, the variable speed display is performed so as to flow spirally from the front side to the back side of the screen as shown by the thick arrow ST2 as a whole. It is displayed in a linked state.

  Further, in the editing screen 50, in a state where one or a plurality of spiral video display unit groups 58 are already displayed in the display unit 55, the editing apparatus 2 displays other clips displayed on the clip list display unit 51 as display units. In response to being dragged and dropped in 55, the spiral corresponding to the newly dragged and dropped clip in addition to one or a plurality of spiral video display groups 58 that have been displayed in the display unit 55 so far The video display unit group 58 can be displayed in the display unit 55.

  On the other hand, in the editing screen 50, the editing device 2 is in the state where one or more spiral video display unit groups 58 are already displayed in the display unit 55, and the editing device 2 is one or more spiral video display unit groups in the display unit 55. When 58 is selected and dragged and dropped on the clip list display unit 51, the selected one or more spiral video display units 58 from the one or more spiral video display units 58 that have been displayed in the display unit 55 are displayed. The spiral image display unit group 58 can also be deleted. Alternatively, the editing device 2 also deletes the one or more spiral image display unit groups 58 when the operator selects one or more spiral image display unit groups 58 in the display unit 55 and presses the Delete key. be able to.

When the editing device 2 displays a plurality of spiral video display unit groups 58 on the display unit 55, after one spiral video display unit group 58 is selected by a mouse operation, the operator clicks on the playback button 56. Accordingly, it is possible to display the playback video of the clip corresponding to each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 at each timing. In this case, the initial values of the playback time difference ΔT and the playback speed v are both values set using the playback time difference / playback speed setting dialog 60 described above.

  By the way, a parallel decoding processing procedure for performing parallel decoding processing using the eight sub CPU cores 3SA to 3SH provided in the microprocessor 3 of the editing device 2 in the editing system 1 will be described below.

  As shown in FIG. 9, the microprocessor 3 of the editing apparatus 2 enters from the start step of the routine RT1, moves to the subsequent step SP1, and receives control signals input from the mouse 38 and the keyboard 39 via the south bridge 6. When the control signal is recognized by the main CPU core 3M, the process proceeds to the next step SP2.

Microprocessor of the editing device 2 at step SP2 3, when the control signal recognized in step SP1 was a command specifying the clip to be displayed on the video display unit _W57 1 _{~W57 n,} the image display section _W57 1 _~W57 The drawing loop of the clip displayed on _n is set by the main CPU core 3M, and the process proceeds to the next step SP3.

  In step SP3, the microprocessor 3 of the editing apparatus 2 performs various physical operations such as a playback image, display position, and display size corresponding to the clip by the main CPU core 3M, and proceeds to the next step SP4.

In step SP4, the microprocessor 3 of the editing apparatus 2 calculates the frame position of the playback video of the clip that should form the drawing loop in the main CPU core 3M, and plays back the clip displayed on the video display units W57 _{1 to} W57 _n. The main CPU core 3M assigns the responsibility of which sub CPU cores 3SA to 3SH are to decode the video, and proceeds to the next step SP5.

  In step SP5, the microprocessor 3 of the editing device 2 reads out a frame necessary for outputting the reproduced video from the storage device 22, distributes it to the sub CPU cores 3SA to 3SH assigned in step SP4, and proceeds to the next step SP6.

  In step SP6, the microprocessor 3 of the editing apparatus 2 simultaneously decodes the frames distributed in step SP5 by the eight sub CPU cores 3SA to 3SH serving as decoders, and proceeds to the next step SP7. .

  In step SP7, the microprocessor 3 of the editing apparatus 2 uses the main CPU core 3M to display the display information (display) indicating the position and size at which the plurality of playback videos decoded in step SP6 should be displayed on the screen of the display unit 55. The high-speed data is transferred to the GPU 4 together with the size and the display position, and the process proceeds to the next step SP8.

In step SP8, the microprocessor 3 of the editing device 2 uses the GPU 4 to determine a predetermined _number of video display units W57 _{1 to} W57 _n according to display information (display size and display position) of a plurality of playback videos transferred from the main CPU core 3M. The information is pasted and displayed at the position, and the process proceeds to the next step SP9 to end the process.

As described above, in the microprocessor 3 of the editing apparatus 2, the eight sub CPU cores 3SA to 3SH each play a role as a decoder of a reproduced video displayed on each of the video display units W57 _{1 to} W57 _n and decode in parallel. The decoded reproduced video can be transferred at high speed to the GPU 4 by a bus 10 having a large bandwidth at a transfer rate of, for example, 30 [Gbyte / sec] at the maximum.

As a result, the microprocessor 3 of the editing apparatus 2 can decode a large number of high-definition reproduced images and transfer the data to the GPU 4 in a short time. Thus, a moving image is displayed between the plurality of video display units W57 _{1 to} W57 _n . As a spiral video display unit group 58 that cooperates in the display order while providing a time difference in the state of the image, the display unit 55 can be smoothly displayed in an interlocking manner with good response.

  On the other hand, on the editing screen 50 (FIG. 2), “setting” on the toolbar 59 displayed at the top of the display unit 55 is clicked, and “sound output mode” is selected from the context menu (not shown) displayed in this way. Accordingly, the main processor 3 selects, for example, one of “main audio output mode” and “all audio output mode” as the audio output mode when the playback video of the clip is displayed on the display unit 55. Can be set.

When the “main audio output mode” is set as the audio output mode, it accompanies the reproduced video displayed on the video display unit W57 _n positioned closest to the spiral video display unit group 58 operated at that time. When only the reproduced audio is output from the speaker 41 and the “all audio output mode” is set, the video display units W57 _m ,..., W57 ₁ away from the video display unit W57 _n located on the most front side of the screen are displayed. In a state in which the volume is adjusted so that the volume of the displayed playback video becomes smaller, all the playback images associated with the playback video displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 are displayed. Reproduced sound is output from the speaker 41.

However, when “all audio output mode” is set as the audio output mode, when the reproduction time difference ΔT between the reproduced images respectively displayed on the image display units W57 _{1 to} W57 _n becomes large, or when a scene change is performed. When a predetermined sound mixture is predicted, such as when it is detected, it is displayed on the video display unit W57 _n located on the foremost side of the screen and several video display units W57 _m ,. The output sound is automatically adjusted so that only the sound of each reproduced video is output from the speaker 41. As a result, the microprocessor 3 can output sound in a good state that is easy for the operator to hear even in the “all sound output mode”.

(2-2) Processing of Microprocessor Regarding Display of Reproduced Video on Edit Screen Here, the microprocessor 3 displays the replayed video on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 as described above. Various processes relating to the output of reproduced audio are executed according to a reproduction processing procedure RT2 shown in FIG. 10 based on a control program stored in the hard disk device 7 (FIG. 1).

  That is, the microprocessor 3 clicks the playback button 56 after one clip is dragged and dropped from the clip list display unit 51 of the editing screen 50 to the display unit 55 (a plurality of spiral video display unit groups 58 are displayed on the display unit 55). If one spiral video display unit group 58 is selected and the playback button 56 is clicked), this playback processing procedure RT2 is entered, and in the next step SP11, the spiral video display unit group 58 is entered. Confirm the clip ID of the clip corresponding to.

  Subsequently, in step SP12, the microprocessor 3 confirms the initial values of the reproduction speed v (or -v) and reproduction time difference ΔT set at that time, and the audio output mode settings, respectively, after that. The process proceeds to the next step SP13, where the playback speed v (or -v) and playback time difference ΔT of the playback video determined by the mouse 38, the keyboard 39 or the operation controller 37 and the display form are confirmed.

At this time, in step SP14, the microprocessor 3 determines whether or not the stop operation or the clip stream has ended. If an affirmative result is obtained, the microprocessor 3 proceeds to the next step SP10 and displays each video in the spiral video display unit group 58. The update process for the parts W57 _{1 to} W57 _n is stopped, and if a negative result is obtained, the process proceeds to the next step SP15.

In step SP15, the microprocessor 3 displays the display position and size of the playback video to be displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58, and the playback audio associated with the playback video. Calculate the output volume respectively.

  Then, the microprocessor 3 moves to step SP16, and controls the storage device 22 based on the clip ID of the clip confirmed in step SP11, so that the video / audio signal of the clip is set when confirmed in step SP12. The video / audio signal is read at the playback speed v, decoded as necessary, and the video / audio signal of the video / audio signal is displayed at the display position and size calculated in step SP15. After the generation, it is temporarily stored in the memory in the GPU 4.

  Further, the microprocessor 3 generates an audio signal of the reproduced audio associated with the reproduced video corresponding to the video signal at the volume level of the output volume calculated for the reproduced video in step SP15, and generates the audio signal as a sound input / output codec. 9 is temporarily stored in the memory in the memory.

In step SP17, the microprocessor 3 determines whether or not the reproduction video and reproduction audio necessary for outputting to all the video display units W57 _{1 to} W57 _n have been prepared, and if a negative result is obtained, the step is performed. Returning to SP16, if a positive result is obtained, the process proceeds to the next step SP18.

In step SP18, the microprocessor 3 monitors the elapse of the screen display cycle time (reproduction time difference ΔT) in each of the video display units W57 _{1 to} W57 _n . If the screen display cycle time has elapsed, the microprocessor 3 proceeds to the next step SP19. The spiral video display unit group 58 is displayed by outputting the playback video to each of the video display units W57 _{1 to} W57 _n , and the playback sound accompanying the playback video is output from the speaker 41 at a predetermined volume and simultaneously with the step. returns to SP13, carries out the following preparations of the reproduced video and audio for each image display unit _W57 1 _{~W57 n.}

  In step SP16, the microprocessor 3 uses the regenerated video signal remaining in the memory of the previously reproduced GPU 4 if it can be reused, generates only the missing video signal, and generates the generated video. The signals are overwritten in order from the old data stored in the memory in the GPU 4.

(2-3) Various Operation Methods on Display Unit of Edit Screen Next, various operation methods on the display unit 55 of the edit screen 50 will be described.

(2-3-1) Basic Operation The microprocessor 3 of the editing apparatus 2 has the video display units W57 _{1 to} W57 _n in the spiral video display unit group 58 in the display unit 55 of the editing screen 50 as shown in FIG. depending on each click (in this case, for example, click the first to the video display unit W57 _e) the desired time to display the scene is for example a video display unit W57 _e of the displayed reproduced video by the operator has been able to Te, the colored to a predetermined color that image display unit W57 _e is the reproduction images corresponding to the editing position with the edit operation (in the figure indicated by hatching) are made so as to highlight.

Then, the microprocessor 3 displays the highlighted display for each of the video display units W57 _{1 to} W57 _n in the spiral video display unit group 58 in accordance with the flow of the scene in the spiral video display unit group 58 (the time series flow of the playback video). As shown in FIGS. 11 and 12, the video display units W57 _h , W57 _j ,..., W57 _n are positioned on the front side of the screen with respect to the video display unit W57 _{e of} the spiral video display unit group 58 with a reproduction time difference ΔT. It is made to move sequentially.

Thus, the operator can display the desired scene on which video display unit W57 _{1 to} W57 _n is currently displayed based on the highlight display flow in each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58. Whether or not it can be visually confirmed instantaneously.

That is, the highlighted display for each of the video display units W57 _{1 to} W57 _{n in the} spiral video display unit group 58 is a scene (for example, the video display unit W57 _e ) corresponding to the editing position associated with the editing operation first focused by the operator. each image display section W57 h when but playing such throughout even when moving _{scenes, ...,} W57 j, _..., since continues to be highlighted while changing to W57 _n, 12 As shown by the long thick arrows in FIG. _1, while the video display that flows to the video display units W57 _{1 to} W57 _n is performed, the operator first recognizes the first focused scene instantly and intuitively by highlighting. Has been made to get.

Note, as shown in short bold arrow in FIG 12, for example, when the operator corresponding scene editing position due to initial focus editing operation is an image display unit W57 _h the scene emphasize that operator attention Since it is displayed, the operator can make an impression as if the flow starts from there.

Further, as shown in FIG. 13A, the microprocessor 3 of the editing apparatus 2 is operated by pressing an “upper” key 70U corresponding to the upper direction among the direction keys 70 on the keyboard 39 (FIG. 13). (A-1)), or in a state where the right button of the mouse 38 is pressed, the cursor 71 is a part of each of the video display units W57 _{1 to} W57 _{n in} the spiral video display unit group 58, for example, the video display unit W57. If along the _{α ~W57 β (i.e.} toward the far side of the screen to the near side) display operation input for moving the cursor 71 so as to trace (FIG. 13 (a-2)) is carried out, the image of the clip The display speed can be increased.

  In this case, the microprocessor 3 acquires the display state change information corresponding to the display operation input by the mouse 38 via the PCI bus 15 and increases the video display speed of the clip according to the contents of the display state change information. ing.

Here, “increasing the video display speed of the clip” means that the playback speed v of the playback video displayed on each of the video display sections W57 _{1 to} W57 _n of the spiral video display section group 58 is increased, and This means that the playback time difference ΔT of the playback video between adjacent video display units W57 _{1 to} W57 _n increases in cooperation with this.

The magnitude of the change in the “display speed of the reproduced video” is determined by pressing the “up (↑)” key 70U of the keyboard 39 at that time or by operating the mouse to display the video display unit W57 ₁ of the spiral video display unit group 58. ˜W57 When the time on the _n is long, it becomes larger. That is, the longer this time is, the more the playback speed v and the playback time difference ΔT both increase, and the display speed of the appearance of each clip as a whole (the flow of each scene of the playback video) increases.

That is, on the display unit 55, the “up (↑)” key 70U of the keyboard 39 is pressed, and the mouse 71 is traced over the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58 via the cursor 71. The longer the time is, the more the display acceleration GS1 of the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 can be increased. Incidentally, in the display unit 55, instead of using time, for example, the display acceleration GS1 of the reproduced video may be increased as the number of times of pressing the “up (↑)” key 70U of the keyboard 39 is increased.

On the other hand, in the microprocessor 3 of the editing apparatus 2, as shown in FIG. 13B, for example, the “down (↓)” key 70D corresponding to the downward direction among the direction keys 70 of the keyboard 39 is pressed. (FIG. 11B-1), or a part of each of the video display units W57 _{1 to} W57 _{n in} the spiral video display unit group 58 by the cursor 71 with the right button of the mouse 38 being pressed. If along the image display section _W57 β ~W57 α _(i.e. toward the front side of the spiral to the back side) display operation input for moving the cursor 71 so as to trace (FIG. 13 (B-2)) is carried out, The video display speed of the clip can be lowered.

  In this case, the microprocessor 3 acquires the display state change information corresponding to the display operation input by the mouse 38 via the PCI bus 15, and reduces the video display speed of the clip according to the contents of the display state change information. ing.

Note that “reducing the video display speed of the clip” here means that the playback speed v of the playback video displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 decreases, In addition, it means that the reproduction time difference ΔT of the reproduced video between the adjacent video display units W57 _{1 to} W57 _n decreases in cooperation with this.

The magnitude of the change in the “display speed of the playback video” is determined by pressing the “down (↓)” key 70D of the keyboard 39 at that time or by operating the mouse to display each video display unit W57 ₁ of the spiral video display unit group 58. ˜W57 When the time on the _n is long, it becomes larger. That is, as this time is longer, both the playback speed v and the playback time difference ΔT decrease, and the apparent display speed of the entire video of the clip decreases.

That is, on the display unit 55, the “down (↓)” key 70D of the keyboard 39 is pressed, and the mouse 71 is traced over the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58 via the cursor 71. The display acceleration GS2 of the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n in the spiral video display unit group 58 can be reduced as the time of the display is longer. Incidentally, in the display unit 55, instead of using time, for example, the display acceleration GS2 of the reproduced video may be decreased as the number of times of pressing the “down (↓)” key 70D of the keyboard 39 is increased.

  In this case, after the playback speed v or playback time difference ΔT becomes “0”, the playback speed v or playback time difference ΔT further decreases (that is, increases in the negative direction). Accordingly, in this case, the flow of each scene of the playback video that has been flowing from the back side of the screen to the near side in the spiral video display unit group 58 until then gradually stops, and then stops before the spiral. It starts to flow from the back to the reverse side and changes to reverse playback, and eventually it seems that the speed gradually increases.

Therefore, for an operator who is visually confirming the plurality of video display units W57 _{1 to} W57 _n in the spiral video display unit group 58, a time difference of the reproduction time difference ΔT is provided from the front side of the spiral toward the back side, and the degree of progress. As if the reproduced video by the video display units W57 _{1 to} W57 _n escapes to the center of the spiral, in other words, the operator himself moves backward from the center of the spiral to the front side. It is made to get a good impression.

Further, the microprocessor 3 of the editing apparatus 2 displays a cursor 71 displayed so as to freely move on the display unit 55 of the editing screen 50 in response to the mouse operation in a state where the left button of the mouse 50 is pressed. Thus, the video of the clip can be stopped in response to an operation to hold a part of the spiral video display unit group 58 at the same position for, for example, 1 second or longer. Here, “stop the video of the clip” means to display a still image on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 while maintaining the reproduction time difference ΔT. Say.

(2-3-2) Extended operation (2-3-2-1) Reproduction time difference and reproduction speed changing operation On the other hand, the microprocessor 3 of the editing device 2 performs the basic operation as described above, as well as “ While the “shift” key is pressed, the above-described “increase the display speed of the clip video” operation, that is, the “up (↑)” key 70U of the direction key 70 of the keyboard 39 is pressed, or the right of the mouse 38 While the button is pressed, a part of the reproduced video displayed on the spiral video display unit group 58 by the cursor 71 is moved along the flow from the back side to the front side of the screen in the spiral video display unit group 58. In accordance with the movement of the cursor 71 so as to trace, the reproduction speed v of the reproduction image displayed on each of the image display units W57 _{1 to} W57 _n of the spiral image display unit group 58 is fixed. Thus, it is possible to increase the reproduction time difference ΔT between the reproduced images displayed on the adjacent image display units W57 _{1 to} W57 _n .

In this case, the microprocessor 3 acquires the display state change information corresponding to the display operation input by the keyboard 39 or the mouse 38 via the PCI bus 15, and displays the video display units W57 ₁ to W57 _1- according to the contents of the display state change information. While the playback speed v of the playback video displayed on W57 _n is fixed, the playback time difference ΔT of the playback video is increased.

At this time, the continuity of the reproduced video displayed on the video display units W57 _{1 to} W57 _n adjacent to each other is reduced, but the spiral video display group 58 as a whole displays a wide range of reproduced video. Can do.

On the other hand, in the microprocessor 3 of the editing device 2, for example, while the “shift” key of the keyboard 39 is being pressed, the above-described “reducing the display speed of the clip image”, that is, the direction key 70 of the keyboard 39 is used. A part of the playback video displayed on the spiral video display unit group 58 by the cursor 71 in a state where the “down (↓)” key 70D is pressed or the right button of the mouse 38 is pressed is displayed as the spiral video. In response to the movement of the cursor 71 so as to follow the flow from the back side to the near side of the screen in the display unit group 58, each video display unit W57 _{1 to} W57 _{n of the} spiral video display unit group 58 is moved. while the reproduction speed v of the displayed playback video is fixed to, the reproduction time difference ΔT of the reproduced images displayed respectively on the adjacent image display unit W57 ₁ ~W57 _n It can cause lack.

  In this case, the microprocessor 3 acquires the display state change information corresponding to the display operation input by the keyboard 39 or the mouse 38 via the PCI bus 15, and sets the reproduction speed v of the reproduced video according to the content of the display state change information. While being fixed, the playback time difference ΔT of the playback video is reduced.

At this time, the entire spiral video display unit group 58 displays a playback video in a narrow range in time, but is displayed on each video display unit W57 _{1 to} W57 _n of the spiral video display unit group 58. The continuity of the reproduced video is high, and the same scene is sequentially displayed on each of the video display units W57 _{1 to} W57 _n so as to flow in the spiral video display unit group 58 from the back side to the near side of the screen with a short playback time difference ΔT. It becomes like this. This flow becomes faster as the reproduction time difference ΔT of the reproduced video displayed on the adjacent video display units W57 _{1 to} W57 _n is decreased.

Further, in the microprocessor 3 of the editing apparatus 2, for example, such an operation of “increasing the display speed of the clip video” is performed while both the “shift” key and the “ALT” key of the keyboard 39 are pressed. Accordingly, the playback time difference ΔT of the playback video displayed on the adjacent video display units W57 _{1 to} W57 _n is fixed, and the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 are displayed. The playback speed v of the playback video can be increased.

  Also in this case, the microprocessor 3 acquires the display state change information corresponding to the display operation input by the keyboard 39 or the mouse 38 via the PCI bus 15, and the reproduction time difference ΔT of the reproduced video according to the content of the display state change information. The playback speed v of the playback video is increased while fixing is fixed.

As a result, the flow of each scene of the playback video in the spiral video display unit group 58 is accelerated while the playback video in the narrow range in time is displayed on the spiral video display unit group 58 from the video of the entire clip. Can do. In this case, the continuity of the reproduced video displayed on the video display units W57 _{1 to} W57 _n adjacent to each other is increased.

On the other hand, in the microprocessor 3 of the editing apparatus 2, for example, such “decrease the display speed of the clip image” is performed while both the “shift” key and the “ALT” key of the keyboard 39 are pressed. depending on the fact, while a fixed playback time difference ΔT of the reproduced images displayed respectively on the image display unit _W57 1 _{~W57 n} mutually adjacent, each of the image display unit _W57 1 _{~W57 n} of spiral image display area group 58 The playback speed v of the playback video displayed on the screen can be reduced.

  Also in this case, the microprocessor 3 acquires the display state change information corresponding to the display operation input by the keyboard 39 or the mouse 38 via the PCI bus 15, and the reproduction time difference ΔT of the reproduced video according to the content of the display state change information. The playback speed v of the playback video is reduced while fixing is fixed.

As a result, the microprocessor 3 causes the spiral video display unit group 58 to display the playback video in the spiral video display unit group 58 while displaying the playback video in a narrow time range as viewed from the video of the entire clip. The flow of each scene can be slowed down. In this case, the continuity of the reproduced video displayed on the video display units W57 _{1 to} W57 _n adjacent to each other is lowered.

(2-3-2-2) Instant Playback or Return Operation of Reproduced Video On the other hand, in the microprocessor 3 of the editing apparatus 2, as shown in FIG. 14A, the “left (←)” of the direction key 70 of the keyboard 39 ”Key 70L is pressed (FIG. 14A-1) or the left button of the mouse 38 is pressed, and the playback video displayed on the spiral video display unit group 58 by the cursor 71 within one second. is part of the (image display section W57 _i) display selection operation (gripping), the cursor from the back side of the screen along the spiral display area group 58 to the image display section W57 _l so as to trace toward the front side In response to the movement operation of the spiral video display unit group when 71 is moved (FIG. 14A-2), the playback speed v of the playback video displayed on the spiral video display unit group 58 and each adjacent image display section _W57 1 ~ Remain both were both fixed between the reproduction time difference ΔT of the reproduced images displayed respectively on 57 _n, the display position of each image display unit W57 ₁ ~W57 _n of spiral image display area group 58, shown in FIG. 15 As described above, the cursor 71 can be moved integrally with the cursor 71 in the same direction as the direction in which the cursor 71 is traced along the spiral image display unit group 58 (the direction from the back side to the near side of the screen).

By the way, although it has been described above that the display positions at which the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58 are arranged on the screen are fixed, an extended operation of instantaneous playback or return operation of the playback video is performed. Is performed, the microprocessor 3 of the editing apparatus 2 can move the display position on the screen in each of the video display units W57 _{1 to} W57 _n .

In this case, the microprocessor 3 acquires the display state change information corresponding to the spiral video display unit group moving operation by the keyboard 39 or the mouse 38 via the PCI bus 15 and reproduces the reproduced video according to the content of the display state change information. While fixing both the playback speed v and the playback time difference ΔT, the display positions of the video display units W57 _{1 to} W57 _n are moved along the spiral video display unit group 58 from the back side of the screen toward the front side with the cursor 71. It is designed to move together.

At that time, as shown in FIG. 16, the video display units W57 _{1 to} W57 _n ,... Of the spiral video display unit group 58 displaying the playback video are shifted to the position traced by the cursor 71 and displayed. Is done. Accordingly, the entire video display units W57 _{1 to} W57 _n that have been displayed at that position are shifted toward the front side of the spiral in accordance with the amount of movement of the cursor 71, and the video display unit after the shift W57 _n, W57 _m, ......, display size of the newly displayed is changed slightly larger than before (Fig. 15).

Note microprocessor 3, display selection operation for the image display unit 57 _i of the video display unit 57 _i with highlighting (enclosed by a broken line) according to (15), then the video display unit 57 _i previous image display unit 57, .... by sequentially highlighted in the display order, and have been made to flow until the movement start point of the video display unit 57 _i so as intuitively recognize the the operator.

At this time, the microprocessor 3 temporarily stops (holds) all the display video data for the video display unit W57 _i or the display video data for each of the video display units W57 _{1 to} W57 _n at the moment when the display selection operation is performed. In addition, the operator can visually confirm the state.

In addition, when the main audio output mode is set as the audio output mode, the reproduced audio associated with the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n is the video display positioned closest to the front at that time. When only the playback audio accompanying the playback video displayed in the part W57 _n (FIG. 14A-2) is output and the full audio output mode is set as the audio output mode, the video positioned closest to the front Each playback sound is set so that the playback sound accompanying the playback video displayed on the video display units W57 _m, W57 _l, W57 _k, ... Separated from the display unit W57 _n by a three-dimensional spiral distance has a lower volume. The output volume of is adjusted.

Incidentally, when a display selection cancellation operation for canceling the display selection operation for the video display unit W57 _i is input, the microprocessor 3 displays the display video data for the video display unit W57 _i or the display for each of the video display units W57 _{1 to} W57 _n . releasing all of the video data from the stop state, while a time difference in the state of a moving image between a plurality of image display section W57 ₁ ~W57 _n, the plurality of image display section W57 ₁ ~W57 _n is coordination in the display order helix The video display unit group 58 is displayed in conjunction again.

On the other hand, in the microprocessor 3 of the editing device 2, as shown in FIG. 14B, the “right (→)” key 70R of the direction key 70 of the keyboard 39 is pressed (FIG. 14B-1). ) Or a part of the spiral image display unit group 58 (image display unit 57 _l ) is selected (grabbed) by the cursor 71 within one second while the left button of the mouse 38 is pressed, and the spiral image display unit The spiral image display unit group moving operation when the cursor 71 is moved to the image display unit 57 _i so as to trace from the back side to the near side of the spiral along the group 58 (FIG. 14 (B-2)). Accordingly, both the playback speed v of the playback video displayed on the spiral video display unit group 58 and the playback time difference ΔT of the playback video displayed on the video display units W57 _{1 to} W57 _n adjacent to each other are fixed. State While, the display position of the image display section W57 ₁ ~W57 _n of spiral image display area group 58, as shown in FIG. 17, the front side of the traced along the spiral display area group 58 direction (helix Can be moved together with the cursor 71 in the same direction as the direction from the back to the back.

In this case, the microprocessor 3 acquires the display state change information corresponding to the spiral video display unit group moving operation by the keyboard 39 or the mouse 38 via the PCI bus 15 and reproduces the reproduced video according to the content of the display state change information. While both the playback speed v and the playback time difference ΔT are fixed, the display positions of the video display units W57 _{1 to} W57 _n are moved along the spiral video display unit group 58 from the front side of the spiral toward the back side with the cursor 71. It is designed to move together.

At that time, as shown in FIG. 18, the video display units W57 _{1 to} W57 _n ,... Of the spiral video display unit group 58 displaying the playback video are shifted to the position traced by the cursor 71 and displayed. The Accordingly, the entire video display units W57 _{1 to} W57 _n that have been displayed at that position are shifted toward the back of the screen according to the amount of movement of the cursor 71, and the video display unit after the shift is performed. The display size of W57 _n, W57 _m, ... Is changed slightly smaller than before and is newly displayed (FIG. 17).

Note microprocessor 3, display selection operation for the image display unit 57 _i of the video display unit 57 _i with highlighting (enclosed by a broken line) according to (17), then the video display unit 57 _i previous image display unit 57, .... by sequentially highlighted in the display order, and have been made to flow until the movement start point of the video display unit 57 _i so as intuitively recognize the the operator.

In this way the display unit 55, while being reproduced image display section _W57 1 _{~W57 n} so as to flow into the forward direction, the display position of the image display section _W57 1 _{~W57 n} in accordance with the spiral display area group move operation Can be moved instantaneously to the backward side, so that the past scenes can be immediately displayed without any deterioration in the responsiveness of the displayed video, such as when reverse playback is performed in the MPEG long GOP format. It can be displayed again.

Microprocessor 3 is also in this case, display selection operation for the image display section W57 _i the image display unit W57 _i with highlighting (enclosed by a broken line) according to (17), the video display unit 57 _i then the previous image display section W57, ......, by sequentially highlighted in the display order, and have been made to flow until the movement start point of the image display section W57 _i so as intuitively recognize is the operator .

The microprocessor 3 this time, temporarily stop all in this state of the display image data to display image data or the image display section W57 _i to the video display unit W57 _i at the moment when the displayed selection operation (holding), The state can be visually confirmed by an operator.

Incidentally, when a display selection cancellation operation for canceling the display selection operation for the video display unit W57 _i is input, the microprocessor 3 displays the display video data for the video display unit W57 _i or the display for each of the video display units W57 _{1 to} W57 _n . releasing all of the video data from the stop state, while a time difference in the state of a moving image between a plurality of image display section W57 ₁ ~W57 _n, the plurality of image display section W57 ₁ ~W57 _n is coordination in the display order helix The video display unit group 58 is displayed in conjunction again.

Further, in this case, when the main audio output mode is set as the audio output mode, the reproduced audio accompanying the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n is at the most front side of the screen at that time. When only the playback audio accompanying the playback video displayed on the video display unit W57 _n positioned at is output and the full audio output mode is set as the audio output mode, the video display unit W57 _n positioned closest to the front is displayed. Output volume of each playback sound so that the playback sound associated with the playback video displayed in the video display sections W57 _m, W57 _l, W57 _k ,. Is adjusted.

(2-3-2-2) Enlarging / Reducing Operation of Video Display Unit Group On the other hand, in the microprocessor 3 of the editing apparatus 2, for example, one spiral video display unit group 58 is selected by a mouse operation, and then the left button of the mouse 38 is selected. When the “down (↓)” key 70D of the direction keys 70 of the keyboard 39 is pressed while the button is clicked, the display state of the spiral image display unit group 58 is reduced as a whole. In addition, one spiral image display unit group 58 is selected by operating the mouse, and then the “up (↑)” key 70U among the direction keys 70 of the keyboard 39 is pressed while the left button of the mouse 38 is clicked. The display state of the spiral image display unit group 58 can be increased as a whole in response to the operation.

  Thereby, in the display unit 55, for example, when insert editing is performed in which a video / audio part of another second clip is cut out and inserted into the first clip during an editing operation of a certain first clip, the first halfway of editing is performed. A large space is created in the display unit 55 by displaying the spiral image display unit group 58 corresponding to the clip of the entire size small, and then the spiral image display unit group 58 corresponding to the new second clip is displayed in the space. To display a desired video / audio part, and then the display size of the spiral video display unit group 58 corresponding to the first clip is returned, and the video / audio part of the clipped second clip is returned to the first. Editing operations such as insertion at a desired position of one clip can be performed.

(2-3-3) Processing of Microprocessor for Various Basic or Extended Operation Inputs Here, the microprocessor 3 stores the processing for various basic or extended operation inputs as described above in the hard disk device 7 (FIG. 1). Based on the control program, the display change processing procedure RT1 shown in FIG. 9 is executed.

For example, when the operation recognized in step SP1 in FIG. 9 is the above-described “increase the display speed of clip video” operation, the microprocessor 3 controls the storage device 22 to correspond to the corresponding spiral video display unit group. Each video to be displayed on the video display units W57 _{1 to} W57 _n is selected and generated according to the playback speed v of the playback video to be displayed on each of the 58 video display units W57 _{1 to} W57 _n .

The microprocessor 3, as well as increase the distance of the reproduced frames between the image display section _W57 1 ～W57 _n, by raising the display-playback speed for each video display unit _W57 1 ～W57 in _n, the video display unit W57 ₁ ～W57 increase the reproduction speed v of the reproduced images displayed respectively on _n, and increases the reproduction time difference ΔT of the reproduced video between the adjacent image display unit _W57 1 ~W57 _n together.

On the other hand, when the operation recognized in step SP1 is the above-described operation “decrease the display speed of the clip image”, the microprocessor 3 controls the storage device 22 to control the corresponding spiral image display unit group 58. Each video to be displayed on the video display units W57 _{1 to} W57 _n is selected and generated according to the playback speed v of the playback video to be displayed on each video display unit W57 _{1 to} W57 _n .

The microprocessor 3 also this time, along with narrowing the distance between the reproduction frames among the image display section _W57 1 _{~W57 n,} by reducing the display-playback speed for each video display unit _W57 1 _~W57 in _n, the video display The playback speed v of the playback video displayed in each of the parts W57 _{1 to} W57 _n is reduced, and the playback time difference ΔT of the playback video between the video display parts W57 _{1 to} W57 _n adjacent to each other is reduced.

Further, when the operation recognized in step SP1 is an operation of “stopping the video of the clip”, the microprocessor 3 controls the storage device 22 to each video display unit of the corresponding spiral video display unit group 58. While the reproduction of the reproduced video to be displayed on W57 _{1 to} W57 _n is stopped, the GPU 4 is controlled, and the last frame just before the reproduction is stopped is respectively displayed on each video display unit W57 _{1 to} W57 _n of the spiral video display unit group 58. Continue to display. As a result, a corresponding still image is displayed on each of the video display units W57 _{1 to} W57 _n while maintaining the original reproduction time difference ΔT.

  Further, when the operation recognized in step SP1 is various expansion operations such as “stop video of clip” while pressing the “shift” key of the keyboard 39, the microprocessor 3 The storage device 22 and the GPU 4 are controlled so that each playback image displayed on the 55 spiral image display unit group 58 and the display image of the spiral image display unit group 58 itself are changed to a state corresponding to the expansion operation. .

  Then, in step SP8, the microprocessor 3 changes each playback video in the spiral video display unit group 58 displayed on the display unit 55 and a display video of the spiral video display unit group 58 to a state corresponding to the operation input. Has been made.

(2-4) Various Operations for Setting Edit Contents on Display Unit Next, the operation procedures of specific individual operations for setting edit contents on the display unit 55 will be described.

First, cut editing within the same clip will be described. When a clip dragged and dropped on the display unit 55 is cut and edited within the clip, as shown in FIG. 19A, first, the playback video of the clip is started, and if necessary, the spiral is started. The playback speed v of each playback video displayed on each of the video display units W57 _{1 to} W57 _n of the video display unit group 58 and the playback time difference of the playback video displayed on each of the video display units W57 _{1 to} W57 _n adjacent to each other. While changing ΔT as described above, the first frame FR _IN of the video / audio portion to be cut is searched.

When the frame FR _IN is determined as the editing position, all the playback videos displayed on the video display units W57 _{1 to} W57 _n are stopped, and the video display unit W57 displaying the image of the frame FR _{IN in} that state. _{1 to} W57 Click _n . As a result of this editing operation input, the microprocessor 3 sets the frame FR _IN as the In point of the video / audio portion to be cut, and sets the frame portion set as the In point among the video display units W57 _{1 to} W57 _n to a predetermined color. Colored (eg yellow).

Next, in the same manner, the next frame FR _OUT of the last frame of the video / audio portion to be cut out is searched for, and when the frame FR _OUT is determined as the editing position, it is displayed on each of the video display units W57 _{1 to} W57 _n . All the reproduced videos to be played back are stopped, and in this state, the operator clicks the video display portions W57 _{1 to} W57 _n on which the image of the frame FR _OUT is displayed. As a result of this editing operation input, the microprocessor 3 sets the frame FR _OUT as the out point of the video / audio portion to be cut out, and the frame portion of the frame to be cut out of the video display portions W57 _{1 to} W57 _n has a predetermined color ( For example, the color is displayed in white).

The microprocessor 3 thus sets the in and out points in the clip as edit positions in this way, and then reproduces the reproduction time difference ΔT of the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n as necessary. Is changed as described above, and the images of the frames FR _IN and FR _{OUT at} the _IN point and the OUT point are displayed in a stationary state on the spiral image display unit group 58, and thereafter, FIG. As shown in FIG. 4, the operation of connecting the In point and the Out point, that is, dragging and dropping the image of the frame FR _OUT set to the Out point so as to overlap the image of the frame FR _IN set to the In point. Let the operator perform editing operation input.

As a result, as shown in FIG. 19C, from the frame FR _IN set at the In point in the video / audio portion of the entire clip to the frame FR _OUT set immediately before the frame FR _OUT set at the Out point. The video / audio part is deleted, and edit data indicating that the frame immediately before the frame FR _IN set at the _IN point and the frame FR _OUT set at the OUT point should be connected is created. Based on this edit data The edited video / audio video obtained by the processing is displayed on the display unit 55 in the same display form as the clip. That is, the spiral video display unit group 58A that displays the playback video for each predetermined playback time difference ΔT of the edited video and audio after the cut editing whose display form has been changed as described above is displayed on each of the video display units W57 _{1 to} W57 _n. 55 is displayed.

When it is desired to leave the video / audio part cut out by such processing without deleting it as a separate clip, it is displayed by first clicking “Setting” displayed on the toolbar 59 of the display unit 55. “Separation mode” is selected and set from the context menu, and thereafter, as shown in FIGS. 20A and 20B, the same as the cut editing described above with reference to FIGS. 19A and 19B. Editing operation to set each frame FR _IN and FR _OUT of the _IN point and OUT point, and then overlay the image of the frame FR _OUT set to the _OUT point on the frame FR _IN set to the _IN point by drag and drop Make input.

As a result, as shown in FIG. 20C, the frame immediately before the frame FR _IN set at the In point and the frame FR _OUT set at the Out point of the entire video / audio portion of the clip. Then, editing data is created to indicate that the video / audio portion from the frame FR _IN set at the In point to the frame immediately before the frame FR _OUT set at the Out point should be a separate clip.

At this time, the display unit 55 displays the edited video and audio video in which the display form after cut editing obtained by the processing based on the edited data is changed, and the out point from the in-point frame FR _IN generated as another clip. The video of the audio / video portion up to the frame immediately before the frame FR _OUT is displayed on the display unit 55 in the same display form as the clip.

That is, the spiral video display unit group 58A in which the playback video for each predetermined playback time difference ΔT of the edited video and audio obtained by the cut editing is displayed on each of the video display units 57 _{1 to} 57 _n and the video set as the separate clip. The spiral video display unit group 58 </ b> B in which the playback video for each predetermined playback time difference ΔT of the audio part is displayed on the plurality of video display units 57 is displayed on the display unit 55.

  In this case, the microprocessor 3 displays the spiral image display unit group 58A and the spiral image display according to the display information (including the display size and display position) in the spiral image display unit group 58A and the spiral image display unit group 58B. The group 58 </ b> B is deformed and displayed in a slightly compressed state in the vertical direction so as to fit in the display unit 55.

As described above, when the editing operation input as described above is performed by the keyboard 39 or the mouse 38, the microprocessor 3 acquires the editing information corresponding to the editing operation input via the PCI bus 15, and the contents of the editing information. Accordingly, the display form of each of the video display units W57 _{1 to} W57 _n can be changed.

Incidentally, when an edit position setting cancel operation for canceling an edit operation input to be set as an edit position is input, the microprocessor 3 cancels all display video data for each of the video display units W57 _{1 to} W57 _n from the stopped state. Then, while providing a time difference in the state of a moving image between the plurality of video display units W57 _{1 to} W57 _n , the plurality of video display units W57 _{1 to} W57 _n are again linked and displayed as a spiral video display unit group 58 linked in display order. It is made to let you.

When it is desired to separate one clip into two clips with a desired frame as a boundary, the “separation mode” is set as described above, and then the desired frame FR _IN as the editing position is set to the in point as described above. Set. As a result, as shown in FIG. 21A, the microprocessor 3 colors the frame portions of the video display portions W57 _{1 to} W57 _n on which the image of the frame FR _IN is displayed in a predetermined color (for example, yellow), As a result, the frame can be set as a frame to be used as a boundary for separation. Then, for example, an editing operation input is performed so as to press both the “Alt” key and the “Space” key of the keyboard 39 thereafter.

As a result, as shown in FIG. 21B, edit data to be divided with the frame FR _IN set at the IN point of the clip as a boundary is generated, and the clip of the clip obtained by the processing based on the edit data is generated. The video images of the first and second video / audio parts are displayed on the display unit 55 in the same display form as the clip.

  That is, the spiral video display unit group 58A in which the playback video for each predetermined playback time difference ΔT in the first half of the video and audio part in the clip is displayed on the plurality of video display units W57, and the second half of the video and audio part in the clip The spiral video display unit group 58B in which the playback video for each predetermined playback time difference ΔT is displayed on each of the video display units W57 is displayed on the display unit 55.

  Also in this case, the microprocessor 3 and the spiral image display unit group 58A and the spiral image display unit group 58A and the spiral image display unit group 58B are displayed in accordance with the display information (including the display size and display position). The video display unit group 58B is deformed and displayed in a slightly compressed state in the vertical direction so as to be accommodated in the display unit 55.

  On the other hand, on the editing screen 50, a spiral video display unit group 58 of two clips is displayed on the display unit 55, and the so-called assemble editing for connecting the two clips using these, or at a desired position of one clip. Cut editing such as insert editing for inserting a part of the other clip can also be performed.

  In practice, when performing assembly editing, the above-described operation displays the spiral video display unit group 58 (58A, 58B) associated with the two clips used for assembly editing, as shown in FIG. Each is displayed in the part 55.

  Subsequently, of the two spiral image display unit groups 58 (58A, 58B) displayed on the display unit 55, the spiral image display unit group 58A associated with one of the first clips to be used as a background is moused. After selection by operation, the in point is set as the first editing position in the first clip in the same manner as in cut editing.

  Thereafter, of the two spiral image display unit groups 58 (58A, 58B) displayed on the display unit 55, the spiral image display unit group 58B associated with the other second clip to be set on the insertion side. After selecting by operating the mouse, an in point and an out point as necessary are set as the second editing position in the second clip in the same manner as in cut editing.

Then, as shown in FIG. 22B, the In point of the first clip set as the first editing position and the In point of the second clip set as the second editing position are connected. Matching, that is, the image of the frame FR _IN-2 set at the in point of the second clip displayed on the spiral video display unit group 58B associated with the second clip is associated with the first clip. An edit connection operation is performed by dragging and dropping so as to overlap the image of the frame FR _IN-1 set at the in point of the first clip displayed in the spiral image display unit group 58A.

As a result, as shown in FIG. 22C, the microprocessor 3 includes a frame set to the in point from the first frame of the first clip, a video / audio portion from one frame before FR _IN-1 , and From the first frame of the second clip to the frame FR _IN-2 set to the in point (when the out point is set in the second clip, the frame FR _IN-2 set to the in point is changed to the out point. Edit data to the effect that the video and audio portions (up to the set frame) are to be combined, and the first and second clips of the edited video and audio obtained by the processing based on this edit data Are displayed on the display unit 55 in the same display form as the video. That is, the microprocessor 3 causes the GPU 4 to display on the display unit 55 a spiral video display unit group 58 </ b> C in which video for each predetermined playback time difference ΔT of the edited video / audio is displayed on each video display unit 57.

  On the other hand, when insert editing is performed, the spiral video display group 58 (58A, 58A, 2A) of two clips to be used for insert editing as shown in FIG. 58B) are displayed in the display unit 55, respectively.

  Subsequently, of these two spiral video display unit groups 58 (58A and 58B) displayed on the display unit 55, the video display group 58A associated with the first clip to be used as the ground is operated by the mouse. After the selection, the In point and Out point are set as the first editing position in the first clip in the same manner as in cut editing, while in the other second clip as well. In-point and out-point are respectively set as the second editing positions.

Then, as shown in FIG. 23 (B), an operation for connecting the In point of the first clip and the In point of the second clip, that is, a spiral video display unit group associated with the second clip. The image of the _IN- point frame FR _IN-2 displayed on 58B is overlaid on the image of the _IN- point frame FR _IN-1 displayed on the spiral video display unit group 58A associated with the first clip. Drag and drop to edit connection operation.

As a result, as shown in FIG. 23C, the microprocessor 3 includes the audio / video portion from the first frame in the first clip to the frame immediately before the frame FR _IN-1 set at the in point. The audio / video part from the frame FR _IN-2 set at the In point of the second clip to the frame FR _OUT-2 set at the Out point and one frame before the frame FR _OUT-2 and the Out point of the first clip are set. Edit data indicating that the video / audio portion from the frame FR _OUT-1 to the last frame of the first clip should be sequentially connected is generated, and the edited video / audio video obtained by the processing based on the edit data is generated. Are displayed on the display unit 55 in the same display form as the images of the first and second clips at that time. That is, the spiral video display unit group 58 </ b> C that displays the video for each predetermined playback time difference ΔT of the edited video and audio on each video display unit W <b> 57 is displayed on the display unit 55.

  On the other hand, when it is desired to set a video special effect at the switching portion between the first and second clips with respect to the edited video / audio of the assemble editing or the insert editing obtained as described above, as shown in FIG. In the same manner as in cut editing, an in point is set as an editing position near the end on the first clip side in the edited video and audio, and an out point is set near the beginning on the second clip side.

Next, as shown in FIG. 24B, the effect icon 13 associated with the desired video special effect is selected from the effect list displayed in the effect information display section 54 of the editing screen 50 (FIG. 2). For example, this is pasted on the image of any frame between the frame FR _IN set as the in point of the corresponding spiral video display unit group 58C and the frame FR _OUT set as the out point by drag and drop. Like that.

  As a result, the microprocessor 3 performs video special effect processing corresponding to the pasted effect icon 13 at the switching portion between the first and second clips as shown in FIG. Edit data to be applied is generated, and the edited video / audio video obtained by the processing based on the edited data is displayed on the display unit 55 in the same display form as the original edited video / audio video.

That is, the spiral video display unit group 58 _</ b _> C is displayed on the display unit 55. The spiral video display unit group 58 _</ b _> C displays the video for each predetermined reproduction time difference ΔT of the edited video / audio subjected to the video special effect processing on the video display units W57 _{1 to} W57 _n. In other words, at the switching portion of the first and second clips, the display contents overlap each other and gradually fade out so that one does not disturb the other forever. .

  In this way, the operator uses the display unit 55 of the editing screen 50 to sequentially connect desired clips and edit data for obtaining edited video audio obtained by setting a desired video special effect at a desired position. Can be created.

  In addition, after creating edit data as described above, when it is desired to save these as an edit list, an “edit list” is selected from a context menu (not shown) displayed by clicking “File” on the toolbar 59 of the display unit 55. Select “Save”. As a result, each edit data created at that time is filed as one edit list and registered in the edit list database in the hard disk device 7 (FIG. 1).

(2-5) Microprocessor Processing for Various Operations Here, the microprocessor 3 changes the above-described various display video and the like processing for the operation input to a control program stored in the hard disk device 7 (FIG. 1). Based on the editing operation response processing procedure RT3 shown in FIG.

  That is, when the microprocessor 3 finishes displaying the spiral video display unit group 58 of the clip designated in accordance with the playback processing procedure RT1 shown in FIG. 9 on the display unit 55 of the editing screen 50, the above-described display change process using FIG. This editing operation response processing procedure RT3 is started in parallel with the procedure RT1, and in the subsequent step SP21, it is determined whether or not the operator has clicked on the spiral video display unit group 58 displayed on the display unit 55 at that time. .

When the microprocessor 3 obtains a negative result in step SP21, the microprocessor 3 proceeds to step SP22 and determines whether or not a predetermined editing operation input has been performed by the operator. Note that the “edit operation input” here is an image of the frame FR _OUT set to the out point after the in point and out point are set in the same clip, as described above with reference to FIGS. In the operation of dragging and dropping on the image of the frame FR _IN set to the In point, or as described above with reference to FIG. 21, the In point is set to a certain clip while the “separation mode” is set. Thereafter, an operation in which both the “Alt” key and the “Space” key of the keyboard 39 are pressed, and as described above with reference to FIGS. 22 and 23, spiral image displays corresponding to the first and second clips, respectively. In a state where the group 58 is displayed on the display unit 55, an in point (and an out point) is set for each of the first and second clips, and then the second clip is set. Dragging and dropping on to the frame FR _IN-1 of the image of the frame FR _IN-2 of the image set to the in-point of the flop first clip-in point, or 24 as described above for, effect icon 13 is an operation of dragging and dropping 13 onto the spiral image display unit group 58.

  If the microprocessor 3 obtains a negative result in step SP22, it returns to step SP21, and thereafter waits for a positive result in step SP21 or step SP22 while repeating the loop of step SP21-SP22-SP21.

When the microprocessor 3 eventually obtains an affirmative result in step SP21 due to the click operation performed on the spiral image display unit group 58 displayed in the display unit 55 by the operator, the microprocessor 3 moves to the next step SP23, The frame that is clicked is set as an In point or an Out point depending on the situation, and by controlling the GPU 4 together with this, the frame set as the In point or Out point in the spiral video display unit group 58 The frame portions of the video display portions W57 _{1 to} W57 _n displaying the images of FR _IN and FR _OUT (FR _IN-1 , FR _IN-2 , FR _OUT-1 , FR _OUT-2 ) are colored in a predetermined color. .

  Further, when the microprocessor 3 obtains a positive result in step SP22 by inputting a predetermined editing operation after the in point (and out point) is set in the clip, the microprocessor 3 moves to the next step SP24, Edit data for obtaining an edit result corresponding to the edit operation is generated.

  Then, the microprocessor 3 moves to step SP25 and controls the GPU 4 to control the video based on the editing data generated in this way, that is, FIG. 19 (C), FIG. 20 (C), FIG. 21 (B). ), The spiral video display unit group 58 of the edited video and audio in which the display form after editing as described above with reference to FIGS. 22C to 24C is displayed in the display unit 55, and then step SP26. Then, the editing operation response processing procedure RT3 is terminated.

  In this manner, the microprocessor 3 generates editing data corresponding to the editing operation input input by the operator, and displays the video of the editing result whose display form is changed according to the editing operation input on the display unit 55. Let

(3) Viewpoint Change Method for Spiral Video Display Unit Group (3-1) Display Form of Spiral Video Display Unit Group The microprocessor 3 of the editing device 2 displays the spiral video display unit group displayed on the display unit 55 of the editing screen 50. 58, the viewpoint from which the visual check is performed is shown to the operator, or the viewpoint is arbitrarily changed, and the spiral image display unit group 58 in a display form corresponding to the changed viewpoint is displayed. It can be displayed on the section 55.

Here, the spiral image display unit group 58 is generated as a result of rendering in 3D space on its internal memory by the GPU 4 according to predetermined 3D image generation software. As shown in FIG. A plurality of video display units W57 _{1 to} W57 _n are sequentially arranged so as to wrap around the virtual time axis TP virtually generated, and a circle formed by the video display units W57 _{1 to} W57 _n Has a spiral structure in which the diameter gradually increases with the progress of the virtual time axis TP.

That is, as a display form set as a default of the spiral video display unit group 58 to be displayed on the display unit 55, assuming that the operator's viewpoint EP1 is at one end on the traveling direction side in the virtual time axis TP, FIG. As shown in FIG. 3, the spiral image display unit group 58 in which the circle formed by the plurality of image display units W57 _{1 to} W57 _n gradually increases as it approaches from the back side to the front side of the screen is three-dimensional. Will be displayed.

For this reason, when the spiral image display unit group 58 is displayed on the display unit 55, the microprocessor 3 of the editing device 2 has a diameter of a circle formed by each of the image display units W57 _{1 to} W57 _n around the virtual time axis TP. Since the operator can visually confirm from the front a spiral structure that gradually increases as the virtual time axis TP progresses, all the video display portions W57 _{1 to} W57 _n are not hidden as any one. Everything can be presented.

Here, the spiral video display unit group 58 includes the video display unit W57 ₁ , the video display unit W57 ₂ , the video display unit W57 ₃ ,..., The video display unit W57 _m , and the video display unit W57 _{n in} the order of the virtual time axis TP. Appearing sequentially in the surroundings, the reproduction images of the video display units W57 _{1 to} W57 _n are started to be reproduced in the order of appearance.

Therefore, at the timing after a predetermined time from the reproduction start of the video display unit W57 ₁ ~W57 _n in spiral image display unit 58, the screen of the display unit 55, the video display unit W57 ₁ present in the deepest side of the helix The most recently reproduced video is displayed in time, and the video display unit W57 _n existing on the most front side of the spiral displays the most recent reproduced video in time.

That shows the progress most willing of the tip-side image display section W57 in _n temporally most of the past are reproduced image of the virtual time axis TP, the image display unit of the rear end side close to the starting point of the virtual time axis TP W57 displays _one future video playback in advanced most temporally, the traveling direction and proceeds degree of reproduction processing in each image display unit W57 ₁ ～W57 _n virtual time axis TP is opposite.

  As shown in FIG. 27, in the editing device 2, the viewpoint when the spiral video display unit group 58 is displayed to the operator while the spiral video display unit group 58 is displayed on the display unit 55 of the editing screen 50. The visual line direction is visually represented by the sub 3D space image IM1 and the character CA1 displayed in the display unit 55. Here, in the sub 3D space image IM1, the sub spiral image display unit group 58S and the sub time axis STP1 which are reduced in the same display form as the spiral image display unit group 58 are displayed.

  That is, the 3D space coordinates of the sub spiral image display unit group 58S displayed in the sub 3D space image IM1, and the 3D space coordinates of the spiral image display unit group 58 generated in the 3D space on the internal memory of the GPU 4 Are basically the same coordinate system because they are only in a 3D space and are in a scaling relationship.

  In practice, when the microprocessor 3 of the editing apparatus 2 recognizes that a desired clip has been selected by the operator from the clip list display unit 51 of the editing screen 50 and then dragged and dropped to the display unit 55, the GPU 4 A sub 3D space that displays a spiral video display unit group 58 corresponding to the clip and includes a sub-spiral video display unit group 58S corresponding to the spiral video display unit group 58 and a sub-time axis STP1 corresponding to the virtual time axis TP by the GPU 4. An image IM1 is generated and displayed together with the spiral image display unit group 58.

  Further, when the character CA1 near the sub 3D space image IM1 is selected via the cursor 71 of the display unit 55 and a drag operation is performed, the microprocessor 3 of the editing device 2 moves the character CA1 in the direction of the drag operation. View coordinate data when the position in the sub 3D space image IM1 of the character CA1 after the movement is used as the view point coordinates (confirmation position) is generated, and can be viewed from the confirmation position at the angle of the line-of-sight direction arrow AL1. A spiral video display unit group 58 having a display form corresponding to the sub-sub spiral video display unit group 58S is displayed on the display unit 55 by the GPU 4.

  As described above, in the editing apparatus 2, the viewpoint of the spiral video display unit group 58 is displayed as an image to the operator by the confirmation position (viewpoint coordinates) of the character CA1 displayed near the sub 3D space image IM1 with respect to the sub spiral video display unit group 58S. By visually presenting to the operator the visual direction arrow AL1 with respect to the sub-spiral video display unit group 58S from the viewpoint as the visual direction of the character CA1, the spiral video display unit group of the display unit 55 is displayed. The operator can intuitively recognize the viewpoint and line-of-sight direction with respect to 58.

  By the way, when the microprocessor 3 of the editing apparatus 2 recognizes that a plurality of clips are dragged and dropped on the display unit 55 after a plurality of desired clips are selected from the clip list display unit 51 of the editing screen 50 by the operator. , The GPU 4 displays a plurality of spiral video display unit groups 58 corresponding to the plurality of clips on the display unit 55 and, as shown in FIG. A sub 3D space image IM1 including the spiral image display unit group 58S2, the sub time axis STP2, the sub spiral image display unit group 58S3, and the sub time axis STP3 can be generated and displayed.

In this case, when the time code information is added to the reproduced video to be displayed on each of the video display units W57 _{1 to} W57 _n constituting the plurality of spiral video display unit groups 58, the microprocessor 3 of the editing device 2 The two sub-spiral video display unit groups 58S2 and the sub-helical video display unit group 58S3 are displayed in a synchronized state based on the time code information.

  Here, when the lengths of the sub time axis STP2 in the sub spiral image display unit group 58S2 and the sub time axis STP3 in the sub spiral image display unit group 58S3 are different from each other, the microprocessor 3 uses the longer sub time axis STP2 ( Alternatively, the shorter sub time axis STP3 (or sub time axis STP2) is displayed in accordance with the sub time axis STP3).

In this way editing apparatus 2, and the entire sub-image display unit WS57 ₁ ~WS57 _n sub spiral image display area group 58S2, the sub-spiral image display unit of the display area group 58S3 WS57 ₁ ~WS57 _n total and sub 3D space image The image can be displayed in a state of being housed inside the IM1, and thus the operator can visually check the entire images of the sub spiral image display unit group 58S2 and the sub spiral image display unit group 58S3.

  As shown in FIG. 29, the microprocessor 3 of the editing device 2 displays the sub 3D space image IM1 including the plurality of sub spiral image display unit groups 58S2 and the sub spiral image display unit group 58S3 on the display unit 55. When a drag operation is performed by the operator, the positions of the sub spiral image display unit group 58S2 and the sub spiral image display unit group 58S3 can be moved, and the sub spiral image display unit group 58S2 and the sub spiral image display unit group 58S2 after the movement can be moved. The arrangement of the plurality of spiral image display unit groups 58 displayed on the display unit 55 in conjunction with the arrangement of the spiral image display unit group 58S3 can also be moved and displayed.

  At this time, the microprocessor 3 of the editing device 2 is arranged so that the sub-spiral video display unit group 58S2 and the sub-helical video display unit group 58S3 are arranged in the longitudinal direction of the sub time axes STP2 and STP3 (future). Direction or past direction).

  Specifically, as shown in FIG. 30A, the microprocessor 3 of the editing device 2 displays the front viewpoint in which the character CA1 is arranged in front of the sub 3D space image IM1 with respect to the spiral video display unit group 58. When the display mode is set to the default setting, in accordance with the drag operation by the cursor 71, the left upper oblique viewpoint in which the character CA1 is arranged obliquely on the left upper side of the sub 3D space image IM1 as shown in FIG. Or change to a diagonally lower right viewpoint in which the character CA1 is arranged diagonally to the lower right of the sub 3D space image IM1 as shown in FIG. 30C, or change to the sub 3D space as shown in FIG. Change to the right lateral viewpoint in which the character CA1 is arranged on the right side of the image IM1, or true that the character CA1 is arranged directly on the sub 3D space image IM1 as shown in FIG. To change the view point, it is adapted to be changed within viewpoint inside the character CA1 sub 3D space image IM1 is arranged as shown in FIG. 30 (F).

  Thereby, the microprocessor 3 of the editing apparatus 2 displays the spiral video display unit group 58 on the display unit 55 of the editing screen 50, and the 3D space coordinates of the sub 3D space image IM1 and the viewpoint coordinates of the character CA1 moved by the cursor 71. The display mode of the spiral video display unit group 58 is changed and displayed by the GPU 4 in accordance with the new viewpoint and line-of-sight direction after the change determined by the relative positional relationship.

  In other words, the microprocessor 3 can display the viewpoint and line-of-sight direction when the viewpoint coordinates of the character CA1 are moved and the viewpoint of the display unit 55 with respect to the spiral video display unit group 58 in a coordinated manner. ing.

  In the microprocessor 3 of the editing apparatus 2, the viewpoint can be changed by key input instead of the drag operation by the cursor 71 in changing the viewpoint. After the character CA1 is selected by the cursor 71, For example, the character CA1 is moved upward in response to the pressing operation on the “W” key on the keyboard 39, the character CA1 is moved in the left direction in response to the pressing operation on the “A” key, and the “X” key is pressed. The character CA1 can be moved downward in response to the pressing operation, and the character CA1 can be moved in the right direction in response to the pressing operation on the “D” key.

  As described above, the microprocessor 3 of the editing device 2 is in the spatial coordinate lock mode in which the viewpoint and its line-of-sight direction are changed by moving the viewpoint coordinates of the character CA1 while fixing (locking) the spatial coordinates of the sub 3D spatial image IM1. Although the viewpoint changing method is used, for example, the mode is switched in response to the pressing operation on the “F1” key on the keyboard 39, and the viewpoint coordinates of the character CA1 are fixed (locked) while being reversed (contained) in contrast to the spatial coordinate lock mode. It is also possible to change to the viewpoint change method in the viewpoint coordinate lock mode in which the viewpoint and the line-of-sight direction are changed by rotating the 3D space image IM1 and moving the space coordinates.

  Incidentally, the microprocessor 3 of the editing device 2 switches the mode in accordance with, for example, a pressing operation on the “F2” key on the keyboard 39, and the 3D space coordinates of the sub 3D space image IM1 and the viewpoint of the character CA1 as shown in FIG. While maintaining the relative positional relationship with the coordinates, the sub 3D space image IM1 and the character CA1 can be rotated together by a drag operation with the cursor 71, whereby the sub 3D space image IM1 and the character CA1 can be rotated for the operator. The angle can be changed to an angle that can be easily visually confirmed.

  By the way, the microprocessor 3 of the editing device 2 changes to the internal viewpoint in which the character CA1 is arranged inside the sub 3D space image IM1, as shown in FIG. In this case, as shown in FIG. 32, the sub 3D space image IM2 can be generated by automatically enlarging the sub 3D space image IM1 to a predetermined magnification.

  In this case, for the enlargement ratio when the sub 3D space image IM1 is enlarged to the sub 3D space image IM2, the sub 3D space images IM1 and IM2 are only sub-displays. It is limited to a certain magnification or less so as not to disturb the group 58.

(3-2) Display Processing Procedure for Spiral Video Display Unit Group According to Change of Viewpoint Subsequently, the viewpoint and its line-of-sight direction change depending on the relative positional relationship between the sub 3D space image IM1 and the character CA1. A display processing procedure for changing and displaying the display form of the spiral video display unit group 58 of the display unit 55 will be described next.

As shown in FIG. 33, the microprocessor 3 of the editing apparatus 2 enters from the start step of the routine RT4, moves to the next step SP41, and receives control signals (for example, spirals) input via the mouse 38, the keyboard 39 or the operation controller 37. The playback speed v of the playback video to be displayed on each of the video display units W57 _{1 to} W57 _n of the video display unit group 58, the playback time difference ΔT, the 3D spatial coordinates of the sub 3D spatial image IM1 after movement according to the drag operation of the operator, The viewpoint coordinates of the character CA1) are taken in via the PCI bus 15 and the south bridge 6 and recognized by the main CPU core 3M, and then the process proceeds to the next step SP42.

In step SP42, when the control signal recognized in step SP41 is a command for changing and displaying the display form of the spiral image display unit group 58, the microprocessor 3 of the editing apparatus 2 displays the spiral image in a new display form. A drawing loop of a clip to be displayed on each of the video display units W57 _{1 to} W57 _n of the display unit group 58 is set by the main CPU core 3M, and the process proceeds to the next step SP43.

  In step SP43, the microprocessor 3 of the editing device 2 performs the display size of the playback video of the clip set in step SP42, the three-dimensional space coordinate calculation in the 3D space of each frame having the display size, the viewpoint coordinate calculation of the character CA1, and the like. Various physical operation processes are executed by the main CPU core 3M, and the process proceeds to the next step SP44.

  In step SP44, the microprocessor 3 of the editing apparatus 2 assigns, by the main CPU core 3M, the sub CPU cores 3SA to 3SH that are responsible for decoding the playback video of the clips constituting the drawing loop by the main CPU core 3M. Control goes to the next step SP45.

  In step SP45, the microprocessor 3 of the editing device 2 reads the frame of the clip necessary for outputting the playback video from the storage device 22, distributes it to the sub CPU cores 3SA to 3SH assigned in step SP44, and proceeds to the next step SP46. .

  In step SP46, the microprocessor 3 of the editing apparatus 2 simultaneously decodes the frames distributed in step SP45 by the eight sub CPU cores 3SA to 3SH serving as decoders, and proceeds to the next step SP47. .

  In step SP47, the microprocessor 3 of the editing device 2 uses the main CPU core 3M to determine how the reproduced video decoded in step SP46 should be arranged on the screen (3D space) of the display unit 55. The 3D space coordinates of the 3D space image IM1 and the viewpoint coordinates of the character CA1 are transferred as high-speed data to the GPU 4 together with the playback video as 3D display position information of the playback video, and the process proceeds to the next step SP48.

In step SP48, the microprocessor 3 of the editing apparatus 2 uses the GPU 4 to determine the viewpoint and line of sight set at that time based on the 3D display position information including the 3D spatial coordinates of the sub 3D spatial image IM1 and the viewpoint coordinates of the character CA1. The spiral image display unit group 58 having a display form corresponding to the direction is converted into two-dimensional coordinate display position information for display on the display unit 55, and the reproduction image transferred from the main CPU core 3M is converted into the two-dimensional coordinate display. A spiral video display unit group 58 having a display form corresponding to a new viewpoint and line-of-sight direction by being pasted and displayed at predetermined positions of the video display units W57 _{1 to} W57 _n according to the display size and display position based on the display position information. Is drawn, and the process proceeds to the next step SP49 to end the process.

As described above, in the editing apparatus 2, the eight sub CPU cores 3SA to 3SH in the microprocessor 3 each play a role as a decoder of a reproduced video displayed on each of the video display units W57 _{1 to} W57 _n and decode in parallel. The decoded playback video can be transferred at high speed to the GPU 4 at a maximum transfer rate of, for example, 30 [Gbyte / sec] by the bus 10 having a large bandwidth. The video can be decoded and transferred to the GPU 4 in a short time.

Thereby, in the GPU 4, when the spiral video display unit group 58 generated in the 3D space on the internal memory is changed to a new viewpoint and line-of-sight direction and displayed on the display unit 55, the 3D space coordinates and characters of the sub 3D space image IM1 are displayed. Since the three-dimensional display position information composed of the viewpoint coordinates of CA1 is converted into the display position information of the two-dimensional coordinates of the display unit 55, the reproduced video of each of the video display units W57 _{1 to} W57 _n can be displayed using this. As a spiral video display unit group 58 that cooperates in the spiral display order while providing a reproduction time difference ΔT in the state of moving images among the video display units W57 _{1 to} W57 _n , the display unit 55 is smoothly and interlockedly displayed. In addition, the viewpoint and line-of-sight direction can be freely controlled by coordinate conversion processing in 3D space.

(3-3) Spiral Video Display Unit Group After Change of Viewpoint The microprocessor 3 of the editing device 2 performs a character for the sub 3D space image IM1 displayed on the display unit 55 of the editing screen 50 as shown in FIG. In response to the position of CA1 being moved by the cursor 71, the spiral image display unit group 58 (FIG. 27) displayed on the display unit 55 can be changed to a display form from below and displayed. .

  In this case, the spiral video display unit group 58 illustrated in FIG. 27 has a display form when the viewpoint of the character CA1 and the line-of-sight direction thereof are arranged facing the sub time axis STP1 of the sub 3D space image IM1. On the other hand, in the spiral image display unit group 58 shown in FIG. 34, the viewpoint of the character CA1 is changed obliquely downward with respect to the sub time axis STP1 of the sub 3D space image IM1, and the viewing direction thereof. The display form is when viewed at the angle.

  That is, in the editing apparatus 2, the viewpoint of the character CA1 with respect to the sub spiral image display unit group 58 of the sub 3D space image IM1 and the direction of the line of sight, and the spiral image display unit group displayed on the actual display unit 55 with the angle changed. The viewpoint with respect to 58 and the line-of-sight direction are linked.

  In this case, since the operator can visually confirm the entire image of the spiral image display unit group 58 (FIG. 34) displayed on the display unit 55, the spiral image display displayed at an angle from the front of the virtual time axis TP. The length of the entire clip, which was difficult to recognize in the group 58 (FIG. 27), can be confirmed as an image, or can be timed via the time code notification frames TC1 to TC4 at important points in the spiral video display group 58. It is designed to recognize the sequence flow.

Further, in the editing device 2, since the scenes of the playback video displayed by the video display units W57 _{1 to} W57 _n are all different, a plurality of video display units W57 that make up the spiral video display unit group 58 are the scenes desired by the operator. _{1 ~W57 n} or the more likely to have been displayed, is configured so as to be able to facilitate also editing together may facilitation work looking image of the operator.

Further editing apparatus 2, because it is the same playback image for each image display unit W57 ₁ ~W57 _n shifted by a predetermined reproduction time difference [Delta] T, the scene the operator desires the first image display unit W57 ₁ even if you miss, the image display section W57 ₁ or later of the video display unit _{W57 2, W57 3, W57 4} , also becomes possible to sequentially display the scene since after the reproduction time difference ΔT to _..., for once lost scene In addition, it is possible to facilitate the image search operation and the editing operation by allowing the operator to view the image again without causing the operator to perform rewinding reproduction.

  By the way, the microprocessor 3 of the editing apparatus 2 further moves the position of the character CA1 with respect to the sub 3D space image IM1 displayed on the display unit 55 of the editing screen 50 by using the cursor 71 as shown in FIG. Accordingly, the display can be changed to the display form when the viewpoint with respect to the spiral image display unit group 58 (FIG. 34) is located further downward.

  Also in this case, in the spiral image display unit group 58, the viewpoint of the character CA1 and the direction of the line of sight of the character CA1 from the diagonally lower direction with respect to the sub time axis STP1 of the sub 3D spatial image IM1 are changed and viewed at that angle. The display format is as follows.

  Further, as shown in FIG. 36, the microprocessor 3 of the editing apparatus 2 further moves the position of the character CA1 with respect to the sub 3D space image IM1 displayed on the display unit 55 of the editing screen 50 by using the cursor 71, so that the sub 3D space image IM1. A position substantially parallel to the virtual time axis TP in response to the movement of the viewpoint of the character CA1 to a position that is near the sub time axis STP1 and has a line-of-sight direction substantially parallel to the sub time axis STP1 The spiral image display unit group 58 (FIG. 34) can be changed and displayed in a display form corresponding to the viewpoint and line-of-sight direction.

  Incidentally, the microprocessor 3 of the editing apparatus 2 changes the viewpoint and the line-of-sight direction by moving the viewpoint coordinates of the character CA1 while fixing (locking) the spatial coordinates of the sub 3D space image IM1 as described above. Instead of using the lock mode viewpoint changing method, as shown in FIG. 37, by rotating the sub 3D space image IM1 with the viewpoint coordinates of the character CA1 fixed (locked), the character CA1 and the sub 3D space image IM1 It is also possible to change to the same display form as the spiral image display unit group 58 shown in FIG. 36 by using the viewpoint change method of the viewpoint coordinate lock mode in which the relative position relationship is changed to change the viewpoint and the line-of-sight direction. it can.

  That is, the spiral image display unit group 58 (FIGS. 36 and 37) displayed on the display unit 55 has a display form in which the image is viewed from the same angle, but the sub 3D space image IM1 in FIG. 36 and the sub 3D in FIG. The spatial image IM1 differs only in the appearance of the sub 3D spatial image IM1 to the operator, and is not essentially different.

Further, as shown in FIG. 38, the microprocessor 3 of the editing apparatus 2 further moves the position of the character CA1 with respect to the sub 3D space image IM1 displayed on the display unit 55 of the editing screen 50 by using the cursor 71, so that the sub 3D space image IM1. in a vicinity of the sub time axis STP1, having substantially parallel viewing direction and its sub time axis STP1, the character CA1 in the vicinity of the image display section W57 _n located on the nearest side of the sub-spiral display area group 58S viewpoint in response to moving the substantially very close viewpoint of the character CA1 foremost side of the image display section W57 _n of spiral image display area group 58 from a position parallel to the virtual time axis TP, from the angle The spiral video display unit group 58 can be changed to the display form and displayed.

In this case, the microprocessor 3 of the editing apparatus 2, the image display unit _W57 n of spiral image display area group 58 on the nearest side of the screen in the display unit _{55, 57 m, W57 l,} ......, has been greatly expanded Therefore, for the reproduced video displayed on these enlarged video display units W57 _n, 57 _m, W57 _l, ..., The frame update speed of the moving image is set, and the other video display units W57 are displayed. The frame update speed is significantly higher than the frame update speed, so that the playback video displayed on the video display units W57 _n, 57 _m, W57 _l, ... Can be presented to the operator as a smooth moving image. Has been made.

However, the microprocessor 3 of the editing device 2 does not change the frame update rate of the moving image for the reproduced video displayed on the enlarged video display units W57 _n, 57 _m, W57 _l, W57 _k ,. While the frame update speed for the portion W57 is significantly increased, the resolution of the frame is reduced by a predetermined level, whereby the video display portions W57 _n, 57 _m, W57 _l, W57 _k, ... Thus, it is possible to reduce the processing load at the time of displaying the playback video and to follow the video playback timing.

Microprocessor 3 of the editing apparatus 2 on the other hand, for the enlarged image display section W57 not _k previous portion, configured so as to output without increasing the frame update rate in a plurality of frames constituting the moving picture cage, thereby the determination difficult image display unit W57 _k previous portion of the image quality for the operator have been made so as to reduce the processing load of GPU4 while dropping the picture quality of the reproduced image.

In this way, the microprocessor 3 of the editing device 2 moves the position of the character CA1 with respect to the sub 3D space image IM1 by the cursor 71, and the video display unit W57n to be noted that is positioned closest to the spiral video display unit group 58 _. 57 _m, W57 _l, W57 _k, ... Are enlarged, and at the same time, the frame frequency of the reproduced video displayed on the enlarged video display units W57 _n, 57 _m, W57 _l, W57 _k ,. By displaying the image, the operator can visually confirm a smooth reproduction image, so that the operator's image search operation and editing operation can be further facilitated.

Thus the operator, portions to perform the looking image scrutiny of the image display section W57 ₁ ~W57 _n of spiral image display area group 58, of spiral image display area group 58 move the character CA1 by the cursor 71 It is only necessary to visually check each of the enlarged video display portions W57 _{1 to} W57 _n , so that the operator can easily perform the image search operation.

At this time, the time code notification frames TC1 to TC4 are added to the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 at predetermined intervals. It is designed to make it easier.

(3-4) Two-dimensional image display processing for spiral image display unit group after changing viewpoint While the microprocessor 3 of the editing device 2 is displaying the spiral image display unit group 58 after changing the viewpoint, as shown in FIG. When for example a video display unit W57g among the plurality of image display section _W57 1 _{~W57 n} constituting the spiral display area group 58 recognizes that the selected operation by the cursor 71, is the video display unit W57 _g A total of nine frames of images SG1 to SG9, each consisting of four frames before and after the corresponding frame of the reproduced video displayed at that time, are displayed in the display unit 55 in a two-dimensional array so as not to overlap each other. Has been made. Note that moving images are also displayed in these images SG1 to SG9.

As a result, the editing apparatus 2 has a plurality of video display units W57 _{1 to} W57 _n that constitute the spiral video display unit group 58 (FIG. 39) after the viewpoint change, and thus the video display units W57 are overlapped with each other. ₁ ～W57 also _n video playback even if difficult to check visually to the operator, back and forth around a corresponding frame of the reproduced video image display unit W57 _g of the selected operation by the operator is displayed at the time 4 Since the images SG1 to SG9 corresponding to a total of nine frames can be two-dimensionally arranged and developed, visibility for a plurality of video display portions W57 _{1 to} W57 _n is remarkably improved and an operator's image search operation is performed. Can be made much easier.

Note editing apparatus 2 displays the image SG1~SG9 of nine frames consisting of each four frames before and after around a corresponding frame of the reproduced video image display unit W57 _g of the selected operation by the operator is displayed at the time In addition, it is also possible to collect and display the images SG1 to SG9 for a total of nine frames with a predetermined number of frames centering on the corresponding frame, depending on the setting by the operator.

Incidentally, the editing device 2 reproduces at a predetermined frame rate using only the remaining frames thinned out in units of a predetermined number of frames (for example, 5 frames) when displaying the reproduced video on each of the video display units W57 _{1 to} W57 _n . If you are such playback speed v is set faster, around the corresponding frame of the reproduced video image display unit W57 _g of the selected operation by the cursor 71 as described above is displayed at that time, during the initial A total of nine frames of images SG1 to SG9 consisting of four frames before and after the drawn frame are developed and displayed.

Note that the images SG1 to SG9 for a total of nine frames mentioned in this example are updated according to the playback speed of each of the plurality of video display units W57 _{1 to} W57 _n constituting the spiral video display unit group 58. . Therefore, if the reproduction is stopped, the image becomes a still image for nine frames, and if the video display units W57 _{1 to} W57 _n are reproduced at 1 × speed, the two-dimensionally expanded image is also reproduced at 1 × speed.

(3-5) Cut Editing Processing for Spiral Video Display Unit Group after Change of Viewpoint Further, the microprocessor 3 of the editing apparatus 2 includes a spiral video display unit group 58 on the display unit 55 of the editing screen 50 as shown in FIG. The two spiral image display units 58 and the spiral image display unit group 78 can be assembled and edited in a state where the spiral image display unit group 78 is displayed side by side at the same time.

Actually, the microprocessor 3 of the editing apparatus 2 corresponds to the first clip to be set as the ground side of the two spiral image display unit groups 58 and the spiral image display unit group 78 displayed on the display unit 55. the spiral display area group 58 attached to after selecting with the mouse, to set the in-point corresponding to the first editing position for the desired image display section W57 _h (IN1).

Thereafter, the microprocessor 3 of the editing device 2 selects the second clip to be set on the insertion side among the two spiral image display unit groups 58 and the spiral image display unit group 78 displayed on the display unit 55. the spiral display area group 78 associated after selecting with the mouse, set the out-point in accordance with a second in-point corresponding to the editing position of the (IN2) and required for the desired image display section W77 _i To do.

  After that, the operator sets the in point (IN1) of the spiral video display unit group 58 set as the first editing position and the in point (IN1) of the spiral video display unit group 78 set as the second editing position. IN2), that is, the frame image set at the IN point (IN2) of the second editing position displayed in the spiral image display unit group 78 is displayed in the first image displayed in the spiral image display unit group 58. An edit connection operation is performed by dragging and dropping so as to overlap the frame image set at the IN point (IN1).

As a result, the microprocessor 3 of the editing apparatus 2, until one of the set video display section W77 _i from the image display unit W77 ₁ In point (IN2) of the second spiral display area group 78 is a clip and video and audio portions of the video display unit W57 _h that is set to the in-point (IN1) of the image display section _W57 1 _{~W57 n} to the video display unit W57 _n of the first spiral display area group 58 is a clip Edit data indicating that the video and audio portions of the edited video and audio portions should be connected, and the edited video and audio images obtained by the processing based on the edit data are newly displayed in the same display form as the first and second clips at that time. The spiral image display unit group (not shown) is displayed on the display unit 55.

As described above, in the editing device 2, the video display units W57 _{1 to} W57 _n and the video display units W77 _{1 to} W77 _n are expressed in a spiral shape, so that the video display units W57 _{1 to} W57 _n and the video display units W77 _{1 to} W77 are displayed. _n can be displayed on the screen of the display unit 55 all at once, thus showing the operator the entire image of the first clip and the second clip as the spiral image display unit group 58 and the spiral image display unit group 78. The operator can easily determine two IN points (IN1, IN2) for connecting the spiral image display unit group 58 and the spiral image display unit group 78 at which position.

(4) Scrub Function In addition to this configuration, the editing apparatus 2 is equipped with a scrub function so that an operator can search for an image while checking the contents of a moving image.

  Scrub means reproduction of a frame or field that is a picture designated by an operator. The designation of the picture by the operator is performed, for example, when the operator performs an operation such as moving a knob as a GUI (Graphical User Interface).

In this embodiment, as shown in FIG. 2, the knob is provided as a scrub knob 67 that is movable with respect to the lowermost scrub bar 66 of the display section 55 in the editing screen 50, and a reproduction time difference ΔT ( Among the video display units W57 _{1 to} W57 _n on which the same playback video is displayed across (FIG. 5), for example, the playback position of the playback video displayed on the video display unit W57 _n (FIG. 3) specified in advance is set. It comes to respond.

When the operation of slowly moving the scrub knob 67 is performed, the picture of the frame at the reproduction position corresponding to the position of the scrub knob 67 changes slowly, so that so-called slow reproduction is performed. On the other hand, when an operation of moving the scrub knob 67 quickly is performed, the picture of the frame at the reproduction position corresponding to the position of the scrub knob 67 changes quickly, so that so-called high-speed reproduction is performed. From this, it can be said that scrub is a kind of variable speed (special) reproduction.

  On the other hand, in the editing device 2, as described above, the display form for the spiral image display unit group 58 that displays the relative positional relationship between the 3D virtual space and the character CA 1 (FIG. 27) on the display unit 55. It has been made to change.

  Therefore, in the editing device 2, scrub (shift (special) reproduction) is also performed in a state in which the display mode of the spiral image display unit group 58 (FIGS. 34 to 40) is changed according to the viewpoint desired by the operator. As a result, it is also possible to search for an image after analyzing in detail the overall flow of the playback video displayed on the spiral video display unit group 58 and the flow of a part of interest in the playback video. Become.

However, when scrubbing is performed, the encoded data of the frame at the reproduction position corresponding to the position of the scrub knob 67 is read, the encoded data is decoded into image data, and the image data (corresponding image) is reproduced. It is necessary to display on the spiral image display unit group 58 (image display units W57 _{1 to} W57 _n ) with a time difference ΔT (FIG. 5) therebetween. At the time when so-called non-linear editing has become commonplace, the scrub function as a means for easily checking the content of a video and determining the editing point has become more important. There is a need for a scrubbing function that enables more detailed and smooth confirmation.

Therefore, the editing device 2 (FIG. 1) has a reproduction time difference ΔT (FIG. ₁ ) with respect to the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ) of the display unit 55 in the editing screen 50 (FIG. 2). The playback images respectively displayed at 5) are appropriately displayed in accordance with the moving operation of the scrub knob 67 while maintaining the playback time difference ΔT.

  Such processing is performed by the microprocessor 3 and the GPU 4 (FIG. 1). The processing content includes processing performed before editing via the editing screen 50 (hereinafter referred to as pre-editing processing), Based on the result of the pre-editing process, a process (hereinafter referred to as “displaying”) corresponding to the moving operation of the scrub knob 67 in a state where the reproduction video is maintained with respect to the spiral video display unit group 58 with a reproduction time difference ΔT (FIG. 5). This is called scrubbing). Hereinafter, pre-editing processing and scrub processing by the microprocessor 3 and the GPU 4 will be sequentially described.

(4-1) Pre-editing processing When the pre-editing processing by the microprocessor 3 and the GPU 4 is expressed as a function, as shown in FIG. 41, the configuration includes an image processing unit 1001, an audio processing unit 1002, and an AV file creation unit 1003. , Fy file creation unit 1004 and recording unit / communication unit 1005.

In this image processing unit 1001, the video / audio signal of the clip which is designated by the operation of the operation controller 37, the mouse 38 or the keyboard 39 and is reproduced from the video tape loaded in the video tape recorders 23 ₁ to 23 _n (FIG. 1). On the other hand, the audio signal is input to the audio processing unit 1002.

  The image processing unit 1001 performs necessary processing such as A / D (Analog / Digital) conversion processing and noise removal processing for removing noise on a video signal in a time-series manner in units of frames. Are supplied to the AV file creation unit 1003 and the Fy file creation unit 1004.

  The audio processing unit 1002 performs necessary processing such as A / D (Analog / Digital) conversion processing and noise removal processing for removing noise in time series on the audio signal in units of frames. Are supplied to the AV file creation unit 1003.

  The AV file creation unit 1003 creates an AV file storing the time-series image data from the image processing unit 1001 and the time-series audio data from the audio processing unit 1002, and supplies the AV file to the recording unit / communication unit 1005. .

The Fy file creation unit 1004 obtains a change amount indicating the degree of change of the image data from the image processing unit 1001 in units of frames. In addition, the Fy file creation unit 1004 selects a display type representing a display method when displaying the frame on the video display units W57 _{1 to} W57 _n (FIG. 2) based on the change amount of the frame, if necessary. Determined in units. Further, the Fy file creation unit 1004 receives a file (hereinafter, appropriately referred to as Fy file) in which either or both of the change amount and the display type in units of frames are stored from the image processing unit 1001. It is created as a metafile (metadata file) (one of them) in which metadata of the series of image data is stored, and is supplied to the recording unit / communication unit 1005.

  A picture means a frame or a field. In the following, a frame is adopted as the picture, but a field can be adopted as the picture.

  The recording / communication unit 1005 is supplied from the AV file supplied from the AV file creation unit 1003 and the Fy file supplied from the Fy file creation unit 1004, that is, the Fy file corresponding to the AV file, that is, the AV file creation unit 1003. The Fy file created from the image data stored in the AV file and supplied from the Fy file creation unit 1004 is recorded in the storage device 22 (FIG. 1), for example.

  Here, a certain AV file fav and an Fy file fFy corresponding to the AV file fav are given file names that indicate that the files correspond to each other, for example. That is, the file names of the AV file fav and the Fy file fFy corresponding to the AV file fav are, for example, file names having different extensions only (file names having different extensions and the same part other than the extension). Is done.

  Next, FIG. 42 shows a configuration example of the AV file creation unit 1003 of FIG. In FIG. 42, the AV file creation unit 1003 includes a main line encoder 1011, a proxy encoder 1012, and a file creation unit 1013.

  The main line encoder 1011 includes a main line image encoder 1011V and a main line audio encoder 1011A.

  Image data as main line data is supplied from the image processing unit 1001 (FIG. 41) to the main line image encoder 1011V. The main line image encoder 1011V encodes (encodes) image data as main line data from the image processing unit 1001 using, for example, MPEG2 (for example, MPEG IMX method) and supplies the encoded data to the file creation unit 1013.

  The main line audio encoder 1011A is supplied with audio data from the audio processing unit 1002 (FIG. 41). The main line audio encoder 1011A converts the audio data from the audio processing unit 1002 into, for example, AES (Audio Engineering Society) 3 type audio data as main line data, and supplies the audio data to the file creation unit 1013.

  The proxy encoder 1012 includes a proxy image encoder 1012V and a proxy audio encoder 1012A.

  The proxy image encoder 1012V is supplied with image data as main line data from the image processing unit 1001 (FIG. 41). The proxy image encoder 1012V thins out the number of pixels of the image data as the main line data from the image processing unit 1001, thereby obtaining, for example, image data as proxy data having a resolution (spatial resolution) lower than that of the main line data. Further, the proxy image encoder 1012V encodes the image data as proxy data using, for example, MPEG4 and supplies the encoded image data to the file creation unit 1013.

  The proxy voice encoder 1012A is supplied with voice data from the voice processing unit 1002 (FIG. 41). The proxy speech encoder 1012A converts the speech data from the speech processing unit 1002 into, for example, ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) G.711 A-Law format speech data as a proxy data file. This is supplied to the creation unit 1013.

  The file creation unit 1013 includes image data (encoded data) as main line data from the main line image encoder 1011V, audio data as main line data from the main line audio encoder 1011A, and image data as proxy data from the proxy image encoder 1012V. The audio data as the proxy data from the proxy audio encoder 1012A is multiplexed, for example, by about 2 seconds in reproduction time, and the AV file in a predetermined format in which the bit stream obtained by the multiplexing is stored, that is, the main line An AV file of a predetermined format in which a bit stream in which data (as image data and audio data) and proxy data (as image data and audio data) are multiplexed for about 2 seconds in reproduction time is stored Create and record / It supplies to the communication part 1005 (FIG. 41).

  Next, FIG. 43 shows a configuration example of the Fy file creation unit 1004 of FIG.

  The Fy file creation unit 1004 includes a data reduction unit 1021, a change amount calculation unit 1022, a display type determination unit 1023, a selection unit 1024, and a file creation unit 1025.

  The data reduction unit 1021 is supplied with image data in units of frames from the image processing unit 1001 (FIG. 41). The data reduction unit 1021 thins out the number of pixels of image data in units of frames from the image processing unit 1001, and supplies image data with a small number of pixels obtained as a result to the change amount calculation unit 1022.

  That is, the data reduction unit 1021 performs thinning so that the number of horizontal and vertical pixels of the frame of the image data from the image processing unit 1001 is 1/8, for example.

  Therefore, if the frame of image data supplied from the image processing unit 1001 (FIG. 41) to the data reduction unit 1021 is composed of 720 × 480 pixels in width × length, for example, the data reduction unit 1021 As shown in FIG. 44, the image data of 720 × 480 pixels from the image processing unit 1001 is thinned out to obtain 90 × 60 pixel image data in which the number of horizontal and vertical pixels is 1/8, respectively.

  In other words, the data reduction unit 1021 divides the image data of 720 × 480 pixels from the image processing unit 1001 into, for example, blocks of 8 × 8 pixels in the horizontal and vertical directions, and averages the pixel values of the pixels constituting the block. By assigning values and the like to blocks, image data of 90 × 60 pixels is generated.

  Here, in the data reduction unit 1021, the number of pixels of the image data from the image processing unit 1001 is reduced in order to reduce the amount of data and the subsequent processing load, and the Fy file creation unit 76. Is sufficient and the processing burden does not need to be reduced, the data reduction unit 1021 supplies the image data from the image processing unit 1001 to the subsequent change amount calculation unit 1022 as it is. Alternatively, the Fy file creation unit 1004 can be configured without providing the data reduction unit 1021.

  The change amount calculation unit 1022 obtains a change amount indicating the degree of change of the image data supplied from the data reduction unit 1021 in units of frames, and supplies the change amount to the display type determination unit 1023 and the selection unit 1024 in time series. Note that, in the change amount calculation unit 1022, as the change amount in units of frames, a change amount indicating the degree of temporal change (hereinafter referred to as time change amount as appropriate) and a change amount indicating spatial change (hereinafter referred to as appropriate). , Referred to as a spatial change amount), or both.

  Here, the time change amount represents the degree of change of the image in the time direction. For example, the time change amount of a certain frame is the change of the image between the frame and the frame one frame before the frame. Degree, that is, the degree of image movement. Therefore, the amount of time change is hereinafter also referred to as motion information as appropriate.

  The amount of spatial change represents the degree of change in the image in the spatial direction. For example, the amount of spatial change in a frame means the degree of change in the pixel value of the pixels constituting the frame, that is, the fineness of the image. Represents (complexity). Therefore, the amount of change in space is hereinafter also referred to as fineness information as appropriate.

  The display type determination unit 1023 determines a display type representing a display method for displaying the frame based on the frame-unit variation supplied from the variation calculation unit 1022 in time series, and selects the display unit. 1024. Here, details of the display type will be described later.

  The selection unit 1024 selects one or both of the change amount in frame units supplied from the change amount calculation unit 1022 and the display type in frame units supplied from the display type determination unit 1023, and This is supplied to the creation unit 1025.

  Here, what type of selection is to be performed in the selection unit 1024 can be set in advance, or can be determined according to the operation of the operator, for example. When the selection unit 1024 selects only the change amount in units of frames supplied from the change amount calculation unit 1022 in a fixed manner, the Fy file creation unit 1004 displays the display type determination unit 1023 and the selection unit 1024. It can comprise without providing.

  The file creation unit 1025 creates a predetermined format Fy file that is supplied from the selection unit 1024 and stores one or both of the change amount and the display type in units of frames, and the recording unit / communication unit 77 (see FIG. 41).

  Next, processing (Fy file creation processing) performed by the Fy file creation unit 1004 in FIG. 43 will be described with reference to the flowchart in FIG.

  First, in step S1001, the data reduction unit 1021 performs data reduction by thinning out the number of pixels of the image data from the image processing unit 1001 (FIG. 41) as necessary, and converts the resulting image data into a change amount. The data is supplied to the calculation unit 1022, and the process proceeds to step S1002.

  In step S1002, the change amount calculation unit 1022 uses the image data from the data reduction unit 1021 to obtain a change amount representing the degree of change in the image data from the image processing unit 1001 (FIG. 41) in units of frames. The data is supplied to the display type determination unit 1023 and the selection unit 1024 in time series, and the process proceeds to step S1003.

  In step S <b> 1003, the display type determination unit 1023 determines a display type representing a display method when displaying the frame based on the change amount in frame units from the change amount calculation unit 1022, and selects the selection unit 1024. To proceed to step S1004.

  In step S <b> 1004, the selection unit 1024 selects one or both of the change amount in frame units supplied from the change amount calculation unit 1022 and the display type in frame units supplied from the display type determination unit 1023. The file is selected and supplied to the file creation unit 1025, and the process proceeds to step S1005.

  In step S1005, the file creation unit 1025 creates a predetermined format Fy file that is supplied from the selection unit 1024 and stores one or both of the change amount and the display type in units of frames, and the recording unit / communication To the unit 1005 (FIG. 41), and the process is terminated.

  Next, FIG. 46 shows a configuration example of the change amount calculation unit 1022 of FIG.

  As described above, the change amount calculation unit 1022 uses motion information (time change amount) indicating the degree of temporal change and fineness information (space change amount) indicating a spatial change as the change amount in units of frames. And / or both.

  46 shows the configuration example of the change amount calculation unit 1022 when only motion information is obtained as the change amount in units of frames, and the second from the top in FIG. 46 shows the change amounts in units of frames. As an example, a configuration example of the change amount calculation unit 1022 when only fineness information is obtained is shown. In addition, the third from the top (first from the bottom) of FIG. 46 illustrates a configuration example of the change amount calculation unit 1022 in the case of obtaining both motion information and fineness information as the change amount in units of frames.

  The first change amount calculation unit 1022 from the top in FIG. 46 has a motion information calculation unit 1031, and the second change amount calculation unit 1022 from the top in FIG. 46 has a fineness information calculation unit 1032. Also, the third change amount calculation unit 1022 from the top in FIG. 46 includes a motion information calculation unit 1031 and a fineness information calculation unit 1032.

  The motion information calculation unit 1031 and the fineness information calculation unit 1032 are supplied with image data from the data reduction unit 1021 (FIG. 43).

  The motion information calculation unit 1031 uses the image data from the data reduction unit 1021 to obtain and output motion information in units of frames. The fineness information calculation unit 1032 uses the image data from the data reduction unit 1021 to obtain and output fineness information in units of frames.

  Next, with reference to FIG. 47, the frame-by-frame motion information obtained by the motion information calculation unit 1031 in FIG.

  For example, as illustrated in FIG. 47, the motion information calculation unit 1031 uses the first frame before the first frame as the motion information of the i-th frame that is the i-th frame from the top of the time-series frames constituting a certain moving image. The amount of change of the image from the i-1 frame to the i-th frame is obtained. In this case, the motion information of the first frame does not exist. However, as the motion information of the first frame, for example, the same motion information as the motion information of the second frame can be adopted. Alternatively, the motion information of the first frame can be obtained assuming that the image of the 0th frame is an image having all pixel values of 0 or the same image as the first frame.

  Next, FIG. 48 illustrates a configuration example of the motion information calculation unit 1031 of FIG.

  48, the motion information calculation unit 1031 includes a motion vector detection unit 1041 and a statistic calculation unit 1042.

  Image data from the data reduction unit 1021 (FIG. 43) is supplied to the motion vector detection unit 1041. The motion vector detection unit 1041 sequentially sets the frames of the image data from the data reduction unit 1021 as a target frame, and a frame one frame before the target frame (hereinafter, referred to as a previous frame as appropriate) is, for example, 16 × 16 pixels. Into blocks (macroblocks in MPEG). Further, the motion vector detection unit 1041 obtains a motion vector representing the motion from the previous frame toward the target frame for each macroblock of the previous frame, and supplies the motion vector to the statistic calculation unit 1042.

  The statistic calculation unit 1042 obtains the statistic of the motion vector obtained for the macroblock of the previous frame from the motion vector detection unit 1041, and outputs it as the motion information of the frame of interest.

  The processing of the motion information calculation unit 1031 in FIG. 48 will be further described with reference to FIG.

  In the motion information calculation unit 1031, the motion vector detection unit 1041 divides the previous frame into 16 × 16 pixel macroblocks, and for each macroblock of the previous frame, 16 × 16 pixels of the target frame most similar to the macroblock Blocks (hereinafter referred to as similar blocks). Then, the motion vector detection unit 1041 obtains, as a macroblock motion vector ΔF0 (h, v), for example, a vector having a macroblock starting from the top left and ending at the top left of the similar block.

  Now, the position of the h-th macroblock from the left of the previous frame and the v-th macroblock from the top is represented as F0 (h, v), and from the macroblock F0 (h, v), the macroblock F0 (h, v) Assuming that the position of the 16 × 16 pixel block of the frame of interest at the position moved by the motion vector ΔF0 (h, v), i.e., the position of the similar block, is expressed as F1 (h, v), the macroblock F0 (h, v, The motion vector ΔF0 (h, v) in v) is expressed by the equation ΔF0 (h, v) = F1 (h, v) −F0 (h, v).

  The statistic calculator 1042 uses, for example, the magnitude of the motion vector ΔF0 (h, v) of all macroblocks in the previous frame as the motion vector statistic obtained for the macroblock in the previous frame | ΔF0 (h , v) |, a sum D0 = Σ | ΔF0 (h, v) | is obtained, and this sum D0 is output as motion information of the frame of interest.

  In the summation D0 = Σ | ΔF0 (h, v) |, the summation Σ is changed from 1 to the number of macroblocks in the horizontal direction of the previous frame, and v is changed from 1 to the previous frame. This represents the summation of up to the number of vertical macroblocks.

  Here, if the magnitude of the motion vector ΔF0 (h, v) of each macroblock F0 (h, v) in the previous frame is large, the motion information D0 that is the sum thereof also becomes large. Therefore, when the movement information D0 of the attention frame is large, the movement of the image of the attention frame is also large (violent).

  In the above case, the magnitude of the motion vector ΔF0 (h, v) of all macroblocks in the previous frame | ΔF0 (h, v) | is calculated as D0 = Σ | ΔF0 (h, v) |. However, as the statistic of the motion vector obtained for the macroblock of the previous frame, for example, the macroblock of the previous frame It is possible to employ the variance of the motion vector determined for.

  In this case, the statistic calculation unit 1042 obtains the average value Δave of the motion vectors ΔF0 (h, v) of all macroblocks in the previous frame, and calculates all the macroblocks F0 (h, v) in the previous frame. The variance σ0 of the motion vector ΔF0 (h, v) is obtained by calculating, for example, the equation σ0 = Σ (ΔF0 (h, v) −Δave) 2.

  Note that the summation Σ in the variance σ0 = Σ (ΔF0 (h, v) −Δave) 2 changes h from 1 to the number of macroblocks in the horizontal direction of the previous frame, and v is changed to 1 To the number of macroblocks in the vertical direction of the previous frame.

  Similarly to the sum D0, the variance σ0 also increases when the motion of the image of the frame of interest is large (violent).

  Here, in the main line image encoder 1011V of FIG. 42, when the motion data ΔF0 (h, v) is obtained when the image data is encoded, the statistic calculation unit 1042 (FIG. 48) displays the main line image. Using the motion vector ΔF0 (h, v) obtained by the encoder 1011V, the above-described statistics can be obtained. In this case, the motion information calculation unit 1031 in FIG. 48 can be configured without providing the motion vector detection unit 1041.

  Next, FIG. 50 shows another configuration example of the motion information calculation unit 1031 of FIG.

  50, the motion information calculation unit 1031 includes a histogram creation unit 1051, a histogram storage unit 1052, and a difference calculation unit 1053.

  Image data from the data reduction unit 1021 (FIG. 43) is supplied to the histogram creation unit 1051. The histogram creation unit 1051 sequentially sets the frames of the image data from the data reduction unit 1021 as the attention frame, creates a so-called simple histogram of the pixel values of the attention frame, and uses the histogram storage unit 1052 as the histogram of the attention frame. And supplied to the difference calculation unit 1053.

  The histogram storage unit 1052 stores the histogram of the frame of interest supplied from the histogram creation unit 1051. Here, the histogram storage unit 1052 has a storage capacity sufficient to store at least a histogram for two frames, and the histogram of the current attention frame supplied from the histogram creation unit 1051 and the previous attention. The frame, that is, the histogram of the previous frame is stored.

  The difference calculation unit 1053 obtains a sum of absolute differences, which will be described later, between the histogram of the frame of interest supplied from the histogram creation unit 1051 and the histogram of the previous frame stored in the histogram storage unit 1052, and serves as motion information of the frame of interest. Output.

  The processing of the motion information calculation unit 1031 in FIG. 50 will be further described with reference to FIG.

  If the pixel value of the image data supplied from the data reduction unit 1021 (FIG. 43) to the histogram creation unit 1051 is represented by, for example, 8 bits that can represent an integer value from 0 to 255, the histogram creation unit 1051 , 0 to 255, for example, divided into 8 sub-ranges of 8 to 31, 32 to 63, 64 to 95, 96 to 127, 128 to 159, 160 to 191, 192 to 223, 224 to 255. Then, by calculating the frequency of the pixel value included in each small range, a simple histogram of the frame of interest is created.

  For example, assuming that the i + 1th frame is a frame of interest, the difference calculation unit 1053 calculates the difference between the i + 1th frame that is the frame of interest and the ith frame that is the previous frame, as shown in FIG. The absolute value Δ of the difference value between the frequencies in the same small range of the histogram (the portion shown with a shadow in FIG. 51) is obtained. Further, the difference calculation unit 1053 obtains the sum of absolute values (difference absolute value sum) ΣΔ of the difference values of the frequencies obtained for the eight small ranges of the histogram, and outputs it as the motion information of the frame of interest.

  Here, when the movement of the attention frame is large (violent), the frequency distribution of the pixel values of the attention frame is different from the frequency distribution of the pixel values of the previous frame. Therefore, when the difference absolute value sum ΣΔ of the target frame is large, the motion of the image of the target frame is also large (severe).

  Next, FIG. 52 shows a configuration example of the fineness information calculation unit 1032 of FIG.

  In FIG. 52, the fineness information calculation unit 1032 includes a DCT (Discrete Cosine Transform) conversion unit 1061, a weight coefficient calculation unit 1062, and an integration unit 1063.

  The DCT conversion unit 1061 is supplied with image data from the data reduction unit 1021 (FIG. 43). The DCT conversion unit 1061 sequentially sets the frames of the image data from the data reduction unit 1021 as the attention frame, and divides the attention frame into blocks of 8 × 8 pixels, for example. Further, the DCT conversion unit 1061 performs DCT conversion on each block of the frame of interest, and supplies 8 × 8 DCT coefficients obtained for each block to the integration unit 1063.

  The weight coefficient calculation unit 1062 obtains a weight assigned to each of 8 × 8 DCT coefficients of the block, and supplies the weight to the integration unit 1063.

  The accumulating unit 1063 adds the weights supplied from the weighting factor calculating unit 1062 to each of 8 × 8 DCT coefficients of the block supplied from the DCT converting unit 1061 and integrates the obtained values. Furthermore, the accumulating unit 1063 obtains the sum of the accumulated values obtained for each block of the frame of interest, and outputs the sum as fineness information of the frame of interest.

  With reference to FIG. 53, the process of the fineness information calculation unit 1032 in FIG. 52 will be further described.

  The left side of FIG. 53 shows a base image of DCT conversion. The base image is composed of 8 × 8 patterns (frequency components), and the pattern on the right side and the pattern on the lower side become higher frequency component patterns.

  Of the 8 × 8 DCT coefficients of the block, i (i = 1, 2,..., 8) from the left and j (j = 1, 2,..., 8) from the top The DCT coefficients Fi-1 and j-1 represent the degree (degree) that the frequency component of the i-th pattern from the left of the base image and the j-th pattern from the top is included in the block.

  The right side of FIG. 53 shows the weights Gi−1 and j−1 calculated by the weight coefficient calculation unit 1062 of FIG.

  The weight Gi-1, j-1 is a weight assigned to the DCT coefficient Fi-1, j-1, and the weight coefficient calculation unit 1062 uses the weight Gi-1, j-1, for example, the expression Gi-1, j Calculated according to -1 = i * j. Therefore, the weighting factor calculation unit 1062 calculates a weight having a larger value as the weighting Gi−1, j−1 attached to the DCT coefficient Fi−1, j−1 of the high frequency component.

  52 multiplies the DCT coefficient Fi-1, j-1 of the block supplied from the DCT conversion unit 1061 by the weight Gi-1, j-1 supplied from the weight coefficient calculation unit 1062, The multiplication value Gi−1, j−1 × Fi−1, j−1 is obtained. The accumulating unit 1063 accumulates the multiplication values Gi−1, j−1 × Fi−1, j−1 obtained for each of the 8 × 8 DCT coefficients Fi−1, j−1 of the block, and the accumulated value V = ΣGi−1, j−1 × Fi−1, j−1 is obtained. Here, the summation Σ in the integrated value V = ΣGi−1, j−1 × Fi−1, j−1 represents the summation in which i and j are changed from 1 to 8, respectively.

  Furthermore, the accumulating unit 1063 obtains the sum K of the accumulated values V obtained for all the blocks of the frame of interest, and outputs the sum K as fineness information of the frame of interest.

  Here, as the high-frequency component is included in the frame of interest, the fineness information that is the sum K of the integrated values V increases. Therefore, the image of the frame of interest is a fine (complex) image.

  Next, FIG. 54 shows another configuration example of the fineness information calculation unit 1032 of FIG.

  54, the fineness information calculation unit 1032 includes an average value calculation unit 1071, a difference value calculation unit 1072, and an integration unit 1073.

  The average value calculation unit 1071 and the difference value calculation unit 1072 are supplied with image data from the data reduction unit 1021 (FIG. 43). The average value calculation unit 1071 sequentially sets the frames of the image data from the data reduction unit 1021 as the attention frame, and divides the attention frame into blocks of 8 × 8 pixels, for example, as shown in FIG. Further, the average value calculation unit 1071 calculates the average value of the pixel values of each block of the frame of interest and supplies the average value to the difference value calculation unit 1072.

  Here, assuming that the pixel value of the kth pixel is expressed as Pk in the raster scan order of the 8 × 8 pixel block, the average value calculation unit 1071 expresses the average value Pave with the formula Pave = 1 / (8 × 8) Calculate according to × ΣPk. Note that the summation Σ in the average value Pave = 1 / (8 × 8) × ΣPk represents the summation with k changed from 1 to 8 × 8 (= 64).

  Similar to the average value calculation unit 1071, the difference value calculation unit 1072 divides the frame of interest into blocks of 8 × 8 pixels, and each pixel value Pk of the block is supplied from the average value calculation unit 1071. The absolute value | Pk−Pave | of the difference value with the average value Pave of the pixel values is obtained and supplied to the integrating unit 1073.

  The integrating unit 1073 integrates the absolute value | Pk-Pave | of the difference values obtained for each pixel of the block supplied from the difference value calculating unit 1072 to obtain an integrated value Q = Σ | Pk-Pave |. Here, the summation Σ in the integrated value Q = Σ | Pk-Pave | represents a summation in which k is changed from 1 to 8 × 8 (= 64).

  Furthermore, the integrating unit 1073 calculates the sum of the integrated values Q obtained for all the blocks of the frame of interest, and outputs the sum as fineness information of the frame of interest.

  Note that the sum of the integrated values Q obtained for the frame of interest is called intra-AC, and the larger the value, the greater the variation in pixel values in the frame of interest. Therefore, the larger the fineness information that is the sum of the integrated values Q, the finer (complex) image of the frame of interest.

  Here, in the present embodiment, the amount of change such as motion information and fineness information described above is used for scrubbing (including variable speed playback) as will be described later. This can be used when detecting the image or when improving the efficiency of MPEG encoding.

  Next, the display type determination unit 1023 in FIG. 43 will be described with reference to FIGS.

  FIG. 56 shows the change amount for each frame obtained by the change amount calculation unit 1022 (FIG. 43). In FIG. 56, the horizontal axis represents the frame (how many frames from the top), and the vertical axis represents the amount of change.

  Assuming that the amount of change is, for example, motion information, the amount of change is large in a frame where the motion is intense (large), and the amount of change is small in a frame where there is no motion (small).

  The display type determination unit 1023 (FIG. 43) displays a display type representing a display method when displaying each frame based on the change amount in units of frames as shown in FIG. 56, which is obtained by the change amount calculation unit 1022. Determine from multiple display types.

  FIG. 57 shows an example of the display type determined by the display type determination unit 1023.

  In FIG. 57, as a display type, a still image type V1 representing a still image display, a normal type V2 having a different resolution for displaying a frame and a display rate (frame rate) for displaying a frame, and There are three types, high display rate and low resolution type V3.

  Here, the resolution when displaying a frame means the spatial resolution of an image displayed on a display device such as the display 40 (FIG. 1). For example, for image data as main line data and proxy data, the main line data has a higher resolution and the proxy data has a lower resolution.

  Moreover, the display rate when displaying a frame means the frequency (cycle) at which the display is updated in a display device such as the display 40. For example, the display is updated at the same display rate as the moving image frame rate (for example, 30 Hz), and the display target frame is changed from one frame to the next frame (the time of a certain frame) at the same rate as the frame rate. (Next frame), the moving image is displayed at 1 × speed.

  Furthermore, for example, if the display is updated at a display rate that is twice the frame rate of the moving image and the display target frame is updated from one frame to the next frame at a rate that is twice the frame rate, the moving image is updated. Displayed at double speed.

  For example, if the display is updated at the same display rate as the frame rate of the moving image and the display target frame is updated from one frame to the next frame at a rate twice the frame rate, Again, it is displayed at double speed.

  However, when the display is updated at a display rate that is twice the frame rate of the moving image and the display target frame is updated at a rate that is twice the frame rate, the frames that make up the moving image are dropped. However, if the display is updated at the same display rate as the video frame rate and the display target frame is updated at twice the frame rate, the frames that make up the video will be displayed. A frame dropout state is displayed every other frame.

  Therefore, when the display target frame is updated at a rate higher than the frame rate, frame dropping can be prevented (reduced) by increasing the display rate.

  For the still image type V1 frame, for example, even if the display target frame is changed, the image of the display target frame may be regarded as the same image as the immediately preceding frame (the image thereof). As much as possible, the frame (image) displayed immediately before is displayed (continues).

  For normal type V2 frames, for example, at the same display rate as the video frame rate (hereinafter referred to as the normal rate as appropriate), the same resolution as the image data as the main line data (higher than the image data as the proxy data) The image is displayed at a resolution (hereinafter referred to as a normal resolution as appropriate).

  For frames of the high display rate low resolution type V3, for example, at a display rate higher than the normal rate, for example, twice the normal rate, the same resolution as the image data as proxy data (from the image data as main line data) Image is displayed at a lower resolution).

  As the display types, as shown in FIG. 57, when there are three still image types V1, normal type V2, and high display rate low resolution type V3, the change amount calculation unit 1022 has a degree of temporal change of the frame. Is obtained as the amount of change. Then, for example, the display type determination unit 1023 compares the change amount (motion information) of each frame with two predetermined thresholds, and determines the display type of the frame based on the result of the comparison, the still image type V1, Either normal type V2 or high display rate low resolution type V3 is determined.

  That is, FIG. 58 shows display types that the display type determination unit 1023 determines in units of frames based on the change amounts in units of frames shown in FIG.

  The display type determination unit 1023 compares the amount of change (motion information) in units of frames with a threshold value L or a threshold value H having a relationship of expression L <H. Then, the display type determination unit 1023 selects a display type of a frame whose motion information is greater than or equal to the threshold value H, that is, a frame indicating that the motion information has a large degree of temporal change, with a low resolution or a high display rate. The high display rate low resolution type V3, which is a kind of low resolution / high display rate type, is determined.

  Here, for example, in a scrub performed in editing, if a frame drop occurs in the display of a frame with a large temporal change, that is, a frame with a large movement, it is easy to overlook the temporal change of the image. It is desirable to display a frame with a large movement at a high display rate.

  However, when the display rate is increased, the burden on the scrubbing apparatus increases. Furthermore, since the amount of data of a frame with a large motion is large, the processing load is larger than the processing load of a frame with a small motion.

  Therefore, in this embodiment, a frame with a large motion is displayed at a high display rate, but instead, a high display rate and low resolution, which is a type of low resolution / high display rate type that displays an image with low resolution. We will adopt type V3. An image with a low resolution has a smaller amount of data than an image with a high resolution. Therefore, by displaying a low-resolution image at a high display rate for a frame with a large movement, temporal changes in the image during editing It is possible to reduce the burden on the scrubbing apparatus while preventing oversight.

  On the other hand, the display type determination unit 1023 sets the display type of a frame whose motion information is less than the threshold L, that is, a frame indicating that the degree of temporal change in the motion information is small, Determine the image type V1.

  Here, for example, in a scrub performed in editing, when frames with a small degree of temporal change, that is, frames with no motion (almost no motion) are continuous, such a frame is displayed. When the target frame is updated, the display on the display device such as the display 40 (FIG. 1) is changed from the frame (image) displayed immediately before to the frame (image) displayed after the update. However, the image displayed on the display device does not change (almost).

  As described above, when the display target frame is updated even though the image displayed on the display device is not changed, the display device displays the display target after the update from the frame displayed immediately before. Changing to this frame places a useless burden on the scrubbing apparatus.

  In addition, when there are continuous (almost) frames that do not move, when the display target frame is updated in such a frame, the display of the display device such as the display 40 (FIG. 1) is displayed. When the frame displayed immediately before is changed to the updated display target frame, the image displayed on the display device may slightly change. In this case, the operator who operates the scrubbing device is wasted in a section of a frame in which frames without motion are continuous and are hardly set as editing points. It will be over.

  Therefore, in this embodiment, for a frame that does not move, even if the display target frame is changed, the image of the display target frame may be regarded as the same image as the frame displayed immediately before. As much as possible, the still image type V1 in which the frame (image) displayed immediately before is displayed is adopted. Thereby, it is possible to prevent an unnecessary burden on the scrubbing apparatus and the operator who performs the operation.

  The display type determination unit 1023 selects a display type of a frame other than a frame whose motion information is greater than or equal to the threshold H and a frame whose motion information is less than the threshold L, that is, a frame whose motion information is greater than or equal to the threshold L and less than the threshold H. Decide on type V2.

  That is, for the frames that have some movement but not so much movement, the normal type V2 that displays the image at the normal rate (the same display rate as the frame rate of the moving image) and at the normal resolution is adopted. .

  Here, in the time-series frames that make up a video, the section in which frames of the still image type V1 are continuous is called the still image section, and the section in which the frames of the normal type V2 are continuous Is called a normal section. A section in which frames of the display type of the high display rate and low resolution type V3 are continuous is referred to as a high display rate and low resolution section.

  Next, FIG. 59 shows a configuration example of the display type determination unit 1023 of FIG.

  59, the display type determination unit 1023 includes a storage unit 1081, a threshold processing unit 1082, a continuity determination unit 1083, and a determination unit 1084.

  The amount of change in units of frames is supplied from the change amount calculation unit 1022 (FIG. 43) to the storage unit 1081. The storage unit 1081 temporarily stores the change amount in frame units from the change amount calculation unit 1022.

  The threshold processing unit 1082 compares the amount of change in units of frames stored in the storage unit 1081 with the threshold value H or L, and determines comparison information representing the result of the comparison with the continuity determination unit 1083 in units of frames. Part 1084.

  The continuity determination unit 1083 is based on the comparison information in units of frames from the threshold processing unit 1082, and whether or not a frame having a change amount greater than or equal to the threshold value H continues for more than N frames, which is the number of a plurality of frames, It is determined whether or not the number of frames of the change amount is N frames or more, and determination information representing the determination result is supplied to the determination unit 1084.

  The determination unit 1084 determines and outputs the display type of each frame constituting the moving image based on the comparison information from the threshold processing unit 1082 and the determination information from the continuity determination unit 1083.

  Next, processing of the display type determination unit 1023 in FIG. 59 will be described with reference to the flowcharts in FIGS. 60 and 61.

  When the change amount for each frame is supplied from the change amount calculation unit 1022 (FIG. 43), the display type determination unit 1023 causes the storage unit 1081 to cache (temporarily store) the change amount for each frame in step S1011. The process proceeds to step S1012.

  Here, for example, motion information is supplied as a change amount from the change amount calculation unit 1022 to the display type determination unit 1023. In addition, the display type determination unit 1023 determines the display type of each frame from, for example, three of the still image type V1, the normal type V2, and the high display rate low resolution type V3 illustrated in FIG. And

  In step S1012, the threshold processing unit 1082 performs threshold processing that compares the amount of change (motion information) in units of frames stored in the storage unit 1081 with the threshold H or L (one or both), and performs frame units. The comparison information representing the result of comparison between the change amount and the threshold value H or L is supplied in units of frames to the continuity determination unit 1083, and the process proceeds to step S1013.

  In step S1013, the continuity determination unit 1083 is, for example, in chronological order that has not yet been set as the frame of interest, among the frames constituting the video for which the Fy file creation unit 1004 (FIG. 43) is currently creating a Fy file. The most preceding frame is selected as the frame of interest, and the process proceeds to step S1014.

  In step S1014, based on the comparison information from the threshold processing unit 1082, the continuity determining unit 1083 has a change amount of the frame of interest equal to or greater than the threshold H, and the change amount immediately before, immediately after, or before and after the frame of interest. It is determined whether or not frames having a threshold value H or more exist continuously for N frames or more, which is the number of frames described above.

  In step S1014, it is determined that the amount of change in the frame of interest is greater than or equal to the threshold value H, and that frames having the amount of change greater than or equal to the threshold value H exist immediately before, immediately after, or before and after the frame of interest. In this case, the continuity determination unit 1083 supplies determination information representing the determination result to the determination unit 1084, and the process proceeds to step S1015.

  In step S1015, the determination unit 1084 determines the display type of the frame of interest to be the high display rate low resolution type V3 based on the determination information from the continuity determination unit 1083, and the process proceeds to step S1020.

  Here, according to steps S1014 and S1015, not only the amount of change of the frame of interest is greater than or equal to the threshold H, but also the frame of which the amount of change is greater than or equal to the threshold H immediately before, immediately after, or before and after the frame of interest. Is continuously displayed for N frames or more, the display type of the target frame is determined to be the high display rate low resolution type V3 for the following reason.

  That is, as will be described later, when the display type is determined from the three types of the still image type V1, the normal type V2, and the high display rate low resolution type V, the display type is the still image type V1 in the scrub. As for the frame and the normal type V2 frame, the image is displayed by processing the high-resolution image (normal resolution image), that is, the image data of the main line data having a larger data amount (than the proxy data). On the other hand, for a frame with a display type of a high display rate and low resolution type V3, an image is displayed by processing an image with a low resolution, that is, image data of proxy data with a smaller data amount (than main line data).

  For example, when the storage device 22 (FIG. 1) is a professional disc on which main line data and proxy data are recorded, scrubbing is performed using the main line data or proxy data recorded on the storage device 22. If this is the case, it is necessary to read the main line data from the storage device 22 for the frame whose display type is the still image type V1 or the normal type V2, and the proxy data from the storage device 22 for the frame of the high display rate low resolution type V3. Need to be read.

  In the storage device 22, since the main line data and the proxy data are recorded at physically separated positions, for example, in an extreme case, the display type is a frame of the still image type V1 or the normal type V2, and a high display rate. When frames of the low resolution type V3 appear alternately, seeks frequently occur when scrubbing the storage device 22 (data recorded in the storage device 22). Depending on the performance of the scrubbing device, the scrub bar 66 ( It may be difficult to smoothly display the frame specified by the operation of FIG.

  Therefore, in this embodiment, in order to prevent frequent seeks, frames of the display type of the high display rate, low resolution type V3 are continuous for N frames or more, and the display type is the still image type V1. Alternatively, the display type is determined so that the frames of the normal type V2 are also continued for N frames or more.

  That is, for example, as described above, in steps S1014 and S1015, the amount of change of the frame of interest is greater than or equal to the threshold value H, and a frame whose amount of change is greater than or equal to the threshold value H immediately before, immediately after, or before and after the frame of interest. When there are N frames or more in succession, the display type of the frame of interest is determined to be the high display rate, low resolution type V3.

  Here, the number N of frames is, for example, the maximum seek time of the drive 5 or the like that reproduces the storage device 22, or the total number of frames constituting the moving image for which the Fy file creation unit 1004 (FIG. 43) is to create the Fy file. It can be determined in consideration of the above. Further, the number N of frames may be determined in accordance with the operation of the operator.

  In addition, when determining the display type, the number of frames N is the minimum number of frames that should have consecutive frames whose change amount is equal to or greater than the threshold value H (and the change amount is less than the threshold value H as will be described later). Hereinafter, the minimum number of frames N will be referred to as appropriate.

  The problem with seeking is that still images of type V1 or normal type V2 in which images are displayed using main line data and images using proxy data in the time-series frames that make up a movie. This is a case where the frame of the high display rate / low resolution type V3 is frequently switched. In either case, seek does not matter when the still image type V1 frame in which an image is displayed using the main line data and the normal type V2 frame are switched.

  On the other hand, in step S1014, it is determined that the amount of change of the frame of interest is not greater than or equal to the threshold value H, or even if the amount of change of the frame of interest is greater than or equal to the threshold value H, On the other hand, if it is determined that there are no frames whose change amount is equal to or greater than the threshold value H more than the minimum number of frames N, the process proceeds to step S1016, and the continuity determination unit 1083 is based on the comparison information from the threshold processing unit 1082. Whether the change amount of the frame of interest is less than the threshold value H, and whether there are consecutive frames with the change amount of less than the threshold value H more than the minimum number of frames N immediately before, immediately after, or before and after the frame of interest. Determine.

  In step S1016, it is assumed that the amount of change of the frame of interest is less than the threshold value H, and frames having the amount of change of less than the threshold value H are continuously present for the minimum number of frames N or more immediately before, immediately after, or before and after the frame of interest. When it is determined, the continuity determination unit 1083 supplies determination information representing the determination result to the determination unit 1084, and the process proceeds to step S1017.

  Result of determination that the amount of change of the target frame is less than the threshold value H, and that there are consecutive frames with the change amount of less than the threshold value H more than the minimum number of frames N immediately before, immediately after, or before and after the target frame In Step S1017, the determination unit 1084 that has received the supply of the determination information that represents the determination on the basis of the comparison information from the threshold processing unit 1082 determines whether or not the amount of change of the frame of interest is greater than or equal to the threshold L. To do.

  If it is determined in step S1017 that the amount of change of the frame of interest is greater than or equal to the threshold L, that is, if the amount of change of the frame of interest is greater than or equal to the threshold L and less than the threshold H, the process proceeds to step S1018. The frame display type is determined to be the normal type V2, and the process proceeds to step S1020.

  If it is determined in step S1017 that the amount of change of the frame of interest is not greater than or equal to the threshold L, that is, if the amount of change of the frame of interest is less than the threshold L, the process proceeds to step S1019. The display type is determined as the still image type V1, and the process proceeds to step S1020.

  Here, according to steps S1016 to S1019, the amount of change of the frame of interest is less than the threshold value H, and frames having the amount of change of less than the threshold value H are continuous for N frames or more immediately before, immediately after, or before and after the frame of interest. If the amount of change in the frame of interest is greater than or equal to the threshold L, the display type of the frame of interest is determined to be the normal type V2, and if the amount of change in the frame of interest is less than the threshold L, a still image The type V1 is determined in order to prevent frequent seeks during scrubbing as described above.

  In step S1020, the continuity determination unit 1083 has a frame that has not yet been selected as a frame of interest among the frames constituting the moving image for which the Fy file creation unit 1004 (FIG. 43) is to create the Fy file. Determine if it exists.

  If it is determined in step S1020 that there is a frame that has not yet been selected as the target frame, the process returns to step S1013, and as described above, a frame that has not yet been selected as the target frame is newly selected as the target frame. Thereafter, the same processing is repeated.

  If it is determined in step S1020 that there is no unselected frame as the frame of interest, the display type determination unit 1023 ends the process.

  On the other hand, in step S1016, it is determined that the amount of change of the frame of interest is not less than the threshold H, or even if the amount of change of the frame of interest is less than the threshold H, either immediately before, immediately after, or before and after the frame of interest. In addition, when it is determined that there are no frames whose change amount is less than the threshold value H more than the minimum number of frames N, the process proceeds to step S1021 in FIG. 61, and the continuity determination unit 1083 determines whether the frame immediately before the frame of interest, Immediately after or before and after, it is determined whether or not frames whose change amount crosses the threshold value H exist continuously for the minimum number of frames N or more.

  Here, FIG. 62 shows the amount of change in units of frames. In FIG. 62, the horizontal axis represents the frame, and the vertical axis represents the amount of change. In FIG. 62, only the threshold value H of the threshold values H and L is shown.

  If the change amount of the frame of interest is equal to or greater than the threshold value H, and there are N frames or more of frames having the change amount of the threshold value H or more immediately before, immediately after, or before and after the frame of interest, FIG. In step S1015, the display type of the target frame is determined to be the high display rate low resolution type V3.

  Also, when the amount of change of the frame of interest is less than the threshold H, and there are consecutive frames with the amount of change of less than the threshold H immediately before, immediately after, or before and after the frame of interest, at least the minimum number N of frames. In step S1018 or S1019 of FIG. 60, the display type of the frame of interest is determined to be the normal type V2 or the still image type V1.

  Therefore, the process of step S1021 in FIG. 61 is performed immediately before, immediately after, and before and after the frame of interest, and there are no more than N frames with a change amount equal to or greater than the threshold value H, and , If there are no frames whose change amount is less than the threshold value H continuously for the minimum number of frames N or more, that is, the change amount as shown in the sections T1, T2, T3, and T4 in FIG. The number of frames in which a frame that crosses the threshold H exists (a section in which frames with a change amount equal to or greater than the threshold H and frames with a change amount less than the threshold H coexist) and frames with the change amount equal to or greater than the threshold H continue. This is a case where the number of frames in which the frames whose change amount is less than the threshold value H is continuous is a frame in a section (hereinafter referred to as a mixed section as appropriate) having a frame number less than the minimum number N at the maximum.

  As shown in FIG. 62, the mixed section always includes a section in which frames having a change amount equal to or greater than the threshold value H continuously exist for N frames or more (hereinafter, referred to as a large change amount section as appropriate), and the change amount is a threshold value. Between intervals where frames less than H exist continuously for the minimum number of frames N or more (hereinafter referred to as small change intervals as appropriate), between two large change intervals, or two small changes It exists in the form of being sandwiched between sections.

  The mixed section can be divided into a section having a section length (number of frames) of the minimum number of frames N or more and a section of less than the minimum number of frames N. Here, of the mixed sections T1 to T4 in FIG. 62, the mixed section T2 is a section whose section length is the minimum frame number N or more, and the other mixed sections T1, T3, T4 have the minimum section length. The interval is less than N frames.

  The mixed section is a section between the large change section and the small change section, a section sandwiched between the large change sections, or a section sandwiched between the small change sections. Can be divided into either. Here, of the mixed sections T1 to T4 in FIG. 62, the mixed sections T1 and T2 are sandwiched between the large variation sections, and the mixed section T3 is the large variation section and the small variation section. It is the section between. The mixed section T4 is a section sandwiched between the small change amount sections.

  Returning to FIG. 61, as described above, in step S1021, the continuity determination unit 1083 determines that the frame whose change amount crosses the threshold value H immediately before, immediately after, or before and after the frame of interest is the minimum number N of frames. It is determined whether or not there is a continuous section, that is, whether or not the mixed section where the frame of interest exists is a section having a section length of the minimum frame number N or more.

  If it is determined in step S1021 that there are consecutive frames whose change amount exceeds the threshold value H immediately before, immediately after, or before and after the target frame, that is, the minimum number of frames N, that is, there is a target frame. If the mixed section is a section whose section length is the minimum frame number N or more, such as the mixed section T2 in FIG. 62, for example, the process proceeds to step S1022, and the determination unit 1084 selects the display type of the target frame as, for example, Then, the normal type V2 is determined, and the process proceeds to step S1020 in FIG.

  That is, for frames in a mixed section where the amount of change spans the threshold value H continuously for the minimum number of frames N or more, the display type of all the frames is used as the main line data (as image data). By using the normal type V2 where the image is displayed, or the high display rate low resolution type V3 where the image is displayed using the proxy data (as the image data), the frequent occurrence of seeking Can be prevented.

  Therefore, in this embodiment, the display types of the frames in the mixed section where the frames whose change amount crosses the threshold value H exist continuously for the minimum number of frames N or more are all set to the normal type V2. In addition, the display type of all the frames in the mixed section where the amount of change that crosses the threshold value H is continuously more than the minimum number of frames N shall be the normal type V2 and the high display rate low resolution type V3. Is possible.

  Here, if only focusing on preventing frequent seeks, it is possible to set the frame display type of the mixed section to a still image type V1 in which an image is displayed using main line data. . However, since the mixed section is a section in which there is a frame in which the change amount exceeds the threshold value H, the display type of the frame in the mixed section is the still image type V1 that is the display type of the frame whose change amount is less than the threshold value L. That is not preferred. Therefore, the display type of the frame in the mixed section is either the normal type V2 or the high display rate / low resolution type V3.

  On the other hand, if it is determined in step S1021 that there are no frames whose change amount crosses the threshold value H immediately before, immediately after, or before or after the frame of interest, more than the minimum number of frames N, that is, When the mixed section in which the frame of interest exists (hereinafter referred to as the mixed section of interest as appropriate) is a section whose section length is less than the minimum number of frames N, such as the mixed section T1, T3, and T4 in FIG. Proceeding to step S1023, the continuity determining unit 1083 is a frame adjacent to the target mixed section among the temporally preceding frames on the left side of the target mixed section whose section length is less than the minimum number of frames N (hereinafter, referred to as “interactive mixed section”). The amount of change in the left frame (as appropriate) and the frame adjacent to the target mixed section among the temporally following frames on the right side of the target mixed section (hereinafter referred to as the right frame as appropriate). Note that whether any one of the change amounts is equal to or greater than the threshold value H and the other change amount is less than the threshold value H, that is, the section length is less than the minimum number of frames N It is determined whether the mixed section is a section sandwiched between the large variation section and the small variation section.

  In step S1023, when it is determined that the target mixed section whose section length is less than the minimum number of frames N is a section sandwiched between the large change amount section and the small change amount section, that is, the section length is the minimum limit. When the attention mixed section having the number of frames less than N is a mixed section sandwiched between the large change amount section and the small change amount section, for example, the mixed section T3 in FIG. 62, the process proceeds to step S1022, and as described above. The determining unit 1084 determines that the display type of the frame of interest is the normal type V2, and proceeds to step S1020 in FIG. 60, where the processing described above is performed.

  That is, both the large change amount section and the small change amount section sandwiching the target mixed section whose section length is less than the minimum number of frames N are sections whose section length is the minimum number of frames N or more. Furthermore, the display type of the frame in the large change amount section is determined to be the high display rate low resolution type V3 in which the image is displayed using proxy data, and the display type of the frame in the small change amount section is determined using the main line data. The normal type V2 or the still image type V1 in which an image is displayed is determined.

  In addition, for frames in the mixed-interest section that are sandwiched between the large variation section and the small variation section and whose section length is less than the minimum number of frames N, the high display determined as the frame display type of the large variation section By making the display type the same as the normal type V2, which is determined as the display type of the frame of the rate low resolution type V3 or its small change amount section, the frame of the high display rate low resolution type V3 or the normal type V2 is minimized. Since the limit frame number N is continued, it is possible to prevent frequent seeks.

  Therefore, in the present embodiment, the frame of the target mixed section sandwiched between the large change section and the small change section is the target mixed section in which the frame whose change amount crosses the threshold value H is less than the minimum number of frames N. The display types are all normally type V2. Note that the display type of the frame of the attention mixed section sandwiched between the large change amount section and the small change amount section is all the attention mixed sections where the change amount crosses the threshold value H is less than the minimum number of frames N, In addition to the normal type V2, it is possible to select a high display rate low resolution type V3.

  On the other hand, in step S1023, when it is determined that the target mixed section whose section length is less than the minimum frame number N is not a section sandwiched between the large change amount section and the small change amount section, the process proceeds to step S1024 and continues. The sex determination unit 1083 includes a change amount of a frame (left frame) adjacent to the target mixed section among temporally preceding frames on the left side of the target mixed section whose section length is less than the minimum frame number N, and Whether the change amount of the frame (right frame) adjacent to the notable mixed section among the temporally succeeding frames on the right side of the notable mixed section is greater than or equal to the threshold value H, that is, Then, it is determined whether or not the target mixed section whose section length is less than the minimum number of frames N is a section sandwiched between the large change amount sections.

  In step S1024, when it is determined that the attention mixed section whose section length is less than the minimum number of frames N is not a section sandwiched between the large change amount sections, that is, the attention whose section length is less than the minimum number of frames N. When the mixed section is a mixed section sandwiched between the small change amount sections, for example, the mixed section T4 in FIG. 62, the process proceeds to step S1022, and as described above, the determination unit 1084 selects the frame of interest. The display type is determined as the normal type V2, and the process proceeds to step S1020 in FIG. 60, where the above-described processing is performed.

  That is, the frame of the attention mixed section sandwiched between the two small change amount sections and having the section length less than the minimum frame number N is determined as the display type of the frames of the two small change amount sections ( By making the display type the same as that of the normal type V2, the frames of the normal type V2 are continued for the minimum number N of frames, and frequent seeks can be prevented.

  On the other hand, in step S1024, when it is determined that the target mixed section whose section length is less than the minimum frame number N is a section sandwiched between the large change amount sections, that is, the section length is the minimum frame number N. If the attention mixed section of less than, for example, is a mixed section sandwiched between large change amount sections, such as the mixed section T1 in FIG. 62, the process proceeds to step S1025, and the determination unit 1084 displays the display type of the target frame. Is determined to be the high display rate and low resolution type V3, and the process proceeds to step S1020 in FIG. 60, and the processing described above is performed.

  That is, for the frame of the attention mixed section that is sandwiched between the two large change amount sections and whose section length is less than the minimum number of frames N, the frame type determined as the frame display type of the two large change amount sections is high. By using the same display type as the display rate low-resolution type V3, the frame of the high display rate low-resolution type V3 will continue to be at least the minimum number of frames N and prevent frequent seeks. Can do.

  Next, in FIG. 63, the change amount calculation unit 1022 in FIG. 43 obtains motion information as the change amount, the display type determination unit 1023 determines the display type based on the motion information, and the selection unit 1024 changes the change type. 43 shows an example of an Fy file created by the file creation unit 1025 in FIG. 43 when both the quantity and the display type are selected.

  In the Fy file of FIG. 63, from left to right, a frame number indicating the number of the frame from the beginning, a time code of a frame corresponding to the frame number, motion information as a frame change amount corresponding to the frame number, The frame display types corresponding to the frame numbers are sequentially arranged.

  In the Fy file of FIG. 63, the display type is determined to be one of the three types of still image type V1, normal type V2, and high display rate / low resolution type V3 shown in FIG.

  Further, in the Fy file of FIG. 63, 1 and 5 are adopted as threshold values L and H, respectively, for determining the display type. As the minimum frame number N, a value obtained by the formula N = F × P from the total number F of the moving image frames in which the Fy file has been created and a predetermined coefficient P is employed. Here, if the total number F of the frames of the moving image that created the Fy file is, for example, 3000 frames, and the predetermined coefficient P is, for example, 0.01, the minimum number of frames N is 30 (= 3000 × 0.01). ) Frame.

  Next, in FIG. 57, motion information is adopted as the amount of change, and based on the motion information of the frame, the display type of the frame is 3 of still image type V1, normal type V2, high display rate low resolution type V3. However, the determination method for determining the display type of the frame is not limited to this.

  That is, FIG. 64 shows another example of the display type determined by the display type determination unit 1023 of FIG.

  In FIG. 64, as a display type, a still image type V11 representing a still image display, a normal type V12 having a different resolution for displaying a frame and a display rate for displaying a frame, and a normal resolution for a high display rate There are five types: V13, ultra-high display rate low resolution type V14, and ultra-high display rate low resolution type V15.

  As for the still image type V11 frame, similarly to the still image type V1 in FIG. 57, even if the display target frame is changed, the image of the display target frame is the same as the frame (the image) displayed immediately before. As long as it can be regarded as the same image, the frame (image) displayed immediately before is displayed (continues).

  As for the normal type V12 frame, similarly to the normal type V2 in FIG. 57, the image has the same display rate (normal rate) as the moving image frame rate and the same resolution (normal resolution) as the main line image data. Is displayed.

  For frames with the high display rate normal resolution type V13, the image is displayed at a display rate higher than the normal rate, for example, twice the normal rate, with the same resolution (normal resolution) as the main line image data. The

  The frame of the super high display rate / low resolution type V14 is higher than the display rate of the high display rate / normal resolution type V13, for example, three times the normal rate and the same resolution as the image data as proxy data ( The image is displayed at a resolution lower than that of the image data as the main line data.

  The frame of the ultra-high display rate low-resolution type V15 is higher than the display rate of the ultra-high display rate low-resolution type V14, for example, four times the normal rate and the same as the image data as proxy data An image is displayed at a resolution (lower resolution than the image data as the main line data).

  Here, if the normal rate is, for example, the same (about) 30 frames / second as the NTSC (National Television System Committee) system, the normal type V12, the high display rate normal resolution type V13, and the ultra-high display rate low resolution The display rates of type V14 and ultra-high display rate low resolution type V15 are 30, 60, 90, and 120 frames / second, respectively.

  As the display type, as shown in FIG. 64, five still image types V11, normal type V12, high display rate normal resolution type V13, super high display rate low resolution type V14, and ultra super high display rate low resolution type V15. If there is, the change amount calculation unit 1022 (FIG. 43) obtains motion information indicating the degree of temporal change of the frame as the change amount. Then, for example, the display type determination unit 1023 (FIG. 43) compares the motion information of each frame with four predetermined threshold values, and determines the display type of the frame based on the result of the comparison, the still image type V11, The normal type V12, the high display rate normal resolution type V13, the super high display rate low resolution type V14, or the super ultra high display rate low resolution type V15 is determined.

  That is, FIG. 65 shows motion information as a change amount in units of frames and a display type determined by the display type determination unit 1023 (FIG. 43) in units of frames based on the change amounts.

  In FIG. 65, the horizontal axis represents the frame, and the vertical axis represents the amount of change.

  The display type determination unit 1023 (FIG. 43) compares the amount of change (motion information) in units of frames with threshold values TH1, TH2, TH3, TH4 having a relationship of TH1 <TH2 <TH3 <TH4. Then, the display type determination unit 1023 selects a display type of a frame in which the motion information is greater than or equal to the threshold TH4, that is, a frame indicating that the degree of temporal change in the motion information is extremely large, with a low resolution or a high display rate. The ultra-high display rate low resolution type V15, which is a kind of low resolution / high display rate type, is determined. In FIG. 65, the display type of the frame in the section D5 in which the frames whose motion information is greater than or equal to the threshold value TH4 continues is determined as the ultra-high display rate low resolution type V15.

  Further, the display type determination unit 1023 displays a frame whose motion information is greater than or equal to the threshold TH3 and less than the threshold TH4, that is, a display type of a frame that indicates that the degree of temporal change of motion information is very large. Is a kind of low resolution or high display rate low resolution / high display rate type, and the ultra-high display rate low resolution type V15 has a display rate lower than the ultra-high display rate low resolution type V15. In FIG. 65, the display type of the frames in the sections D4 and D6 in which the motion information is greater than or equal to the threshold TH3 and less than the threshold TH4 is determined as the ultra-high display rate / low resolution type V14.

  Further, the display type determination unit 1023 represents a frame in which the motion information is greater than or equal to the threshold value TH2 and less than the threshold value TH3, that is, the motion information does not reach a very large degree of temporal change but is still large. The display type of the frame is determined to be a high display rate normal resolution type V13 having a lower display rate and higher resolution than the ultra high display rate low resolution type V14. In FIG. 65, the display type of the frames in the sections D3 and D7 in which the motion information is greater than or equal to the threshold TH2 and less than the threshold TH3 is determined as the high display rate normal resolution type V13.

  Further, the display type determination unit 1023 selects a display type of a frame in which the motion information is greater than or equal to the threshold TH1 and less than the threshold TH2, that is, a frame indicating that the degree of temporal change in the motion information is not so high. The normal type V12 having a lower display rate than the normal resolution type V13 is determined. In FIG. 65, the display type of the frames in the sections D2 and D8 in which the frames whose motion information is greater than or equal to the threshold TH1 and less than the threshold TH2 is continuous is determined as the normal type V12.

  Further, the display type determination unit 1023 sets the display type of a frame whose motion information is less than the threshold TH1, that is, a frame indicating that the degree of temporal change in the motion information is small, The image type is determined as V11. In FIG. 65, the display type of the frames in the sections D1 and D9 in which the frames whose motion information is less than the threshold TH1 continues is determined as the still image type V11.

  66 and 67, the display types are still image type V11, normal type V12, high display rate normal resolution type V13, ultra-high display rate low resolution type V14 shown in FIG. Alternatively, the processing of the display type determination unit 1023 in FIG. 59 when it is determined to be one of the ultra-high display rate and low resolution type V15 will be described.

  When the change amount for each frame is supplied from the change amount calculation unit 1022 (FIG. 43), the display type determination unit 1023 caches the change amount for each frame in the storage unit 1081 (FIG. 59) in step S1031. Temporarily store), and the process proceeds to step S1032.

  Here, for example, motion information is supplied as a change amount from the change amount calculation unit 1022 to the display type determination unit 1023.

  In step S1032, the threshold processing unit 1082 (FIG. 59) performs threshold processing that compares the amount of change (motion information) in units of frames stored in the storage unit 1081 with the thresholds TH1, TH2, TH3, or TH4. Comparison information representing a result of comparison between the change amount in frame units and the threshold TH1, TH2, TH3, or TH4 is supplied in frame units to the continuity determination unit 1083, and the process proceeds to step S1033.

  In step S1033, when the continuity determining unit 1083 (FIG. 59) is not currently set as the frame of interest among the frames constituting the moving image for which the Fy file creating unit 1004 (FIG. 43) is to create the Fy file. The frame that precedes in the sequence order is selected as the frame of interest, and the process proceeds to step S1034.

  In step S1034, based on the comparison information from the threshold processing unit 1082, the continuity determination unit 1083 has the change amount of the frame of interest equal to or greater than the threshold TH3, and the change amount immediately before, immediately after, or before and after the frame of interest. It is determined whether frames having a threshold value TH3 or more exist continuously for a minimum number N of frames.

  In step S1034, the amount of change in the frame of interest is greater than or equal to the threshold TH3, and frames with the amount of change greater than or equal to the threshold TH3 continuously exist at least the minimum number of frames N immediately before, immediately after, or before and after the frame of interest. When it is determined, the continuity determination unit 1083 supplies determination information representing the determination result to the determination unit 1084 (FIG. 59), and the process proceeds to step S1035.

  Result of determination that the amount of change in the frame of interest is greater than or equal to the threshold TH3, and that frames with the amount of change greater than or equal to the threshold TH3 continuously exist at least the minimum number of frames N immediately before, immediately after, or before and after the frame of interest In Step S1035, the determination unit 1084 that has received the supply of the determination information that represents the determination based on the comparison information from the threshold processing unit 1082 determines whether or not the amount of change of the frame of interest is equal to or greater than the threshold TH4. To do.

  If it is determined in step S1035 that the amount of change of the frame of interest is greater than or equal to the threshold TH4, the process proceeds to step S1036, and the determination unit 1084 determines the display type of the frame of interest to be the ultra-high display rate low resolution type V15. The process proceeds to step S1043.

  If it is determined in step S1035 that the amount of change in the frame of interest is not greater than or equal to the threshold TH4, that is, if the amount of change in the frame of interest is greater than or equal to the threshold TH3 and less than the threshold TH4, the process proceeds to step S1037. The display type of the attention frame is determined to be the ultra-high display rate / low resolution type V14, and the process proceeds to step S1043.

  Here, according to steps S1034 to S1037, not only the amount of change of the frame of interest is equal to or greater than the threshold TH3, but also the frame whose amount of change is equal to or greater than the threshold TH3 immediately before, immediately after, or before and after the frame of interest. When there are more than N frames continuously, the display type of the target frame is determined to be either the ultra-high display rate low resolution type V15 or the ultra-high display rate low resolution type V14. The reason is as follows.

  That is, the display type is selected from among the still image type V11, the normal type V12, the high display rate normal resolution type V13, the ultra high display rate low resolution type V14, and the ultra super high display rate low resolution type V15 shown in FIG. In the scrub, when the display type is a frame of the still image type V11, a frame of the normal type V12, and a frame of the high display rate normal resolution type V13, a high resolution image (normal resolution image) That is, the image data is displayed by processing the image data of the main line data having a larger data amount (than the proxy data).

  On the other hand, for the frames with the display type of the ultra-high display rate low resolution type V14 and the ultra-high display rate low resolution type V15 frame, the proxy has a low resolution, that is, a proxy with less data (than main line data). The image data is processed to display an image.

  For example, when the storage device 22 (FIG. 1) is a professional disc on which main line data and proxy data are recorded, scrubbing is performed using the main line data or proxy data recorded on the storage device 22. As in the case described with reference to FIG. 60, in the time-series frames constituting the moving image, the frame of the super high display rate low resolution type V14 in which the image is displayed using the proxy data, or the super ultra high display. Seek frequently occurs when the frame of the low resolution type V15 and the frame of the still image type V11, normal type V12, or high display rate normal resolution type V13, in which an image is displayed using main line data, are frequently switched. However, it is difficult to smoothly display the frame designated by the operation of the scrub bar 66 (FIG. 3). It made it there is.

  In order to prevent frequent seeks as described above, the frame of the super high display rate low resolution type V14, or the frame of the super ultra high display rate low resolution type V15, in which the image is displayed using proxy data. And the frame of the still image type V11, the normal type V12, or the high display rate normal resolution type V13 in which an image is displayed using the main line data must not be frequently switched.

  On the other hand, in both the ultra-high display rate low resolution type V14 frame and the ultra-high display rate low resolution type V15 frame, the image is displayed using proxy data, so the ultra-high display rate low resolution type V14 Seek is not a problem when switching between the frame and the ultra-high display rate low resolution type V15 frame.

  Furthermore, the still image type V11 frame, the normal type V12 frame, and the high display rate normal resolution type V13 frame all display images using main line data, so the still image type V11 frame, Seek is not a problem for switching between a frame of normal type V12 or a frame of any one of the frames of high display rate normal resolution type V13 and a frame of another display type.

  Therefore, in FIGS. 66 and 67, a frame of a display type (super high display rate / low resolution type V14, ultra super high display rate / low resolution type V15) in which an image is displayed using proxy data, and main line data are used. In order to prevent switching between frames of the display type (still image type V11, normal type V12, high display rate normal resolution type V13) in which images are displayed, at least in the frame interval that is the minimum number of frames N, In steps S1034 to S1037 described above, the frame whose amount of change is greater than or equal to the threshold TH3 and whose amount of change is greater than or equal to the threshold TH3 immediately before, immediately after, or before and after the frame of interest is the minimum frame. When the number of consecutive frames is more than N, the display type of the frame of interest is the super high display rate, low resolution type V14, or super super high It is determined in one of Display rate low resolution type V15.

  In addition, the switching between the display type frame in which the image is displayed using the proxy data and the display type frame in which the image is displayed using the main line data is at least in a frame section in which the minimum number of frames N continues. In order not to occur, in steps S1038 to S1042, which will be described later, a frame in which the amount of change of the frame of interest is less than the threshold TH3 and the amount of change is less than the threshold TH3 immediately before, immediately after, or before and after the frame of interest. However, when the minimum number of frames N continues, the display type of the frame of interest is determined as one of the still image type V11, the normal type V12, or the high display rate normal resolution type V13.

  That is, in step S1034, it is determined that the amount of change of the frame of interest is not greater than or equal to the threshold TH3, or even if the amount of change of the frame of interest is greater than or equal to the threshold TH3, either immediately before, immediately after, or before and after the frame of interest. On the other hand, if it is determined that there are no frames having the change amount equal to or greater than the threshold TH3 continuously for the minimum number N of frames, the process proceeds to step S1038, and the continuity determination unit 1083 is based on the comparison information from the threshold processing unit 1082. Whether the change amount of the frame of interest is less than the threshold TH3, and there are consecutive frames having a change amount of less than the threshold TH3 immediately before, immediately after, or before and after the frame of interest, at least the minimum number N of frames. Determine.

  In step S1038, the amount of change of the frame of interest is less than the threshold value TH3, and frames having the amount of change of less than the threshold value TH3 are continuously present at least the minimum frame number N immediately before, immediately after, or before and after the frame of interest. When it is determined, the continuity determination unit 1083 supplies determination information representing the determination result to the determination unit 1084, and the process proceeds to step S1039.

  Result of determination that the amount of change in the frame of interest is less than the threshold TH3, and that frames with the amount of change of less than the threshold TH3 exist continuously for the minimum number of frames N immediately before, immediately after, or before and after the frame of interest In Step S1039, the determination unit 1084 that has received the supply of the determination information that represents the frame of interest based on the comparison information from the threshold processing unit 1082 determines whether the change amount of the frame of interest is less than the threshold TH1 or not. It is determined whether it is greater than or equal to TH1 and less than threshold TH2, or is greater than or equal to threshold TH2 and less than threshold TH3.

  If it is determined in step S1039 that the amount of change of the frame of interest is less than the threshold TH1, the process proceeds to step S1040, and the determination unit 1084 determines the display type of the frame of interest to be the still image type V11, and then proceeds to step S1043. move on.

  If it is determined in step S1039 that the amount of change in the frame of interest is greater than or equal to the threshold TH1 and less than the threshold TH2, the process proceeds to step S1041, and the determination unit 1084 determines the display type of the frame of interest as the normal type V12. The process proceeds to step S1043.

  If it is determined in step S1039 that the amount of change in the frame of interest is greater than or equal to the threshold TH2 and less than the threshold TH3, the process proceeds to step S1042, and the determination unit 1084 sets the display type of the frame of interest as the high display rate normal resolution type V13. The process proceeds to step S1043.

  Here, according to steps S1038 to S1042, as described above, the change amount of the target frame is less than the threshold value TH3, and the change amount is less than the threshold value TH3 immediately before, immediately after, or before and after the target frame. When there are N or more frames continuously, the display type of the frame of interest is determined as the still image type V11, normal type V12, or high display rate normal resolution type V13, in which images are displayed using main line data. However, as described above, this is to prevent frequent seeks during scrubbing.

  In step S1043, the continuity determination unit 1083 includes a frame that has not yet been selected as a frame of interest among the frames constituting the moving image for which the Fy file creation unit 1004 (FIG. 43) is to create the Fy file. Determine if it exists.

  If it is determined in step S1043 that there is a frame that has not yet been selected as the target frame, the process returns to step S1033, and as described above, a frame that has not yet been selected as the target frame is newly selected as the target frame. Thereafter, the same processing is repeated.

  If it is determined in step S1043 that there is no unselected frame as the frame of interest, the display type determination unit 1023 ends the process.

  On the other hand, in step S1038, it is determined that the amount of change of the frame of interest is not less than the threshold TH3, or even if the amount of change of the frame of interest is less than the threshold TH3, either immediately before, immediately after, or before and after the frame of interest. In addition, if it is determined that there are no frames whose change amount is less than the threshold TH3 continuously for the minimum number of frames N or more, the process proceeds to step S1051 in FIG. 67, and the continuity determination unit 1083 determines whether the frame immediately before the frame of interest is Immediately after or before and after, it is determined whether or not frames whose change amount exceeds the threshold value TH3 are continuously present for the minimum number of frames N or more.

  Here, when the amount of change of the frame of interest is greater than or equal to the threshold TH3, and immediately before, immediately after, or before and after the frame of interest, the frame having the amount of change greater than or equal to the threshold TH3 continuously exists for N frames or more. In step S1036 or S1037 of FIG. 66, the display type of the frame of interest is determined to be the ultra high display rate low resolution type V14 or the ultra ultra high display rate low resolution type V15.

  Also, when the amount of change of the frame of interest is less than the threshold TH3, and there are consecutive frames with the amount of change of less than the threshold TH3 immediately before, immediately after, or before and after the frame of interest at least the minimum number of frames N In steps S1040 to S1042 of FIG. 66, the display type of the frame of interest is determined to be the still image type V11, the normal type V12, or the high display rate normal resolution type V13.

  Therefore, the process of step S1051 in FIG. 67 is performed immediately before, immediately after, and before and after the frame of interest, and there are no frames with a change amount equal to or greater than the threshold TH3 continuously, and , If there are no frames whose change amount is less than the threshold TH3 continuously for the minimum number of frames N, that is, the frame in which there is a frame whose change amount exceeds the threshold TH3 (frames whose change amount is the threshold TH3 or more) And the number of frames in which the amount of change is greater than or equal to the threshold TH3 and the number of frames in which the frames less than the threshold TH3 continue are at least the minimum This is a case where the frame is in a section less than the number of frames N (hereinafter also referred to as a mixed section as appropriate).

  As in the case described with reference to FIG. 62, the mixed section is a section in which frames with a change amount equal to or greater than the threshold TH3 are continuously present for N frames or more (this is also referred to as a large change amount section as appropriate hereinafter). And a section in which frames with a change amount less than the threshold TH3 exist continuously for the minimum number of frames N or more (this is also referred to as a small change section as appropriate hereinafter) or between two large change sections. Or between two small change intervals.

  From the above, the mixed section can be divided into a section having a section length (the number of frames) equal to or greater than the minimum frame number N and a section having a minimum frame number N.

  The mixed section is a section between the large change section and the small change section, a section sandwiched between the large change sections, or a section sandwiched between the small change sections. Can be divided into either.

  In step S1051, the continuity determination unit 1083 determines whether there is a frame having a change amount that exceeds the threshold TH3 continuously immediately before, immediately after, or before and after the frame of interest, ie, whether there is a minimum number of frames N or more. Then, it is determined whether or not the mixed section where the frame of interest exists is a section whose section length is the minimum number of frames N or more.

  If it is determined in step S1051 that frames whose change amount crosses the threshold value TH3 exist immediately before, immediately after, or before and after the frame of interest, the process proceeds to step S102 and is determined. The unit 1084 determines the display type of the attention frame as, for example, the high display rate normal resolution type V13 among the display types in which the image is displayed using the main line data, and proceeds to step S1043 in FIG. The above-described processing is performed.

  That is, for frames in a mixed section where the amount of change that crosses the threshold TH3 is continuously more than the minimum number of frames N, the display type of all the frames is used as the main line data (as image data). By using a display type in which an image is displayed or a display type in which an image is displayed using proxy data (as image data), frequent seeks can be prevented.

  Therefore, in FIG. 67, the display types of the frames in the mixed section in which the frames whose change amount crosses the threshold value TH3 exist continuously for the minimum number of frames N or more are all display types in which an image is displayed using the main line data. Among them, for example, the high display rate normal resolution type V13 is used.

  In addition, the display type of the frame of the mixed section in which the amount of change crosses the threshold TH3 continuously exists more than the minimum number of frames N, among other display types in which an image is displayed using main line data, for example It can be the normal type V12. In addition, the display type of a mixed section frame in which the amount of change spans the threshold value TH3 continuously exists for a minimum number of frames N or more, for example, an extremely high display rate at which an image is displayed using proxy data. Either the resolution type V14 or the ultra-high display rate low resolution type V15 can be used.

  On the other hand, if it is determined in step S1051 that there are no frames in which the amount of change crosses the threshold TH3 immediately before, immediately after, and before and after the frame of interest, more than the minimum number of frames N, that is, When the mixed section where the target frame exists (target mixed section) is a section whose section length is less than the minimum number of frames N, the process proceeds to step S1053, and the continuity determination unit 1083 has a section length of less than the minimum number of frames N. The amount of change in the frame (left frame) adjacent to the target mixed section among the temporally preceding frames on the left side of the target mixed section and the temporally subsequent frame on the right side of the target mixed section Of these, the change amount of one of the change amounts of the frame (right frame) adjacent to the attention mixed section is equal to or greater than the threshold TH3, and the other change amount is not the threshold TH3. Whether it is, that is, it determines whether a section length is the minimum limit number of frames N less attention mixed section is a section sandwiched between a large variation section and a small variation section.

  In Step S1053, when it is determined that the target mixed section whose section length is less than the minimum number of frames N is a section sandwiched between the large change amount section and the small change amount section, the process proceeds to Step S1052 and described above. As described above, the determination unit 1084 determines the display type of the frame of interest to be the high display rate normal resolution type V13, proceeds to step S1043 in FIG. 66, and the processing described above is performed thereafter.

  That is, both the large change amount section and the small change amount section sandwiching the target mixed section whose section length is less than the minimum number of frames N are sections whose section length is the minimum number of frames N or more. Furthermore, the display type of the frame of the large change amount section is determined to be a display type (super high display rate / low resolution type V14 or ultra super high display rate / low resolution type V15) in which an image is displayed using proxy data. The display type of the frame in the small change amount section is determined as a display type (still image type V11, normal type V12, or high display rate normal resolution type V13) in which an image is displayed using the main line data.

  In addition, for the frame of the attention mixed section that is sandwiched between the large variation section and the small variation section and whose section length is the minimum number of frames N, the display type of the frame of the large variation section, or its small variation amount By setting the display type to be the same as the display type of the frame in the section, frequent seeks can be prevented.

  Therefore, in FIG. 67, the display type of the frame of the attention mixed section in which the frame whose change amount crosses the threshold TH3 is less than the minimum number of frames N and is sandwiched between the large change amount section and the small change amount section. Are all, for example, the high display rate normal resolution type V13 among the display types in which an image is displayed using the main line data.

  Note that the display type of the frame of the attention mixed section that is between the large change amount section and the small change amount section that is the attention mixed section whose change amount crosses the threshold TH3 is less than the minimum number of frames N. For example, the normal type V12 of the display types that display images using main line data and the ultra-high display rate low resolution type V14 of the display types that display images using proxy data can be used. It is.

  On the other hand, if it is determined in step S1053 that the target mixed section whose section length is less than the minimum number of frames N is not a section sandwiched between the large change amount section and the small change amount section, the process proceeds to step S1054 and continues. The sex determination unit 1083 includes a change amount of a frame (left frame) adjacent to the target mixed section among temporally preceding frames on the left side of the target mixed section whose section length is less than the minimum number of frames N. Whether the change amount of the frame (right frame) adjacent to the target mixed section in the temporally succeeding frame on the right side of the target mixed section is greater than or equal to the threshold value TH3. Then, it is determined whether or not the target mixed section whose section length is less than the minimum number of frames N is a section sandwiched between the large change amount sections.

  In Step S1054, when it is determined that the target mixed section whose section length is less than the minimum frame number N is not a section sandwiched between the large change amount sections, the process proceeds to Step S1052, and as described above, the determination unit 1084 is performed. Determines the display type of the frame of interest to be, for example, the high display rate normal resolution type V13 among the display types in which an image is displayed using the main line data, and proceeds to step S1043 in FIG. Processing is performed.

  That is, for the frame of the attention mixed section that is sandwiched between the two small change sections and whose section length is less than the minimum frame number N, it is determined as the display type of the frames of the two small change sections. By making the display type the same as the display type in which an image is displayed using main line data, frequent seeks can be prevented.

  On the other hand, in Step S1054, when it is determined that the attention mixed section whose section length is less than the minimum frame number N is a section sandwiched between the large change amount sections, the process proceeds to Step S1055, and the determination unit 1084 Of the display types in which an image is displayed using proxy data, for example, the super high display rate / low resolution type V14 is determined as the display type of the frame of interest, and the process proceeds to step S1043 in FIG. 66. Processing is performed.

  That is, for the frame of the attention mixed section sandwiched between the two large change amount sections and the section length is less than the minimum frame number N, it is determined as the display type of the frame of the two large change amount sections. By using the same display type as that for displaying an image using proxy data, frequent seeks can be prevented.

  Next, FIG. 68 shows another example of the display type determined by the display type determination unit 1023 of FIG.

  In FIG. 68, there are three display types: a low resolution normal display rate type C1, a normal type C2, and a normal resolution low display rate type C3, which have different resolutions for displaying frames and display rates for displaying frames. is there.

  Low resolution normal display rate Type C1 frames have the same display rate (normal rate) as the video frame rate, and the same resolution as the image data as proxy data (lower resolution than the image data as main line data). , The image is displayed.

  For the normal type C2 frame, the image is displayed at the normal display rate and at the same resolution as the main line image data (referred to as normal resolution).

  For frames of the normal resolution low display rate type C3, the resolution is lower than the normal rate, for example, half the normal rate, and the same resolution as the image data as the main line data (from the image data as the proxy data) (High resolution).

  As the display types, as shown in FIG. 68, when there are three low resolution normal display rate type C1, normal type C2, and normal resolution low display rate type C3, the change amount calculation unit 1022 (FIG. 43) Fine information representing the degree of spatial change is obtained as the amount of change. Then, for example, the display type determination unit 1023 compares the amount of change (fineness information) of each frame with two predetermined threshold values, and based on the comparison result, the display type of the frame is displayed as a low-resolution normal display. One of the rate type C1, the normal type C2, and the normal resolution low display rate type C3 is determined.

  That is, FIG. 69 shows fineness information as a change amount in frame units and a display type that the display type determination unit 1023 (FIG. 43) determines in frame units based on the change amount.

  In FIG. 69, the horizontal axis represents the frame, and the vertical axis represents the amount of change.

  The display type determination unit 1023 compares the amount of change (fineness information) in units of frames with a threshold value L or a threshold value H having a relationship of expression L <H. The display type determination unit 1023 displays a display type of a frame whose fineness information is greater than or equal to the threshold value H, that is, a frame indicating that the fineness information has a large degree of spatial change, with a high resolution or a low display. The normal resolution low display rate type C3, which is a type of high resolution / low display rate type, is determined.

  Here, for example, in scrubbing performed in editing, if a frame of a large degree of spatial change, that is, a frame of an image (complex image) with a large change in pixel value is displayed at a low resolution, In order to easily overlook spatial changes, it is desirable to display a complex image frame at a high resolution.

  However, the burden of processing for displaying a complex image at a high resolution is greater than the burden of processing for displaying a non-complex, for example, flat image.

  For this reason, complex image frames are displayed at a high resolution, but instead, a normal resolution low display rate type C3, which is a type of a high resolution / low display rate type that displays an image at a low display rate, is adopted. I will do it. When displaying an image at a low display rate, the processing burden is less than when displaying an image at a high display rate, so a high-resolution image can be displayed at a low display rate for complex image frames. By displaying, it is possible to reduce the burden on the scrubbing apparatus while preventing the spatial change of the image from being overlooked during editing.

  On the other hand, the display type determination unit 1023 displays a display type of a frame whose fineness information is less than the threshold L, that is, a frame indicating that the fineness information has a small degree of spatial change, as a normal resolution low display rate type. The low-resolution normal display rate type C1 that displays an image at a lower resolution than C3 and at a higher normal-rate display rate than the normal-resolution low-display-rate type C3 is determined.

  Here, for a frame with a small degree of spatial change, that is, a flat image frame, even if the image is displayed at a low resolution even when the image is displayed at a low resolution, It does n’t change that much.

  In this way, displaying images with high resolution, whether they are displayed at high resolution or low resolution, with a high resolution, is a wasteful burden on the scrubbing device. It will be over.

  Therefore, the display type of the flat image frame is a low resolution normal display image with a lower resolution than the normal resolution low display rate type C3 and a normal display rate higher than the normal resolution low display rate type C3. The display rate type is C1. Thereby, it is possible to prevent an unnecessary burden on the scrubbing apparatus.

  The display type determination unit 1023 displays a frame other than a frame whose fineness information is greater than or equal to the threshold H and a frame whose fineness information is less than the threshold L, that is, a frame whose fineness information is greater than or equal to the threshold L and less than the threshold H. Is normally determined as type C2.

  That is, the display type of an image frame that is not so complicated but not flat is a normal type C2 that displays an image at a normal rate (the same display rate as the frame rate of a moving image) and at a normal resolution. .

  Next, referring to the flowcharts of FIGS. 70 and 71, the display type is the low resolution normal display rate type C1, the normal type C2, or the normal resolution low shown in FIG. The processing of the display type determination unit 1023 in FIG. 59 when it is determined as any one of the display rate types C3 will be described.

  When the change amount for each frame is supplied from the change amount calculation unit 1022 (FIG. 43), the display type determination unit 1023 caches the change amount for each frame in the storage unit 1081 (FIG. 59) (step S1061). Temporary storage), and the process proceeds to step S1062.

  Here, in FIG. 70 and FIG. 71, the fineness information is supplied from the change amount calculation unit 1022 to the display type determination unit 1023 as the change amount.

  In step S1062, the threshold processing unit 1082 performs threshold processing for comparing the amount of change (fineness information) in units of frames stored in the storage unit 1081 with the threshold H or L, and the amount of change in units of frames and the threshold Comparison information representing the result of comparison with H or L is supplied to the continuity determination unit 1083 in units of frames, and the process proceeds to step S1063.

  In step S1063, the continuity determination unit 1083 is the most time-sequential of the frames constituting the moving image for which the Fy file creation unit 1004 (FIG. 43) is currently creating an unaware frame. The preceding frame is selected as the target frame, and the process proceeds to step S1064.

  In step S 1064, the continuity determination unit 1083 determines that the amount of change of the frame of interest is less than the threshold L based on the comparison information from the threshold processing unit 1082, and the amount of change immediately before, immediately after, or before and after the frame of interest. It is determined whether or not frames having a value less than the threshold continuously exist for a minimum number N of frames.

  In step S1064, the amount of change of the frame of interest is less than the threshold value L, and there are frames having the amount of change of less than the threshold value L immediately before, immediately after, or before and after the frame of interest continuously for the minimum number N of frames. When it is determined, the continuity determination unit 1083 supplies determination information indicating the determination result to the determination unit 1084, and the process proceeds to step S1065.

  In step S1065, the determination unit 1084 determines the display type of the frame of interest to be the low resolution normal display rate type C1 based on the determination information from the continuity determination unit 1083, and the process proceeds to step S1070.

  Here, according to steps S1064 and S1065, not only the amount of change of the frame of interest is less than the threshold L, but also the frame of which the amount of change is less than the threshold L immediately before, immediately after, or before and after the frame of interest. When N is continuously present for N frames or more, the display type of the target frame is determined to be the low resolution normal display rate type C1, for the following reason.

  That is, when the display type is determined from the low resolution normal display rate type C1, the normal type C2, and the normal resolution low display rate type C3 shown in FIG. For a frame of the normal display rate type C1, an image is displayed by processing an image having a low resolution, that is, proxy data having a smaller data amount (than main line data).

  On the other hand, for frames with the display type normal type C2 and normal resolution low display rate type C3, images with high resolution, that is, main line data with a larger amount of data (than proxy data data) are processed. An image is displayed.

  For example, when the storage device 22 (FIG. 1) is a professional disc on which main line data and proxy data are recorded, scrubbing is performed using the main line data or proxy data recorded on the storage device 22. As in the case described with reference to FIG. 60, in the time-series frames constituting the moving image, the low-resolution normal display rate type C1 frame in which the image is displayed using the proxy data and the main line data are used. When the frame of the normal type C2 where the image is displayed or the frame of the normal resolution low display rate type C3 is frequently switched, the seek frequently occurs, and the display of the frame specified by the operation of the scrub bar 66 (FIG. 3) is performed. It can be difficult to do smoothly.

  To prevent frequent seeks as described above, low-resolution normal display rate type C1 frames that display images using proxy data and normal types that display images using main line data It is necessary to prevent frequent switching between frames of C2 or normal resolution low display rate type C3.

  On the other hand, the normal type C2 frame and the normal resolution low display rate type C3 frame both display images using the main line data, so the normal type C2 frame and the normal resolution low display rate type C3 frame Seek is not a problem when switching between frames.

  Therefore, in FIGS. 70 and 71, a frame of a display type (low resolution normal display rate type C1) in which an image is displayed using proxy data and a display type (normal type C2) in which an image is displayed using main line data. In step S1064 and S1065 described above, the change amount of the frame of interest is set to a threshold value so that switching to the frame of the normal resolution low display rate type C3) does not occur at least in the frame section that is continuous by the minimum number of frames N. If the frame that is less than L and the change amount is less than the threshold L immediately before, immediately after, or before and after the frame of interest continues for the minimum number of frames N or more, the display type of the frame of interest is proxy The low-resolution normal display rate type C1 in which an image is displayed using data is determined.

  In addition, the switching between the display type frame in which the image is displayed using the proxy data and the display type frame in which the image is displayed using the main line data is at least in a frame section in which the minimum number of frames N continues. In order not to occur, in steps S1066 to S1069, which will be described later, a frame in which the amount of change of the frame of interest is equal to or greater than the threshold L and the amount of change is equal to or greater than the threshold L immediately before, immediately after, or before and after the frame of interest. However, when the minimum number of frames N continues, the display type of the frame of interest is determined as the normal type C2 in which an image is displayed using the main line data, or the normal resolution low display rate type C3.

  That is, in step S1064, it is determined that the amount of change of the frame of interest is not less than the threshold L, or even before the frame of interest, immediately after, or before or after the frame of interest is less than the threshold L. On the other hand, if it is determined that there are no frames whose change amount is less than the threshold L continuously for the minimum number of frames N or more, the process proceeds to step S1066, where the continuity determination unit 1083 is based on the comparison information from the threshold processing unit 1082. Whether the change amount of the frame of interest is greater than or equal to the threshold value L, and whether or not there are consecutive frames having the change amount of the threshold value L or more immediately before, immediately after, or before and after the frame of interest more than the minimum number N of frames. Determine.

  In step S1066, if the amount of change of the frame of interest is greater than or equal to the threshold L, and frames immediately before, immediately after, or before and after the frame of interest exist continuously over the minimum number N of frames. When it is determined, the continuity determination unit 1083 supplies determination information indicating the determination result to the determination unit 1084, and the process proceeds to step S1067.

  Result of determination that the amount of change in the frame of interest is greater than or equal to the threshold L, and that frames with the amount of change equal to or greater than the threshold L immediately before, immediately after, or before and after the frame of interest continuously exist for the minimum number N of frames. In Step S1067, the determination unit 1084 that has received the supply of the determination information that represents the determination on the basis of the comparison information from the threshold processing unit 1082 determines whether or not the amount of change of the frame of interest is greater than or equal to the threshold H. To do.

  If it is determined in step S1067 that the amount of change in the frame of interest is greater than or equal to the threshold H, the process proceeds to step S1068, and the determination unit 1084 determines the display type of the frame of interest as the normal resolution low display rate type C3. The process proceeds to step S1070.

  If it is determined in step S1067 that the amount of change of the frame of interest is not greater than or equal to the threshold H, that is, if the amount of change of the frame of interest is greater than or equal to the threshold L and less than the threshold H, the process proceeds to step S1069. The display type of the attention frame is determined to be the normal type C2, and the process proceeds to step S1070.

  Here, according to steps S1066 to S1069, the amount of change of the frame of interest is equal to or greater than the threshold value L, and frames having the amount of change equal to or greater than the threshold value L continue immediately before, immediately after, or before and after the frame of interest. The display type is determined to be either the normal resolution low display rate type C3 or the normal type C2 where the image is displayed using the main line data, as described above. This is to prevent frequent seeks during scrubbing.

  In step S1070, the continuity determination unit 1083 does not yet select a frame that has not yet been selected as a frame of interest among the frames constituting the moving image for which the Fy file creation unit 1004 (FIG. 43) is to create the Fy file. Determine if it exists.

  If it is determined in step S1070 that there is still a frame that has not been selected as the target frame, the process returns to step S1063, and as described above, a frame that has not yet been selected as the target frame is newly selected as the target frame. Thereafter, the same processing is repeated.

  If it is determined in step S1070 that there is no unselected frame as the frame of interest, the display type determination unit 1023 ends the process.

  On the other hand, in step S1066, it is determined that the amount of change of the frame of interest is not greater than or equal to the threshold L, or even before the frame of interest, immediately after, or before or after the frame of interest is greater than or equal to the threshold L. On the other hand, if it is determined that there are no frames whose change amount is equal to or greater than the threshold L continuously beyond the minimum number of frames N, the process proceeds to step S1071 in FIG. 71, and the continuity determination unit 1083 immediately before or immediately after the frame of interest. In addition, it is determined whether or not the frames whose change amount crosses the threshold value L exist continuously over the minimum number N before and after.

  Here, when the amount of change of the attention frame is less than the threshold L, and immediately before, immediately after, or before and after the attention frame, frames having the amount of change less than the threshold L exist continuously for N frames or more, In step S1065 of FIG. 70, the display type of the target frame is determined to be the low resolution normal display rate type C1.

  Also, when the amount of change of the frame of interest is greater than or equal to the threshold L, and there are consecutive frames with the amount of change of the threshold L or greater immediately before, immediately after, or before and after the frame of interest, at least the minimum number of frames N In step S1068 or S1069 of FIG. 70, the display type of the frame of interest is determined to be the normal resolution low display rate type C3 or the normal type C2.

  Therefore, the process of step S1071 in FIG. 71 is performed because there are no frames with a change amount less than the threshold L continuously for more than N frames immediately before, immediately after, and before and after the frame of interest, and When frames with a change amount equal to or greater than the threshold value L do not exist continuously for the minimum number of frames N or more, that is, the target frame is a section in which there is a frame where the change amount exceeds the threshold value L (a frame whose change amount is less than the threshold value L). , And the number of frames in which the amount of change is less than the threshold value L and the number of frames in which the frame value of the threshold value L or more continues are at most the minimum number of frames. This is a case where the frame is in a section less than the number N (also referred to as a mixed section as appropriate hereinafter).

  As in the case described with reference to FIG. 62, the mixed section is a section in which frames having a change amount less than the threshold L are continuously present for N frames or more (also referred to as a small change amount section as appropriate hereinafter). And a section where frames with a change amount equal to or greater than the threshold L exist continuously for the minimum number of frames N or more (this is also referred to as a large change section as appropriate hereinafter) or between two large change sections. Or between two small change intervals.

  From the above, the mixed section can be divided into a section having a section length (the number of frames) equal to or greater than the minimum frame number N and a section having a minimum frame number N.

  The mixed section is a section between the large change section and the small change section, a section sandwiched between the large change sections, or a section sandwiched between the small change sections. Can be divided into either.

  In step S1071, the continuity determination unit 1083 determines whether there is a frame having a change amount that crosses the threshold value L immediately before, immediately after, or before and after the frame of interest, that is, the minimum number of frames N or more. Then, it is determined whether or not the mixed section where the frame of interest exists is a section whose section length is the minimum number of frames N or more.

  If it is determined in step S1071 that frames whose change amount exceeds the threshold value L immediately before, immediately after, or before and after the frame of interest continue to exist for the minimum number of frames N or more, the process proceeds to step S1072. The unit 1084 determines, for example, the normal type C2 among the display types in which an image is displayed using main line data, and proceeds to step S1070 in FIG. Is done.

  That is, for frames in a mixed section in which the amount of change spans the threshold value L continuously for the minimum number of frames N or more, the main frame data (as image data) is used as the display type of all the frames. By using a display type in which an image is displayed or a display type in which an image is displayed using proxy data (as image data), frequent seeks can be prevented.

  Therefore, in FIG. 71, the display types of the frames in the mixed section in which the frames whose change amount crosses the threshold value L exist continuously for the minimum number of frames N or more are all display types in which an image is displayed using main line data. Of these, for example, the normal type C2.

  In addition, the display type of the frame of the mixed section in which the amount of change spans the threshold value L continuously exists for the minimum number of frames N or more, for example, a low-resolution normal display rate at which an image is displayed using proxy data, for example It can be C1.

  On the other hand, if it is determined in step S1071 that there are no frames whose change amount exceeds the threshold value L immediately before, immediately after, or before or after the frame of interest, ie, there is no minimum frame number N, that is, When the mixed section in which the frame exists (the attention mixed section) is a section whose section length is less than the minimum number of frames N, the process proceeds to step S1073, and the continuity determination unit 1083 has a section length of less than the minimum number of frames N. Among the temporally preceding frames on the left side of the target mixed section, the change amount of the frame adjacent to the target mixed section (left frame) and the temporally subsequent frame on the right side of the target mixed section Of these, the change amount of one of the change amounts of the frame (right frame) adjacent to the target mixed section is less than the threshold value L, and the other change amount is not less than the threshold value L. Whether, i.e., determines whether a section length is the minimum limit number of frames N less attention mixed section is a section sandwiched between a large variation section and a small variation section.

  In Step S1073, when it is determined that the target mixed section whose section length is less than the minimum frame number N is a section sandwiched between the large change amount section and the small change amount section, the process proceeds to Step S1072 and described above. As described above, the determination unit 1084 determines the display type of the frame of interest to be the normal type C2, and proceeds to step S1070 in FIG. 70, where the processing described above is performed.

  That is, both the large change amount section and the small change amount section sandwiching the target mixed section whose section length is less than the minimum number of frames N are sections whose section length is the minimum number of frames N or more. Furthermore, the display type of the frame in the large change interval is determined by the display type (normal type C2 or normal resolution low display rate type C3) in which the image is displayed using the main line data. The type is determined as a display type (low-resolution normal display rate type C1) in which an image is displayed using proxy data.

  In addition, for the frame of the attention mixed section that is sandwiched between the large variation section and the small variation section and whose section length is the minimum number of frames N, the display type of the frame of the large variation section, or its small variation amount By setting the display type to be the same as the display type of the frame in the section, frequent seeks can be prevented.

  Therefore, in FIG. 71, the frame type of the target mixed section between the large change amount section and the small change amount section, which is the target mixed section whose change amount crosses the threshold value L is less than the minimum number of frames N. Are all the normal type C2 among the display types in which an image is displayed using the main line data.

  Note that the display type of the frame of the attention mixed section sandwiched between the large change amount section and the small change amount section is the attention mixed section in which the change amount crosses the threshold value L is less than the minimum number of frames N. For example, the display type of the frame in the small change amount section, that is, the low resolution normal display rate type C1 in which an image is displayed using proxy data can be used.

  On the other hand, in Step S1073, when it is determined that the target mixed section whose section length is less than the minimum frame number N is not a section sandwiched between the large change amount section and the small change amount section, the process proceeds to Step S1074 and continues. The sex determination unit 1083 includes a change amount of a frame (left frame) adjacent to the target mixed section among temporally preceding frames on the left side of the target mixed section whose section length is less than the minimum number of frames N. Whether the change amount of the frame (right frame) adjacent to the target mixed section in the temporally following frame on the right side of the target mixed section is less than the threshold L, that is, Then, it is determined whether or not the target mixed section whose section length is less than the minimum number of frames N is a section sandwiched between the small change amount sections.

  If it is determined in step S1074 that the target mixed section whose section length is less than the minimum number of frames N is not a section sandwiched between the small change amount sections, that is, the target whose section length is less than the minimum number of frames N When the mixed section is a section sandwiched between the large change amount sections, the process proceeds to step S1072, and as described above, the determination unit 1084 sets the display type of the attention frame to the display type of the frame of the large change amount section. That is, among the display types in which an image is displayed using the main line data, for example, the normal type C2 is determined, and the process proceeds to step S1070 in FIG. 70, and the above-described processing is performed.

  That is, for the frame of the attention mixed section sandwiched between the two large change amount sections and the section length is less than the minimum frame number N, it is determined as the display type of the frame of the two large change amount sections. By making the display type the same as the display type in which an image is displayed using main line data, frequent seeks can be prevented.

  On the other hand, if it is determined in step S1074 that the target mixed section whose section length is less than the minimum frame number N is a section sandwiched between the small change amount sections, the process proceeds to step S1075, and the determination unit 1084 The display type of the frame of interest is determined to be the low-resolution normal display rate type C1 in which an image is displayed using proxy data, and the process proceeds to step S1070 in FIG. 70, where the above-described processing is performed.

  That is, for the frame of the attention mixed section that is sandwiched between the two small change sections and whose section length is less than the minimum frame number N, it is determined as the display type of the frames of the two small change sections. By using the same display type as the low-resolution normal display rate type C1 in which images are displayed using proxy data, frequent seeks can be prevented.

  Next, in FIG. 72, the change amount calculation unit 1022 in FIG. 43 obtains the motion information and the fineness information as the change amount, and the display type determination unit 1023 displays the display type based on the movement information and the fineness information. 43 shows an example of an Fy file created by the file creation unit 1025 of FIG. 43 when the type is determined and both the change amount and the display type are selected by the selection unit 1024.

  In the Fy file of FIG. 72, from left to right, a frame number indicating the number of the frame from the head, a time code of a frame corresponding to the frame number, motion information as a frame change amount corresponding to the frame number, A display type based on the fineness information and the frame motion information corresponding to the frame number and a display type based on the fineness information are sequentially arranged.

  In FIG. 73, when the change amount calculation unit 1022 in FIG. 43 obtains the motion information and the fineness information as the change amount, and the selection unit 1024 selects only the change amount, the file creation unit 1025 in FIG. An example of the Fy file to be created is shown.

  In the Fy file of FIG. 73, from left to right, a frame number indicating the frame number from the head, a time code of a frame corresponding to the frame number, motion information as a frame change amount corresponding to the frame number, Fineness information is arranged sequentially.

  74, the change amount calculation unit 1022 in FIG. 43 obtains motion information and fineness information as the change amount, and the display type determination unit 1023 determines the display type based on the motion information and the display type based on the fineness information. FIG. 43 shows an example of an Fy file created by the file creation unit 1025 in FIG. 43 when only the display type is selected in the selection unit 1024.

  In the Fy file of FIG. 74, the display type based on the frame number indicating the frame number from the top in the left to right direction, the time code of the frame corresponding to the frame number, and the motion information of the frame corresponding to the frame number Display types based on fineness information are sequentially arranged.

  Next, in the above case, the display type is determined based on either the motion information or the fineness information, but the display type is based on both the motion information and the fineness information. It is also possible to decide.

  That is, FIG. 75 shows another example of the display type determined by the display type determination unit 1023 of FIG.

  In FIG. 75, as a display type, a still image type VC1 representing display in a still image, a normal type VC2 having a different resolution when displaying a frame and a display rate when displaying a frame, a high display rate and a low resolution There are four types: type VC3 and normal display rate low resolution type VC4.

  As for the still image type VC1, as with the still image type V1 in FIG. 57, even if the display target frame is changed, the image of the display target frame is the same as the frame (the image) displayed immediately before. As long as it can be regarded as the same image, the frame (image) displayed immediately before is displayed (continues).

  As for the normal type VC2 frame, similarly to the normal type V2 in FIG. 57, the image has the same display rate (normal rate) as the moving image frame rate, and the same resolution (normal resolution) as the main line image data. Is displayed.

  As for the high display rate / low resolution type VC3 frame, similarly to the high display rate / low resolution type V3 in FIG. 57, the image data as proxy data is higher than the normal rate, for example, twice the normal rate. The image is displayed at the same resolution as that of the image data (a resolution lower than the image data as the main line data).

  For the frame of the normal display rate low resolution type VC4, an image is displayed at the normal rate display rate and at the same resolution as the image data as proxy data (lower resolution than the image data as main line data).

  Here, the high display rate low resolution type VC3 and the normal display rate low resolution type VC4 display images at a display rate higher than the normal rate in the high display rate low resolution type VC3. The display rate low resolution type VC4 is different only in that the image is displayed at the normal rate (display rate).

  As the display types, as shown in FIG. 75, when there are four still image types VC1, normal type VC2, high display rate low resolution type VC3, and normal display rate low resolution type VC4, change amount calculation unit 1022 (FIG. In 43), motion information and fineness information are obtained as the amount of change. Then, the display type determination unit 1023 (FIG. 43) is based on both the motion information and the fineness information, that is, for example, compares the motion information with two threshold values and compares the fineness information with one threshold value. Based on the comparison result, the frame display type is determined to be one of the still image type VC1, the normal type VC2, the high display rate low resolution type VC3, or the normal display rate low resolution type VC4.

  That is, FIG. 76 shows the motion information and fineness information as the change amount in frame units, and the display type determined by the display type determination unit 1023 (FIG. 43) in frame units based on the motion information and fineness information. Show.

  Note that the upper part of FIG. 76 represents motion information in units of frames, and the lower part of FIG. 76 represents fineness information in units of frames.

  In FIG. 76, the horizontal axis represents the frame, and the vertical axis represents the amount of change (motion information, fineness information).

  The display type determination unit 1023 (FIG. 43) compares the frame-by-frame motion information with a threshold value L or H having a relationship of the expression L <H, and determines a temporary display type based on the comparison result (display). Tentatively determine the type).

  That is, the display type determination unit 1023 determines the temporary display type of the frame based on the motion information of the frame, for example, as described with reference to FIGS. 57 to 62, as the still image type V1 shown in FIG. The normal type V2 or the high display rate low resolution type V3 is determined (the display type is provisionally determined).

  Accordingly, as described in FIGS. 57 to 62, roughly, the display type of the frame whose motion information is less than the threshold value L is the still image type V1, and the display of the frame whose motion information is greater than or equal to the threshold value L and less than the threshold value H is displayed. The type is temporarily determined as the normal type V2, and the display type of the frame whose motion information is greater than or equal to the threshold value H is temporarily determined as the high display rate and low resolution type V3.

  Also, the display type determination unit 1023 (FIG. 43) compares the frame unit fineness information with a predetermined threshold K, and based on the comparison result and the provisional display type provisionally determined based on the motion information. The frame display type is finally determined as one of the still image type VC1, the normal type VC2, the high display rate low resolution type VC3, or the normal display rate low resolution type VC4 shown in FIG.

  That is, the display type determination unit 1023 finally determines the display type of the frame whose temporary display type is the still image type V1 as the still image type VC1, and the temporary display type is the frame whose high display rate is the low resolution type V3. Is finally determined to be a high display rate, low resolution type VC3.

  Further, the display type determination unit 1023 selects the display type of a frame whose fineness information is greater than or equal to the threshold value K among frames whose temporary display type is the normal type V2, that is, a frame of an image that cannot be said to be a flat image. The final decision is made on the normal type VC2. Further, the display type determination unit 1023 selects the display type of the frame whose fineness information is less than the threshold value K among the frames whose provisional display type is the normal type V2, that is, the display type of the flat image frame, as the normal display rate and the low resolution. Final decision on type VC4.

  That is, a frame whose provisional display type is the normal type V2 is a frame that does not move so much but has a certain amount of movement. A frame whose fineness information is less than the threshold value K is a flat image frame. Therefore, the frame whose fineness information is less than the threshold value K among the frames whose temporary display type is the normal type V2 is a flat image frame although there is some movement.

  Even if the image has a certain amount of movement, if it is a flat image, even if it is displayed at a low resolution, the spatial change of the image cannot be easily overlooked. Further, when an image is displayed at a low resolution in scrubbing, it is possible to reduce the load on the scrubbing apparatus.

  Therefore, for frames with a certain amount of movement and the temporary display type is the normal type V2, the display type is finally determined to the normal type VC2 in which the image is displayed at the normal resolution in principle. Only when the information is less than the threshold value K, the normal display rate low resolution type VC4 in which an image is displayed at a resolution lower than the normal resolution is finally determined.

  Here, in FIG. 76, the display type of the frame whose section information DP is the frame DP whose fineness information is less than the threshold value K and whose temporary display type based on the motion information is the normal type V2 is the normal display rate low resolution. Type VC4 is finally decided.

  Note that one of the still image type VC1, normal type VC2, high display rate low resolution type VC3, or normal display rate low resolution type VC4 shown in FIG. Hereinafter, the display type finally determined is referred to as a final display type as appropriate.

  Next, referring to the flowchart of FIG. 77, the display type (final display type) is based on both the motion information and the fineness information, and the still image type VC1, the normal type VC2, and the high display rate low shown in FIG. The processing of the display type determination unit 1023 in FIG. 59 when it is determined to be either the resolution type VC3 or the normal display rate low resolution type VC4 will be described.

  Note that the change amount calculation unit 1022 (FIG. 43) obtains the motion information and the fineness information as the change amount and supplies them to the display type determination unit 1023.

  In step S1081, the display type determination unit 1023, based on the motion information in units of frames from the change amount calculation unit 1022 (FIG. 43), displays the display type (provisional) based on the motion information as described with reference to FIGS. Display type) and the process proceeds to step S1082.

  In step S1082, the display type determination unit 1023 causes the storage unit 1081 (FIG. 59) to cache the frame unit fineness information supplied from the change amount calculation unit 1022, and the process proceeds to step S1083.

  In step S1083, the threshold processing unit 1082 (FIG. 59) performs threshold processing for comparing the frame unit fineness information stored in the storage unit 1081 with the threshold K, and the frame unit fineness information change amount and the threshold K are compared. The comparison information representing the comparison result is supplied to the continuity determination unit 1083 in units of frames, and the process proceeds to step S1084.

  In step S1084, when the continuity determining unit 1083 (FIG. 59) is not the frame of interest yet among the frames constituting the moving image for which the Fy file creating unit 76 (FIG. 43) is currently creating. The frame that precedes in the sequence order is selected as the frame of interest, and information on the frame of interest is supplied to the determination unit 1084 (FIG. 59), and the process proceeds to step S1085.

  In step S1085, the determination unit 1084 determines whether the display type based on the motion information of the frame of interest (the temporary display type determined in step S1081) is the still image type V1.

  If it is determined in step S1085 that the display type based on the motion information of the target frame is the still image type V1, the process proceeds to step S1086, and the determination unit 1084 determines the final display type of the target frame (the target frame motion information and The display type based on the fineness information) is finally determined to be the still image type VC1, and the process proceeds to step S1093.

  If it is determined in step S1085 that the display type based on the motion information of the target frame is not the still image type V1, the process proceeds to step S1087, and the determination unit 1084 determines that the display type based on the motion information of the target frame is high. It is determined whether or not the display rate is the low resolution type V3.

  If it is determined in step S1087 that the display type based on the motion information of the frame of interest is the high display rate low resolution type V3, the process proceeds to step S1088, and the determination unit 1084 displays the final display type of the frame of interest as the high display. The rate low resolution type VC3 is finally determined, and the process proceeds to step S1093.

  In step S1087, when it is determined that the display type based on the motion information of the target frame is not the high display rate low resolution type V3, that is, the display type based on the motion information of the target frame is the normal type V2. In step S1089, the continuity determination unit 1083 (FIG. 59), based on the comparison information from the threshold processing unit 1082, has fineness information of the attention frame whose display type based on the motion information is the normal type V2 is less than the threshold K. It is determined whether or not there are consecutive frames having fineness information less than the threshold value K more than the minimum number of frames N immediately before, immediately after, or before and after the frame of interest.

  In step S1089, even if the fineness information of the attention frame is not less than the threshold value K, or the fineness information of the attention frame is less than the threshold value K, the fineness information is obtained immediately before, immediately after, and before and after the attention frame. When it is determined that there are no frames whose information is less than the threshold value K continuously for the minimum number of frames N or more, the continuity determination unit 1083 supplies the determination information indicating the determination result to the determination unit 1084 (FIG. 59). Then, the process proceeds to step S1090.

  As a result of determining that the fineness information of the attention frame is not less than the threshold value K, or the number of frames whose fineness information is less than the threshold value K, immediately before, immediately after, and before and after the attention frame, is the minimum number of frames. The determination unit 1084 (FIG. 59) that has received the supply of the determination information indicating the determination result that it does not exist continuously for N or more from the continuity determination unit 1083, the display type based on the motion information is the normal type V2 in step S1090. The final display type of the current frame of interest is finally determined to be the normal type VC2, and the process proceeds to step S1093.

  In step S1089, the fineness information of the frame of interest is less than the threshold value K, and the frames with the fineness information of less than the threshold value K are continuous for the minimum number of frames N or more immediately before, immediately after, or before and after the frame of interest. If it is determined that the frame exists, the process advances to step S1091, and the continuity determination unit 1083 (FIG. 59) determines the minimum number of frames immediately before, immediately after, or before and after the frame of interest whose fineness information is less than the threshold value K. The same display type continues when the display type of the normal type V2 frame is changed to the normal display rate low resolution type VC4 among the frames in the interval of N or more consecutive frames. It is determined whether or not there is a section in which the number of frames is less than the minimum number of frames N.

  That is, in step S1091, for example, in FIG. 76, if the section DP is a section in which the frames whose fineness information is less than the threshold value K are continuous for the minimum number N of frames, the motion of the frames in the section DP When you change the display type of the frame whose information display type is the normal type V2 to the normal display rate low resolution type VC4, the motion information and fineness information that is finally obtained for the movie that is going to create the Fy file In the time series of display types based on the above, it is determined whether or not a section in which the same display type continues in the section where the same display type continues is less than the minimum frame number N is determined.

  In step S1091, the display type based on the motion information is normal among the frames in the section of the frame that is continuous for the minimum number of frames N immediately before, immediately after, or before and after the target frame whose fineness information is less than the threshold value K. When the display type of the frame of type V2 is changed to the normal display rate low resolution type VC4, when it is determined that the interval in which the same display type continues is less than the minimum number of frames N, that is, If the final display type of the frame of interest whose display type based on the motion information is the normal type V2 is the normal display rate low resolution type VC4, if there is a risk of frequent seeks, the process proceeds to step S1090 and the determination unit 1084 (FIG. 59). As described above, the final display type of the frame of interest whose display type based on motion information is the normal type V2, Finally decided to ordinary type VC2, the process proceeds to step S1093.

  Further, in step S1091, the display type based on the motion information of the frames in the section of the frame that is continuous for the minimum number of frames N, immediately before, immediately after, or before and after the target frame whose fineness information is less than the threshold value K. However, even if the display type of the normal type V2 frame is changed to the normal display rate low resolution type VC4, it is determined that there is no section where the number of frames with the same display type is less than the minimum number of frames N. In step S1092, the determination unit 1084 (FIG. 59) finally determines the final display type of the target frame whose display type based on the motion information is the normal type V2 as the normal display rate low resolution type VC4. The process proceeds to step S1093.

  Here, when the display type (final display type) is determined from among the still image type VC1, the normal type VC2, the high display rate low resolution type VC3, and the normal display rate low resolution type VC4 shown in FIG. In scrubbing, for the still image type VC1 frame and the normal type VC2 frame, a high-resolution image (normal resolution image), that is, a main line with a larger amount of data (than proxy data) The image data is processed to display an image.

  In addition, for frames with a high display rate, low resolution type VC3 and a frame with a normal display rate, low resolution type VC4, the display type is an image with a low resolution, that is, proxy data with a smaller amount of data (than main line data). An image is displayed by processing the image data.

  For example, when the storage device 22 (FIG. 1) is a professional disc on which main line data and proxy data are recorded, scrubbing is performed using the main line data or proxy data recorded on the storage device 22. As in the case described with reference to FIG. 60, the frame of the high display rate / low resolution type VC3 in which the image is displayed using the proxy data in the time-series frames constituting the moving image or the normal display rate of the low resolution. When the type VC4 frame and the still image type VC1 in which an image is displayed using the main line data or the frame of the normal type VC2 are frequently switched, a seek frequently occurs and the scrub bar 66 (FIG. 3) is operated. It may be difficult to smoothly display the frame specified by.

  To prevent frequent seeks as described above, high-display-rate, low-resolution type VC3 frames that display images using proxy data, or normal-display-rate, low-resolution type VC4 frames, and the main line It is necessary to prevent frequent switching between the still image type VC1 in which an image is displayed using data or the frame of the normal type VC2.

  Therefore, in FIG. 77, in steps S1090 to S1092, the frames in the section of the frame that is continuous more than the minimum number of frames N immediately before, immediately after, or before and after the target frame whose fineness information is less than the threshold value K. Even if the display type based on motion information is the normal type V2 and the display type is changed to the normal display rate low resolution type VC4, the interval in which the same display type continues is less than the minimum number of frames N The final display type of the attention frame whose display type based on motion information is the normal type V2 is determined as the normal display rate low resolution type VC4, and the other cases (the same display type (final display type)) If the number of consecutive frames is less than the minimum number of frames N), the display type based on motion information is not accepted. Type final display type of the frame of interest V2, it adapted to determine the normal display rate low resolution type VC4.

  In step S1093, the continuity determination unit 1083 includes a frame that has not yet been selected as a frame of interest among the frames constituting the moving image for which the Fy file creation unit 1004 (FIG. 43) is to create the Fy file. Determine if it exists.

  If it is determined in step S1093 that there is still a frame that has not been selected as the target frame, the process returns to step S1084, and as described above, a frame that has not yet been selected as the target frame is newly selected as the target frame. Thereafter, the same processing is repeated.

  If it is determined in step S1093 that there is no unselected frame as the frame of interest, the display type determination unit 1023 ends the process.

  Next, in FIG. 78, the change amount calculation unit 1022 in FIG. 43 obtains the motion information and the fineness information as the change amount, and the display type determination unit 1023 displays the display type (final display) based on the movement information and the fineness information. 43 shows an example of an Fy file created by the file creation unit 1025 in FIG. 43 when both the change amount and the display type are selected in the selection unit 1024.

  In the Fy file of FIG. 78, from left to right, a frame number indicating the number of the frame from the beginning, a time code of a frame corresponding to the frame number, motion information as a frame change amount corresponding to the frame number, The display type (final display type) based on the fineness information, the frame motion information corresponding to the frame number and the fineness information is sequentially arranged.

  In the pre-editing process described above, the type of pixel value used to determine the frame change amount is not particularly limited. That is, for example, when the pixel value includes a luminance signal (Y) and a color difference signal (Cb, Cr), the amount of change can be obtained using the luminance signal, or the amount of change can be obtained using the color difference signal. You can also. However, since the change in the luminance signal has a greater influence on human vision than the change in the color difference signal, the amount of change is preferably obtained using the luminance signal.

  When the pixel value is composed of R, G, and B color components, the amount of change can be obtained using, for example, the sum of squares of the R, G, and B color components.

  Furthermore, as the motion information as the amount of change, a value that quantitatively represents the intensity of motion other than the values described with reference to FIGS. 48 to 51 can be employed. Similarly, as the fineness information as the amount of change, a value that quantitatively represents the fineness of the image other than the values described in FIGS. 52 to 55 (for example, a value called “difficulty” or “flatness”). Can be adopted.

  Further, for example, as described in FIG. 52, when the change amount is obtained using image data encoded by the MPEG2 system, the encoded bits obtained by encoding the image data by the MPEG2 system The stream can be analyzed (parsed), and the amount of change can be obtained using the result of the analysis.

  That is, since the encoded bit stream includes a motion vector, when the motion information as the amount of change is obtained using the motion vector as described with reference to FIGS. 48 and 49, the encoded bit stream is By analyzing, as a result of the analysis, a motion vector included in the encoded bitstream is obtained, and motion information can be obtained using the motion vector.

  In addition, since the encoded bitstream includes DCT coefficients obtained by DCT transforming an 8 × 8 pixel block, the fineness information as the amount of change can be obtained as described with reference to FIGS. 52 and 53. When obtaining using coefficients, by analyzing the encoded bitstream, the DCT coefficient included in the encoded bitstream is obtained as a result of the analysis, and fineness information is obtained using the DCT coefficient. be able to.

  Furthermore, among the DCT coefficients obtained by DCT transforming an 8 × 8 pixel block, the so-called DC component, which is the upper left DCT coefficient, is the average value of the pixel values of each 8 × 8 pixel of the block, As described with reference to FIGS. 54 and 55, when the fineness information as the amount of change is obtained using the average value of the pixel values of the block, by analyzing the encoded bit stream, The DC component of the DCT coefficient included in the encoded bitstream is obtained, and the fineness information can be obtained using the DC component, that is, the average value of the pixel values of the block.

(4-2) Scrub Processing Next, scrub processing by the microprocessor 3 and the GPU 4 will be described. When the processing by the microprocessor 3 and the GPU 4 is functionally expressed, the functional configuration thereof is as shown in FIG. 79. The GUI control unit 1111, the stream decoding position detection unit 1112, the data supply control unit 1113, and the Fy file management A unit 1114, a display type acquisition unit 1115, a decoder 1116, and a display control unit 1117.

  The GUI control unit 1111 controls the display control unit 1117 to display a GUI such as an editing screen (FIG. 2) shown in FIG. In addition, the GUI control unit 1111 controls the operation signal supplied from the operation controller 37, the mouse 38, or the keyboard 39 in response to the operation of the operation controller 37, the mouse 38, or the keyboard 39 with respect to the GUI displayed on the display 40. In response to the operation signal, necessary information and the like are supplied to the stream decoding position detection unit 1112, the data supply control unit 1113, the display control unit 1117, and the like.

  In other words, the GUI control unit 1111, for example, specifies the main data or proxy data stream (AV file) to be scrubbed according to the operation signal supplied from the operation controller 37, the mouse 38 or the keyboard 39. The stream decoding position detection unit 1112 is supplied with a command (instruction) for requesting frame reproduction together with information indicating the frame to be reproduced in scrubbing.

  Further, the GUI control unit 1111 sets a threshold value used when determining the display type based on the change amount in the data supply control unit 1113 according to the operation signal supplied from the operation controller 37, the mouse 38, or the keyboard 39. Supply.

  Further, the GUI control unit 1111 sends the playback time difference and playback speed v supplied from the operation controller 37, mouse 38, or keyboard 39 to the data supply control unit 1113 based on the playback time difference / playback speed v setting dialog (FIG. 7). Supply.

  In response to a command from the GUI control unit 1111, the stream decoding position detection unit 1112 receives a frame number as information for specifying a frame to be displayed on the spiral video display unit group 58 and data of the frame (main line data or proxy data). A stream number is generated as information for specifying a stream including “” and supplied to the data supply control unit 1113.

  The data supply control unit 1113 relays data exchanged between blocks.

  That is, for example, the data supply control unit 1113 receives a threshold value from the GUI control unit 1111 and supplies the threshold value to the Fy file management unit 1114. Also, the data supply control unit 1113 receives the frame number and stream number from the stream decoding position detection unit 1112, for example, and supplies the frame number and stream number to the display type acquisition unit 1115. Further, the data supply control unit 1113 acquires the Fy file read from the storage device 22, for example, and supplies the Fy file to the Fy file management unit 1114.

  The Fy file management unit 1114 manages (stores) Fy files supplied from the data supply control unit 1113.

  The display type acquisition unit 1115 refers to the Fy file managed by the Fy file management unit 1114, thereby specifying the frame specified by the frame number (and necessary stream number) supplied from the data supply control unit 1113. That is, the display type of the frame to be displayed on the spiral image display unit group 58 is acquired and supplied to the decoder 1116, the display control unit 1117, and other necessary blocks constituting the editing system, such as the GUI control unit 1111. To do.

  The decoder 1116 includes a memory control unit 1116A, decodes the stream (main line data or proxy data) supplied from the data supply control unit 1113, and obtains image data of the frame (baseband image data) obtained as a result Is supplied to the display control unit 1117.

  The decoder 1116 decodes the stream while storing the data necessary for decoding the stream in the XDRAM 5 (FIG. 1). The memory control unit 1116A built in the decoder 1116 reads and writes data to the XDRAM 5. Control. As will be described later, the decoder 1116 may not perform decoding when the display type of the frame to be displayed on the spiral video display unit group 58 is a still image type. It is recognized by referring to the display type from the display type acquisition unit 1115 that the display type of the frame to be performed is a still image type.

  The display control unit 1117 displays an editing screen (FIG. 2) or the like on the display 40 according to information supplied from the GUI control unit 1111, Fy files managed by the Fy file management unit 1114, or the like.

  The display control unit 1117 also displays an image corresponding to the image data of the frame supplied from the decoder 1116, the display type supplied from the display type acquisition unit 1115, the playback time difference supplied from the data supply control unit 1113, and playback. Based on the velocity v, the image is displayed on the spiral video display unit group 58 of the display unit 55 on the editing screen 50.

That is, the display control unit 1117 thins out the image data of the frame supplied from the decoder 1116 by the number of frames corresponding to the reproduction speed v, and displays the image of the image data of the frame that is not the thinning target as the display of the frame. The display method represented by the type is displayed on the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ) with a time difference of the reproduction time difference ΔT (FIG. 5) set by the operator.

Note that the display control unit 1117 is set in advance as a default when the reproduction time difference and the reproduction speed v as reproduction parameters are not supplied from the data supply control unit 1113 (that is, when the setting operation is not performed by the operator). Playback parameters are used. The display 40 displays an image corresponding to data stored in a part of the storage area of the XDRAM 5 (FIG. 1), and the display control unit 1117 displays the XDRAM 5 (FIG. 1). By writing the image data, an image corresponding to the image data is displayed on the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ).

  By the way, when the frame designated as the frame to be displayed on the spiral video display unit group 58 in scrubbing is the frame of interest, the Fy file management unit 1114 manages (stores) the Fy in the display type acquisition unit 1115. The display type of the target frame is acquired based on the file, and this is supplied to the display control unit 1117.

In this case, the decoder 1116 decodes the data supplied from the storage device 22 via the data supply control unit 1113 into the image data of the frame of interest, and supplies this to the display control unit 1117. Therefore, the display control unit 1117 displays the time difference of the reproduction time difference ΔT (FIG. 5) at the timing corresponding to the reproduction speed v by displaying the image corresponding to the image data of the frame of interest with the display method represented by the display type of the frame. The images are displayed on the spiral image display unit group 58 (image display units W57 _{1 to} W57 _n ).

  By the way, as described above, since either one or both of the change amount and display type for each frame are selected and stored in the Fy file, both the change amount and the display type are stored in the Fy file. When the display type is stored, the change amount is stored, but when the display type is not stored, the display type is stored, but the change amount may not be stored.

  When the display type is stored in the Fy file, that is, when both the change amount and the display type are stored, and when the display type is stored but the change amount is not stored, Using the display type stored in the Fy file as it is, an image corresponding to the image data of the frame of interest can be displayed on the spiral video display unit group 58.

  In addition, when the change amount is stored in the Fy file, that is, when both the change amount and the display type are stored, and when the change amount is stored but the display type is not stored, The Fy file management unit 1114 obtains a display type in units of frames based on the amount of change in units of frames stored in the Fy file, and uses the display type to obtain an image corresponding to the image data of the frame of interest. The image can be displayed on the spiral image display unit group 58.

  Next, FIG. 80 shows a configuration example of the Fy file management unit 1114 and the display type acquisition unit 1115 of FIG.

  The Fy file management unit 1114 includes a file storage unit 1121, a change amount acquisition unit 1122, a display type determination unit 1123, and a display type writing unit 1124.

  The file storage unit 1121 stores the Fy file (data stored therein) supplied from the data supply control unit 1113 (FIG. 80).

  The change amount acquiring unit 1122 acquires the change amount in units of frames from the Fy file stored in the file storage unit 1121, and supplies it to the display type determining unit 1123.

  The display type determination unit 1123 includes a storage unit 1131, a threshold processing unit 1132, a continuity determination unit 1133, and a determination unit 1134, and is based on the change amount in units of frames supplied from the change amount acquisition unit 1122. Similarly to the display type determination unit 1023, the display type for each frame is determined (re-determined) and supplied to the display type writing unit 1124.

  Here, in the display type determination unit 1123, the storage unit 1131, the threshold processing unit 1132, the continuity determination unit 1133, or the determination unit 1134 is the storage unit 1081, threshold processing unit 1082 of the display type determination unit 1023 in FIG. Each is configured in the same manner as the sex determination unit 1083 or the determination unit 1084.

  Similarly to the threshold processing unit 1082, the threshold processing unit 1132 compares the amount of change for each frame with the threshold. The threshold processing unit 1132 compares the threshold with the amount of change from the GUI control unit 1111 in FIG. The data is supplied to the threshold processing unit 1132 via the data supply control unit 1113.

  The display type writing unit 1124 corresponds to the frame type display type supplied from the Fy file management unit 1114 to the frame number and time code stored in the Fy file (FIG. 63) stored in the file storage unit 1121. Add and store (write).

  The display type acquisition unit 1115 includes a display type reading unit 1141. The display type reading unit 1141 is supplied from the data supply control unit 1113 from the Fy file managed by the Fy file management unit 1114, that is, the Fy file stored in the file storage unit 1121 of the Fy file management unit 1114. The display type associated with the frame number of the frame of interest is acquired by reading out and supplied to the decoder 1116, the display control unit 1117, and the like in FIG.

  Next, in the scrub performed in FIG. 79, for example, depending on the position of the scrub knob 67 of the scrub bar 66 shown in FIG. 2, it is displayed on the display 40 (the spiral video display unit group 58 displayed on the display 40). ) Frame (frame of interest) is designated.

  That is, the GUI control unit 1111 in FIG. 80 causes the display control unit 1117 to display the scrub bar 66 and recognizes the frame specified by the position of the scrub knob 67 of the scrub bar 66 as the frame of interest.

  Here, the relationship between the position of the scrub knob 67 and the frame specified by the position will be described with reference to FIG.

  FIG. 81 shows the scrub bar 66.

  In the scrub bar 66, the horizontal movable range in which the scrub knob 67 can move (the horizontal direction has a length of about a few tens of centimeters, for example) is the editing target displayed on the spiral video display unit group 58. In other words, it is divided into small sections (hereinafter referred to as designated sections as appropriate) that are the same as the number of frames constituting the playback video (stream) of (the scrub target).

  In FIG. 81, the movable range is divided into F designated sections.

  If the horizontal length of the designated section (the width when the movable range is divided into designated sections) is referred to as the section length, in FIG. 81, the movable range is divided into designated sections having the same section length. Is (equally divided).

  In each designated section that divides the movable range, for example, a playback video of an edit target (that is, a scrub target) displayed on the spiral video display unit group 58 is configured in the direction from the left designated section to the right designated section. Frames to be assigned are assigned in time series. Therefore, if the i-th specified section from the left of the movable range is referred to as the i-th specified section and the i-th frame from the top of the frames constituting the playback video is referred to as the i-th frame, the i-th specified section Is assigned the i-th frame.

  The GUI control unit 1111 detects the designated section where the scrub knob 67 is located at a timing according to the display rate represented by the display type of the frame displayed immediately before on the spiral video display unit group 58, and the designated section is displayed in the designated section. The assigned frame is recognized as the frame of interest.

  Here, as the display type, the still image type V1, the normal type V2, and the high display rate low resolution type V3 shown in FIG. 57 are adopted, and the display rates of the still image type V1 and the normal type V2 are, for example, 30 frames / second, and the display rate of the high display rate / low resolution type V3 is, for example, 60 frames / second. Note that the display rate of the normal type V2 is a display rate corresponding to the playback speed v among the playback parameters.

  FIG. 82 illustrates a scrub display method for each of the normal type V2 and high display rate / low resolution type V3 frames.

  When the display type of the frame displayed immediately before on the spiral video display unit group 58 is the still image type V1 or the normal type V2 with a display rate of 30 frames / second, the GUI control unit 1111 is 1/30 second interval. The designated section where the scrub knob 67 is located is detected, and the frame assigned to the designated section is recognized as the frame of interest. Accordingly, when the display type is the normal type V2, the frame (frame of interest) displayed on the spiral video display unit group 58 is updated at a 1/30 second interval, that is, a display rate of 30 frames / second.

  Further, the GUI control unit 1111 is 1/60 when the display type of the frame displayed immediately before in the spiral image display unit group 58 is, for example, the high display rate / low resolution type V3 with a display rate of 60 frames / second. At a second interval, the designated section where the scrub knob 67 is located is detected, and the frame assigned to the designated section is recognized as the frame of interest. Therefore, when the display type is the high display rate low resolution type V3, the frames (frames of interest) displayed on the spiral video display unit group 58 are 1/60 second intervals, that is, a display rate of 60 frames / second. Updated.

  From the above, when the scrub knob 67 is moved at a speed of moving from the left to the right by the length of two specified sections in 1/30 seconds, for example, the display type is a frame of the normal type V2, for example. In the normal section in which the frames are continuous, every other frame, i.e., i-th frame, i + 2 frame, i + 4 frame,... In the video display unit group 58, an image is displayed at double speed in a frame dropping state.

  Similarly to the case described above, when the scrub knob 67 is moved at the speed of moving from the left to the right by the length of two designated sections in 1/30 seconds, that is, in 1/60 seconds. When moved at the speed of moving from the left to the right by the length of one specified section, the display type is, for example, the high display rate low resolution section where the frames of the high display rate low resolution type V3 are continuous, At 1/60 second intervals, every frame, i.e., i-th frame, i + 1-th frame, i + 2-th frame,... Becomes the frame of interest, and as a result, in the spiral video display group 58, The image is displayed at double speed without dropping frames.

  As described above, when the scrub knob 67 is moved at such a high moving speed as to perform double-speed display (playback), a display type frame in which frame dropping occurs and a display type frame in which frame dropping does not occur There is. That is, for a frame of the high display rate low resolution type V3 whose display rate is higher than that of the normal type V2, that is, a frame with a large movement, even if the scrub knob 67 is moved quickly, frame dropping is less likely to occur. Thereby, it is possible to prevent the temporal change of the image from being overlooked with respect to a frame having a large movement.

  As described above, in the normal section, the frame of interest is updated at 1/30 second intervals, whereas in the high display rate and low resolution section, 1/2 of the normal section is 1/60 second intervals. The attention frame is updated.

  Accordingly, the decoder 1116 (FIG. 79) converts the frame of the high display rate / low resolution section (the frame of the display type is the high display rate / low resolution type V3) into the 2 of the frame of the normal section (the frame of the display type is the normal type V2). Decoding is required twice as fast. Further, since the frame in the high display rate and low resolution section is a frame having a larger motion than the frame in the normal section, when the resolution is constant, the amount of data required for decoding by the decoder 1116 is high. There are more frames in the display rate low resolution section than in the normal section. As described above, displaying a frame in a high display rate / low resolution section is more burdensome than displaying a frame in a normal section.

  Therefore, in the editing apparatus 2, when displaying a frame in the normal section, the image data as the main line data is decoded, and when displaying a frame in the high display rate and low resolution section, the data amount is larger than that of the main line data. The image data as proxy data with a small amount of data is decoded, thereby reducing the load on the display of the frame of the high display rate / low resolution section.

  Further, in the above-described case, the scrub knob 67 is moved at a speed of moving from the left to the right by the length of two specified sections in 1/30 seconds, for example. When the scrub knob 67 is moved at the speed of moving from the left to the right by the length of one designated section in 1/30 second, the frame of the normal type V2 is displayed. In the continuous normal section, every frame becomes a frame of interest at 1/30 second intervals. As a result, the spiral video display unit group 58 displays an image at 1 × speed without dropping frames.

  Further, when the scrub knob 67 is moved at a speed of moving from the left to the right by the length of one designated section in 1/30 seconds, the scrub knob 67 is 1 in 1/60 seconds. Since the display type moves from the left to the right by 1/2 of the length of one specified section, the display type is 1/30 seconds in the high display rate / low resolution section in which the frames of the high display rate / low resolution type V3 are continuous. The same frame becomes the frame of interest, and as a result, in the spiral video display unit group 58, the image is displayed at 1 × speed without dropping frames.

  Next, with reference to FIG. 83, display of a frame whose display type is the still image type V1 will be described.

  In this editing device 2, regarding a frame of a still image section in which a frame of the still image type V1 is displayed continuously, even if the target frame (display target frame) is changed in the frame of the still image section, The image of the frame displayed immediately before is displayed on the spiral video display unit group 58.

  That is, for the frame in the still image section, as long as the frame in the still image section is the target frame, the image of the frame that first becomes the target frame in the still image section is continuously displayed.

  Therefore, for example, as shown first from the top in FIG. 83, at the start of scrubbing, the scrub knob 67 pays attention to a frame in a section other than a still image section (in this case, a normal section or a high display rate low resolution section). When the frame is designated, the operator moves the scrub knob 67 from the left to the right, so that the frame located at the left end of the still image segment (the still image segment When the first frame in time) is first designated as the attention frame, the decoder 1116 (FIG. 79) decodes the image data of the frame designated as the first attention frame, and the spiral image In the display unit group 58, a corresponding image is displayed.

  After that, as long as the frame in the still image section is designated as the attention frame, the image of the frame designated as the attention frame first is displayed as the current attention frame image. Therefore, in the decoder 1116, as long as the frame of the still image section is designated as the frame of interest, the decoding of the image data of the current frame of interest is performed except for the decoding of the image data of the frame first designated as the frame of interest. I will not.

  Further, for example, as shown in the second from the top in FIG. 83, when the scrub knob 67 designates the frame of the still image section as the frame of interest at the start of scrubbing, the decoder 1116 (FIG. 79) The image data of the frame designated as the frame of interest at the start of scrubbing is decoded, and the corresponding image is displayed in the spiral video display unit group 58.

  After that, as long as the frame in the still image section is designated as the frame of interest, the image of the frame designated as the frame of interest at the start of scrubbing is displayed as the image of the current frame of interest. Therefore, in the decoder 1116, as long as the frame of the still image section is designated as the attention frame, the decoding of the image data of the current attention frame excludes the decoding of the image data of the frame designated as the attention frame at the start of scrubbing. Not done.

  Further, for example, as shown in the third from the top (first from the bottom) in FIG. 83, at the start of scrubbing, the scrub knob 67 is set to a section other than the still picture section (here, the normal section or the high display rate low resolution). When the frame of (section) is designated as the frame of interest, the operator moves the scrub knob 67 from the right to the left, so that it is positioned at the right end of the still image section in the frame of the still image section. When a frame (a frame that is temporally following in a still image section) is first designated as a frame of interest, an image of the frame designated as the frame of interest first in the decoder 1116 (FIG. 79). The data is decoded and the corresponding image is displayed in the spiral video display unit group 58.

  After that, as long as the frame in the still image section is designated as the attention frame, the image of the frame designated as the attention frame first is displayed as the current attention frame image. Therefore, in the decoder 1116, as long as the frame of the still image section is designated as the frame of interest, the decoding of the image data of the current frame of interest is performed except for the decoding of the image data of the frame first designated as the frame of interest. I will not.

  Next, the scrub process will be described with reference to the flowchart of FIG.

  For example, when an operator operates the operation controller 37, mouse 38, or keyboard 39 to specify an AV file to be edited, an operation signal corresponding to the operation is GUI controlled from the operation controller 37, mouse 38, or keyboard 39. Supplied to the unit 1111.

  In step S1111, the GUI control unit 1111 recognizes the AV file to be edited in response to an operation signal from the keyboard 39 or the mouse 38, and proceeds to step S1112.

  In step S1112, the Fy file management unit 1114 caches (temporarily stores) the Fy file corresponding to the AV file recognized as the editing target by the GUI control unit 1111 and proceeds to step S1113.

  That is, the GUI control unit 1111 controls the data supply control unit 1113 via the stream decoding position detection unit 1112 so as to supply the Fy file corresponding to the AV file recognized as the editing target. The data supply control unit 1113 reads the Fy file from the storage device 22 and supplies it to the Fy file management unit 1114 according to the control from the GUI control unit 1111. In step S1112, the Fy file management unit 1114 causes the file storage unit 1121 (FIG. 54) to store the Fy file supplied from the data supply control unit 1113 as described above.

  In step S1113, the change amount acquisition unit 1122 (FIG. 80) of the Fy file management unit 1114 determines whether a display type exists in the Fy file stored in the file storage unit 1121 in the immediately preceding step S1112.

  If it is determined in step S1113 that the display type is present in the Fy file stored in the file storage unit 1121, step S1114 described later is skipped, and the process proceeds to step S1115.

  If it is determined in step S1113 that the display type does not exist in the Fy file stored in the file storage unit 1121, the change amount acquisition unit 1122 performs frame units from the Fy file stored in the file storage unit 1121. Is read and supplied to the display type determining unit 1123 (FIG. 80), and the process proceeds to step S1114.

  Here, it is assumed that the Fy file stored in the file storage unit 1121 stores at least motion information as a change amount in units of frames.

  In step S1114, the display type determination unit 1123 determines the display type in units of frames based on the change amount in units of frames supplied from the change amount acquisition unit 1122 in the same manner as the display type determination unit 1023 in FIG. This is supplied to the display type writing unit 1124 (FIG. 80). Then, the display type writing unit 1124 writes the display type from the display type determining unit 1123 to the Fy file in the file storage unit 1121, and proceeds from step S1114 to step S1115.

  Note that the threshold value processing unit 1132 (FIG. 80) of the display type determining unit 1123 compares the amount of change with the threshold value in determining the display type in step S1114. The same threshold value used by the threshold processing unit 1082 of 1023 is used.

  Further, as described above, when the process proceeds from step S1113 to step S1115, or when the process proceeds from step S1114 to step S1115, the editing screen 50 of FIG.

  In step S1115, the GUI control unit 1111 determines whether the operation controller 37, the mouse 38, or the keyboard 39 has been operated so as to end the editing program.

  If it is determined in step S1115 that the operation controller 37, the mouse 38, or the keyboard 39 is not operated so as to end the editing program, the process proceeds to step S1116, and the GUI control unit 1111 proceeds from step S1113 to step S1115. Or whether the scrub knob 67 included in the scrub bar 66 of the editing screen 50 (FIG. 2) displayed when the process proceeds from step S1114 to step S1115, that is, the operation controller 37, mouse 38 or keyboard 39 is operated. By doing so, it is determined whether or not the scrub knob 67 has been moved.

  If it is determined in step S1116 that the scrub knob 67 has been moved, that is, the operator operates the operation controller 37, the mouse 38, or the keyboard 39 so as to move the scrub knob 67, and an operation signal corresponding to the operation. Is supplied from the operation controller 37, the mouse 38 or the keyboard 39 to the GUI control unit 1111, the GUI control unit 1111 sets the frame assigned to the designated section (FIG. 81) where the scrub knob 67 is located as the frame of interest. The information on the frame of interest is supplied to the display type acquisition unit 1115 via the stream decoding position detection unit 1112 and the data supply control unit 1113, and the process proceeds to step S1117.

  In step S1117, the display type acquisition unit 1115 sets the display type of the target frame to Fy based on the information on the target frame supplied from the GUI control unit 1111 via the stream decoding position detection unit 1112 and the data supply control unit 1113. It is acquired by reading from the file storage unit 1121 (FIG. 80) of the file management unit 1114 and supplied to the GUI control unit 1111, the decoder 1116, and the display control unit 1117. Further, in step S1117, the decoder 1116 and the display control unit 1117 determine the display type of the frame of interest from the display type acquisition unit 1115, and display control processing for displaying an image of the frame of attention based on the determination result (described later). Steps S1118 to S1123) are performed, and the process returns to Step S1115.

  That is, if it is determined in step S1117 that the display type of the frame of interest is the still image type V1, the process advances to step S1118, and the display control unit 1117 displays the frame that was the frame of interest immediately before (hereinafter, appropriately the previous time). Whether or not the display type of the frame) is the still image type V1 and the previous frame and the target frame are frames in the same still image section is stored in the file storage unit 1121 of the Fy file management unit 1114. Judgment by referring to the Fy file.

  If it is determined in step S1118 that the previous frame and the target frame are not frames of the same still image section, the process proceeds to step S1119, and the data supply control unit 1113 receives the main line data of the target frame from the storage device 22 in the decoder 1116. (Furthermore, the data necessary for decoding the frame of interest) is read out, supplied to the decoder 1116, the main line data of the frame of interest is acquired (received), and the process proceeds to step S1120.

In step S1120, the decoder 1116 decodes the main line data of the frame of interest acquired from the data supply control unit 1113, supplies the image data obtained as a result to the display control unit 1117, and proceeds to step S1121. In step S1121, the display control unit 1117 buffers the image data of the frame of interest from the decoder 1116, and proceeds to step S1122, and displays an image corresponding to the buffered image data with a reproduction time difference ΔT (FIG. 5). The images are displayed on the spiral image display unit group 58 (image display units W57 _{1 to} W57 _n ) with a time difference.

  Therefore, an image corresponding to the image data as the main line data, that is, an image having a higher resolution (than an image corresponding to the image data as proxy data) is displayed as the frame image of the still image type V1. The

On the other hand, if it is determined in step S1118 that the previous frame and the target frame are frames of the same still image section, the process proceeds to step S1122, and the display control unit 1117 performs step S1121 performed immediately before (close). The images corresponding to the image data buffered in the above are displayed on the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ) as the image of the frame of interest with a time difference of the reproduction time difference ΔT (FIG. 5). Let

  In other words, when the previous frame and the frame of interest are frames of the same still image section, the image of the frame of the still image section is the first frame of interest among the frames of the still image section. A frame image is displayed. Therefore, in this case, since it is not necessary to perform decoding in the decoder 1116, the load on the editing apparatus 2 can be reduced.

  On the other hand, if it is determined in step S1117 that the display type of the attention frame is the normal type V2, the process proceeds to steps S1119 to S1122, and the image of the attention frame is displayed as described above.

  That is, in step S1119, the decoder 1116 waits for the data supply control unit 1113 to read main line data of the target frame from the storage device 22 and supply it to the decoder 1116, and acquires the main line data of the target frame. The process proceeds to S1120.

In step S1120, the decoder 1116 decodes the main line data of the frame of interest acquired from the data supply control unit 1113, supplies the image data obtained as a result to the display control unit 1117, and proceeds to step S1121. In step S1121, the display control unit 1117 buffers the image data of the frame of interest from the decoder 1116, and proceeds to step S1122, and displays an image corresponding to the buffered image data with a reproduction time difference ΔT (FIG. 5). The images are displayed on the spiral image display unit group 58 (image display units W57 _{1 to} W57 _n ) with a time difference.

  Therefore, an image corresponding to the image data as the main line data, that is, an image having a higher resolution (than an image corresponding to the image data as proxy data) is displayed as the frame image of the normal type V2. .

  On the other hand, if it is determined in step S1117 that the display type of the target frame is the high display rate low resolution type V3, the process proceeds to step S1123, and the decoder 1116 causes the data supply control unit 1113 to store the target frame from the storage device 22. The proxy data (and data necessary for decoding the frame of interest) is read out, waits for supply to the decoder 1116, the proxy data of the frame of interest is acquired, and the process proceeds to step S1120.

In step S1120, the decoder 1116 decodes the proxy data of the frame of interest acquired from the data supply control unit 1113, supplies the image data obtained as a result to the display control unit 1117, and proceeds to step S1121. In step S1121, the display control unit 1117 buffers the image data of the frame of interest from the decoder 1116, and proceeds to step S1122, and displays an image corresponding to the buffered image data with a reproduction time difference ΔT (FIG. 5). The images are displayed on the spiral image display unit group 58 (image display units W57 _{1 to} W57 _n ) with a time difference.

  Therefore, as an image of a frame of the display type of the high display rate low resolution type V3, an image corresponding to the image data as proxy data, that is, an image having a lower resolution (than an image corresponding to the image data as main line data). Is displayed.

On the other hand, if it is determined in step S1116 that the scrub knob 67 has not been moved, the process advances to step S1122, and the display control unit 1117 displays the buffered image data in step S1121 performed immediately before (coming soon). Are displayed on the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ) as a frame image of interest with a time difference of the reproduction time difference ΔT (FIG. 5).

  That is, when the scrub knob 67 has not been moved since the previous determination in step S1116, the display control unit 1117 displays the image corresponding to the image data stored in the frame buffer 1117A, that is, the immediately preceding image. The image displayed in (the image displayed in the previous step S1122) is displayed again.

  As described above, after the image of the attention frame is displayed in step S1122, the process proceeds to step S1124, and the GUI control unit 1111 determines the display type of the attention frame supplied from the display type acquisition unit 1115 in step S1117. To do.

  If it is determined in step S1124 that the display type of the frame of interest is the high display rate low resolution type V3, the process proceeds to step S1125, and the GUI control unit 1111 determines whether the scrub knob 67 has been moved in the previous step S1116. For example, it is determined whether 1/60 seconds, which is the time corresponding to the display rate of the high display rate / low resolution type V3, has elapsed.

  If it is determined in step S1125 that 1/60 seconds have not elapsed, the process returns to step S1125.

  If it is determined in step S1125 that 1/60 second has elapsed, the process returns to step S1115, and the above-described processing is repeated.

  Accordingly, when the frame of the display type is the high display rate / low resolution type V3 is the frame of interest, the GUI control unit 1111 determines in step S1116 that 1/60 second corresponding to the high display rate / low resolution type V3. It is determined whether or not the scrub knob 67 has been moved in the cycle.

When it is determined that the scrub knob 67 has been moved, the display control unit 1117 is obtained by decoding the proxy data of the frame assigned to the designated section (FIG. 55) where the scrub knob 67 after the movement is located. Images corresponding to the image data (low-resolution images) are respectively displayed on the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ) with a time difference of the reproduction time difference ΔT (FIG. 5).

  When it is determined that the scrub knob 67 has not been moved, the display control unit 1117 displays the image displayed on the display 40 immediately before again.

  As described above, an image is displayed at a display rate of 60 frames / second with respect to a frame whose display type is a high display rate and low resolution type V3.

  On the other hand, when it is determined in step S1124 that the display type of the target frame is either the still image type V1 or the normal type V2, the process proceeds to step S1126, and the GUI control unit 1111 performs the process in the previous step S1116. Then, after determining whether or not the scrub knob 67 has been moved, it is determined whether or not, for example, 1/30 second has elapsed, which is the time corresponding to the display rate of the still image type V1 and the normal type V2.

  If it is determined in step S1126 that 1/30 second has not elapsed, the process returns to step S1126.

  If it is determined in step S1126 that 1/30 second has elapsed, the process returns to step S1115, and the above-described processing is repeated.

  Therefore, when any frame of the still image type V1 or the normal type V2 is the frame of interest, the GUI control unit 1111 switches to the still image type V1 and the normal type V2 in step S1116. It is determined whether or not the scrub knob 67 has been moved at the corresponding 1/30 second period.

When it is determined that the scrub knob 67 has been moved, the display control unit 1117 is obtained by decoding the main line data of the frame assigned to the designated section (FIG. 55) where the scrub knob 67 after the movement is located. Images corresponding to the image data (high-resolution images) are respectively displayed on the spiral video display unit group 58 (video display units W57 _{1 to} W57 _n ) with a time difference of the reproduction time difference ΔT (FIG. 5).

  When it is determined that the scrub knob 67 has not been moved, the display control unit 1117 displays the image displayed on the display 40 immediately before again.

  As described above, for the frames of the still image type V1 and the normal type V2, the image is displayed at the display rate of 30 frames / second.

  On the other hand, if it is determined in step S1115 that the keyboard 39 or the mouse 38 has been operated so as to end the editing program, that is, for example, the operator operates the keyboard 39 or the mouse 38 so as to end the editing program. When an operation signal corresponding to the operation is supplied from the keyboard 39 or the mouse 38 to the GUI control unit 1111, the scrub process (execution of the editing program) ends.

As described above, the display type of the frame of interest, which is the frame specified by the scrub knob 67, is obtained on the basis of the display rate corresponding to the playback speed v, and the playback time difference ΔT is displayed by the display method represented by the display type. Since the images are displayed on the spiral image display unit group 58 (image display units W57 _{1 to} W57 _n ) with a time difference, scrubbing can be performed appropriately.

  That is, if the frame whose display type is the still image type V1, that is, the frame that does not move is the frame of interest, the frame of interest is the first in the still image section including the frame of interest and the frame of the still image type V1 is continuous. Since the image data of the frame that has been obtained is obtained by decoding the main line data and the image corresponding to the image data is displayed, a high-quality image of the same frame is displayed. As a result, the operator Can be easily confirmed. Furthermore, as long as the frames in the same still image section are consecutively frames of interest, it is not necessary to decode the main line data, so the burden on the editing device 2 can be reduced.

  In addition, when the display type is a high display rate, low resolution type V3 frame, that is, a frame with large (strong) motion is the frame of interest, the image is displayed at a high display rate. Can be confirmed accurately.

  Furthermore, when the display type is a high display rate, low resolution type V3 frame, the proxy data with a smaller data amount than the main line data is decoded, and the image corresponding to the resulting image data is displayed. The burden on the editing apparatus 2 can be reduced, and the editing program can be prevented from responding (the editing apparatus 2 is hung up) due to the heavy load on the editing apparatus 2.

  Also, if the display type is a normal type V2 frame, that is, a frame that is not violent but has a certain amount of movement is the frame of interest, the display type is lower than the frame of the high display rate low resolution type V3. However, an image having a higher resolution than the frame of the high display rate / low resolution type V3 is displayed. Since the display type is normal type V2, the display type is high display rate low resolution type V3 because the image movement (change in pixel value between frames) is smaller than the frame of display type high display rate low resolution type V3. Even if the image is displayed at a display rate lower than that of the frame, the operator can confirm the content of the image accurately.

  Furthermore, the frame of the normal type V2 has a smaller image movement than the frame of the high display rate low resolution type V3 of the display type, so the data amount of the main line data of the frame of the normal type V2 of the display type is the display type. There is less than the data amount of main line data of high display rate low resolution type V3 frame. Therefore, when the frame of the normal type V2 is the frame of interest, even if the main line data is decoded and a high-resolution image corresponding to the image data obtained as a result is displayed, the load on the editing device 2 is The load applied when decoding main line data of a frame whose display type is a high display rate, low resolution type V3 is smaller.

  As described above, the scrub process does not display all the frames in the same display method, but changes the predetermined parameters according to the amount of change in the image (motion information, fineness information) and plays the reproduced video. Is displayed. Specifically, these parameters are such that the resolution of the image is changed as a display parameter (display parameter), and the display rate (frame rate) and the playback speed v are changed as playback parameters (playback parameters). In addition, only necessary frames are decoded according to the amount of change in the image. These allow proper scrubbing.

(4-3) Other Scrub Processing Forms In the scrub processing described above, the display type is determined based on the motion information, and the still image type V1, the normal type V2, and the high type shown in FIG. It is assumed that there are three display rate low resolution types V3. However, for scrub processing, the display types are, for example, the still image type V11, the normal type V12, the high display rate normal resolution type V13, and the ultra high display shown in FIG. The display rate low resolution type V14 and the ultra-high display rate low resolution type V15 can be used, and the display type is determined based on the fineness information, for example, as shown in FIG. The low resolution normal display rate type C1, the normal type C2, and the normal resolution low display rate type C3 can be used.

  In the scrub process described above, the main line data of the high resolution image and the proxy data of the low resolution image are recorded in the storage device 22, that is, two types of resolution images having the same content are recorded. Depending on the display type of the frame, an image having a high resolution or an image having a low resolution is displayed. In addition, for example, images of three types of resolutions of high, medium, and low are recorded in the storage device 22. Depending on the display type of the frame, it is possible to display an image with a high resolution, an image with a medium resolution, or an image with a low resolution.

  Further, in the above-described scrub process, the section length of the designated section to which the frame is assigned is constant, but the section length weighted according to the motion information is set as the amount of change of the frame to be assigned to the designated section. Is also possible.

In the above-described scrub process, the position of the scrub knob 67 is detected at a period corresponding to the display rate, and the frame assigned to the designated section where the scrub knob 67 is located is regarded as a frame of interest, and the frame of interest is displayed as a spiral image. Displayed on the video display units W57 _{1 to} W57 _n of the group 58 with a reproduction time difference ΔT therebetween.

  As described with reference to FIG. 81, the movable range of the scrub bar 66 is the same as the number of frames constituting the playback video of the edit target (that is, the scrub target) displayed on the spiral video display unit group 58. When the section length is divided into designated sections having the same length, when the scrub knob 67 is moved at a constant speed, the scrub knob 67 has a certain distance, and thus a certain number of designations in a certain unit time. Move the section. Therefore, if the display rate is constant, the scrub knob 67 is moved by a constant distance at a constant speed, whereby a certain number of frames are displayed.

  That is, for the sake of simplicity, assuming that the display rate is constant, the playback video displayed in the spiral video display unit group 58 is a section in which a frame with intense motion continues (hereinafter, there is appropriate motion). Frame) and a section in which frames without motion are continuous (hereinafter, referred to as a section without motion as appropriate), a frame with a motion within the movable range of the scrub bar 66 is assigned. Even if the scrub knob 67 is moved at a constant speed at a constant speed in each of the designated area range and the designated area range to which a frame of no movement is assigned, the spiral image display unit group The number of frames displayed at 58 remains the same.

  However, when the scrub knob 67 is moved within the range of the designated section to which the frame with the motion is assigned, the frame displayed with the motion video is displayed on the spiral image display unit group 58 because the frame with the motion is intense. Changes greatly (moves).

  On the other hand, when the scrub knob 67 is moved in the range of the designated section to which the frame of the non-motion section is allocated, the frame of the non-motion section is displayed on the spiral video display unit group 58 because there is (almost) no motion. The image that is made is (almost) unchanged.

  Therefore, when the movable range of the scrub bar 66 is divided into designated sections having the same section length, when the scrub knob 67 is moved by a certain distance at a certain speed, the spiral image display unit group 58 is displayed. A range in which the displayed image changes greatly and a range that does not change occur.

  In the range of the designated section to which the frame of the non-motion section is assigned, the image displayed on the spiral video display unit group 58 does not change even if the scrub knob 67 is moved to some extent. The operator may feel annoyed.

  On the other hand, when the scrub knob 67 is moved to some extent in the range of the designated section to which the frame with movement is assigned, the image displayed on the spiral video display unit group 58 changes greatly. The operator who is doing this may be required to operate a fine (fine) scrub knob 67 in order to find a desired image frame.

  Therefore, the section length of the designated section to which the frame is assigned is not constant, but is set as a section length weighted according to the motion information, for example, as the amount of change in the frame to be assigned to the designated section. The possible range can be divided into designated sections having a larger section length as a designated section to which a frame having a larger amount of motion information as a change amount is assigned. Specific examples are given below.

  FIG. 85 shows an example of weighting when weighting the section length of the designated section as described above.

  In FIG. 85, the horizontal axis represents a frame (how many frames from the top), and the vertical axis represents motion information as a change amount.

  In FIG. 85, using the threshold values L1, L2, L3, and L4 that satisfy the expression L1> L2> L3> L4, the section length weight w of the specified section to which the frame whose motion information is the threshold value L1 or more is assigned is, for example, 1 For example, the weight w of the section length of the designated section to which the frame whose motion information is greater than or equal to the threshold value L2 and less than the threshold value L1 is assigned is, for example, 0.7. Further, the weight w of the section length of the designated section to which the frame whose motion information is greater than or equal to the threshold value L3 and less than the threshold L2 is 0.5, for example, and the designated section to which the frame whose motion information is greater than or equal to the threshold value L4 and less than the threshold value L3 is allocated. For example, the weight w of the section length is 0.3. Furthermore, the weight w of the section length of the designated section to which the frame whose motion information is less than the threshold value L4 is assigned is, for example, 0.1.

  The section length weighted using the weight w as described above can be obtained, for example, as follows.

  That is, a section in which designated sections weighted by the same weight w are added to the section length is referred to as a same weight section, and the playback video of the edit target (that is, the scrub target) displayed on the spiral video display unit group 58 is displayed. Suppose that the sequence of designated sections allocated to the frames that constitutes is divided into Q identically weighted sections.

  In addition, the number of designated sections constituting the qth same weight section from the top of the Q identical weight sections, and hence the number of frames allocated to the designated section constituting the qth same weight section In addition to representing Fq, the weight given to the section length of each designated section constituting the qth same weight section is represented as wq.

  It should be noted that the total number F of the playback video frames to be edited (that is, the scrub target) displayed on the spiral video display unit group 58 is expressed by the equation F = F1 + F2 +... FQ.

  Assuming that the length of the movable range is 1, the section length of the designated section is represented by 1 / F when weighting is not performed.

  On the other hand, when weighting is performed, the section length of the designated section constituting the qth same weight section from the beginning is represented by wq / (ΣwiFi). That is, since the length of the qth same weight section from the beginning is represented by wqFq / (ΣwiFi), wqFq / (ΣwiFi) is assigned to the designated section (which constitutes the qth same weight section). By dividing by the number Fq of frames), the section length of the designated section constituting the qth same weight section from the head when weighting can be obtained. However, ΣwiFi represents the sum of wiFi with the suffix i changed from 1 to Q.

  Note that the section length of the designated section when weighting is not performed can be obtained by assuming that the weights w1, w2,..., WQ when weighting are all the same weight w. That is, when weighting is performed, the section length of the designated section constituting the qth same weight section from the head is wq / (ΣwiFi) as described above, but w1, w2,. Assuming the same weight w, wq / (ΣwiFi) = w / (wΣFi) = 1 / ΣFi = 1 / F.

  Next, in FIG. 85, the section length weight w of the specified section allocated to the frame is obtained using four threshold values L1, L2, L3, and L4 that are not related to the threshold values used to determine the display type. However, the weight w of the section length can be performed using, for example, a threshold value used for determining the display type.

  That is, FIG. 86 shows an example of the weights obtained using the threshold values H and L that are used only for determining the display type based on the motion information.

  In FIG. 86, the horizontal axis represents a frame (how many frames from the beginning), and the vertical axis represents motion information that is a change amount.

  In FIG. 86, the weight w of the section length of the designated section to which the frame whose motion information is greater than or equal to the threshold H is 0.5, for example, and the designated section to which the frame whose motion information is greater than or equal to the threshold L and less than the threshold H is assigned. The section length weight w is, for example, 0.3. Further, the weight w of the section length of the designated section to which the frame whose motion information is less than the threshold value L is assigned is, for example, 0.1.

  Note that the section length weight w of the specified section assigned to the frame may be determined according to the display type, for example, when the display type of the frame is determined based on the motion information of the frame. it can.

  That is, when the display type is determined based on the motion information, the display type represents the degree of frame motion. Specifically, for example, as shown in FIG. 57, the display type is determined as one of the still image type V1, the normal type V2, or the high display rate low resolution type V3 based on the motion information. In principle, the frame of the still image type V1 is a frame whose motion information is less than the threshold L, and the frame of the normal type V2 is a frame whose motion information is greater than or equal to the threshold L and less than the threshold H. The frame of the high display rate / low resolution type V3 is a frame whose motion information is equal to or higher than the threshold value H.

  Therefore, as in the case of using the thresholds H and L described with reference to FIG. 86, using the display type, the section length weight w of the specified section to which the frame of the still image type V1 is assigned is set to 0.1, and the normal type V2 The section length weight w of the designated section to which the frame of the frame is assigned can be set to 0.3, and the section length weight w of the designated section to which the frame of the high display rate low resolution type V3 can be assigned to 0.5, respectively. .

  In addition, when the weight w of the section length of the specified section assigned to the frame is determined according to the display type of the frame, the still image section where the frame of the still image type V1 is continuous, the frame of the normal type V2 is continuous The normal display section and the high display rate / low resolution section in which the frames of the high display rate / low resolution type V3 continue are both the same weight sections.

  Next, FIG. 87 shows each designated section when the section length is not weighted and when the section length is weighted.

  When the section length is not weighted, as shown in the upper half side of FIG. 87, the movable range of the scrub bar 66 is reproduced as the edit target (that is, the scrub target) displayed on the spiral video display unit group 58. As shown in FIG. 81, the frames constituting the playback video displayed on the spiral video display unit group 58 are divided into designated zones having the same zone length as the number of frames constituting the video. Are assigned in time series.

  When the section length is not weighted, the still image section where the frames of the still image type V1 are continuous, the normal section where the frames of the normal type V2 are continuous, the high display rate and the low resolution section where the frames of the low resolution type V3 are continuous The section length of the designated section assigned to any of the frames is d.

  On the other hand, when the section length is weighted, as shown in the lower half side of FIG. 87, the movable range of the scrub bar 66 is divided into designated sections. The frames constituting the video to be edited (ie, the scrub target) displayed in the spiral video display unit group 58 are assigned in time series, but the designated section in which the frame having the large amount of motion information as the change amount is assigned. The section length is getting longer.

  That is, in FIG. 87, the frame length of the specified section allocated to the frame of the still image section in which the frames of the still image type V1 are continuous, that is, the frame having no motion is 1/2 times the section length d when weighting is not performed. D / 2. In addition, the section length of the designated section assigned to the frame of the normal section in which the frames of the normal type V2 are continuous, that is, the frame having a motion that cannot be said to be intense, is the same as the section length d without weighting. It has become. Furthermore, the section length of the designated section assigned to the frame of the high display rate and low resolution section where the frames of the high display rate and low resolution type V3 are continuous, that is, the frame with intense motion, is three times the section length d without weighting. It has become 3d.

  Therefore, when weighting the section length, when the scrub knob 67 is moved by a movement amount that is ½ of that when the section length is not weighted, the display of the frame of the still image section is ended. As a result, even if the scrub knob 67 is moved to some extent, the image displayed on the spiral image display unit group 58 does not change, and the operator operating the scrub knob 67 can be prevented from feeling annoying. it can.

  Furthermore, when weighting the section length, when the scrub knob 67 is moved by a movement amount three times that when the section length is not weighted, the frame of interest displayed on the spiral video display unit group 58 is The frame is changed from one frame having a high display rate and low resolution section to the next frame. As a result, the operator can easily find a frame of a desired image without operating the scrub knob 67 so finely (finely).

  In FIG. 87, type information (a GUI of a band-like area in which display types are displayed in time series) in which the horizontal scale is matched with the scale of the movable range of the scrub bar 66 is illustrated.

  In the type information in which the horizontal scale matches the scale of the movable range of the scrub bar 66, the type information (pattern) at the position of the scrub knob 67 is assigned to the designated section where the scrub knob 67 is located. Indicates the display type.

  In FIG. 87, an area D1 indicated by hatching in the band-like area as type information represents a still image section in which frames of the still image type V1 are continuous, and is an area indicated without a pattern. D2 represents a normal section in which normal type V2 frames continue. A region D3 indicated by a horizontal line represents a high display rate / low resolution section in which frames of the high display rate / low resolution type V3 are continuous.

  In FIG. 87, as described above, when the section length is weighted, the section length of the designated section assigned to the frame of the still image section where the frames of the still image type V1 are continuous is the section length d when weighting is not performed. The horizontal length of the area D1 that represents the still image section when the section length is weighted is also halved when the section length is not weighted. Become.

  Also, in FIG. 87, when the section length is weighted, the section length of the designated section allocated to the normal section frame in which the normal type V2 frames are continuous is set to d which is the same as the section length d without weighting. Therefore, the horizontal length of the region D2 representing the normal section when the section length is weighted is also the same as when the section length is not weighted.

  Further, in FIG. 87, when the section length is weighted, the section length of the designated section allocated to the frame of the high display rate / low resolution section in which the frames of the high display rate / low resolution type V3 are continuous is not weighted. Since it is 3d, which is 3 times the section length d, the horizontal display area D3 representing the high display rate and low resolution section when the section length is weighted is also three times that when the section length is not weighted. Become.

  As described above, when the section length is weighted, the type information is different from the case where the section length is not weighted according to the weighted section length (in other words, the section length is weighted). If not, the type information is different from the case where the section length is weighted).

  In the editing device 2, as the operation mode, the above-described weighted on mode for weighting the section length of the designated section and the weighted off mode without weighting are provided, and the operation mode is set according to the operation of the operator. It is possible to switch to a weighted on mode or a weighted off mode.

  Therefore, with reference to a flowchart of FIG. 88, processing (processing related to the section length) performed by the microprocessor 3 of the editing device 2 regarding the section length of the specified section will be described.

  It is assumed that a weight button (not shown) that is operated when the operation mode is switched to the weighted on mode or the weighted off mode is provided at a predetermined position on the editing screen 50 (FIG. 2). By operating the weight button, the operation mode is switched to the weighted on mode or the weighted off mode.

  The length of the movable range of the scrub bar 66 is assumed to be 1.

  In the process related to the section length, in step S1131, the GUI control unit 1111 determines whether the operation mode has been switched. If it is determined in step S1131 that the operation mode has been switched to the weighted on mode, the process advances to step S1132, and the GUI control unit 1111 stores the Fy file stored in the file storage unit 1121 of the Fy file management unit 1114 (FIG. 80). Based on the change amount or the display type in, as described above, the weight attached to the section length of each designated section assigned to the frame is obtained, and the process proceeds to step S1133.

  In step S1133, the GUI control unit 1111 obtains a weighted section length for each designated section based on the weight given to the section length of each designated section assigned to the frame.

  That is, in step S1132, when the weight attached to the section length of each specified section allocated to the frame is obtained based on the change amount or the display type as described above, the GUI control unit 1111 has the same section length. A section in which designated sections that are weighted by the weight w are continuously recognized as the same weight section.

  As described above, assuming that the sequence of designated sections assigned to the frames constituting the moving image to be scrubbed is divided into Q equal weight sections, the GUI control unit 1111 includes the Q identical weight sections. Of these, the number Fq of the designated sections constituting the qth same weight section from the head, and the weight wq attached to the section length of the designated section constituting the qth same weight section obtained in step S1133. The section length wq / (ΣwiFi) of the designated section constituting the qth same weight section from the head is obtained.

  When the GUI control unit 1111 obtains the section length wq / (ΣwiFi) of the designated section constituting each of the Q equal weight sections in step S1133, the display control unit 1117 obtains the section length wq / (ΣwiFi) of the designated section. (FIG. 80), the process proceeds to step S1134, and the section length of the designated section that divides the movable range of the scrub bar 66 displayed on the display unit 55 of the editing screen 40 is obtained in step S1133. Is set (changed) to return to step S1131.

  On the other hand, when it is determined in step S1131 that the operation mode has been switched to the weighted off mode, the process proceeds to step S1135, and the GUI control unit 1111 sets the section length of each specified section allocated to the frame to the same 1 / F. (Change) and supply the section length 1 / F to the display control unit 1117, and the process returns to step S1131. Note that, as described above, F represents the total number of frames constituting the playback video of the editing target (that is, the scrub target) displayed on the spiral video display unit group 58.

  By the way, with respect to the movable range from the top Bst to the back Bed of the scrub bar 66 described above, it corresponds from the beginning to the end of the playback video of the editing target (that is, the scrub target) displayed on the spiral video display unit group 58. For example, as shown in FIG. 89, the range from the start point indicator IGst for specifying the start position to the end point indicator IGed for specifying the end position is set as a part of the playback video. A frame corresponding to (allocation range) can also be allocated.

  In this case, not all of the range from the top frame to the end frame of the playback video of the editing target (that is, the scrub target) displayed in the spiral video display unit group 58 is assigned a part of the range (scrubbing). The range of frames allocated from the first designated section to the last designated section of the movable range of the bar 66).

  In this way, by setting a partial range of the reproduced video as the allocation range with respect to the movable range of the scrub bar 66, the scrub bar 66 can be moved compared to the case where all the ranges are the allocation range. The number of frames allocated from the first designated section to the last designated section of the range is reduced, and as a result, the length of the designated section is increased.

  Therefore, it is possible to prevent the frame used as the frame of interest to be displayed on the spiral video display unit group 58 from greatly changing between before and after the movement of the scrub knob 67 due to the short section length of the designated section. As a result, a desired image frame can be easily found.

  The start point indicator IGst and the end point indicator IGed represent allocation ranges that are ranges of frames that are allocated from the first designated section to the last designated section of the movable range of the scrub bar 66, and can be referred to as allocation range information. Since it is operated when an allocation range is designated, it can also be referred to as a range designation operation means.

  Next, referring to the flowchart of FIG. 90, as described in FIG. 89, the frame range (allocation range) to be allocated to the movable range (specified section) of the scrub bar 66 is set to the start point indicator IGst and the end point indicator IGed. The allocation range setting process to be set according to will be described.

  When a predetermined scrub range setting start operation is performed from the operation controller 37, the mouse 38, or the keyboard 39, the GUI control unit 1111 (FIG. 53) controls the display control unit 1117 in step S1141, thereby controlling the scrub bar 66. The start point indicator IGst and the end point indicator IGed are displayed at predetermined positions, and the process proceeds to step S1142.

  In step S1142, the GUI control unit 1111 (FIG. 53) corresponds to the position of the start point indicator IGst in the frame constituting the playback image of the editing target (that is, the scrub target) displayed on the spiral video display unit group 58. The range from the frame to the frame corresponding to the position of the end point indicator IGed is set as an allocation range, and the frame in the allocation range is allocated to the movable range of the scrub bar 66.

  That is, for the sake of simplicity, if the weighting for the section length of the specified section described above is not taken into consideration, the GUI control unit 1111 determines the movable range of the scrub bar 66 and assigns the length to the allocation range. The division value divided by the number of frames is divided into specified sections having a section length, and the frames in the allocation range are allocated from the first specified section to the end specified section of the movable range.

  Then, the process proceeds from step S1142 to step S1143, and the GUI control unit 1111 moves the start point indicator IGst or the end point indicator IGed (FIG. 89), that is, the operator moves the start point indicator IGst or the end point indicator IGed. In this way, it is determined whether or not the operation controller 37, mouse 38 or keyboard 39 has been operated.

  If it is determined in step S1143 that neither the start point indicator IGst nor the end point indicator IGed has started moving, the process returns to step S1143.

  If it is determined in step S1143 that the movement of the start point indicator IGst or the end point indicator IGed has started, that is, the operator moves the start point indicator IGst or the end point indicator IGed so that the operator moves the start point indicator IGst or the end point indicator IGed. When the operation signal corresponding to the operation is supplied from the operation controller 37, the mouse 38 or the keyboard 39 to the GUI control unit 1111, the GUI control unit 1111 receives the operation signal from the keyboard 39 or the mouse 38. Accordingly, a position to move the start point indicator IGst or the end point indicator IGed is obtained, and information indicating the position is supplied to the display control unit 1117, and the process proceeds to step S1144.

  In step S1144, the display control unit 1117 causes the start point indicator IGst or the end point indicator IGed to be displayed at a position corresponding to the information from the GUI control unit 1111 instead of the position displayed immediately before, and the process proceeds to step S1145.

  In step S1145, the GUI control unit 1111 determines whether or not the movement of the start point indicator IGst or the end point indicator IGed is finished, that is, the operator moves the start point indicator IGst or the end point indicator IGed. Determine whether the operation is finished.

  If it is determined in step S1145 that the operation of moving the start point indicator IGst or the end point indicator IGed has not been completed, that is, the operator moves the start point indicator IGst or the end point indicator IGed so that the operator moves the start point indicator IGst or the end point indicator IGed. Alternatively, when the keyboard 39 continues to be operated and an operation signal corresponding to the operation is supplied from the operation controller 37, mouse 38 or keyboard 39 to the GUI control unit 1111, the GUI control unit 1111 includes the operation controller 37, In response to an operation signal from the mouse 38 or the keyboard 39, a position for moving the start point indicator IGst or the end point indicator IGed is obtained, and information indicating the position is supplied to the display control unit 1117, and the process proceeds to step S1144. Ri, and the same processing is repeated.

  Here, the start point indicator IGst or the end point indicator IGed is moved in the horizontal direction in accordance with the operation of moving the start point indicator IGst or the end point indicator IGed by the processing of steps S1143 to S1145.

  On the other hand, if it is determined in step S1145 that the operation of moving the start point indicator IGst or the end point indicator IGed has been completed, that is, the operator moves the start point indicator IGst or the end point indicator IGed, the operation controller 37, the mouse 38, or the keyboard 39. When the operation signal corresponding to the operation of moving the start point indicator IGst or the end point indicator IGed is not supplied from the operation controller 37, mouse 38, or keyboard 39 to the GUI control unit 1111, the GUI control unit 1111 The position of the start point indicator IGst or the end point indicator IGed (in the horizontal direction) that has been moved is recognized, the process returns to step S1142, and the editing target (displayed in the spiral image display unit group 58) The range from the frame corresponding to the position of the start point indicator IGst to the frame corresponding to the position of the end point indicator IGed, among the frames composing the playback video of the target image of Mari Scrub), is the frame of the allocation range. Is assigned to the movable range of the scrub bar 66, and the same processing is repeated thereafter.

  As described above, the operator operates the start point indicator IGst or the end point indicator IGed as a GUI to select the scrub target among the frames constituting the edit target playback video displayed in the spiral video display unit group 58. The frame range (the frame range (allocation range) allocated to the movable range of the scrub bar 66) can be easily changed.

  72 is terminated when, for example, a predetermined scrub range setting end operation is performed.

In this way, by setting the section length of the designated section to which the frame is assigned to a section length that is weighted according to the motion information or the like, the movable range of the scrub bar 66 is changed to a frame with a large amount of motion information as a change amount. The assigned section to be assigned can be divided into designated sections having a larger section length.

  Further, in the above-described scrub process, the size of the frame image displayed in the spiral video display unit group 58 is fixed, but can be changed according to the display type of the frame, for example. That is, it is possible to further add an image size (display size) as a display parameter that is changed according to the amount of change in the image. Specific examples will be described below.

  In this example, the frame display type is determined to be, for example, one of the still image type V1, the normal type V2, and the high display rate / low resolution type V3 described with reference to FIG. Suppose that

  In addition, for a still image type V1 frame image, for example, a certain default size is set for the length of the diagonal line or the horizontal and vertical lengths, and the image is displayed in the default size.

  For normal type V2 frame images, for example, 1.5 times the default size is set, and the diagonal length or horizontal and vertical length is displayed at 1.5 times the default size. Is done.

  For the image of the frame of the high display rate and low resolution type V3, for example, the size of twice the default size is set, and the length of the diagonal line or the horizontal and vertical length is twice the default size. It is displayed in the size.

  As described above, in order to display an image with a size corresponding to the display type, the image may be displayed in step S1122 of the scrub process described with reference to FIG.

  Here, with reference to the flowchart in FIG. 91, the processing (display processing) in step S1122 in FIG. 84 in the case of displaying an image with a size corresponding to the display type will be described.

  Here, as described above, the display type is determined to be, for example, one of the still image type V1, the normal type V2, and the high display rate low resolution type V3 described in FIG. To do. In addition, a certain default size is set for the still image type V1 frame image, and for the normal type V2 frame image, for example, 1.5 times the default size is set. To do. Further, for the frame image of the high display rate / low resolution type V3, for example, a size twice as large as the default size is set.

  In step S1151, the display control unit 1117 (FIG. 79) displays the frame (target frame) assigned to the designated section where the scrub knob 67 is located, supplied from the display type acquisition unit 1115 (FIG. 79). It is determined whether the type is a still image type V1, a normal type V2, or a high display rate low resolution type V3.

If it is determined in step S1151 that the display type of the target frame is the still image type V1, the process proceeds to step S1152, and the display control unit 1117 selects the corresponding video display unit W57 _{1 in the} spiral video display unit group 58. the ～W57 _n, the default size, and displays an image based on image data stored in the frame buffer 1117a (FIG. 79).

If it is determined in step S1151 that the display type of the target frame is the normal type V2, the process proceeds to step S1153, and the display control unit 1117 in the spiral video display unit group 58 corresponds to the corresponding video display unit W57. ₁ , 57 ₂ ,..., Or W57 _n is enlarged to 1.5 times the default size, and a frame buffer is added to the enlarged video display unit (hereinafter referred to as an enlarged display unit as appropriate). An image based on the image data stored in 1117A is displayed.

If it is determined in step S1151 that the display type of the frame of interest is the high display rate low resolution type V3, the process proceeds to step S1154, and the display control unit 1117 corresponds to the spiral video display unit group 58. The video display units W57 _{1 to} W57 _n are enlarged to twice the default size, and an image based on the image data stored in the frame buffer 1117A is displayed on the enlarged video display unit.

  In addition, the size of the spiral video display unit group 58 (image displayed on it) is changed according to the display type as described above only while the scrub knob 67 is operated (for example, while being dragged). When the operation of the scrub knob 67 is stopped, the default size can be set.

  In this way, since the image of the normal type V2 frame has a certain amount of movement, the image of the normal type V2 frame is displayed in a size larger than the default size, so that the operator can Details can be confirmed.

  Also, because there is intense movement in the frame of the high display rate low resolution type V3, the image of the frame of the high display rate low resolution type V3 must be displayed in a larger size than the image of the normal type V2 frame. Thus, the operator can confirm the movement of the image in more detail.

  Furthermore, by displaying an image with a different size for each display type, the operator can intuitively determine the degree of movement of the image in the entire video (reproduced video) displayed in the spiral video display unit group 58. I can grasp it.

In particular, when the display mode of the spiral image display unit group 58 (FIG. 3) is changed to the display mode described above with reference to FIG. As shown. However, in FIGS. 92 to 101, for convenience, a case where one image with a severe pattern appears in isolation is shown, and the display rates in the video display portions W57 _{1 to} W57 _n are the same. Assume the case.

  As can be seen from FIGS. 92 to 101, the video display unit W57 (W57c to W57l in these drawings), which is the display target of the image whose picture changes greatly, is sequentially changed as the enlarged video display unit W57X, and the enlarged video display is performed. With respect to the portion W57X, an image whose picture changes drastically is emphasized by being displayed in a larger size than the other images, and the reproduction time difference ΔT is separated from the back side by the virtual time axis TP. Will continue to flow.

Further, when there is a continuous section of images with a significant change in design with respect to the reproduced video, as shown in FIG. 102, the video display section W57 (in this figure, W57c to W57g), as the enlarged video display portions W57X _{1 to} W57X _5, are shifted so as to flow with a reproduction time difference ΔT in a continuous state. However, in FIG. 102, as in FIGS. 92 to 101, it is assumed that the display rates of the video display units W57 _{1 to} W57 _n are the same.

Further, when there are a plurality of sections in which images with drastic changes in the image are continuous with respect to the reproduced video, as shown in FIG. 103, the video display unit W57 (in this figure, W57c1) displays each image in the section. ~W57g1, W57c2~W57g2, W57c3~W57g3, W57c4~W57g4) is, as an enlarged image display section _{W57X 11 ~W57X 15, W57X 21 ~W57X} 25, W57X 51 ~W57X 35, W57X 41 ~W57X 45, continuous to each section In this state, the flow changes so as to flow at a reproduction time difference ΔT. However, in FIG. 103, as in FIGS. 92 to 101, it is assumed that the display rates of the video display units W57 _{1 to} W57 _n are the same.

  In this way, the operator is conscious of the entire content of the spiral video display unit group 58 by displaying the image with a large change in pattern flowing in an enlarged manner, for example, the content of the image with a large change in pattern such as a scene change. While being able to grasp intuitively.

In addition, when the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58 are reproduced as moving images, the image size can be changed according to the amount of change of the image without performing the operation of scrubbing. Can do. Therefore, even when the operation of scrubbing is not performed, the operator can intuitively grasp the content of the image whose picture changes drastically while being conscious of the entire spiral video display unit group 58.

  In the above specific example, the image is displayed in a different size for each display type determined based on the motion information. Instead, for example, for each display type determined based on the fineness information. You may make it display with a different size, or you may make it display with a different size for every display type determined based on both fineness information and motion information.

In addition, when displaying an image with a different size for each display type determined based on the fineness information, for example, an image of a display type whose degree of fineness is determined based on the large fineness information For example, the operator can recognize the details of the image intuitively.

In addition, when displaying in different sizes for each display type determined based on both the fineness information and the motion information, for example, an image of a display type in which the degree of motion is determined based on the large detail information It is possible to display the image in a larger size, and display the corresponding video display unit W57X ₁ , W57 ₂ ,..., Or W57 _n frame in color so that the color becomes darker as the degree of fineness increases. Thus, the operator can recognize the details of the image more intuitively.

  As described above, the case where the image size is changed according to the display type of the frame of interest has been described. However, for example, it is also possible to perform the change according to the operation of the scrub knob 67 by the operator.

  That is, generally, when searching for a desired image, the operator operates the scrub knob 67 to move quickly when the image displayed on the spiral image display unit group 58 is not so focused. When attention is paid, the scrub knob 67 is operated so as to move slowly.

  Therefore, when the scrub knob 67 is moved fast, the image is displayed in a default size, and when the scrub knob 67 is moved slowly, the image is displayed in a size larger than the default size. can do. By doing so, the operator can easily find a desired image.

  Further, in the above-described scrub process, a scrub bar 66 having a scrub knob 67 is adopted as means operated when designating a frame to be displayed on the spiral image display unit group 58, and the spiral is changed depending on the position of the scrub knob 67. Although the frame to be displayed on the video display unit group 58 is designated, the frame to be displayed on the spiral video display unit group 58 can be specified by, for example, a so-called (real, not GUI) jog dial or shuttle ring. It can be done by manipulating it.

Note that when the video display portions W57 _{1 to} W57 _{n of} the spiral video display portion group 58 are reproduced as moving images, even if the operation of scrubbing is not performed by the jog dial, shuttle ring, or the like, depending on the amount of change in the image. You can change the image size. Therefore, the operator can intuitively grasp the content of the image whose design changes drastically while being conscious of the entire spiral video display unit group 58.

  In the scrub process described above, a change amount (motion information, fineness information) indicating the degree of change in the image data of the moving image is obtained for each frame, and the display type is determined for each frame based on the change amount. Various processing related to scrubbing is performed using the display type. However, for example, when audio data is attached to a moving image, the display type is the level (power) of the audio data itself, It can be determined based on a value representing the degree of change. In other words, for example, in a frame in which the level of audio data changes from small to large (there is a sound excitement), there is a high possibility that an image of a scene to be noticed is reflected. For example, if the motion information is large to some extent, a display type with a high display rate is determined. For example, if the fineness information is large to some extent, the display type with high resolution is selected. Can be determined.

  In the scrub process described above, scrubbing is performed using main line data and proxy data recorded in the storage device 22, that is, image data of two types of resolutions. This can be performed using image data of one type of resolution or three or more types of resolution.

  Further, in the above-described scrub process, the display type determination unit 1023 in FIG. 59 (the same applies to the display type determination unit 1123 in FIG. 80), the change amount is greater than or equal to a certain threshold value in order to deal with the seek problem. Judgment is made whether or not the minimum number of frames is more than the minimum number of frames N (continuity determination), but if there is no need to deal with the seek problem, the continuity determination is not performed. The display type can be determined by whether the amount is above a certain threshold or below a certain threshold.

  In the above-described scrub process, the amount of change is obtained for each frame, and the display type is determined for each frame based on the amount of change for each frame. Is possible. Similarly, the display type can be obtained for each of a plurality of frames.

  That is, FIG. 104 shows the amount of change for each frame (unit) constituting, for example, one GOP as a plurality of frames. In FIG. 104, the horizontal axis represents the frame, and the vertical axis represents the amount of change.

  In FIG. 104, a thick line indicates the amount of change for each frame constituting 1 GOP. As the amount of change for each frame constituting a 1 GOP, for example, an average value of the amount of change of all the frames constituting the 1 GOP or the amount of change of an I picture in the frames constituting the 1 GOP can be employed. .

  As shown in FIG. 104, when determining the amount of change for each frame constituting one GOP, based on the amount of change, in the same manner as when the display type for each frame is determined from the amount of change for each frame, The display type for each frame (unit) that constitutes can be determined.

  In addition, for example, the display type for each frame constituting one GOP as a plurality of frames can be determined based on the amount of change for each frame in addition to the amount of change for each frame constituting one GOP. is there.

  That is, FIG. 105 shows an Fy file in which both the amount of change for each frame and the display type for each frame are stored.

  For example, assuming that 1 GOP is composed of 15 frames, the display type for each 15 frames constituting 1 GOP can be determined, for example, as the display type having the highest ratio in 15 frames constituting 1 GOP.

  In this case, for example, in 1 GOP composed of the first to fifteenth frames from the top shown in the Fy file of FIG. 105, the ratio of the display type V2 among the display types V1, V2, and V3 for each frame Since there are the most, the display type of the 1GOP is determined to be V2.

  For example, for the ninth frame from the top shown in the Fy file of FIG. 105, the display type for each frame is V1, but the display type for each GOP is V2.

  As described above, when the change amount and display type are obtained as a plurality of frames, for example, for each frame constituting 1 GOP (determined) and stored in the Fy file, the change amount and display type for each frame Compared to storing the file in the Fy file, the capacity (file size) of the Fy file can be reduced, and the processing load required for the analysis (file parsing) of the Fy file can be reduced.

  When the image data is encoded in the MPEG format when creating the Fy file, when calculating the change amount and display type for each frame that constitutes 1 GOP, the change amount and display type are encoded in the MPEG format. Can be included in the GOP header in the resulting stream.

  Here, in the above-described scrub process, image data is encoded by the MPEG method and recorded in the storage device 22, and therefore the decoder 1116 in FIG. 79 needs to decode the image data by the MPEG method. In the MPEG method, for example, 15 frames are set as 1 GOP, and each frame is encoded as a picture of any picture type of I (Intra) picture, P (Predictive) picture, or B (Bidirectionally Predictive) picture. . Of the I, P, and B pictures, the P picture and the B picture are encoded using the I or P picture encoded before them as a reference image (the image that serves as a base for generating a predicted image). Therefore, it cannot be decoded unless the reference image is decoded.

  That is, for example, if 1 GOP is composed of 15 frames, and each frame of the 15 frames is represented by I, P, or B indicating the picture type and a number indicating the display order, 1 GOP The arrangement of 15 frames can be expressed as B1, B2, I3, B4, B5, P6, B7, B8, P9, B10, B11, P12, B13, B14, and P15, for example.

  Now, for example, the sixth P picture P6 from the top of the GOPs composed of the 15 frames B1 to P15 as described above is encoded using the third I picture I3 as a reference image, and the ninth P picture P9. Are encoded using the sixth P picture P6 as a reference image, the twelfth P picture P12 is encoded using the ninth P picture P9 as a reference image, and the fifteenth P picture P15 is encoded as the twelfth P picture. It is assumed that the picture P12 is encoded as a reference image. Furthermore, it is assumed that the 13th B picture B13 is encoded using the 12th P picture P12 and the 15th P picture P15 as reference images.

  In this case, for example, when the 13th B picture B13 is the frame of interest to be displayed on the viewer 15, the 3rd I picture I3 is decoded and the 6th P picture P6 is referred to the 3rd I picture I3. The 9th P picture P9 is decoded with reference to the 6th P picture P6, the 12th P picture P12 is decoded with reference to the 9th P picture P9, and the 15th P picture is decoded. Unless the picture P15 is decoded with reference to the twelfth P picture P12, the twelfth P picture P12 and the fifteenth P picture P15 cannot be referenced. It cannot be decrypted. Therefore, it takes time to decode the B picture B13.

  Therefore, P pictures P6, P9, P12, and P15 referred to for decoding other pictures are stored in separate files called P-to-I files as I pictures I6, I9, I12, and I15, respectively. The decoder 1116 can perform decoding by referring to the picture stored in the P-to-I file as necessary. In this case, for example, the B picture B13 can be decoded in a short time with reference to the I pictures I12 and I15 stored in the P-to-I file.

  As described above, the scrub process described above applies the resolution and image size to the display parameters (display parameters) as parameters to be changed according to the amount of change, and matches the playback parameters (reproduction parameters). In this case, the display rate (frame rate) and the playback speed v are applied, but various combinations of these display parameters and playback parameters can be set as appropriate.

  In the above-described scrub process, a playback time difference ΔT can be applied in addition to the display rate (frame rate) and playback speed v in the playback parameters. In this case, for example, the reproduction time difference ΔT may be increased as the image change is increased, and the distance between the video display units W57 may be increased or decreased according to the reproduction time difference ΔT. In this way, when the image changes drastically, the display form for the portion changes, so that the image search can be further facilitated.

  The case of scrubbing has been described above, but the present invention can also be applied to the case of performing variable speed reproduction (n-times speed reproduction) other than scrubbing.

As described above, when the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58 are reproduced as moving images, the spiral video display unit group is not required to be scrubbed. In 58, since the image itself appears to flow, the same effect can be obtained.

(5) Drawing Interval Automatic Change Function By the way, when the editing device 2 displays the reproduced video on the video display units W57 _{1 to} W57 _n by thinning out every predetermined frame, the constant change amount in the reproduced video is displayed. A video display unit W57 (W57 ₁ , W57 ₂ ,..., Or W57 _n−1 ) that targets a frame that exceeds (threshold) as a current display target, and a video display unit W57 (W57 _n−1 ) that is a display target after the playback time difference ΔT. W57 ₂ , W57 ₂ ,..., Or W57 _n ), the number of video display units corresponding to the number of frames thinned out (hereinafter referred to as new video display units as appropriate) is newly created. The thinned frames can be displayed on each of the created new video display sections.

  Here, processing by the microprocessor 3 and the GPU 4 will be described. For convenience, frames constituting a moving image are thinned out for each predetermined frame, and, for example, 19 frames among a plurality of frames left as frames constituting a reproduced image are used. Assume that only the th frame is a frame that exceeds a certain amount of change (threshold), and this frame will be mainly described.

That is, the microprocessor 3 and the GPU 4 thin out every predetermined frame for the 1st to 18th frames that are less than a certain amount of change (threshold) among the frames constituting the moving image while the editing screen 50 is displayed. After decoding the frames that are not to be thinned out, they are sequentially displayed on the video display units W57 _{1 to} W57 _n of the display unit 55 with a reproduction time difference ΔT.

  Incidentally, the amount of change can be acquired from the Fy file stored in the storage device 22 by the pre-editing process of the scrub process described above.

  On the other hand, as shown in FIG. 106, for example, the microprocessor 3 and the GPU 4 stop the thinning-out process when the 19th frame having a predetermined change amount (threshold value) or more is recognized, and perform the scrub process as described above. Without changing the display rate, m frames (hereinafter referred to as thinned frames) that were supposed to be thinned are also sequentially decoded.

Then, the microprocessor 3 and GPU4 are 19 th image (frame) and displays on the image display unit W57 _1, new novel image display unit W57NX ₁ ~W57NX _m number corresponding to m pieces of thinning frames The corresponding thinned frame is displayed on each of the created new video display sections W57NX _{1 to} W57NX _m .

The 19th image (frame) is sequentially displayed on the video display units W57 ₂ , W57 ₃ ,... With a reproduction time difference ΔT, and the new video display units W57NX _{1 to} W57NX _m are images of the 19th image. and at a similar playback time difference [Delta] T, the image display section W57 of the current display target, between the between (W57 ₂ and W57 ₃ of the image display section W57 to be displayed on the next, W57 ₃ and W57 ₄ and Will be displayed at…).

As a result, for example, as shown in FIG. 107 and FIG. 108, the current display target video display unit W57c displaying an image whose picture changes drastically, and the virtual time axis TP (FIG. 26) are moved in the direction of travel. The display transition between the playback time difference ΔT with respect to the video display unit W57d is finely presented to the new video display units W57NX _{1 to} W57NX _m . Can grasp. Incidentally, FIGS. 107 and 108 show a case where m = 5.

Note that the microprocessor 3 and the GPU 4 start the thinning process again for the frames after the m thinned frames, and after decoding the frame that is not the thinning target, the reproduction time difference ΔT is separated and the display unit 55 The images are sequentially displayed on the video display units W57 _{1 to} W57 _n .

In this way, when the editing apparatus 2 displays the reproduced video in a predetermined frame on the video display units W57 _{1 to} W57 _n and displays them, frames exceeding a certain amount of change (threshold) in the reproduced video. without thinned out for the for the frame, the image display section W57 of the current display target, between the between (W57 ₂ and W57 ₃ of the image display section W57 to be displayed on the next, W57 ₃ and W57 ₄ ), new video display portions W57NX _{1 to} W57NX _m can be created and displayed.

  In the above-described method for changing the drawing interval, all the frames to be thinned (thinned frames) are the objects to be displayed between the current video display unit W57 to be displayed and the video display unit W57 to be displayed next. However, not all the frames to be thinned out, and any number of frames may be used.

  Further, the number of frames to be displayed (the number of thinned frames) displayed between the current display target video display unit W57 and the next video display unit W57 to be displayed is, for example, equal to or more than a first threshold value and second. When it is less than the threshold value, the first frame number is set, and when it is equal to or greater than the second threshold value and less than the third threshold value, the second frame number is set to be greater than the first frame number and is equal to or greater than the third threshold value. In some cases, the number of frames can be switched in stages according to the amount of change, such as a third frame number greater than the second frame number.

  In this way, the editing device 2 changes the number of frames displayed between the current video display unit W57 to be displayed and the video display unit W57 to be displayed next, depending on the degree of change in the pattern. Therefore, the operator can grasp not only the displayed image itself but also the changed part in the reproduced video not only by the number of frames, and as a result, the image can be searched more intuitively. it can.

  Further, the above-described method for changing the drawing interval is the number of thinned frames corresponding to the amount of change, and the thinned frame between the current video display unit W57 to be displayed and the video display unit W57 to be displayed next. And the thinned frame can be displayed in a size corresponding to the amount of change.

  In this way, the editing device 2 determines the number of frames and the size of the frames displayed between the video display unit W57 that is the current display target and the video display unit W57 that is the next display target, depending on the degree of change in the pattern. Therefore, the operator can grasp not only the displayed image itself but also the changed part in the playback video not only by the number of frames and the frame size, but as a result, it is more intuitive. Lets you search for a picture.

  Furthermore, the above-described method for changing the drawing interval can be combined with the above-described scrub process. That is, a thinned frame is displayed between the current display target video display unit W57 and the next display target video display unit W57 with the number of thinned frames according to the change amount, and the change amount It is also possible to change the resolution and display rate of an image (a frame other than a thinned frame) according to a corresponding display type.

  In this way, the editing device 2 determines the number of frames displayed between the current display target video display unit W57 and the next display target video display unit W57 according to the degree of change in the pattern, and Since the resolution and display rate of the frame displayed on the video display unit W57 can be changed, the operator can smoothly display the playback video regardless of the moving speed of the scrub knob 67 and display the playback video. The details of the image itself can be grasped, and as a result, the image can be searched more intuitively.

However, as long as the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58 are reproduced as moving images, the image itself flows in the spiral video display unit group 58 without performing scrubbing. Therefore, as shown in FIGS. 107 and 108, the same effect can be obtained in the recognition of the moving image in the 3D space.

(6) Operation and Effect In the above configuration, the editing device 2 of the editing system 1 selects a desired clip from the clip list display unit 51 of the editing screen 50 displayed on the display 40, and drags it to the display unit 55. When the playback button 56 is clicked after dropping and selecting the spiral video display unit group 58 associated with the clip as necessary, each video display unit W57 _{1-of the} spiral video display unit group 58 is clicked. to W57 _n to display respectively the reproduced video of the clip with a certain time difference.

Then, according to such a video display method, each scene of the playback video of the clip has a plurality of video display units W57 _{1 to} W57 _n constituting the spiral video display unit group 58 in front of the screen. By starting the reproduction process for each of the video display units W57 _{1 to} W57 _n with the reproduction time difference ΔT set at that time, a three-dimensional sense of depth can be given.

Therefore, in the editing apparatus 2, even when the operator searches for a frame such as a scene change point, the same reproduced video is sequentially displayed on the video display units W57 _{1 to} W57 _n at the time interval of the reproduction time difference ΔT. Therefore, as compared with the conventional editing screen 1 (FIG. 121), the operator looks at the two-dimensional playback image displayed on the monitor unit 3 while moving one frame at a time and looks for a desired frame. The image search operation can be executed easily and intuitively without forcing the operator to perform a complicated frame advance operation for moving the reproduced video one frame at a time.

In particular, the microprocessor 3 of the editing apparatus 2 displays the spiral image display unit group 58 having a spiral structure by the image display units W57 _{1 to} W57 _n arranged in a spiral shape, so that one screen of the display unit 55 is displayed. All the video display portions W57 _{1 to} W57 _n can be displayed without being hidden, and the operator can easily perform the image search operation and the editing operation of the desired scene.

This becomes an important factor when the editing device 2 stops all the reproduced videos of the video display units W57 _{1 to} W57 _n , and the video display units W57 _{1 to} W57 constituting the spiral video display unit group 58. _Since the still images of the peripheral frames that are shifted with respect to _n are sequentially displayed with respect to _n , there is a special effect that it is not necessary to perform the image search operation by the frame advance operation.

Further, the editing device 2 displays the playback video that is linked to each other in the state of the moving image while displaying the playback time difference ΔT in the display order for each of the video display units W57 _{1 to} W57 _{n of} the spiral video display unit group 58, thereby spiraling. Play will continue to progress reproduced video sequentially toward the rear side of the display area group 58, for example at the time of entering update change of scene change to a new scene from the front side of each image display section W57 ₁ ~W57 _n to the back side As the video progresses sequentially, the operator can intuitively search and edit images while letting the operator experience the time (playback speed and playback direction) that flows through the spiral video display group 58 throughout the playback video. Can be executed.

The editing device 2 makes the operator feel that the entire spiral video display unit group 58 is lowered to the back side when the playback video to be displayed on the video display units W57 _{1 to} W57 _n is reversely played back. Image search work and editing work can be executed.

  Furthermore, since the editing device 2 arranges the spiral image display unit group 58 in the 3D virtual space on the internal memory of the GPU 4, the editing device 2 receives the sub 3D space image IM1 and the character CA1 having the same coordinate system as the 3D virtual space. The viewpoint can be moved toward the inner side of the spiral vortex, and the spiral image display unit group 58 can be changed to a display form that can be seen from the internal viewpoint after the movement and can be displayed on the display unit 55.

Accordingly, the editing device 2 moves the viewpoint toward the back side of the spiral vortex and the spiral video display unit group 58 even when the playback video in each of the video display units W57 _{1 to} W57 _n is paused. By changing the display form of the display, it is possible to give the operator the impression that the playback process is proceeding toward the future while passing through the center of the spiral vortex, and a highly entertaining GUI (Graphical User Interface) Can be provided.

In addition to this, the editing apparatus 2 changes the viewpoint and line-of-sight direction with respect to the spiral video display unit group 58 according to the relative positional relationship between the sub 3D space image IM1 and the character CA1, thereby providing the spiral video display unit group 58. When the viewpoint is moved to the outside of the circle formed by the plurality of video display units W57 _{1 to} W57 _n constituting the video, the display mode of the spiral video display unit group 58 is changed to a belt shape (FIGS. 34 to 40, etc.) Therefore, the playback time of the entire clip can be imaged by the operator from the entire length of the spiral image display unit group 58, or the entire clip can be changed to a position or angle that is easy for the operator to edit.

  Furthermore, since the editing device 2 can perform operations such as individual clip extraction, multiple clip editing processing, and edited video confirmation processing in one display unit 55, clip extraction as in a conventional editing device is possible. It is performed using the monitor unit 3 of the editing screen 1, editing processing is performed using the storyboard unit 4 and the timeline unit 5, and editing video confirmation processing is performed using the monitor unit 3 without requiring complicated work. Such a series of editing operations can be performed collectively in the display unit 55. In this way, not only the so-called image search operation but also the editing operation as a whole can be facilitated.

Further, in the editing apparatus 2, when displaying the reproduced video on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58, “main audio output mode” and “all audio output mode” are selected as the audio output mode. Therefore, by switching to the audio output mode in accordance with the content of the reproduced audio of the target clip at that time, it is possible to search for a desired video / audio portion using the output audio.

In this case, in this editing apparatus 2, when the “all audio output mode” is set as the audio output mode, the reproduction video displayed on the video display units W57 _{1 to} W57 _m apart from the reference video display unit W57 _n is displayed. In a state in which the volume is adjusted so that the volume of the sound is lower, all the playback sounds associated with the playback images displayed on the respective video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 operated at that time are displayed. Is output from the speaker 41, so that the audio can be expressed three-dimensionally, and the search operation for a desired video / audio portion based on the output audio can be further facilitated depending on the content of the clip. Of course, not only the ease of editing but also the audio expression that makes a sense of depth unprecedented can be realized only by performing the reproduction by the spiral band display.

The editing device 2 of the present invention generates a plurality of display video data to be displayed on the plurality of video display units W57 _{1 to} W57 _{n on} the editing screen 50. In addition, the editing device 2 displays a display method for displaying the picture (frame) based on the amount of change indicating the degree of change of each image data in the video / audio signals displayed on the video display units W57 _{1 to} W57 _n. The type is determined in units of pictures (FIGS. 41 to 78, etc.).

  Then, the editing device 2 changes the resolution and display rate of the image corresponding to each display video data in accordance with the type information indicating the display type in units of pictures, and displays each display with the changed resolution and display rate of the image. Video data is displayed at a predetermined position on the editing screen 50 in a moving image state with a time difference provided in the display order (FIGS. 79 to 106, etc.).

  Therefore, the editing apparatus 2 can display each image of the moving image on the editing screen 50 in the display order with a time difference, so that the content (story) of the image can be drawn. As a result, the operator can be made to search for an image while intuitively grasping the contents of the moving image, and thus the editing points (in point and out point) can be easily searched.

  Further, as a display method for displaying each image of the moving image in a display order with a time difference, the editing device 2 does not display all images in the same display method, but according to the display type of the image. Then, the resolution and display rate of the image are changed, and not all the frames that have become the frame of interest are decoded, but only the necessary frames are decoded according to the display type.

  Therefore, even if the playback speed v is variable, the editing device 2 draws the contents (story) of the moving image so as to flow, and changes the display mode according to the change of each image. As a result, the operator can be made to search for an image while intuitively grasping the contents of the moving image, and thus the editing points (in point and out point) can be easily searched.

  Further, the editing apparatus 2 arranges the moving images to be displayed in the display order with a time difference arranged in a three-dimensional spiral. Therefore, this editing device 2 can present many images while maintaining their temporal continuity as compared with the case where they are simply arranged vertically or horizontally. As a result, the efficiency of image search can be improved.

  According to the above configuration, the playback video of the selected clip is displayed in a 3D spiral shape in a moving image state while displaying the playback time difference ΔT in the display order. It is possible to provide a display form that makes it easy to search for depth and images, and thus, as compared with the conventional editing screen 1, a desired video portion is searched while viewing a reproduced video of a two-dimensional one screen. It is possible to realize an editing system capable of facilitating editing work by allowing an operator to easily and intuitively search for a video portion.

Also, according to the above configuration, the images of the reproduced video displayed on all the video display units W57 _{1 to} W57 _n are not all displayed in the same display method, but are displayed according to the display type of the image. By changing the resolution and display rate, etc., and by decoding only the necessary frames according to the display type, instead of decoding all the frames that became the frame of interest, Thus, it is possible to search for an image while intuitively grasping the content of the reproduced video, and thus it is possible to easily search for an editing point.

(7) Second Embodiment (7-1) Configuration of Editing System According to this Embodiment In FIG. 1, reference numeral 90 denotes an editing system according to the second embodiment, which is displayed on the display unit 55 of the editing screen 50. The size of each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 (FIG. 2) is not only gradually increased from the back side to the front side of the screen, but also the video display unit W57 _1. ˜W57 _{n The} configuration is the same as that of the editing system 1 (FIG. 1) according to the first embodiment except that it fluctuates in accordance with the audio level of the reproduced audio accompanying the reproduced video displayed on _n . Various processes similar to those of the editing apparatus 2 in the first embodiment can be executed.

That is, in the case of this editing system 90, the microprocessor 3 of the editing device 2 is shown in FIG. 2 based on the video / audio signal of the clip provided from the storage device 22 via the PCI bus 15 and the south bridge 6 as described above. As described above, the playback video of the clip is displayed on the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58, respectively, while the audio level (audio signal) of each playback audio accompanying the playback video is displayed. The signal level) is sequentially notified to the GPU 4.

Then, based on the notification from the microprocessor 3, the GPU 4 determines whether or not the peak level of the reproduced sound is larger than a preset threshold Thd for each of the reproduced sounds. When the image of the frame is displayed, the corresponding video display portions W57 _{1 to} W57 _n are displayed larger than the size that should be normally displayed.

For example, the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n is FIG. 109A, and the audio level of the reproduced audio (the signal level of the audio signal) accompanying the reproduced video is shown in FIG. In the case of (B), it is displayed on the second to fourth video display parts W57 _l , W57 _{k, and} W57 _j counting from the reference video display part W57 _n located on the most front side of the screen to the back side. Since the audio level of each reproduced audio accompanying the reproduced video is larger than the threshold Thd, at this moment, as indicated by a cross arrow in FIG. 109 (C), these video display units W57 _l , W57 _k, W57 _j is displayed much larger than the others.

Further, in this case, the microprocessor 3 selects the reproduced audio accompanying the reproduced video displayed on the video display units W57 _l , W57 _k, W57 _j for the video display units W57 _l , W57 _k, W57 _j to be displayed in a large size. The GPU 4 is controlled so that the larger the audio level is, the larger the enlargement ratio of the video display units W57 ₁ , W57 _{k, and} W57 _j is displayed.

Thus, for example, in the example of FIG. 109 (B), highest sound level of the reproduced sound accompanying the reproduced images displayed from the reference image display section W57 _n in the second image display unit W57 _l to the right in this moment, third and fourth image display section W57 _k, W57 approximately the same size as the sound level of the reproduced sound accompanying the reproduced images displayed respectively slightly lower than this range _j from the reference image display section W57 _n counted to the right Therefore, the second video display portion W57 _l is enlarged and displayed at the maximum magnification on the right side from the reference video display portion W57 _n , and the other reference video display portions W57 _{k and} W57 _j are slightly smaller than this. Will be enlarged.

In this way, in the editing system 90, the audio level of the reproduced audio associated with the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 on the display unit 55 of the editing screen 50. Is displayed so that it can be visually recognized by the operator, so that the operator can intuitively recognize the contents of the clip and easily execute image search and editing operations.

(7-2) Video Display Unit Enlargement Display Processing Procedure Here, in practice, the microprocessor 3 of the editing device 2 in the editing system 90 applies the reproduced video displayed on the video display units W57 _{1 to} W57 _n as described above. The processing for enlarging and displaying the video display units W57 ₁ , W57 _{k, and} W57 _j according to the audio level of the accompanying reproduced audio is based on the control program stored in the hard disk device 7 (FIG. 1) and the video display shown in FIG. This is executed according to the part enlargement display processing procedure RT5.

  That is, when the microprocessor 3 finishes displaying the playback video of the clip on the spiral video display unit group 58 according to the playback processing procedure RT2 shown in FIG. 10, the above-described display change processing procedure RT2 and FIG. In parallel with the editing operation response processing procedure RT3, the GPU 4 starts the video display unit enlarged display processing procedure RT5.

In step SP51, the microprocessor 3 of the editing device 2 reproduces one of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 displayed on the display unit 55 along with a display frame (for example, one frame). The peak level of the voice is acquired, and the process proceeds to the next step SP52.

  In step SP52, the microprocessor 3 of the editing device 2 determines whether or not the peak level of the reproduced sound is larger than a preset threshold value Thd. If a negative result is obtained, the process returns to step SP51. The process proceeds to step SP53.

In step SP53, the microprocessor 3 of the editing apparatus 2 sets the enlargement ratio according to the peak level of the video display units W57 _{1 to} W57 _n that have been determined that the peak level of the reproduced audio is larger than the preset threshold value Thd. Thereafter, the process proceeds to step SP54, and the enlargement ratio of the video display units W57 _{1 to} W57 _n is controlled based on the calculation result.

Thus, the video display portions W57 _{1 to} W57 _{n of} the spiral video display portion group 58 displayed on the display portion 55 of the editing screen 50 at this time are normal sizes (sizes when the peak level is equal to or less than the threshold Thd). In comparison with the above, the image is enlarged and displayed in accordance with the enlargement ratio calculated in step SP53.

Next, the microprocessor 3 of the editing apparatus 2 returns to step SP51, and thereafter executes the processing in step SP51 to step SP54 in a similar manner while sequentially switching the target video display units W57 _{1 to} W57 _n. Has been made.

In this way, the microprocessor 3 of the editing device 2 displays the video when the audio level of the reproduced audio accompanying the reproduced video displayed on the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 is high. The parts W57 _{1 to} W57 _n can be displayed in a size corresponding to the sound level.

(7-3) Operation and Effect in Second Embodiment In the configuration described above, the editing device 2 of the editing system 1 displays on the video display units W57 _{1 to} W57 _n in the spiral video display unit group 58, respectively. When the audio level is high according to the audio level of the reproduced audio accompanying the reproduced video being displayed, the display size of the corresponding video display units W57 _{1 to} W57 _n is enlarged greatly and displayed accordingly.

And according to this image display method, the time is displayed to each image display unit _W57 1 _{~W57 n} based on the size of each image display unit _W57 1 _{~W57 n} of spiral image display area group 58 The audio level of the reproduced audio accompanying the reproduced video can also be visually recognized according to the display size of each of the video display units W57 _{1 to} W57 _n. It is possible for the operator to easily execute an editing process that links the two.

According to the above configuration, when the audio level is high according to the audio level of the reproduced audio associated with the reproduced video displayed on each of the video display units W57 _{1 to} W57 _n in the spiral video display unit group 58, respectively. In response to this, the corresponding video display units W57 _{1 to} W57 _n are also displayed in a large size, whereby the video display units W57 _{1 to} W57 _n are displayed in a linked manner as a moving image in a three-dimensional spiral display form. And the display effect in which the playback video and playback audio of each of the video display units W57 _{1 to} W57 _n cooperate with each other can make the operator's image search and editing operations easier to execute. As compared with the editing system 1 according to the first embodiment, an editing system 90 that can further facilitate editing work can be realized.

(8) Other Embodiments In the first and second embodiments described above, the case where the present invention is applied to the editing apparatus 2 which is a nonlinear editing apparatus has been described. However, the present invention can be widely applied to various other forms of editing devices and various video display control devices other than editing devices.

In the above-described first and second embodiments, the storage device 22 is controlled each time a reproduced image is displayed on each of the image display units W57 _{1 to} W57 _n of the spiral image display unit group 58. Although the case where the video / audio signal of the corresponding clip is read has been described, the present invention is not limited to this. For example, a buffer memory is provided in the GPU 4 and each video display unit W57 _{1 of the} spiral video display unit group 58 is provided. in display each reproduced image to ～W57 _n reads only once video and audio signals of the corresponding clip from the storage device 22 to hold it in the buffer memory in the GPU 4, which is then held in this buffer memory image Based on the audio signal, the reproduced video may be displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58. In this case, the storage means for storing the video / audio signals of one or a plurality of video materials includes the storage device 22 and the buffer memory. In the first and second embodiments, the description has been given of storing the video / audio signal in the storage device 22, but the hard disk device 7 may store all or a part thereof.

Further, in the first and second embodiments described above, a case has been described in which a playback video composed of moving images is displayed on each of the video display units W57 _{1 to} W57 _n . Not limited to this, the playback speed v of the video display portions W57 _{1 to} W57 _n which are present on the back side of the screen and are not highly visible to the operator are slowed down, the resolution is reduced, or a still image is displayed. May be displayed, or sometimes nothing may be displayed. As a result, the editing apparatus 2 can reduce the processing load of the microprocessor 3 as long as it does not forcefully display a high-resolution moving image if the image size is not recognizable by the operator.

Further, in the first and second embodiments described above, the case _where the time code notification frames TC1 to TC4 at predetermined intervals are displayed on the plurality of video display units W57 _{1 to} W57 _n has been described. The invention is not limited to this, and if the playback elapsed time can be recognized by the operator, the frame number is used as the speed display data indicating the speed of displaying the display video data in time series on the video display units W57 _{1 to} W57 _n . Various other notification frames such as a notification frame and a field number notification frame may be displayed.

Further, in the first and second embodiments described above, the spiral video display unit group 58 arranged in a spiral manner on the video display units W57 _{1 to} W57 _n is arranged on the display unit 55 of the editing screen 50. Although the case of displaying is described, the present invention is not limited to this, and as shown in FIG. 111, a plurality of video display units W57 _{1 to} W57 _n constituting the spiral video display unit group 58 are arranged in time series. The first spiral image display unit group 58A is displayed on the display unit 55A, the second spiral image display unit group 58B is displayed on the display unit 55B, and the third spiral image display unit group 58C is displayed on the display unit 55C. The fourth spiral image display unit group 58D may be displayed on the display unit 55D.

  In this display portion 55A, various operation button groups 55A1 such as play, stop, pause, fast forward, and fast reverse are provided at the bottom of the screen, and the playback timing display bar for the clip portion corresponding to the entire 1/4 hour is provided at the bottom. 55A2 is provided, and a slider 55A3 is provided on the timeline below the 55A2. The same operation button groups 55B1, 55C1 and 55D1, and a playback timing display bar are also provided for the display unit 55B, the display unit 55C and the display unit 55D. 55B2, 55C2 and 55D2, and sliders 55B3, 55C3 and 55D3 are provided.

  In particular, the playback timing display bars 55A2, 55B2, 55C2 and 55D2 are displayed in the display area 55E at the bottom of the screen in a state in which all of the clips are unified in a time series, for example, the playback timing display bar When 55D2 is selected, as shown in FIG. 112, the display unit 55D is enlarged and displayed on the display area 55E.

In the playback timing display bars 55A2, 55B2, 55C2, and 55D2, which side of the 1/4 hour corresponds to the portion of the playback video currently displayed on the plurality of video display units W57 _{1 to} W57 _n . together with expressed by Shiro抜area WE, is made a part corresponding to the image display unit section W57 _k, W57 _n being has been highlighted selection by an operator therein as shown by the predetermined color bars B1 and B2 of Yes. Thus, in the editing systems 1 and 90, the spiral playback video corresponding to the clip portion of the four time ranges at a time is provided to the operator via the four display units 55A, 55B, 55C, and 55D. It is possible to present it to the editor and promote the efficiency of editing work.

Furthermore, in the first and second embodiments described above, a plurality of video display units W57 _{1 to} W57 _n are expressed on the display unit 55 as a spiral video display unit group 58 arranged in a spiral three-dimensional manner. However, the present invention is not limited to this, and as shown in FIG. 113, when the plurality of video display units W57 _{1 to} W57 _n sequentially approach from the back side to the front side, The reproduced video of the video display unit W57 _n located at the position may be expressed on the display unit 55 so as to be finally dropped and disappeared.

At this time, the editing device 2 displays the time code notification frames TC3 and TC4 representing the time code at that time for each predetermined interval on the video display units W57 _{1 to} W57 _n , for example, so that the temporal flow of the clip May be presented intuitively to the operator. Also in this case, the editing device 2 arbitrarily sets the playback speed v and the playback time difference ΔT of the playback video displayed on the plurality of video display units W57 _{1 to} W57 _n , so that the operator can easily edit the spiral video display unit group. 58 can be expressed on the display unit 55.

Further, in the first and second embodiments described above, a plurality of video display units W57 _{1 to} W57 _n are expressed on the display unit 55 as a spiral video display unit group 58 arranged in a three-dimensional manner in a spiral manner. However, the present invention is not limited to this, and as shown in FIG. 114, between the left hand and the right hand between both hands, from the left hand to the right hand, the video display units W57 _{1 to} W57 _n The spiral video display unit group 58 expressed by sequentially skipping the playback video in time series may be displayed on the display unit 55.

In this case, when the microprocessor 3 of the editing device 2 recognizes that the palm of the left hand has been operated so as to be tilted toward the front side indicated by the arrow e as shown in FIG. 115, a plurality of video display units W57 _{1 to} W57 _n. 116, the display angle is changed so as to fall from the near side to the far side in accordance with the speed at which the playback video is flowed in time series, and the left palm is shown as an arrow f as shown in FIG. When it is recognized that the operation has been performed so as to be tilted to the side, the playback images for the plurality of video display units W57 _{1 to} W57 _n are displayed in association with each other so that the display angle is changed so as to be tilted from the back side to the near side. Can do.

Incidentally, the editing device 2 displays, for example, time code notification frames TC1 to TC4 at a given time interval indicating the speed of a playback video to be sent in time series to a plurality of video display unit video display units W57 _{1 to} W57 _n . Thus, the operator may intuitively recognize the temporal flow of the clip. Also in this case, the editing device 2 arbitrarily sets the playback speed v and the playback time difference ΔT of the playback video displayed on the video display units W57 _{1 to} W57 _n , so that the spiral video display group 58 that is easy for the operator to edit. Can be displayed on the display unit 55.

Furthermore, in the first and second embodiments described above, a plurality of video display units W57 _{1 to} W57 _n are expressed on the display unit 55 as a spiral video display unit group 58 arranged in a spiral three-dimensional manner. However, the present invention is not limited to this, and as shown in FIGS. 117 (A) and (B), the reproduced images of the plurality of video display units W57 _{1 to} W57 _n are displayed from the back side to the near side. In this case, the spiral image display unit group 58 may be displayed on the display unit 55 in a display form that slides down the slide sequentially from the middle.

In this case, the editing apparatus 2 sequentially stacks and displays the reproduced images of the video display units W57 _{1 to} W57 _n that have already been displayed, and also displays the reproduced images of the video display units W57 _{1 to} W57 _n that are scheduled to be displayed. Since a plurality of images are displayed in an overlaid state, it is possible to intuitively recognize which part of the entire clip is the reproduced video portion currently displayed.

  Incidentally, on the contrary, the editing device 2 may be expressed on the belt-like display unit 55 so that the reproduced images of the video display unit 57 stacked on the right side run up sequentially on the slide.

Further, in the first and second embodiments described above, of the reproduced images respectively displayed on the respective video display units W57 _{1 to} W57 _n in the spiral video display unit group 58 as shown in FIG. When the video display section W57 _{1 to} W57 _n displaying the desired scene is clicked (in this case, for example, by first clicking the video display section W57 _n ), the video display section W57 _n is edited in accordance with the editing operation. The case where the reproduction video corresponding to the position is colored and highlighted in a predetermined color has been described, but the present invention is not limited thereto, and the video display unit displays the display video data of the clip including the scene change. W57 ₁ ~W57 when displaying the _n, good be highlighted only an image display unit _W57 1 _{~W57 n} for displaying the display image data of the scene change .

Further, in the first and second embodiments described above, the largest video display unit W57 _n that most located on the front side of the spiral display area group 58, and now the video display to be gradually reduced as the distance by displaying section W57 ₁ ~W57 _n, has been dealt with the case where the increase of the degree of attention closer to the image display section W57 _n located on the front side of the spiral display area group 58, the present invention will now not limited to, lowering the extreme resolution as this addition distance from the image display section W57 _n located e.g. nearest side, each such narrow or blur the arrangement interval of the video display unit W57 ₁ ~W57 _n The reproduced video may be displayed on the video display units W57 _{1 to} W57 _n . By doing so, it is possible to further To improve enough attention approaches the image display section W57 _n located on the nearest side of the spiral display area group 58. Incidentally, as a method for improving the degree of attention closer to the image display section W57 _n located on the nearest side of the plurality of video display unit W57 ₁ ~W57 _n, it is possible to widely apply various other techniques.

Further, in the first and second embodiments described above, the rear video display unit is displayed through the portion where the front side and the rear side overlap among the plurality of video display units W57 _{1 to} W57 _n. However, the present invention is not limited to this, and when it is difficult to see through the display, the image display on the rear side is displayed instead of the image display on the rear side. Depending on the part, it may be displayed while being hidden.

Furthermore, in the first and second embodiments described above, the playback time difference ΔT between the playback videos displayed on the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 and the playback speed v of each playback video are set. The case where the playback time difference ΔT between all playback videos and the playback speed v of each playback video are made constant has been described. However, the present invention is not limited to this, and this is not limited to each playback video or each playback video. May be set independently.

In this case, the reproduced video displayed respectively to the playback speed v and the image display section _W57 1 _{~W57 n} of playback images displayed respectively on the image display section _W57 1 _{~W57 n} on the display screen of the display unit 55 The reproduction time difference ΔT may be controlled in accordance with the coordinate position (absolute coordinate position) of each of the video display units W57 _{1 to} W57 _n on the display screen.

Specifically, GPU 4 is, detects the coordinate position of the video display unit _W57 1 _{~W57 n} on the display screen, respectively, on the basis of the detection result is displayed on the image display section _W57 1 _{~W57 n} Play The storage device 22 is controlled so that the playback speed v of the video and the playback time difference ΔT of the playback video displayed on each of the video display units W57 _{1 to} W57 _n become the set values at that time.

For example as a change process of the reproduction time difference ΔT increases the higher the reproduction time difference ΔT go to the back side from the image display unit W57 _n located on the nearest side of the spiral display area group 58. As a visual effect, the operator appears to display the reproduced image at a reduced speed as the reproduced image flies spirally from the back side of the screen to the near side and approaches the near side.

Further, when the positions of the video display units W57 _{1 to} W57 _n and the spiral video display unit group 58 are changed (moved) by a predetermined operation input by the operator, the microprocessor 3 and the GPU 4 correspondingly change the positions on the display 40. while changing the display position of the image display section _W57 1 _{~W57 n} and a helical display area group 58 sequentially detects coordinate positions of the respective image display section _W57 1 _{~W57 n} and a helical display area group 58 after the change Thus, the playback speed v and playback time difference ΔT of the playback video displayed on each of the video display units W57 _{1 to} W57 _n may be controlled in accordance with the change in the coordinate position. Further, a playback state detection function for detecting a playback state may be installed as a partial function of the microprocessor 3 and the GPU 4. Note that the playback state may be a set of parameters such as playback speed and playback time (such as relative time in the video material).

Furthermore, in the first and second embodiments described above, the playback speed v and the playback time difference ΔT of the playback video displayed on each of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 are operated by the operator. Although the present invention is not limited to this, the present invention is not limited to this. For example, when video that has been compression-encoded at a variable bit rate is reproduced, compression encoding is performed at a high bit rate with a large amount of generated code. Since it is possible to infer that the recorded portion is a fast moving video portion, the playback speed v and / or the playback time difference ΔT are reduced so that the video can be confirmed slowly, while the generated bit amount is low. Since it can be estimated that the portion that has been compression-encoded with is a video portion with little motion, the playback speed v Alternatively, the reproduction time difference ΔT may be increased. By doing so, it becomes easy for the operator to visually confirm and edit the reproduced image of the spiral image display unit group 58. Note that the reproduction control and display control in this case may be configured such that a threshold value such as the amount of generated code is set in advance from a menu screen or the like, or automatically controlled in the editing system.

  Furthermore, in the first and second embodiments described above, the case where only the spiral video display unit group 58 is displayed in the display unit 55 of the editing screen 50 has been described, but the present invention is not limited to this. The time scale may be displayed in the display unit 55 along the spiral video display unit group 58. This makes it possible for the operator to easily check the time information of the edited video and audio.

Furthermore, in the above-described first and second embodiments, the case where the timeline unit 53 is supplementarily provided on the editing screen 50 (FIG. 2) has been described. However, the present invention is not limited to this, and the time The line unit 53 may have a function for displaying the time axis. In this case, it is time to determine which part of the entire corresponding clip the reproduced video displayed on the video display unit W57 _n positioned on the most front side as a reference of the spiral video display unit group 58 corresponds to. What is necessary is just to give the function displayed on the line part 53. FIG.

  Further, in the first and second embodiments described above, each time the operation described above with reference to FIGS. 19 to 24 is performed, the processing result of the processing according to the operation is edited in the same display form as the display form. The case where the images are sequentially displayed in the display section 55 of the screen 50 has been described. In this case, for example, the immediately preceding edited content is automatically backed up so that the state immediately before the operation can be restored after such operation. May be.

Furthermore, in the above-described first and second embodiments, the cursor is operated in a state where the “up” key 70U of the direction key 70 of the keyboard 39 is pressed or the right button of the mouse 38 is pressed. When the operation of moving the cursor 71 so as to draw a part of the reproduced video displayed on the spiral video display unit group 58 by 71 to the left along the video display unit group 58 is performed. The playback speed v of the playback video displayed on each of the video display units W57 _{1 to} W57 _n in the display unit group 58 is increased, and the playback video between the adjacent video display units W57 _{1 to} W57 _n is linked with this. However, the present invention is not limited to this. For example, only one of the playback speed v and the playback time difference ΔT may be increased. Thus, the editing apparatus 2 may be constructed.

Similarly, in the above-described first and second embodiments, the “down (↓)” key 70D of the direction key 70 of the keyboard 39 is pressed, or the right button of the mouse 38 is pressed. When the operation of moving the cursor 71 so as to pull a part of the reproduced video displayed on the spiral video display unit group 58 by the cursor 71 to the right along the spiral video display unit group 58 is performed, The playback speed v of the playback video displayed on each of the video display units W57 _{1 to} W57 _n in the spiral video display unit group 58 is reduced, and in conjunction with this, the video display units W57 _{1 to} W57 _n are adjacent to each other. However, the present invention is not limited to this. For example, only one of the playback speed v and the playback time difference ΔT is reduced. The editing device 2 may be constructed so that

Further, in the first and second embodiments described above, the playback speed v and playback time difference ΔT of the playback video displayed on the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 can be set freely. As described above, the appearance of the playback video displayed on the spiral video display unit group 58 can be changed. However, the present invention is not limited to this, and the playback speed v and playback time difference of the playback video are not limited thereto. In addition to ΔT, for example, it is possible to change the appearance of the reproduced video displayed on the spiral video display unit group 58 by setting a numerical value for the frame / field interval. Other parameters may be freely settable as parameters for changing.

Furthermore, in the first and second embodiments described above, as a result of the instantaneous playback or instantaneous return operation of the playback video described above with reference to FIGS. 15 (A), (B) or FIGS. 17 (A), (B), By moving and displaying the display positions of the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 together with the cursor 71 in the drawn direction along the spiral video display unit group 58, The case where the playback video displayed on each video display unit W57 _{1 to} W57 _n of the spiral video display unit group 58 is made to appear as if it was instantaneously sent to the future side or the past side has been described. invention is not limited thereto, the display position without changing each image display unit W57 ₁ ~W57 _n, momentarily only reproduction image displayed on the image display section W57 ₁ ~W57 _n when such operation is input Fast forward or So as to return, it may be show the playback video that has been displayed on the video display unit W57 ₁ ~W57 _n as if it were returned to instantly proceed to the future side or in the past side.

Furthermore, in the above-described second embodiment, each of the corresponding video display units is displayed when displaying the image of the frame only when the peak level of the audio level of the reproduced audio is larger than the preset threshold value Thd. Although the case where W57 _{1 to} W57 _n are displayed larger than the size that should be normally displayed has been described, the present invention is not limited to this, and the sound level of the corresponding reproduced sound can be set without setting the threshold Thd. Accordingly, the video display units W57 _{1 to} W57 _n may be displayed so as to be enlarged or reduced.

Further, in the second embodiment described above, when the peak level of the sound level of the reproduced sound is larger than the threshold value Thd, the corresponding video display units W57 _{1 to} W57 _n are displayed when the image of the frame is displayed. Although the case where the display is larger than the normal display size has been described, the present invention is not limited to this, and the average of the audio level is compared with the threshold Thd, and when the average is larger than the threshold Thd, the video display is performed. While the portions W57 _{1 to} W57 _n are enlarged and displayed, when the average is smaller than the threshold Thd, the video display portions W57 _{1 to} W57 _n may be reduced and displayed.

Furthermore, in the first and second embodiments described above, the video display units W57 _{1 to} W57 _n are displayed so as to flow from the back side to the front side of the screen during playback in the forward direction in FIG. As described above, the present invention is not limited to this, and the flow direction can be arbitrarily switched according to the setting of the operator, and depending on the clip, the flow from the front side of the screen to the back side is defined as the positive direction, and the front side of the screen is to flow to the side may be displayed each image display unit _W57 1 _{~W57 n.}

Further, in the first and second embodiments described above, the case where the respective video display units W57 _{1 to} W57 _n are displayed on the screen of the display unit 55 in a fixed state has been described. Not limited to this, when the playback speed v of the playback video of each of the video display units W57 _{1 to} W57 _n is extremely low and the playback time difference ΔT is 1 frame or less, the video display units W57 _{1 to} W57 _n are currently displaying on the video display units W57 _{1 to} W57 _n . by sequentially shifting the position of each image display unit W57 ₁ ~W57 _n as if each entire image display unit W57 ₁ ~W57 _n will move toward the reproducing direction without updating the image, at the time of slow play You may make it visually check the image display group of smoother movement.

For example, in the editing apparatus 2, as shown in FIGS. 118A and 118B, the video images W57 _{1 to} W57 _n are stopped without being updated and the video images W57 _{1 to} W57 _n are stopped without being updated. it is possible to represent low-speed movement, such as if they were shifted from the far side to the near side of the screen by simultaneously one frame by shifting to ₁ ～W57 _n positions sequentially adjacent on the screen itself.

  Furthermore, in the first and second embodiments described above, as shown in FIG. 24, the effect operation method for the edit point is the case where effect processing is performed simultaneously at the In point and the Out point. As described above, the present invention is not limited to this, and effects may be individually set for the in point and the out point. Specifically, in FIG. 24B, after the effect icon 13 is selected, the setting on the in-point side is dragged and dropped near the frame of the In point by dragging and dropping near the frame of the In point. Set the Out point side.

  In addition, the effect processing can be performed for several frames before and after the frame to which the effect effect is applied and for several seconds before and after the frame to which the effect effect is applied. The method is set in advance by the operator and adjusted after the effect is set. Try to select both methods. As a specific method of the latter, it can be made possible by clicking an effect frame at an edit point while pressing the shift key, and then extending the effect section while pressing the shift key or clicking a frame to be shortened.

  Further, in the first and second embodiments described above, the position of the character CA1 is changed as the confirmation position icon for the sub 3D space image IM1 in order to change the display form on the spiral video display unit 58. However, the present invention is not limited to this, and it is also possible to use a confirmation position icon in various other forms such as an icon representing a human eye or a simple arrow icon.

  Further, in the first and second embodiments described above, the case has been described in which the sub 3D space image IM1 used for changing the display form on the spiral image display unit 58 is formed in a substantially cubic shape. The present invention is not limited to this, and the sub 3D space image IM1 may be formed by various other shapes such as a sphere.

  Further, in the first and second embodiments described above, the case where the present invention is applied to the editing devices 2 and 90 configured as shown in FIG. 1 has been described. However, the present invention is not limited to this. Any image processing apparatus capable of providing a function as an editing apparatus can be widely applied to image processing apparatuses having various configurations. The image processing apparatus includes, for example, a computer, a video camera, a digital camera, a game machine, a portable information terminal (a portable computer, a mobile phone, a portable game machine), a storage device (for example, an optical disk device, a home server), and an invention. A processing board or a processing card equipped with a function related to the above is included. In any image processing apparatus, the casing, the signal processing unit, and the external interface are configured in common, and are configured by combining peripheral devices according to the product form. For example, in the case of a video camera or a digital camera, in addition to the above-described configuration, a camera unit and a writing circuit for storing captured video data in a storage medium are provided. For example, an electronic device having a communication function such as a cellular phone has a transmission / reception circuit and an antenna in addition to the above-described structure.

  Furthermore, in the first and second embodiments described above, the microprocessor 3 activates a control program as an image processing program stored in the hard disk device 7 and changes the display according to the viewpoint with respect to the spiral video display unit 58. The case where the processing is executed has been described, but the present invention is not limited to this, and the above-described display change processing is executed by installing the program storage medium storing the control program in the microprocessor 3. Anyway.

  As a program storage medium for installing the control program for executing the above-described display change processing and the like in the hard disk device 7 and making it executable, for example, a floppy (registered trademark) disk, a CD-ROM ( In addition to package media such as Compact Disc-Read Only Memory (DVD) and DVD (Digital Versatile Disc), the present invention may be realized by a semiconductor memory or a magnetic disk in which a control program is temporarily or permanently stored. In addition, as a means for storing the control program in these program storage media, a wired and wireless communication medium such as a local area network, the Internet, and digital satellite broadcasting may be used, and stored via various communication interfaces such as a router and a modem. You may do it.

Further, in the first and second embodiments described above, the playback image is instantaneously sent or returned to the video display units W57 _{1 to} W57 _n using the mouse 38 or the keyboard 39, and the spiral video display unit group 58 is displayed. Although the case where the shape change operation or the enlargement / reduction operation of the spiral image display unit group 58 is performed has been described, the present invention is not limited thereto, and these operations may be performed using the operation controller 37. .

  In addition, as shown in FIG. 119, this operation controller 37 is a crescent-shaped solid with a streamline shape, and is formed symmetrically. If one of the surfaces of the operation controller 37 having a crescent shape is used as the upper surface, the cross key CK and the “△”, “×”, “◯” and “ □ ”buttons B1 to B4 are formed approximately symmetrically.

  The cross key CK is adapted to accelerate or decelerate the reproduction speed by an up / down pressing operation and move the cursor 71 by a left / right pressing operation. The “△” button B1 displays the cursor menu, the “×” button B2 cancels the designation of the cursor 71 or deletes the cursor menu, the “◯” button B3 designates or determines the cursor 71, and “□” The button B4 can display various menus other than the cursor menu.

On the other hand, on the inner peripheral side of the upper surface of the operation controller 37, a right jog lever RJ and a left jog lever LJ are provided symmetrically. The right jog lever RJ and the left jog lever LJ move the video display units W57 _{1 to} W57 _n of the spiral video display unit group 58 selected by the cursor 71 to the spiral video display unit group 58 by pushing them down. It can be moved in the direction from the near side to the far side along the direction or from the far side to the near side.

  On the other hand, a right button RB and a left button LB are provided on the outer peripheral side surface of the operation controller 37 symmetrically. The right button RB or the left button LB can move the spiral image display unit group 58 to a position desired by the operator by a pressing operation.

  As an example of the operation in the operation controller 37, the cursor 71 is integrated with the cursor 71 in the same direction as the direction traced along the spiral image display unit group 58 (the direction from the back side to the near side) from the basic operation. The operation for moving (FIG. 15) will be described with reference to FIG.

  That is, the operator first selects the spiral image display unit group 58 by positioning the cursor on the spiral image display unit group 58 by the left and right pressing operation of the cross key CK (FIG. 120A).

  At this time, when the cursor 71 is positioned so as to be removed from the spiral image display unit group 58, the selection for the spiral image display unit group 58 is released. On the other hand, when the selection is determined, the spiral image display unit group 58 can be designated (grabbed) by the cursor 71 by pressing the “◯” button B3 (FIG. 120B).

In this state, when the operator pushes down the right jog lever RJ along the spiral image display unit group 58 in the direction corresponding to the front side from the back side (FIG. 120C), the operator displays the right jog lever RJ on the spiral image display unit group 58. The spiral video display unit group while the playback speed v of the playback video and the playback time difference ΔT of the playback video displayed on the video display units W57 _{1 to} W57 _n adjacent to each other are both fixed. The display positions of the 58 video display units W57 _{1 to} W57 _n move together with the cursor 71 in the same direction as the direction traced along the spiral video display unit group 58 (the direction from the back side toward the front side). Will be.

  As described above, the editing apparatus 2 can perform various extended operations other than the basic operation even when the operation controller 37 is used in addition to the mouse 38 or the keyboard 39, and makes the operator intuitively as if it were a game. Image search work and editing work can be executed.

  Further, in the above-described first and second embodiments, the editing apparatus as the image processing apparatus of the present invention is configured by the microprocessor 3 as the image generation means and the GPU 4 as the image processing means. As described above, the present invention is not limited to this, and the image processing apparatus may be configured by image generation means and image processing means having various other circuit configurations.

  Various modifications other than those described above are possible within the scope of the invention. Various modifications and applications created based on the description of the present specification are also conceivable.

  The present invention can be widely applied to non-linear editing devices, other editing devices, and various video display control devices that display videos other than editing devices.

1 is a schematic block diagram showing a configuration of an editing system according to the present invention. It is a basic diagram which shows the structure of the edit screen by this Embodiment. It is a basic diagram with which it uses for description of a spiral image display part group. It is a conceptual diagram with which it uses for description of the reproduction | regeneration order with respect to each video display part of a spiral video display part group. It is a timing chart with which it uses for description of the reproduction time difference of the reproduction | regeneration image | video displayed on each video display part, respectively. It is a basic diagram with which it uses for description of the video display which flows to a Forward direction. It is a basic diagram which shows a reproduction time difference / reproduction speed setting dialog. It is a basic diagram with which it uses for description of the video display which flows to a Reverse direction. It is a flowchart which shows a parallel decoding process procedure. It is a flowchart which shows the reproduction | regeneration processing procedure (display change processing procedure). It is a basic diagram with which it uses for description of basic operation (1) in a display part. It is a timing chart with which it uses for description of the flow of the colored image | video display part. It is a basic diagram with which it uses for description of basic operation (2) in a display part. It is a basic diagram with which it uses for description of extended operation (1) in a display part. It is a conceptual diagram with which it uses for description of the movement of a video display part by extended operation (1-1). It is a timing chart with which it uses for description of the movement (1-2) of the video display part by extended operation. It is a conceptual diagram with which it uses for description of the movement (2-1) of the video display part by extended operation. It is a timing chart with which it uses for description of the movement (2-2) of the video display part by extended operation. It is a basic diagram with which it uses for description at the time of performing cut edit (1) within one clip. It is a basic diagram with which it uses for description at the time of performing cut edit (2) within one clip. It is an approximate line figure used for explanation of separation operation of a clip. It is an approximate line figure used for explanation of cut edit using a plurality of clips. It is an approximate line figure used for explanation of insert edit using a plurality of clips. It is an approximate line figure used for explanation of a picture special effect setting. It is a flowchart which shows an edit operation response process sequence. It is a basic diagram with which it uses for description of the spiral structure of a spiral image display part group. It is a basic diagram with which it uses for description of the sub 3D space image which shows a viewpoint. It is a basic diagram with which it uses for description of a some sub spiral image display part group. It is a basic diagram with which it uses for description of the movement of a sub spiral image display part group. It is a basic diagram with which it uses for description of the viewpoint change which fixed the sub 3D space image. It is a basic diagram with which it uses for description of a rotation of a sub 3D space image and a character. It is a basic diagram with which it uses for description of the enlarged display of a sub 3D space image. It is a flowchart which shows the display processing procedure with respect to the spiral image display part group according to a viewpoint change. It is a basic diagram which shows the spiral image display part group (1) after a viewpoint change. It is a basic diagram which shows the spiral image display part group (2) after a viewpoint change. It is a basic diagram which shows the spiral image display part group (3) after a viewpoint change. It is a basic diagram which shows the spiral image display part group after the viewpoint change by viewpoint coordinate lock mode. It is a basic diagram which shows the spiral image display part group (4) after a viewpoint change. It is a basic diagram which shows the example of a two-dimensional image display during an in point search. It is an approximate line figure used for explanation of an example of cut edit processing using a plurality of clips. It is a block diagram which shows the function structure of the edit pre-processing by a microprocessor and GPU. It is a block diagram which shows the structural example of an AV file preparation part. It is a block diagram which shows the structural example of a Fy file creation part. It is a figure for demonstrating the process of a data reduction part. It is a flowchart explaining Fy file creation processing. It is a block diagram which shows the structural example of a variation calculation part. It is a figure explaining the motion information of the frame unit calculated | required in a motion information calculation part. It is a block diagram which shows the structural example of a motion information calculation part. It is a figure explaining the process of a motion information calculation part. It is a block diagram which shows the other structural example of a motion information calculation part. It is a figure explaining the process of a motion information calculation part. It is a block diagram which shows the structural example of a fineness information calculation part. It is a figure explaining the process of a fineness information calculation part. It is a block diagram which shows the other structural example of a fineness information calculation part. It is a figure explaining the process of an average value calculation part. It is a figure which shows the variation | change_quantity for every flame | frame calculated | required in the variation | change_quantity calculation part. It is a figure explaining the example of a display type. It is a figure explaining the display type which a display type determination part determines per frame. It is a block diagram which shows the structural example of a display type determination part. It is a flowchart explaining a display type determination process (1). It is a flowchart explaining a display type determination process (2). It is a figure which shows the variation | change_quantity per frame. It is a figure which shows the example (1) of a Fy file. It is a figure explaining the example (1) of a display type. It is a figure which shows the motion information and display type of a frame unit. It is a flowchart explaining a display type determination process (3). It is a flowchart explaining a display type determination process (4). It is a figure explaining the example (2) of a display type. It is a figure which shows the fineness information and display type of a frame unit. It is a flowchart explaining a display type determination process (5). It is a flowchart explaining a display type determination process (6). It is a figure which shows the example (2) of a Fy file. It is a figure which shows the example (3) of a Fy file. It is a figure which shows the example (4) of a Fy file. It is a figure explaining the example (3) of a display type. It is a figure which shows movement information and fineness information, and a display type. It is a flowchart explaining a display type determination process (7). It is a figure which shows the example (5) of a Fy file. It is a block diagram which shows the function structure of the scrub process by a microprocessor and GPU. It is a block diagram which shows the structural example of a Fy file management part and a display type acquisition part. It is a figure explaining the relationship between the position of a scrub knob, and the flame | frame designated by the position. It is a figure which shows the display method in each scrub of the frame of a normal type and a high display rate low resolution type. It is a figure explaining the display of the frame whose display type is a still picture type. It is a flowchart explaining the process of a scrub. It is a figure which shows the example (1) of the weight which weights the area length of a designated area. It is a figure which shows the example (2) of the weight which weights the area length of a designated area. It is a figure which shows each designation | designated area when not weighting to the section length, and when weighting. It is a flowchart explaining the process regarding a section length. It is a figure explaining the setting of the allocation range. It is a flowchart explaining an allocation range setting process. It is a flowchart explaining a display process. It is a figure which shows the example of a display of transition (1) of an enlarged image display part. It is a figure which shows the example of a display of transition (2) of an enlarged image display part. It is a figure which shows the example of a display of transition (3) of an expansion image display part. It is a figure which shows the example of a display of transition (4) of an expansion image display part. It is a figure which shows the example of a display of transition (5) of an enlarged image display part. It is a figure which shows the example of a display of transition (6) of an expansion image display part. It is a figure which shows the example of a display of transition (7) of an enlarged image display part. It is a figure which shows the example of a display of transition (8) of an enlarged image display part. It is a figure which shows the example of a display of transition (9) of an expansion image display part. It is a figure which shows the example of a display of transition (10) of an enlarged image display part. It is a figure which shows the example of a display of transition (11) of an enlarged image display part. It is a figure which shows the example of a display of transition (12) of an enlarged image display part. It is a figure which shows the variation | change_quantity for every flame | frame which comprises 1 GOP. It is a figure explaining the method of determining the display type for every flame | frame which comprises 1 GOP. It is a figure explaining densification of a drawing interval. It is a figure which shows densification (1) of the description between the images corresponding to a reproduction time difference. It is a figure which shows densification (2) of the description between the images corresponding to a reproduction time difference. It is a basic diagram with which it uses for description of the display method of the reproduction | regeneration image | video in 2nd Embodiment. It is a flowchart which shows a video display part enlarged display processing procedure. It is a basic diagram which shows the structure (1-1) of the video display part in other embodiment. It is a basic diagram which shows the structure (1-2) of the video display part in other embodiment. It is a basic diagram which shows the reproduction | regeneration image | video (1) of the display part in other embodiment. It is a basic diagram which shows the reproduction | regeneration image | video (2) of the display part in other embodiment. It is an approximate line figure showing reproduction picture (2-1) of a display part in other embodiments. It is an approximate line figure showing reproduction picture (2-2) of a display part in other embodiments. It is a basic diagram with which it uses for description of the reproduction | regeneration image | video (3) of the display part in other embodiment. It is a basic diagram with which it uses for description of the movement of the video display part at the time of low speed reproduction in other embodiment. It is a basic diagram which shows the structure of the operation controller in other embodiment. It is a basic diagram which shows the example of operation order using the operation controller in other embodiment. It is a basic diagram which shows the structural example of the conventional edit screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,90 ... Editing system, 2 ... Editing apparatus, 22 ... Memory | storage device, 3 ... Microprocessor, 4 ... GPU, 5 ... XDR-RAM, 37 ... Operation controller, 38 ... Mouse, 39 …… Keyboard, 40 …… Display, 41 …… Speaker, 50 …… Edit screen, 55 …… Display unit, W57 _{1 to} W57 _n …… Video display unit, 58, 77 …… Helix image display unit group, IM1… ... sub 3D space image, CA1 ... character.

Claims (30)

Image generating means for generating a plurality of display video data for display on a plurality of video display units on the display screen from the video data;
Image processing means for displaying the plurality of display video data to be displayed on the plurality of video display units in a three-dimensional spiral form on the display screen in a moving image state while providing a time difference in display order; An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the image generation unit generates the plurality of display video data by decoding the encoded stream. Storage means for storing the encoded stream;
Reading means for reading out the encoded stream from the storage means;
The image processing apparatus according to claim 2, wherein the image generation unit generates the plurality of display video data by decoding the encoded stream read by the reading unit. The image processing means displays the plurality of display video data on the display screen as a spiral video display unit group in which the plurality of video display units arranged in a three-dimensional spiral on the display screen cooperate in the display order. The image processing apparatus according to claim 1, wherein: The image processing means acquires display information including display sizes and display positions in the plurality of video display units, and causes the display video data to be displayed on the display screen using the acquired display information. The image processing apparatus according to claim 1. The image processing apparatus according to claim 1, wherein the image processing unit displays the plurality of display video data on the display screen in a state where the plurality of display video data is displayed at a variable speed. 7. The image processing according to claim 6, wherein the image processing means displays the plurality of display video data on the display screen in a state where display speeds of the plurality of display video data are equal to each other. apparatus. 2. The image according to claim 1, wherein the image processing unit displays the plurality of display video data on the display screen while providing the time difference of equal intervals between the plurality of video display units. Processing equipment. The image processing means includes, among the plurality of display video data, reference display video data displayed on a reference video display unit serving as a display reference among the plurality of video display units included in the spiral video display unit group. The image processing apparatus according to claim 4, wherein the image processing apparatus is displayed so as to be maximized. The image processing means is configured to gradually reduce the display size of the plurality of display video data excluding the reference display video data as the distance on the three-dimensional spiral increases from the reference video display unit. The image processing apparatus according to claim 9, wherein the image processing apparatus is displayed. The image processing apparatus according to claim 9, wherein the image processing unit displays the reference display video data so as to be positioned closest to the spiral video display unit group. The image processing means is configured to display the plurality of display video data excluding the reference display video data so that the resolution decreases as the distance on the three-dimensional spiral increases from the reference video display unit. The image processing apparatus according to claim 9, wherein the apparatus is an image processing apparatus. The image processing means displays the plurality of display video data in an overlapping state in the video display units adjacent to each other, and displays the overlapping portions in the video display unit in a transparent state. The image processing apparatus according to claim 1. The image processing means may arrange the plurality of display video data excluding the reference display video data with an arrangement interval in the plurality of video display units as the distance on the three-dimensional spiral increases from the reference video display unit. The image processing apparatus according to claim 9, wherein the display is narrowed. The image processing means is characterized in that the display angle of the plurality of video display units is changed and displayed in accordance with a speed at which the plurality of display video data in the spiral video display unit group is flowed in time series. Item 8. The image processing apparatus according to Item 1. The image generation means further generates speed display data indicating a speed at which the plurality of display video data in the spiral video display unit group is flowed in time series,
The image processing apparatus according to claim 1, wherein the image processing unit displays the speed display data as an icon on the display screen. The image processing means highlights the video display unit corresponding to the editing position or the display video data corresponding to the editing position when displaying the display video data corresponding to the editing position on the video display unit. The image processing apparatus according to claim 1, wherein: The image processing means highlights the display video data corresponding to the scene change or the display video data corresponding to the scene change when displaying the display video data corresponding to the scene change on the video display unit. The image processing apparatus according to claim 1. The image generation unit generates the divided encoded stream by dividing the encoded stream into a plurality of parts on the time axis according to the number of display areas configured by the plurality of video display units, and generates the divided encoded stream. The image processing apparatus according to claim 2, wherein the stream is decoded to generate the plurality of display video data for each display area. The image generation means includes
A plurality of decoding units for decoding the encoded stream;
The image processing apparatus according to claim 2, further comprising: a control unit that assigns a decoding process of the encoded stream to the plurality of decoding units according to the number of the plurality of video display units. Confirmation position information generating means for generating confirmation position information indicating a confirmation position when the operator visually confirms the plurality of display video data on the display screen;
The image processing means displays the plurality of display video data on the display screen by arranging the display video data according to the confirmation position information generated by the confirmation position information generation means. An image processing apparatus according to 1. The confirmation position information generating means performs a confirmation position movement operation for moving the confirmation position with respect to a reference position in the plurality of video display units arranged in a three-dimensional spiral on the display screen. Change the confirmation position information according to the confirmation position operation input,
The image processing means displays the plurality of display video data on the display screen by rearranging the display video data according to the confirmation position information changed by the confirmation position information generation means. The image processing apparatus according to 21. The confirmation position moving operation is an operation of enlarging or reducing the plurality of video display units arranged in a three-dimensional spiral from the reference position in a three-dimensional virtual space provided on the display screen. The image processing apparatus according to claim 22, characterized in that: The check position moving operation is an operation of rotating the plurality of video display units arranged in a three-dimensional spiral in a three-dimensional virtual space provided on the display screen. An image processing apparatus according to 1. The confirmation position moving operation is an operation of moving in a future direction or a past direction with respect to a reference axis in the plurality of video display units arranged in a three-dimensional spiral in a three-dimensional space provided on the display screen. The image processing apparatus according to claim 22, wherein: Confirmation position information generating means for generating confirmation position information indicating confirmation positions on the display screen when the operator visually confirms the plurality of display image data corresponding to the plurality of image display sections on the display screen. Have
The image processing apparatus according to claim 1, wherein the image processing unit displays the confirmation position information generated by the confirmation position information generation unit on the display screen as a predetermined confirmation position icon. The confirmation position information generation means, when a confirmation position movement operation for moving the confirmation position is performed, with respect to reference positions in the plurality of video display units arranged in a three-dimensional spiral on the display screen, Change the confirmation position information according to the confirmation position operation input,
27. The image processing means displays the position corresponding to the confirmation position information changed by the confirmation position information generation means on the display screen as a predetermined confirmation position icon. The image processing apparatus described. An image generating step for generating a plurality of display video data for display on a plurality of video display units on the display screen from the video data;
An image processing step for displaying the plurality of display video data to be displayed on the plurality of video display units in a moving image state in a moving image state while providing a time difference in the display order, and displaying them in a three-dimensional spiral form on the display screen; An image processing method comprising the steps of: For information processing equipment
An image generating step for generating a plurality of display video data for display on a plurality of video display units on the display screen from the video data;
An image processing step for displaying the plurality of display video data to be displayed on the plurality of video display units in a moving image state in a moving image state while providing a time difference in the display order, and displaying them in a three-dimensional spiral form on the display screen; An image processing program for executing An image generating step for generating a plurality of display video data for display on a plurality of video display units on the display screen from the video data;
An image processing step for displaying the plurality of display video data to be displayed on the plurality of video display units in a moving image state in a moving image state while providing a time difference in the display order, and displaying them in a three-dimensional spiral form on the display screen; A program storage medium storing an image processing program, characterized by causing an information processing apparatus to execute a process including: