JPH0581406A - Animation display processing device - Google Patents

Abstract

(57) [Summary] [Object] The present invention relates to a moving picture display processing apparatus for displaying moving pictures in a plurality of windows on a display, and an object thereof is to make it possible to display moving pictures having a related meaning in each window. .. [Structure] The moving picture display control means 6 reads the moving picture display unit from the moving picture display unit management means 4 by interpreting the scenario description in the scenario description management means 5, and relates to a plurality of windows opened on the display. Display time-series data in video display units that have different meanings. Further, the moving image display control means 6 performs a backward display in moving image display units based on the management data of the moving image history management means 7 in accordance with the user's request. On the other hand, the user can edit the moving image display unit in the moving image display unit managing means 4 and the scenario description in the scenario description managing means 5 by using the data input device 2. In particular, the scenario description editing means 13 can edit the scenario description graphically.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture display processing apparatus for displaying moving pictures in a plurality of windows on a display.

[0002]

2. Description of the Related Art Techniques for displaying moving images on a display such as a workstation are being developed.

As a conventional technique for such moving image display processing, a method is constructed in which images are constructed as a bit map for screen display at each time according to a certain time unit, and those images are displayed in time order on a display. Is most commonly used. In this case, the moving image is independently displayed for each window on the display.

Another conventional technique for moving image display processing is as follows. First, the state of the screen at a certain time is converted into data by some method. next,
The state of the screen at any time thereafter is similarly converted into data. Then, the state of the screen at each time during that time is obtained by interpolating from the above two sets of screen data. A moving image is displayed by sequentially displaying the state of the screen at each time thus obtained on the display. Also in this case, the moving image is independently displayed for each window on the display.

[0005]

However, in the above-described conventional technology (Note: plural form), since moving pictures having a related meaning cannot be displayed on the display at the same time, it is not possible to use moving pictures in a workstation or the like. There was a problem that it was limited. For example, when displaying an operation description of software in a computer system on a display, if a structure is adopted in which the text of the description content and a moving image illustrating the operation according to the description are simultaneously displayed in synchronization with each other, It is expected that the software operation can be easily explained, but in the method of displaying moving images independently for each window as in the above-mentioned conventional technique,
We were not able to meet such demands at all.

Further, in the conventional moving image display processing, a series of moving images are displayed on the display along a temporal flow as if the film were rotated. Therefore, there is a problem in that the display contents of the moving image are fixed, and the use of the moving image in the workstation or the like is limited. For example, even if a moving image can be displayed when displaying the operation description of the software in the computer system on the display as described above, if the user wants more detailed explanation about the desired explanation part, the dynamic image Since the displayed contents cannot be changed, such a request could not be met at all.

Moreover, in the conventional moving image display processing, since the displayed moving images are collectively managed as if they were films, for example, in order to change a part of the contents of the moving image, the long moving image data is used. There was also a problem that it took a lot of time and effort to find and edit the corresponding part from among them.

An object of the present invention is to make it possible to display a moving image having a related meaning in a plurality of windows on a display.

[0009]

FIG. 1 illustrates the principle configuration of the present invention. The data processing device 1 executes a moving image display process. The data input device 2 allows the user to input a scenario description for displaying a moving image. The man-machine interface (MMI) 3 includes a data processing device 1 and a data input device 2.
Connected to.

The data processing device 1 is, for example, a moving picture display unit managing means 4 and a scenario description managing means 5, a moving picture display controlling means 6, a moving picture history managing means 7, and an operation mode controlling means which are realized by an auxiliary storage device or the like. 8 and interface means 9.

The moving image display unit managing means 4 manages a plurality of moving image display units each including one or a plurality of frames for describing the screen display definition. Here, the frame that constitutes each moving image display unit is a data set that describes each screen display definition for each prescribed display unit time, for example, the type of figure displayed on the display and the characteristic information of the figure. Is described with a figure ID.

The scenario description managing means 5 displays the opening and closing timings and positions of each window on the display, the temporal order relationship of the moving image display units displayed in each window, or the display in each window. It manages the scenario description that describes the synchronization relationship between each video display unit. Here, the above-described temporal order relationship is configured to include an arbitrary temporal transition relationship from one moving image display unit to another moving image display unit displayed in the same window as the window in which it is displayed or in a different window. can do. Further, the above temporal transition relationship can be configured to include identification information that becomes effective when a predetermined screen portion in the currently displayed moving image display unit is designated by the user. Furthermore, the above scenario description can be configured to include group information such that a group of a plurality of moving image display units is treated as one moving image display unit.

The moving picture display control means 6 interprets the scenario description managed by the scenario description management means 5 and sequentially displays the time series data in moving picture display units in each window according to this interpretation.

The moving picture history management means 7 manages the history of moving pictures displayed on the basis of moving picture display units that make arbitrary temporal transitions. The operation mode control means 8 responds to the instruction from the man-machine interface 3 in response to the moving picture history management means 7.
The management function of is controlled not to operate.

The interface means 9 controls the execution state of the moving picture display control means 6 in response to an instruction from the man-machine interface 3. More specifically, the interface unit 9 has a function of stopping the progress of the moving image display processing of the moving image display control unit 6, and moves the stopped moving image display unit one by one by controlling the moving image display control unit 6. And a function of moving the frames of the stopped moving image display unit one by one by controlling the moving image display control means 6.

Next, the data input device has a moving image input control means 10, a graphic editing means 11, a moving image display unit editing means 12, and a scenario description editing means 13. The moving image input control means 10 includes a man-machine interface 3 and
It controls the input and output of data to and from the figure editing means 11, the moving image display unit editing means 12 or the scenario description editing means 13.

The graphic editing means 11 is a general-purpose or special-purpose graphic editor, and allows the user to edit the graphic definition of each frame of each moving image display unit in the graphic format by the man-machine interface 3.

The moving picture display unit editing means 12 collects the graphic definitions of each frame of each moving picture display unit edited by the figure editing means 11 as one moving picture display unit, and the moving picture display unit managing means. Store in 4.

The scenario description editing means 13 provides the user with
The man-machine interface 3 edits the scenario description managed by the scenario description managing means 5 by an editing process including a graphic symbol editing process.

Specifically, the scenario description editing means 13
The user edits the position of each window at the time of displaying a moving image in text format on the editing window opened on the display at the time of editing, and the temporal order relation of moving image display units displayed in each window or Edit the synchronization relationship between each video display unit displayed in each window in a graphic format using graphic symbols. Further, the scenario description editing means 13 may allow the user to further edit the display timing of the moving image display unit displayed in each window when displaying the moving image in the text format on the editing window.

In addition to the above-described editing of moving image display units, the scenario description editing means 13 allows the user to display a moving image on a rectangular parallelepiped whose display screen displayed on the editing window is three-dimensionally displayed in the time axis direction. Edit the open / close timing and its position on the display of each window at the time of display. Alternatively, the scenario description editing means 13 allows the user to edit the opening or closing timing of each window on the display when displaying a moving image in a text format on the editing window, and edits each window when displaying the moving image. The position on the display of each window for displaying a moving image is edited by arranging the window in a predetermined area on the display window.

[0022]

The moving picture display unit managing means 4 manages a plurality of moving picture display units each including one or a plurality of frames which describe the screen display definition. Further, the scenario description management means 5 follows the scenario description, the state of each window on the display, the temporal order relation of each moving image display unit to be displayed in each window, or between each moving image display unit of each window. Manage synchronization relationships. Then, in response to this management processing, the moving picture display control means 6 interprets the scenario description managed by the scenario description management means 5, and according to this interpretation, a time series of moving picture display units is displayed in a plurality of windows opened on the display. By displaying the data, you can display moving images with relevant meanings on the display.

As described above, according to the present invention, since a moving image having a related meaning can be displayed on the display according to the multi-window, for example, the text of the explanation content and the operation according to the explanation are illustrated. It becomes possible to build an animation manual that displays the video and the video while synchronizing them at the same time. Moreover, since the moving images to be displayed are managed in units of moving image display units, there is an advantage that the editing efficiency of the moving images can be improved.

Here, the temporal order relation of the moving image display units described by the scenario description is arbitrary from one moving image display unit to another moving image display unit displayed in the same window as the window in which it is displayed or a different window. In this configuration, the arbitrary temporal transition relationship includes identification information that becomes effective when a specific screen portion in the displayed video display unit is designated by the user. In this case, the moving image display control means 6 can dynamically change the display content of the moving image according to this arbitrary temporal transition relationship.

Further, when the scenario description has a configuration including group information such that a group of a plurality of moving image display units is treated as one moving image display unit, the moving image display control means 6 causes the moving image display control means 6 to group the moving images. In display units,
It is possible to move forward and backward on the display.

The moving picture history management means 7 manages the history of moving pictures displayed on the basis of moving picture display units that make arbitrary temporal transitions, and the moving picture display control means 6 responds to a request from the user to display moving pictures. In accordance with the management data of the history management means 7, a backward moving display process is performed for each moving image display unit that makes an arbitrary time transition. Thereby, even when the scenario description includes an arbitrary temporal transition relation to another moving image display unit, it is possible to return the temporal order.

When the moving picture history management means 7 enters the operating state,
A large amount of memory is consumed to manage the movie history. Therefore, in the case where the scenario description includes an arbitrary temporal transition relation to another moving image display unit, if the user does not request to return to the temporal order, the operation mode control means 8 may, for example, when the device is operating, The management function of the moving picture history management means 7 is controlled so as not to operate. As a result, it is possible to eliminate waste of memory capacity.

Then, the moving picture display control means 6 stops the progress of the moving picture display processing according to the control processing of the interface means 9, or moves the stopped moving picture display unit back and forth one by one, or stops it. Moves the frames of the video display unit back and forth one by one. This allows
The user can accurately understand the moving image displayed on the display.

On the other hand, the user can edit the moving image display unit managed by the moving image display unit managing means 4 and the scenario description managed by the scenario description managing means 5 by using the data input device 2.

In particular, the scenario description editing means 13 can graphically edit the scenario description. Specifically, the user edits the position of each window at the time of displaying the moving image in a text format on the editing window, and displays the temporal order relationship of the moving image display units displayed in each window or each window. It is possible to edit the synchronization relationship between the moving image display units in a graphic format using graphic symbols.

Further, if a configuration is adopted in which the display timing of the moving image display unit displayed in each window when displaying the moving image on the editing window can be edited in the text format, the user can synchronize the moving image display units with each other. It is possible to generate a scenario description in which the display timing of each moving image display unit displayed on each window in the state of being saved is explicitly specified.

In addition to the above-described editing of moving image display units, the user can display each window when displaying a moving image on a rectangular parallelepiped whose display screen displayed on the editing window is three-dimensionally graphic in the time axis direction. It is possible to edit the timing of opening or closing and the position thereof. Alternatively, the user can edit the open / close timing on the display of each movie display window when displaying the movie in a text format on the edit window, and at the same time open each movie display window on the edit window by a predetermined value. By arranging it in the area, the position of each moving image display window on the display can be edited.

[0033]

EXAMPLES Next, examples of the present invention (note: plural) will be described. FIG. 2 illustrates an embodiment of the device configuration of a moving image display device that realizes the present invention. In FIG. 2, 10 is a moving image display device that realizes the present invention, and 20 is a moving image display device 1.
0 is an input / output device, 30 is a moving image display unit input device for inputting the moving image display unit described in FIG. 1, 31 is a scenario description input device for inputting the scenario description described in FIG.
Reference numeral 40 is a moving image display unit storage file that manages the moving image display unit input by the moving image input device 30, and 41 is a scenario description storage file that manages the scenario description input by the moving image input device 30.

First, the moving image display device 10 has the following configuration. The moving image display unit interpretation device 11 interprets the moving image display unit stored in the moving image display unit storage file 40. The scenario description interpretation device 12 interprets the scenario description stored in the scenario description storage file 41 while using the interpretation result of the moving image display unit interpretation device 11. The window system 13 is, for example, an X window system, and opens a window on the display of the input / output device 20. In the moving picture internal data storage file 14, the moving picture internal data which is the interpretation result of the scenario description interpreting apparatus 12 is expanded. The moving image operating device 15 executes interface processing with the input / output device 20. The screen state creating device 16 creates screen data to be displayed on the display from the moving image internal data expanded in the moving image internal data storage file 14. The screen display device 17 activates the window system 13 to display the screen data created by the screen state creation device 16 in a window on the display.

As will be described in detail later, the moving image input device 30 has a built-in dedicated editor or general-purpose editor, and a moving image display unit composed of one or a plurality of frames for describing the screen display definition is input / output device to the user. Then, the moving image display unit is stored in the moving image display unit storage file 40.

FIG. 3 shows an example of moving image display units stored in the moving image display unit storage file 40. This embodiment describes the screen display state shown in FIG. Frames that make up the video display unit
Is a data set that describes each screen display definition for each prescribed display unit time, and the type of figure displayed on the display and the characteristic information of that figure are figure ID (identifie
r) is added and described.

First, the first frame describing the screen display state at time a describes seven figures. That is, the symbol "arc" to which the graphic IDs "arc0" and "arc1" are attached respectively describes arc-shaped graphics. Symbol "circle" with figure ID "cir1"
Describes the shape of a circle. Graphic IDs "box1" and "b
ox1 Each symbol "bo" with "orig"
Each x "describes a rectangular graphic, but both are overlapping. The symbol" s "with the graphic ID" s3 "is added.
"string" describes a graphic of a character string. Graphic ID
The symbol "arrow" with "al" describes an arrow figure. Further, following the symbol indicating each type of figure, characteristic information indicating the position and size of each figure is described. For example, the numerical sequence “40” described after the symbol “cir1”
0 300 20 ”means that the intermediate coordinates of the circle are“ 400, 30
It is described that the radius of the circle is "20" with "0".
Further, the next frame describes the screen display state at the time (a + 1) following the time a according to the same description as above. Here, in order to reduce the load of the input processing of the moving image display unit by the user, the screen display definition for the once defined graphic is configured to be reused by referring to the graphic ID.

Next, the moving image input apparatus 30, as described later in detail, describes a scenario description describing the temporal order relation of moving image display units displayed in a plurality of windows opened on the display, and these windows. The user is prompted to input a scenario description that describes the synchronization relationship of the moving image display units displayed on the screen from the input / output device 20, and the scenario description is stored in the scenario description storage file 41.

There are two types of models of the scenario description input by the moving image input device 30, that is, the film type and the network type. In the embodiment shown in FIG. 2, the device configuration for handling the film type scenario description is disclosed. ing.
The film type scenario description is as shown in Fig. 5 (a).
This is a model in which moving image display units displayed in the same window are arranged in time order, and as shown by the broken line in FIG. 5 (a), the synchronization relationship between windows is defined without breaking this time order. .. Here, the rectangles in the figure represent moving image display units, and “→” represents the time order relationship.

On the other hand, the network type scenario description is a model in which the degree of freedom of the relationship between the moving image display units is expanded, and the time sequence shown by the broken line in FIG. Interaction with
This is a model that allows moving to another moving image display unit.

Next, the basic format of the scenario description stored in the scenario description storage file 41 will be described. (1) glueOpen <window name><width><height><x><y> is described as a scenario description for instructing the creation of a window. This description opens a window with the size of <width> and <height> at the coordinate position of <x> and <y>, and then the window has an ID of <window name>.
Instruct to give. This <window name> is used when the window containing it is referenced in other scenario descriptions.

(2) Glue <glue name><windowname><file name in moving image display unit> is described as a scenario description for instructing display in moving image display unit. In this description, the moving image display unit stored in <file name of moving image display unit> is defined as <window name> by I
Display it in a window called D and add <
It is instructed to give an ID of "glue name>". In this description format, when descriptions including the same <window name> are described consecutively, the moving image display unit specified for each window with the <window name> as the ID is displayed in the order of the description. Means to instruct to go. Here, <glue name> must be unique.

(3) glueClose <window name> is described as a scenario description for instructing deletion of a window. This description instructs the deletion of the window whose ID is the opened <window name>.

(4) glueSync <glue name 1><glue name 2> is described as a scenario description for instructing simultaneous display of moving image display units between different windows. This description indicates that a moving image display unit having <Glue Name 1> as an ID and a moving image display unit having <Glue Name 2> as an ID are simultaneously displayed in the respective windows. Here, <glue name 1> and <glue name 2>
Must be defined.

(5) group <group name> with {glue name | group name} is described as a scenario description for instructing to group a plurality of moving image display units as illustrated in FIG. This description is {glue name | group name}
It is instructed that the glue or the group having the ID (either the glue name or the group name) as one group is given and the ID of the <group name> is given. This defined group name can be used in the above-mentioned glueSync, in which case the moving image display unit having the first glue name of the group is used.

(6) In the network type scenario description,
As described above, arbitrary time transition to another moving image display unit is allowed, and as a scenario description corresponding to this, gluebranch from <glue name 1> to <glue name 2> or gluebranch from <figure name> in <glue 1> to <glue name 2> is described. The former description indicates the transition from the moving image display unit having <Glue Name 1> as an ID to the moving image display unit having <Glue Name 2> as an ID, and the latter description specifies <Glue Name 1> as an ID. <Figure name> in the video display unit
When the figure D is designated by the user (by a mouse click operation), it is instructed to make a transition to a moving image display unit having <glue name 2> as an ID.

(7) When a comment is described in the scenario description, "#" is inserted at the beginning of the line. By inputting this comment, the scenario description can be easily decoded. The scenario description stored in the scenario description storage file 41 of the embodiment of FIG. 2 is described in the basic format described above.

FIG. 7 and FIG. 8 illustrate an example of scenario description stored in the scenario description storage file 41 when a manual of a data processing device is realized as a display display by the moving image display processing device of FIG. To do.
It is assumed that the scenario description shown in FIG. 7 is followed by the scenario description shown in FIG.

7 and 8, the coordinate position "100,100" is displayed to display "outline explanation".
"S" having a width of "640" and a height of "600"
After instructing to open the window with the ID "tagel", this window displays "sho
tman / kaisetsul ”,“ shotman /
kaisetsu2 "," shotman / kaise "
"tsu3", "shotman / kaisetsu4"
It is instructed to sequentially display the four moving image display units "".

Next, in order to display the "realization diagram" following the display of the "outline explanation", "book / ko" is displayed.
useiz0 ”,“ shotman2 / kousei ”
zu1 ”,“ shotman2 / kouseizu ”
2 "," shotman2 / kouseizu4 ", and four moving image display units are the same as the above-mentioned" stage "
It is instructed to sequentially display the windows having the IDs. After that, it is instructed to delete this window.

On the other hand, after being instructed to open the window having the ID "stage2" to display the "output example" as a supplement to the above-mentioned "outline explanation", three moving image display units are displayed in this window. Are instructed to be displayed in order.

Next, in order to display the "realization explanation" in accordance with the display of the "realization diagram" described above, it is instructed to sequentially display the six moving image display units. After that, it is instructed to delete this window.

Further, in order to display the "source file" in accordance with the display of a part of the "realization explanation" described above, "s" is displayed.
It is instructed to open the window having the ID "stage3" and then to display one moving image display unit in this window. Then, it is instructed to delete this window.

Then, following the comment "synchronization", the synchronization relation of each moving image display unit displayed in each of the above-mentioned windows is described. For example, the glue name "kai2"
Video display unit having "shotman / kaiset"
It is instructed to simultaneously display "su2" and the moving image display unit "shotman / example1" having the glue name "ex1" on the display.In Fig. 9, the description of the synchronization relationship following the comment "Synchronization" (Note: 2) shows the synchronization relationship between each video display unit defined by the plural form. Note that, in this figure, the actual time length of the video display unit is appropriately corrected and shown for the sake of clarity. ing.

The moving picture display device 10 of FIG. 2 interprets the scenario description stored in the scenario description storage file 41 and the moving picture display unit stored in the moving picture display unit storage file 40, and utilizes the multi-window. The processing is effectively performed so that the moving image (in the above example, the manual of the data processing device) is displayed.

FIG. 10 shows an embodiment of the processing flow of the scenario description interpretation device 12 for interpreting the scenario description. First, in step (ST) 10, the scenario description is read from the scenario description storage file 41.

Next, at step 11, it is judged whether or not there are no scenario descriptions in the scenario description storage file 41. If there is still a scenario description in the scenario description storage file 41 and the determination in step 11 is NO, the format of the read scenario description is identified in step 12, each window data, each moving image display unit,
By creating those link relationships, global variables (described later), and the like, moving image internal data that is the interpretation result of the scenario description is created. Here, regarding the moving image display unit, since the actual screen data is linked by starting the moving image display unit interpretation device 11 in step 14 described later, at the stage when step 12 is executed, one of the moving image internal data is A part (hereinafter, referred to as partial moving image internal data) is created.

FIG. 12 shows an example of partial moving image internal data created based on the operation of the scenario description interpreting apparatus 12 described above. For example, when the scenario description read from the scenario description storage file 41 is the scenario description “glueOpen” described in (1) above, a “window” structure as shown in FIG. The window system 13 of FIG. 2 is started with the width, height, x coordinate, and y coordinate of the window based on the description as parameters, and control data for opening the window, for example, Wi in the case of the X Window system
Control data called dget is created. And
A pointer to that control data is set as the data for the "window" structure described above, and other data in that structure is also set appropriately.

If the scenario description read from the scenario description storage file 41 is the scenario description "glue" described in (2) above, a "moving image display unit" structure as shown in FIG. 12 is created. , A file name, a glue name, etc. are set as data, and a pointer to this structure is set as the last data of the moving image display unit list of the “window” structure for the window specified by the window name.

If the scenario description read from the scenario description storage file 41 is the scenario description "glueSync" described in (4) above, synchronization of the "moving image display unit" structure as shown in FIG. One new element is added as list data, and a pointer to the "moving image display unit" structure having a synchronous relationship is set as the element. The same setting is made in the "moving image display unit" structure having a synchronous relationship.

The scenario description read from the scenario description storage file 41 is the scenario description “gluebranch from <glue name 1> to <glue name 2” in (6) above.
> ”, The new element is 1 as the data of the trigger destination list of the“ video display unit ”structure of the glue name 1.
A pointer to the “moving image display unit” structure of the glue name 2 is stored as the element.

When the scenario description read from the scenario description storage file 41 is the scenario description “gluebranch from <figure name> in <glue 1> to <glue name 2>” in (6) above, One new element is added as the data of the trigger destination list of the "video display unit" structure of the glue name 1, the specified text format figure name and the "video display unit" structure of the glue name 2 The pointer to the body is stored.

When partial moving image internal data is created according to the read scenario description as in the above example,
The processing is returned to step 10 in FIG. 10, the next scenario description is read from the scenario description storage file 41, and partial moving image internal data corresponding to the scenario description is created in step 12.

As a result of repeating the above-mentioned processing, there is no scenario description to be read from the scenario description storage file 41, and if the determination in step 11 is YES,
The process of step 13 is executed.

In step 13, each moving image display unit of the created partial moving image internal data is traced to extract the file name of each moving image display unit. When the file name is extracted, the determination in step 14 is NO, and in step 15, the moving image display unit interpreting device 11 is started based on the thus extracted file name of each moving image display unit.

FIG. 11 shows an embodiment of the processing flow of the moving image display unit interpreting apparatus 11 which interprets the moving image display unit. First, in step 20, the file name of the moving image display unit described above is referred to, so that the data of the moving image display unit corresponding to the file name is read from the moving image display unit storage file 40.

Next, at step 21, it is judged whether or not all the data in the referred moving image display unit are gone. If there is still data in the moving image display unit and the determination in step 21 is NO, in step 22, screen data to be displayed is created according to the read data in the moving image display unit, and the created screen data is a scenario. The partial moving image internal data corresponding to the file name created by the description interpreting device 12 is linked. That is, the screen data is created, for example, in the form of a list including the shape, size, position, etc. of each figure in the form of a list and included in the partial moving image internal data, and a pointer to the list is shown in FIG. It is set as the screen data of a "video display unit" structure.

As a result, complete moving picture internal data has been created, and the created moving picture internal data is stored in the moving picture internal data storage file 14. afterwards,
The control is returned to the scenario description interpretation device 12, and the scenario description interpretation device 12 returns the control to the process of step 13 of FIG. Then, each moving image display unit of the created partial moving image internal data is traced, the file name of the next moving image display unit is extracted, and the moving image display unit interpreting device 11 is activated based on the extracted file name to create the screen data. Then, the video internal data is created.

As a result of repeating the above-described processing, there is no file name of the moving image display unit to be traced, and if the determination in step 14 is YES, in step 16, the moving image operating device 15 of FIG. Based on the moving image internal data created in this way, a moving image (in the above-described example, the manual of the data processing device) is displayed on the display of the input / output device 20. That is, the moving image operating apparatus 15 first displays a control panel window, which will be described later, on the display of the input device 20. When the user instructs to display a moving image according to the control panel, the screen state creating device 16 creates the screen data by tracing the moving image internal data stored in the moving image internal data storage file 14, and then the screen display device 17
Uses the window system 13 to display this screen data on the display. In this way, the moving image can be displayed on the display in a multi-window format. Regarding the operation of the screen state creation device 16,
This will be described in detail later with reference to FIG. 27.

13 to 22 show a part of the manual of the data processing apparatus which is an example of the moving image displayed on the display according to the scenario description of FIGS. 7 and 8 based on the above-described moving image display processing. To do.

First, FIG. 13 is a diagram showing the arrangement of windows on the display screen. In the figure, 50 is the control panel shown in FIG. 14, and 51 is the aforementioned "stag".
A window having an ID of "e1", 52 is a window having an ID of "stage2" described above, and 53
Is a window having the ID "stage3" described above.

FIG. 14 is a diagram showing the screen structure of the control panel 50. The control panel 50 has a "play" key for instructing the reproduction of the animation, a "back" key for instructing the rewinding, and an "s" for instructing the stop.
"top" key, "head" key for instructing movement to the beginning, "tail" key for instructing movement to the end, "fr ++" key for instructing frame advance, "fr ---" key for instructing frame return, A “sh ++” key for instructing a forward movement between moving image display units, a “sh−−” key for instructing a backward movement between moving image display units, and the like are provided.

Now, in FIG. 13, the window display at time t1 corresponds to the screen example 1 in FIG. A window 51 having an ID "stage1" is opened on this screen, and "sho" is displayed in this window.
The “outline explanation” illustrated in FIG. 15 corresponding to the moving image display unit having the ID “tman / kaisetsu1” is displayed.

Next, the window display at time t2 in FIG. 13 corresponds to the screen example 2 in FIG. On this screen, the window 5 with the ID "stage1"
In FIG. 1, the “outline explanation” illustrated in FIG. 16 corresponding to the moving image display unit “shotman / kaisetsu3” is displayed. At the same time, "stage2"
A window 52 having an ID of "open" is opened, and "shotman / example"
The "output example" illustrated in Fig. 17 corresponding to the moving image display unit "2" is displayed. When the window 52 is opened, the bell is ringed to call the user's attention. ..

Subsequently, the window display at time t3 in FIG. 13 corresponds to the screen example 3 in FIG. On this screen, a window 51 having an ID of "stage1" corresponds to a moving image display unit of "book / kouseizu0", and "realization diagram" illustrated in FIG.
Is displayed. At the same time, in the window 52 having the ID "stage2", "shotman2 /
It corresponds to the video display unit "jitsugen2",
“Explanation of realization” illustrated in FIG. 19 is displayed. At the same time, a window 53 having an ID of "stage3" is opened and "sho" is displayed in this window.
The "source file" illustrated in FIG. 20 corresponding to the moving image display unit "tman2 / examplefile" is displayed.

Furthermore, the window display at time t4 in FIG. 13 corresponds to the screen example 4 in FIG. On this screen, in the window 51 with the ID "stage1",
The “realization diagram” illustrated in FIG. 21 corresponding to the moving image display unit “book / kouseizu0” is displayed, and the window 5 with the ID “stage2” is displayed.
2, "Explanation of realization" illustrated in FIG. 22 corresponding to the moving image display unit of "shotman / jitsugen3" is displayed. At the same time, the window 53 having the ID “stage3” is deleted.

As in the above example, the manual of the data processing device can be effectively displayed as a moving image in accordance with the multi-window. Another Embodiment of Video Display Device Next, FIG. 23 shows another embodiment of the video display device for realizing the present invention. In FIG. 23, the parts denoted by the same symbols as described in FIG. 2 have the same functions.

In this embodiment, a device configuration for handling the network type scenario description described in FIG. 5B is disclosed, which is an embodiment having a function of dynamically changing a moving image.

In the film-type scenario description described with reference to FIG. 5A, since the moving image display units are arranged in time order, the screen state at any time can be easily reproduced and the moving image can be reversed in time order. On the other hand, in the network-type scenario description, the display contents change depending on the situation, and therefore the configuration of the embodiment of FIG. Therefore, in the embodiment of FIG. 23, the history data storage file 18 is provided, and the transition of the moving image display unit is recorded in the history data storage file 18, and the backward movement operation of the time sequence of the moving images in the network type scenario description is recorded according to this recording. Is realized.

Here, if the transitions of moving image display units are recorded in the history data storage file 18 indefinitely, a large memory capacity is consumed. Therefore, in the embodiment shown in FIG. 23, a model selection device 19 is provided for selecting whether or not the user has to request to return the time sequence according to the interaction processing with the user, and the model selection device 19 returns the time sequence. When it is selected not to request, the control is performed so that the saving process in the history data storage file 18 is not executed, so that the memory is not wasted.

24 and 25, an example of the scenario description stored in the scenario description storage file 41 of the embodiment of FIG. 23 is shown in FIG. 26, and in the moving picture internal data storage file 14 of the embodiment of FIG. 3A and 3B respectively show examples of stored moving image internal data. It is assumed that the scenario description shown in FIG. 24 is followed by the scenario description shown in FIG.

In FIGS. 24 and 25 (i) to (iv), the scenario description "glue" of (6) above, which is peculiar to the network type, is used.
"branch from" is described. Also, as shown in (a) to (c) of FIGS. 24 and 25, the scenario description "gr" of (5) described above instructing grouping of moving image display units is given.
"up" is described. When such grouping is performed, for example, a moving image showing a document having a chapter structure can be displayed, and the chapters can be moved so that the user can use it. 27. Detailed operation of screen state creating device 16 Next, the process flow shown in Fig. 27 executed by the screen state creating device 16 will be described in detail. If the device 16 is implemented on the embodiment of FIG. 2, only film-type scenario descriptions are dealt with, so steps 31 ...
When only the processing of 36 is unconditionally executed and the screen state creation device 16 is realized in the embodiment of FIG. 23, both the film type scenario description and the network type scenario description are handled, The processing from 30 to 42 is executed. For simplicity of explanation, the screen state creation device 16 will be described below as realized on the embodiment of FIG.

First, in step 30, it is determined whether the scenario description is a film type or a network type by reading the above-mentioned global variable from the moving image internal data storage file 14 and determining the setting value. To be done.

When it is determined in this determination process that the scenario description is of the film type, in step 31, the input from the user on the control panel is awaited.
If there is an input, in step 32, it is determined whether or not it is an input instructing the end of input.

In this determination processing, it is assumed that the input is an input other than the input end and is realized above. By reading the global variable described above and determining the setting value, it is determined whether the scenario description is a film type or a network type. This determination process is performed. At this time, if the time exceeds the start / end time, an appropriate moving image display unit before and after is searched, and similar processing is executed for that. If the input is, for example, an instruction to "move to the beginning / end of the video",
The time is set as the start / end time of the scenario description, and the screen data of the moving image display unit at that time is created in the window opened at that time.

Subsequently, in step 34, the screen display device 1
7 is activated. As a result, the screen display device 17 displays each figure indicated by the screen data in the corresponding window using the drawing function of the window system 13.

In the following step 35, it is determined whether the input from the control panel is "play / reverse" or other input. If the input is "play / reverse", then in step 36 a "play / reverse" input is automatically generated and control is returned to step 33. As a result, one-time "playback / reverse" input allows continuous reproduction / reverse processing of moving image display units.

If the input is something other than "play / reverse", the process returns from step 35 to step 31 to wait for the input on the control panel. on the other hand,
When the scenario description is determined to be of the network type in the determination processing of step 30, the input from the user on the control panel is awaited in step 37.

When there is an input, it is determined in step 38 whether or not the input is an instruction to end the input. In this determination process, if it is determined that the input is an input other than the end of input, screen data is created in step 39. Specifically, if the input is, for example, a "play / reverse" instruction, the moving image display unit list currently displayed in the moving image internal data storage file 14 is traced, and the next / previous screen data is displayed. Screen data is created. At this time, if there is no screen data to be displayed, the screen data at the beginning of the next / previous moving image display unit of the current moving image display unit is created. Further, when the input is, for example, an instruction of “click on the screen”, the movement list (see FIG. 26) is retrieved from the clicked moving image display unit and the figure ID and displayed in the window.

Subsequently, in step 40, the screen display device 1
7 is activated. As a result, the screen display device 17 displays each figure indicated by the screen data in the corresponding window using the drawing function of the window system 13.

In the following step 41, it is determined whether the input from the control panel is "play / reverse" or other input. If the input is "play / reverse", then in step 42 a "play / reverse" input is automatically generated and control is returned to step 39. As a result, one-time "playback / reverse" input allows continuous reproduction / reverse processing of moving image display units.

In this way, the screen state creating device 16
Executes a process of creating screen data according to the type of scenario description. If the input is something other than "play / reverse", the process returns from step 41 to step 37 to wait for the input on the control panel. One Example of Video Input Device 30 Finally, one example of the video input device 30 of FIG. 2 or FIG. 23 is shown in FIG. 28, and its operation is performed in accordance with the processing flow of FIGS. 29 to 31 and the explanatory diagram of FIG. 32. Explain.

First, in step 50 of FIG. 29, the moving image input device 301 selects an edit target of a moving image display unit or a scenario description according to an instruction from the input / output device 20 by the user.

When the user gives an instruction to edit the moving image display unit, the moving image input device 301 activates the moving image display unit file control device 302. Currently, each moving image display unit is managed in file units. Then, the moving image display unit file control device 302 searches in step 51 of FIG. 30 whether or not a file corresponding to the file name designated by the user exists in the moving image display unit storage file 40, and the file exists. For example, the file is selected as an edit target, and if it does not exist, a new file is created and the edit target is created. After that, the moving image display unit file control device 302 determines that the moving image display unit inner frame specifying device 30
Start 3.

Now, editing in moving image display units is performed in frame (time) units. Therefore, the moving image display unit internal frame designation device 303 designates the frame designated by the user as an edit target on the file set by the moving image display unit file control device 302 in step 52, and in step 53, the graphic editor Start 304. At this time, if the file already exists in the moving image display unit storage file 40 and the designated frame exists in the file, the moving image display unit data of the frame is read and displayed in the graphic editor 304. It

The graphic editor 304 uses steps 54 and 5
By repeating step 5, the user edits the screen data of the corresponding frame. An existing general-purpose editor can be used as the graphic editor 304, or a dedicated graphic editor can be newly developed based on a known technique. At this time, the moving image input control device 301 controls the transfer of the graphic data between the data input / output device 20 and the graphic editor 304 of FIG. 2 or FIG.

When the user gives an instruction to end editing with the graphic editor 304, the determination in step 55 becomes YES, and the editing result is saved as graphic editor creation data 305 in a disk storage device or the like. After that, the moving image display unit file control device 302 causes the graphic data analysis device 306 to
To start.

The graphic data analysis device 306 executes step 5
In step 6, the frame data of the moving image display unit represented in the internal text format as already illustrated in FIG. 3 is generated with reference to the format definition 307 of the moving image display unit.

The moving image display unit file control device 302
In step 57, the frame data of the moving image display unit generated as described above is added to the specified file,
Alternatively, the frame data on the file is replaced by the frame data.

When the user instructs to edit another frame, the moving image display unit file control device 302 returns the control from step 58 to step 52 and repeats the control process for editing a new frame.

If the user gives an instruction to edit another moving image display unit, the moving image display unit file control device 3
02 returns control from step 59 to step 51,
The control process for editing a new moving image display unit is repeated.

When the user gives an instruction to edit the scenario description, the moving image input device 301 activates the scenario description editing device 308. In step 60, the scenario description editing device 308 follows the edit screen format previously defined as the scenario description format definition 309.
The scenario description edit screen is displayed, and by repeating steps 61 and 62, the user is allowed to edit the scenario description on the edit screen as shown in FIG. 32, for example. At this time,
The moving image input control device 301 controls the passing of data between the data input / output device 20 and the scenario description editing device 308 of FIG.

The user specifies the name, coordinates or size / width / height of each window, for example, by filling the corresponding item in the table on the screen, and the order relation of each moving image display unit in each window is As shown in FIG. 32, the order of the rectangles displayed together with the character string representing the file name is edited by operating the mouse or the like.

Further, when the user wants to trigger the transition from a predetermined figure in a predetermined moving image display unit to another moving image display unit by clicking the mouse at the time of displaying a moving image, for example, as shown in FIG. As shown in the figure, while inputting a desired figure name (ID) in the rectangle corresponding to the moving image display unit that allows the trigger, the arrow from the rectangle corresponding to the moving image display unit that allows the trigger to the rectangle corresponding to the desired moving image display unit To set.

When the user gives an instruction to end the editing of the scenario description, the determination in step 62 becomes YES, and in step 63, the editing result is saved as scenario description screen data 310 in a disk storage device or the like. Then, in step 64, the scenario description editing device 308 activates the scenario description input data analysis device 311.

Scenario description input data analysis device 311
In step 65, the scenario description screen data 310,
7 and 8 while referring to the edit screen format predefined as the scenario description format definition 309 and the internal text format scenario description format predefined as the scenario description format definition 309. The scenario description data expressed in the internal text format is generated. As the scenario description to be generated, as described above with reference to FIGS. 7 and 8, the position of each window, the order relationship of the moving image display units in the window, the synchronization relationship between the moving image display units on different windows, Alternatively, each information such as a graphic for activating the transition to any moving image display unit and the moving image display unit of the transition destination is described in a format that can be recognized by the scenario description interpretation device 12 of FIG. 2 or FIG. 23 described above.

Scenario description input data analysis device 311
In step 66, the scenario description generated as described above is stored as the scenario description storage file 41 in FIG. 2 or FIG.

As described above, the scenario description storage file 4
The scenario description stored in 1 is interpreted by the scenario description interpretation device 12 in FIG. 2 or FIG. in this case,
For the progress time of each frame of each moving image display unit displayed on each window according to the scenario description, for example, an appropriate display time is assigned to each frame so that the synchronization relationship between each moving image display unit described above is saved. By this, it is automatically determined. Other Embodiment of Video Input Device 30 The configuration and operation of the video input device 30 of FIG. 2 or FIG.
It is not limited to those shown in FIGS. 28 to 32 described above.

In the first other embodiment, for example, FIG.
The logical time is displayed on the “time” axis of the order relationship of the moving image display units of No. 2, and the user sets the order relationship of each moving image display unit to the above-mentioned rectangle and the numerical value representing the above-mentioned logical time. Use to enter. In this way, the user creates a scenario description in which the display timing of each video display unit displayed on each window is explicitly specified in a state where the synchronization relationship between the above-mentioned video display units is saved. be able to. In this case, in the moving picture internal data obtained as the moving picture internal data storage file 14 by the scenario description interpretation device 12 of FIG. 2 or 23 interpreting the scenario description, the display time of each moving picture display unit is, for example, as shown in FIG.
It is set as start time data and end time data in the "moving image display unit" structure of 2.

In the second other embodiment, as shown in FIG. 33, a rectangular parallelepiped in which planes representing the display screen are superimposed in the direction of the time axis is displayed on the graphic editor, and the user is in the rectangular parallelepiped. Use to edit the window position, size, display timing, etc. As a result, the user can generate a scenario description including a description for controlling the window. Then, for each state of each window set in this way, the user edits the state of each moving image display unit on the editing screen shown in FIG. 32, for example, and the scenario description corresponding to the editing operation is performed. To generate. In this case, in the moving picture internal data obtained as the moving picture internal data storage file 14 by the scenario description interpretation device 12 of FIG. 2 or 23 interpreting the scenario description, the display time of each window is, for example, as shown in FIG.
It is set as the open time data and the close time data in the "window" structure of.

In the third other embodiment, as shown in FIG. 34, the area of the two-dimensional window screen and the areas "A" and "B" for inputting the logical time are displayed on the graphic editor. The user uses both of these areas to specify and edit the position, size, display timing, etc. of the window. As a result, the user can generate a scenario description including a description for controlling a window, as in the second alternative embodiment. Then, for each state of each window set in this way, the user edits the state of each moving image display unit on the editing screen shown in FIG. 32, for example, and the scenario description corresponding to the editing operation is performed. To generate.

[0112]

According to the present invention, a plurality of moving picture screens can be displayed in windows that are associated with each other by utilizing a multi-window, and it is possible to dramatically improve the understanding of the contents of a moving picture. Become.

Further, by associating the moving images temporally and spatially with a smaller moving image display unit and by associating them with the window, it is possible to dramatically improve the degree of freedom of expression.

Moreover, since the moving picture can be reused as parts by disassembling the moving picture into several parts, it becomes possible to easily edit the moving picture for a long time. The efficiency of is also dramatically improved.

Furthermore, since the scenario description can be edited graphically, editing of the moving image becomes easier.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is an overall configuration diagram of an embodiment of the present invention.

FIG. 3 is a diagram showing an example of a description format of a moving image display unit.

FIG. 4 is an explanatory diagram of a description format of a moving image display unit.

FIG. 5 is an explanatory diagram of a model of scenario description.

FIG. 6 is an explanatory diagram of a structured moving image display unit.

FIG. 7 is a diagram showing an example of scenario description (No. 1)
Is.

FIG. 8 is a diagram showing an example of scenario description (No. 2)
Is.

FIG. 9 is an explanatory diagram of a synchronization relationship of moving image display units in an example of scenario description (FIGS. 7 and 8).

FIG. 10 is a diagram showing a processing flow of a scenario description interpretation device.

FIG. 11 is a diagram showing a processing flow of the moving image display unit interpretation device.

FIG. 12 is a diagram showing an example of created moving image internal data.

FIG. 13 is an explanatory diagram of a manual moving image display process.

FIG. 14 is a diagram showing an example of a control panel display.

FIG. 15 is an explanatory diagram (part 1) of screen data displayed in a window.

FIG. 16 is an explanatory diagram (part 2) of screen data displayed in a window.

FIG. 17 is an explanatory diagram (part 3) of screen data displayed in a window.

FIG. 18 is an explanatory view (No. 4) of screen data displayed in a window.

FIG. 19 is an explanatory diagram (No. 5) of screen data displayed in a window.

FIG. 20 is an explanatory diagram (No. 6) of screen data displayed in a window.

FIG. 21 is an explanatory diagram (No. 7) of screen data displayed in a window.

FIG. 22 is an explanatory diagram (No. 8) of screen data displayed in a window.

FIG. 23 is an overall configuration diagram of another embodiment of the present invention.

FIG. 24 is a diagram (part 1) illustrating an example of a scenario description.

FIG. 25 is a diagram (No. 2) showing an example of the scenario description.

FIG. 26 is a diagram showing an example of created moving image internal data.

FIG. 27 is a diagram showing a processing flow executed by the screen state creation device.

FIG. 28 is a diagram showing an example of a configuration of a moving image input device.

FIG. 29 is a diagram (No. 1) showing the processing flow of the moving image input apparatus.

FIG. 30 is a diagram (No. 2) showing the processing flow of the moving image input apparatus.

FIG. 31 is a diagram (No. 3) showing the processing flow of the moving image input apparatus.

FIG. 32 is a diagram showing an editing screen of a moving image display unit based on a two-dimensional table.

[Fig. 33] Fig. 33 is a diagram showing a window editing screen in a rectangular parallelepiped display.

[Fig. 34] Fig. 34 is a diagram showing a window editing screen by two-dimensional screen display.

[Explanation of symbols]

 1 data processing device 2 data input device 3 man-machine interface 4 video display unit management means 5 scenario description management means 6 video display control means 7 video history management means 8 operation mode control means 9 interface means 10 video input control means 11 graphic editing Means 12 Movie display unit editing means 13 Scenario description editing means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location G09G 5/36 8121-5G

Claims (13)

[Claims] 1. A moving picture display processing apparatus for displaying a moving picture on a plurality of windows opened on a display, wherein a plurality of moving picture display units each consisting of one or a plurality of frames describing a screen display definition are provided. Minute display managing unit (4) for managing the minutes, the state of each window on the display, the temporal order relationship of the moving image display units displayed in each window, or each moving image displayed in each window A scenario description management unit (5) for managing scenario descriptions each describing a synchronization relationship between display units and a scenario description managed by the scenario description management unit (5) are interpreted, and the moving image is displayed in each window according to this interpretation. A moving image display processing device, comprising: a moving image display control means (6) for sequentially displaying time-series data of a display unit. 2. The temporal order relation includes an arbitrary temporal transition relation from one moving image display unit to another moving image display unit displayed in the same window as the window in which it is displayed or in a different window. The moving image display processing device according to claim 1. 3. The temporal transition relationship includes identification information that becomes effective when a predetermined screen portion in the currently displayed moving image display unit is designated by the user. Item 2. The moving image display processing device according to item 2. 4. A moving picture history management means (7) for managing the history of each moving picture display unit, which is displayed in each window and makes an arbitrary temporal transition in accordance with the temporal transition relationship, further comprising: The moving image display control means (6) controls each moving image display unit displayed in each window according to the history data managed by the moving image history management means (7) and performing an arbitrary temporal transition according to the temporal transition relationship. The moving image display processing apparatus according to claim 2, wherein the moving image display reverses display processing. 5. An operation mode control means (8) for controlling the history management function of the moving picture history management means (7) so as not to operate in response to an instruction from the man-machine interface (3). The moving image display processing device according to claim 4. 6. The scenario description includes group information such that a group of a plurality of moving image display units is treated as one moving image display unit, according to any one of claims 1 to 5. The moving image display processing device described. 7. An interface means (9) for controlling the execution state of the moving picture display control means (6) in response to an instruction from a man-machine interface, further comprising: The moving image display processing device according to any one of claims. 8. The interface means (9) has a function of stopping the progress of the moving picture display processing of the moving picture display control means (6) and the function of stopping the moving picture display control means (6). A function of moving the moving image display unit back and forth one by one, and a function of moving the frame of the moving image display unit stopped by controlling the moving image display control means (6) one by one back and forth, The moving image display processing device according to claim 7, wherein 9. A moving picture display processing apparatus for displaying a moving picture on a plurality of windows opened on a display, wherein a plurality of moving picture display units composed of one or a plurality of frames describing a screen display definition. Minute display managing unit (4) for managing the minutes, the state of each window on the display, the temporal order relationship of the moving image display units displayed in each window, or each moving image displayed in each window A scenario description management unit (5) for managing scenario descriptions each describing a synchronization relationship between display units and a scenario description managed by the scenario description management unit (5) are interpreted, and the moving image is displayed in each window according to this interpretation. The moving image display control means (6) for sequentially displaying the time-series data of the display unit, and the scenario description management means (5) for the user. The scenario description that sense, moving image display processing apparatus comprising a scenario description editing means (13) for editing by the editing process including a graphic symbol editing process. 10. The scenario description editing means (13)
Allows the user to edit the position of each window at the time of displaying the moving image in a text format on the editing window opened on the display at the time of editing, and the temporal position of the moving image display unit displayed in each window. The moving picture display processing apparatus according to claim 9, wherein the order relationship or the synchronization relationship between the moving picture display units displayed in the windows is edited in a graphic format using a graphic symbol. 11. The scenario description editing means (13)
11. The moving image according to claim 10, wherein the user further edits the display timing of the moving image display unit displayed in each window at the time of displaying the moving image in a text format on the editing window. Display processor. 12. The scenario description editing means (13)
Is a rectangular parallelepiped in which the display screen displayed on the editing window opened on the display at the time of editing is displayed three-dimensionally in the time axis direction. Alternatively, the closing timing and its position are edited, and the moving image display processing apparatus according to claim 10 or 11. 13. The scenario description editing means (13)
Allows the user to edit, in a text format, the timing of opening or closing each window on the display when displaying a moving image in the editing window opened on the display when editing, and each window when displaying the moving image. 12. The position on the display of each window at the time of displaying the moving image is edited by arranging in a predetermined area on the editing window. 12. The method according to claim 10, wherein Video display processor.