JPH08249196A - Redundant execution method of tasks - Google Patents

Abstract

(57) [Summary] [Purpose] To secure the reliability of tasks in a multi-computer system. [Constitution] In a multi-computer system formed of a plurality of nodes, a task control means 6 on each node allocates a task on the system and further activates and manages the task on its own node. The event notification unit 7 and the execution error detection unit 10 notify the task control unit 6 of the event that has occurred in the system and the comparison result of the task execution results. The user task I / F unit 8 is an I / F for the user task and the task control means 6 on the own node. With such a configuration, the redundant execution method is realized. [Effect] Since a task can be executed continuously with redundancy, a highly reliable system can be constructed. In addition, tasks running on a node can be safely ejected outside the node, so that a node failure, H / W or S
In the maintenance work of / W, it is not necessary to stop the system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention masks node disappearance and task error failures in a multi-computer system consisting of a plurality of computers and an external storage device shared between them, and further The present invention relates to a technique for constructing a system having a fault tolerance function by providing checkpoint generating means which are synchronized with each other.

[0002]

2. Description of the Related Art FIG. 18 shows "Fau" by Daniel P. Siewiorek.
lt Tolerance in Commercial Computers ”COMPUTER, Ju
This is the configuration of the fault-tolerant computer (hereinafter referred to as FT computer) introduced in ly 1990.pp.26-37. In the FT computer shown in FIG. 18, main H / W modules are made redundant, which is effective in preventing malfunction of the system due to H / W failure.

In the conventional FT computer shown in FIG. 18, for example, execution results are compared in redundant processor elements, and the data of the processor element pair in which the execution results match are adopted as the normal value. . If a fixed failure is detected after the occurrence of a comparison error in the execution results, the failed H / W module is disconnected from the system. Then, the failed H / W module is re-installed in the system after the module is replaced in the online state. On the other hand, in a computer which does not have a fault tolerant function, the computer itself is generally constructed with a double system to increase the reliability of the system. And
In such a dual system, a process of saving the execution history of the process in the main system and a process of transmitting it to the standby system are performed in case of an unexpected stop of the main system. In addition, SIFT which is an S / W implemented fault tolerant function.
(Software Implemented Fau
lt Tolerance) and MAFT (Multitic)
computer Architecture for
The concept of FT computer called “Fault Tolerance” is introduced, and an implementation of a device for drawing these flows is disclosed in, for example, Japanese Patent Publication No. 2-503122. FIG. 19 is a block diagram of the multi-computer architecture in Tokuhyo 2-503122. In the same architecture, failure of multiple nodes does not prevent execution of any operation task.

FIG. 20 is a block diagram of a conventional checkpoint mechanism disclosed in Japanese Patent Laid-Open No. 2-287858. 12, 1
Reference numerals 7 are a processing device A and a processing device B, respectively, which are nodes constituting a distributed system, and which are connected to each other via a communication line 2
Bound by two. The processing device A12 includes a communication control unit A13, a checkpoint task execution unit A14, and a restart task execution unit A15. Also, the checkpoint file A is stored in the processing device A12.
16 is connected as an external storage device. Similarly, the processing device B17 includes a communication control unit B18, a checkpoint task execution unit B19, and a restart task execution unit B20. A checkpoint file B21 is connected to the processing device B17 as an external storage device.

In FIG. 20, when the communication control unit A13 of the processing device A12 issues a data transmission request to the communication control unit B18 of the processing device B17 via the communication line 22, the checkpoint task execution unit A14 outputs the request. The execution program information is stored in the checkpoint file A16 as new checkpoint data. At this time, the storage area for the new checkpoint data is different from the old checkpoint already stored in the checkpoint file A16. Next, the checkpoint task execution unit A14 instructs the checkpoint task execution unit B19 in the processing device B17 to communicate with the communication control unit B18.
Request to collect the execution program information of the program being executed (received). In response to this request, the checkpoint task execution unit B19 stores the relevant program information in the checkpoint file B21 as new checkpoint data. The storage area for the new checkpoint data at this time is a different area from the old checkpoint data already stored in the checkpoint file B21.

When the checkpoint is successfully collected, the checkpoint task execution unit B19 determines that the processing unit A12
The acknowledge (ANK) is returned to the checkpoint task execution unit A14. If the collection fails, the acknowledgment will not be returned. When the acknowledge is returned, the checkpoint task execution unit A14 discards the old checkpoint data in the checkpoint file A16 and sets the new checkpoint data as the old checkpoint data. Similarly, the checkpoint task execution unit B19 discards the old checkpoint data in the checkpoint file B21 and sets the new checkpoint data as the old checkpoint data. If the acknowledge is not returned, the old checkpoint data remains stored.

In the restart processing, the restart task execution section A15 takes out the new checkpoint data stored in the checkpoint file A16 and takes out the corresponding checkpoint data.
A restart request is issued to the restart task execution unit B20 of the processing device B17. Upon receiving the restart request, the restart task execution unit B20 extracts the corresponding new checkpoint data from the checkpoint file B21. When this extraction is successful, the restart task execution unit B20 returns an acknowledge to the restart task execution unit A15. If you fail to retrieve it, do not return it. When the acknowledge is returned, the restart task execution unit A15 restarts the task execution with the new checkpoint data. When the acknowledge is not returned,
The restart task execution unit A15 restarts the task execution based on the old checkpoint data.

In this way, by confirming the synchronization with each other at the time of collecting checkpoints and at the time of restarting, a synchronized checkpoint restart process is realized.

[0009]

The conventional FT computer has the following problems. (1) It is resistant to H / W failures, but is vulnerable to failures caused by I / O device operation timing and S / W errors. Furthermore, when a secondary failure occurs in the H / W module that has lost the redundancy, the system is stopped. (2) There is no detection mechanism for the task that caused the execution error. In task processing, it is difficult to know what is correct and what is incorrect, and recovery processing after a failure occurs is difficult. (3) In order not to stop the system during maintenance work such as expansion and replacement of H / W module, H / W
Online expansion of W module, hot-swap technology, and I / O reconfiguration capability of supervisor are required. (4) Ultimately, it is necessary to multiplex all H / W modules. (5) The system must be stopped when replacing supervisors and task modules.

When the system has a double system, it is possible to avoid the system from being stopped by continuing the continuous operation in one system to solve the above problem. However, in the redundant system of the node itself which is seen in such a dual system, (6) it is necessary to configure a plurality of computers having the same performance for the target, which results in high cost. Of course, it is difficult to configure the system symmetrically with different models. (7) In a dual system, which has a master-slave relationship, it is always necessary to save execution history and checkpoint save processing in preparation for an unexpected accident. This requires a recovery time and a complicated recovery process depending on the amount of recovery data. (8) Furthermore, in the dual system, the spatial majority logic does not hold, and it is not suitable for constructing a system that secures high reliability such as giving spatial redundancy to the execution of tasks. (9) Generally, the H / W diagnostic process is performed after the online work is stopped, and thus the redundancy is reduced in the system process.

Further, in the computer architecture disclosed in Tokuhyo 2-503122, although failure detection and subsequent failure part isolation are performed, (10) recovery processing after occurrence of failure is not mentioned. . Therefore, the system to which this method is applied does not guarantee a permanent continuous operation. Furthermore, a failure of one device on a node may result in a failure of the entire one node. Therefore, (11) it is not compatible with a system in which devices are complicatedly combined.

Further, since the conventional checkpoint mechanism is configured as described above, (12) there is no method for synchronizing checkpoints for communication between tasks in the same processing unit. It was (13) Since a checkpoint is taken every communication, there is a problem that the overhead is large. (14) Since one checkpoint file is connected to each processing device, there is a problem that restart from a checkpoint cannot be performed by another processing device.

The present invention has been made to solve the above problems, and an object thereof is to obtain the following functions in a multi-computer system. (1) To prevent the system from being stopped due to an error in H / W or S / W, an object thereof is to obtain a function of making task execution spatially / temporally redundant. (2) An execution error detecting means having a function of monitoring the execution error of the task is arranged on the node so that the judgment standard of the execution error can be defined for each task. (3) After the execution error occurs in the redundantly executed task, create a checkpoint with the normally executed task,
The execution error task is transferred to another executable node after being stopped. Subsequent task processing enables the recovery processing with a short stop time by taking over and executing the checkpoint data. In this method, compared to the conventional method that uses the execution history that was kept in preparation for a sudden system stop for recovery processing, the check is performed only when a redundantly executed task has an execution error and loses the redundancy of task execution. By taking points, the checkpoint save / recovery process is simplified. (4) The maintenance work in the node is performed by waiting for the completion of the task being executed on the node or guiding the completion thereof, and then detaching / attaching the node by the online degeneracy / expansion function of the node. Therefore, the node in the system to which the present invention is applied does not require the hot-swap mechanism of the H / W module, the online expansion function of the H / W module, and the I / O reconfiguration capability of the supervisor. (5) It is possible to specify a node that executes a task, and realize load distribution and role sharing in the node. Thus, it is an object of the present invention to enable incorporation in the same system even if the nodes do not have the same function. (6) In the method of submitting a task to this system, a programming interface by an indirect call method is given. This allows the task execution destination node to be changed without recompiling the program by changing the task definition that is separate from the program description of the task module. Furthermore, the node manages the state for assigning each task. With these functions, it is possible to guarantee that a specific task is not temporarily assigned to a node, so that it is possible to deal with S / W maintenance work. (7) When a node suddenly disappears, in order to facilitate the recovery processing of tasks that have not been redundantly executed, restart the tasks that have no spatial redundancy and that have been executed on other nodes. To enable. (8) It is possible to execute a specific diagnostic procedure on the faulty node after the task has made an execution error. In addition, the redundant task that has caused an execution error can also perform online diagnosis of the faulty node by executing the online work behind the scenes without participating in the task execution voting. Further, (9) when taking a checkpoint, it is possible to take a checkpoint in synchronization even if the communication partner is in the same node. (10) It is possible to eliminate the overhead of checkpoint collection in normal processing. (11) After taking a checkpoint, the purpose is to restart on a different node.

In the present invention, by realizing the above functions, it is possible to construct a system in which a fault tolerant function is realized in a multi-computer system.

[0015]

A task redundancy execution system according to the present invention is a task redundancy execution system of a multi-computer system composed of a plurality of computers, wherein each computer has the following elements. I am trying. (A) A task definition unit that stores in advance information for managing tasks executed by the computer, and (b) task control means that controls the execution of the task based on the information stored in the task definition unit.

Further, the task redundancy execution system of the present invention is characterized in that the task definition section stores information for making the task redundant in time and executing the task.

Further, the task redundancy execution system of the present invention is characterized in that the task definition section stores information for spatially redundantly executing a task.

Further, in the task redundancy execution system according to the present invention, the computer further comprises an execution error detecting means for detecting an error occurring during task execution and notifying the detected error to the task control means, The task definition unit predefines control at the time of error detection, and the task control unit, based on the notified content,
It is characterized in that the execution of the task is controlled according to the definition contents of the task definition section.

In the task redundancy execution method according to the present invention, when the task control unit notifies the execution error detection unit of an error of a task being executed on a specific computer, the redundancy of the task is determined. It is characterized by comprising a task redundancy execution means for restarting the task so as to secure the task.

Further, in the task redundancy execution system according to the present invention, the redundancy execution means stops the execution of the task in the computer in which the error of the task being executed occurs.
The above-mentioned task is restarted to another computer.

According to the task redundancy execution method of the present invention, the computer is constituted by redundant hardware, and a failure occurs in the hardware so that the computer executes a task having no spatial redundancy. If you are
It is characterized in that the task control means is provided with a task transfer means for once stopping a task which is being normally processed on the computer and restarting the task on another computer.

The task redundancy execution method according to the present invention is the task redundancy execution method, further comprising an external storage unit for storing information necessary for mid-execution of the task.
The task control means includes information collecting means for collecting the information necessary for mid-execution of the task from the task being normally processed, and storing the collected information in the external storage unit,
It is characterized by comprising re-execution means for retrieving information necessary for mid-execution of the task from the external storage unit and re-executing the task midway based on the retrieved information.

In the task redundancy execution system according to the present invention, the task redundancy execution system further comprises a plurality of input / output devices, and the task control means is the task being executed by the execution error detection means. When an error is detected in the above, it is provided with an allocation prohibition means for prohibiting allocation of other tasks using the I / O device scheduled to be used by the above task to the computer among the above I / O devices. Characterize.

Further, in the task redundancy execution system according to the present invention, the task definition unit includes:
A diagnostic procedure is provided for diagnosing which input / output device has an error, and the task control section is provided with online diagnostic means for executing the diagnostic procedure.

Further, in the task redundancy execution system according to the present invention, the external storage section further stores modules, and the task control means does not allocate a specific task to a specific computer for a specific period. It is characterized in that it is provided with a task maintenance unit that guarantees and takes out the module stored in the external storage unit and replaces the taken out module with the specific task.

Further, in the task redundancy execution system according to the present invention, the task control means stops the task which is normally processed on the computer, disconnects the computer from the system, and executes it on another computer. It is characterized by having an online degeneracy means for re-mobilizing tasks.

The task redundancy execution system according to the present invention is characterized in that the task control means further comprises an online expansion means for re-inputting the separated computer into the system.

In the task redundancy execution system according to the present invention, the plurality of computers further include communication means for exchanging messages between the computers, and it is detected that the computers performing the communication disappear. A message for preventing the message from disappearing due to the disappearance of the computer, based on the notification content from the event notifying unit It is characterized by having a management means.

The task redundancy execution system according to the present invention is characterized in that the task control means further comprises task re-input means for preventing the task from disappearing as the computer disappears.

In the task redundancy execution method according to the present invention, the information collecting means is in the process of communication between the tasks, the process of receiving the communication between the tasks, the process of waiting for a response in the communication between the tasks, and When performing any of the input / output processing of the task, it is characterized in that information necessary for mid-execution of the task is collected.

Further, in the task redundancy execution system according to the present invention, when the information collecting means has a communication partner of inter-task communication on another computer, a signal is sent to the information collecting means provided in the other computer. It is characterized by sending and collecting the information necessary for mid-execution of the task.

[0032]

In the task redundancy execution system according to the first aspect of the invention, the task control means controls the execution of the task based on the information stored in the task definition section. For this reason,
If information for ensuring the reliability of the task is defined in the task definition section, the task control means can refer to this information and execute the task redundantly.

In the task redundancy execution method according to the second aspect of the present invention, the task definition section stores the information to be executed by making the task redundant in time. Therefore, by referring to the task definition section by the task control means,
You can add temporal redundancy to the tasks you are performing.

In the task redundancy execution system according to the present invention, the task definition unit stores the information to be executed by spatially making the task redundant. Therefore, the task control unit can give the task being executed a spatial redundancy by referring to the task definition unit.

In the task redundancy execution method according to the present invention, the execution error detection means determines whether there is an error or not based on the execution result of the task that has been redundantly executed, and if there is an error, the task control is executed. Notify the means of the error. Then, the task control unit that has received the error notification controls the execution of the task according to the definition content of the task definition unit. Therefore, even if an error occurs during execution of the redundantly executed task, the executed task can be easily handled by the task control means when an error is detected.

In the task redundancy execution method according to the present invention, the task redundancy execution means restarts the task so as to secure the redundancy of the task. Therefore, even if the execution error detection unit notifies the task control unit of the execution error of the task, the task can be continuously executed without impairing the redundancy of the task.

In the task redundancy execution system according to the present invention, the redundancy execution means stops the execution of the task on the computer in which the error has occurred, and restarts the task on another computer. Therefore, for example, if the error of a task is an error related to an I / O device connected to a computer, the task is restarted by another computer, and the probability of an error related to the same I / O device is low. Become.

In the task redundancy execution system according to the present invention, the re-execution means retrieves information necessary for mid-execution of the task stored in the external storage unit, and re-executes the task midway based on the retrieved information. To do. Therefore, even if the task is restarted in another computer, the execution result until the error occurs can be surely taken over.

In the task redundancy execution method according to the present invention, the allocation prohibiting means uses the input / output device which was or is planned to be used by the task when an error occurs during execution of the task. It is prohibited to assign the task of (1) to the computer on which the above task was executed. Therefore, when the task error is related to the input / output device, the task that uses the same input / output device cannot be assigned to the computer, thus improving the operating rate of the system.

In the task redundancy execution system according to the present invention, the online diagnostic means diagnoses which of the input / output devices making up the computer is in error. Therefore, the input / output device in which the error has occurred can be easily specified.

In the task redundancy execution system according to the present invention, the task maintenance means guarantees that the specific task is not allocated to the specific computer for the specific period, and the module stored in the external storage unit is taken out. Swap the specified task with the removed module. For this reason, for example, when upgrading the version of a module, it is possible to easily replace only a specific task with a new module without stopping the entire system.

In the task redundancy execution system according to the present invention, the online degeneracy means stops the task during normal execution on the computer, disconnects the computer from the system,
Restart the stopped task on another computer. Therefore, for example, when performing maintenance on a computer, it is possible to allow another computer to take over the processing without stopping the task being executed, and to disconnect the computer for maintenance from the system.

In the task redundancy execution system according to the present invention, the online expansion means re-inputs the separated computer into the system. Therefore, the computer decoupled from the system can be easily reconnected to the system by the online degeneracy means.

In the task redundancy execution system according to the present invention, the event notification unit detects that the computer has disappeared,
The task control means is notified of the disappearance. Then, the message management means prevents the message being exchanged from disappearing as the computer disappears. For this reason,
In the system using the task redundancy execution system according to the present invention, it is possible to guarantee that the system is operated safely.

In the task redundancy execution method according to the present invention, the task re-injection means prevents the task from disappearing as the computer disappears. Therefore, it is possible to guarantee that the system using the task redundancy execution system according to the present invention can be safely operated.

In the task redundancy execution system according to the present invention, the information collecting means is performing communication processing of inter-task communication, transmitting processing of inter-task communication, receiving processing of inter-task communication, and inter-task communication. When performing any of the response waiting process and the input / output process between tasks in step 1, the information necessary for mid-execution of the task is collected. Therefore, even if the execution of the task is interrupted in the middle, based on the information collected by the information collecting means,
You can easily re-execute the task from the middle.

In the task redundancy execution system according to the present invention, even when the communication partner of the inter-task communication is on another computer, the information collecting means sends a signal to the information collecting means provided in the other computer. Send and collect the information required for mid-execution of the task. Therefore, for example, when an error occurs in one of the tasks that are being spatially redundantly executed and the execution of the task is interrupted, the task that is being executed by another computer that is being redundantly executed may be interrupted. Information necessary for execution can be easily collected, and a stable task redundancy execution method can be realized.

[0048]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In each node constituting a multi-computer system according to the present invention, a task control means for controlling execution of a task on the node and an execution error detecting means for comparing the execution result of the task and notifying the task control means And an event notification unit for notifying the task control unit of an event that occurs in the system, and an I / F unit for the user task which is a communication unit between the user task and the task control unit.

The task control means has a function of activating a task within its own node, but maintains and manages the task activated by itself so that the existence thereof can be grasped. In other words, the task control means has a task end detection mechanism and can grasp the task being executed. The end of the task on the own node may be detected by the task control means itself or may be notified by the task execution error detection means or the event notification unit, but at the time of task creation and end on the own node, The information is notified to the task control means on another node, and the status of the task being executed is adjusted in the entire node. Therefore, the task control means on one node grasps the task being executed on all the nodes. The message exchange between the task control means is carried out on a one-to-one basis with the task control means on another node by using a reliable communication function provided at a higher layer by the supervisor mounted on each node.

In addition to the above, the function of the task control means includes a process of assigning the input task to a node of the system. Each node has a state for allocating a task related to the occurrence status of a failure in the node, and is managed by the task control means. The state of this node is notified to the task control means on another node every time the state transition of its own node occurs, and the state is adjusted. In the processing of the task control means, the node to which the task is assigned is determined based on the task definition that describes the dynamically changing node state and task attribute. The task definition can be changed even during system operation, and it is managed so that the same contents can be referenced by all nodes.

As described above, the task control means on each node in the present invention has a redundancy management function for providing a spatial / temporal redundancy for the execution of the task in order to secure the reliability of the task, By knowing the tasks being executed on all nodes, the status for task allocation in all nodes, and the attributes of the tasks that are submitted to the system, the task that was submitted by the user task on the local node The control means itself can determine the optimum node for task allocation.

The execution error detection means detects an execution error task based on the voting of the execution result of the task by the S / W means, and operates based on, for example, a voting algorithm in which an error criterion is registered in advance. The execution error detection means has a function of notifying the task control means of the comparison result of the task execution results.

The event notification unit has a function of detecting an event such as the disappearance of another node or an H / W abnormality occurring on its own node, and notifying the task control means. The type of abnormality that occurs in the own node and the means for detecting the abnormality depend on the realization of the node, and the detailed description in the embodiments will not be given in the present invention.

The I / F part with the user task has a function of instructing the task control means to input a task, change the task definition, and change the state of the node, and can be incorporated into the user task. The format is

In the present invention, the following processing is performed in the checkpoint acquisition processing. The task control unit issues a checkpoint collection request, and the task receiving the checkpoint collection request continues processing until intertask communication or an input / output instruction appears. When the above command appears, one of the following processes is performed.

When the inter-task communication (transmission or reception) process is reached, checkpoint data is collected, the checkpoint data is stored in the shared external storage device, and the task is stopped before the communication process.

When the inter-task communication (rendezvous) process is reached, checkpoint data is collected, the checkpoint data is stored in the shared external storage device, and the task is stopped before the rendezvous process.

When the input / output processing is reached, a checkpoint is taken, the checkpoint data is stored in the shared external storage device, and the task is stopped before the input / output processing. When a checkpoint collection request is received during communication processing, the checkpoint data at that time is collected and the task is stopped. Further, when the partner of the inter-task communication is on another node, a checkpoint collection signal is sent to the partner task. The configuration requirements for each node that makes up the multicomputer described above are:
It shall be common to all the examples.

Example 1. In the first embodiment, the task control means of the present invention will be described. FIG. 1 shows a multi-computer system to which an embodiment of the present invention is applied. 1
FIG. 3 is a block diagram of a part that realizes the task redundancy execution system of the present invention in a node (computer, hereinafter referred to as “node”) that is a component of a multi-computer system. Each node is a Front that performs 2 I / O control
I have an End Processor and I / O of 3
Controls four I / O devices and the like connected to the network. In addition, the nodes share the secondary storage device, which is the external storage device of 5, and can be used for tasks executed on the nodes. The task control means 6 on the node is realized as one task executed on the supervisor, and message exchange is performed using the inter-node communication function 7 with the task control means on another node. The I / F part with the user task of 8 can be incorporated into the user task, and the task control means on the own node can use the intra-node communication means to activate the task or
It has a function of changing the definition contents defined in the first task definition section and transmitting a message of node state transition. Similarly to the task control unit 6, the event notification unit 9 and the execution error detection unit 10 on the node are also realized as one task executed on the supervisor. These have a function of notifying the task control means 6 of an event, but can be divided into a plurality of tasks according to the difference in notification information.

In the system, the task definition shown in FIG. 2 is performed for each task to be executed / managed. This task definition is stored in the task definition unit 11 and managed by the task control means so that it is unique in all the nodes.
The description members in the task definition have the following meanings. The task ID 101 is an identifier that distinguishes tasks during system operation. The task processing form 102 is
When a task execution node has a failure or the like and the task execution on the node is canceled, the task control means 6 handles the task as described below. In this system, tasks are classified into natural termination type, forced termination type, forced termination / restart type according to the processing type, and defined in the task processing type 102. According to these classifications, the tasks are treated in such a manner that when the execution node is detached from the system, the task is not involved in the task, is killed, and is restarted by another node after the kill. The task spatial redundancy 103 is for designating the number of tasks to be input to the system when a task activation request is issued. The above spatial redundancy is 1
Performing a task using multiple computers. The spatial redundancy 104 of the task during the degenerate operation is
It specifies the spatial redundancy of the task when the number of nodes decreases. Here, the spatial redundancy required when the number of nodes falls below the border line is specified. The priority execution destination node 105 assigns a priority to a node to which a task is assigned and explicitly specifies the node. If the task allocation node is not explicitly specified, the task control means 6
Node name to know that (for example, "DONTMIN
By assigning D ″), task allocation is performed according to the load distribution.
This is to specify the temporal redundancy in task execution. The above-mentioned temporal redundancy means that one computer processes one task a plurality of times. In the time-redundant execution, the same task is input on the same node until the number of repeated executions 106 is satisfied after the processing of one task is completed. Signal number 1 used for notification when own node fails
07 designates the number of the signal sent from the task control means 6 to the task when the task is brought to the end due to the occurrence of a failure on its own node. The signal number 108 used for the notification of the checkpoint save is a signal sent for the notification of setting the checkpoint save to the task that is normally processed by the task control means 6 at the timing when the redundancy of the task is lost. The number is specified. If necessary, the task receives the signals 107 and 108 that can be detected, and then registers the process to be executed. As a specific example, the OS (supervisor) is UN
When this system is implemented using IX, the task program is provided with a signal detection mechanism, and the process executed when the signal is detected can be defined as a function in the task program. Is. That is, it becomes possible to describe in a part of the program the processing to be performed on the detected signal at a convenient time after detecting the signal. The setting of what to do (which function to execute) when receiving what kind of signal is registered by the program using a supervisor call. The program part to be executed when these signals are received is UN
In the world of IX, it is called a signal handler or a signal catching function. The device bitmap 109 is bitmap data for registering a device used by the task. FIG. 3 shows an example of the device bitmap. According to FIG. 3, the device bitmap can store 32-bit device connection information. 0
Bit 1 is fixed disk, Bit 1 is expansion disk 1
, The bit “1” indicates the connected state, and the bit “0” indicates the non-connected state. In this device bitmap, the devices in the system are associated and operated so as to be uniquely determined, and thus it is possible to prevent the failure of one device from the failure of the entire node. This is because the task control means invalidates only the execution path of the task using the failed device throughout the system. Command name 110 used at task startup
Shows the procedure when the task control means requests the supervisor to start the task. The task execution error procedure 111 indicates a procedure that the task control unit requests the supervisor to execute when an execution error occurs in the task. The procedure 112 used for detecting the execution error of the task is a task activation procedure for detecting the execution error of the task, and is activated on the same node when the task is activated. Also, the task grouping expression 1
13 shows a group of tasks to be executed on the same node. Also, the grouping expression 113 is
There is a plurality for each grouping, and the task I described above
The procedure 112 used for detecting a task execution error from D101 is different definition information.

The above task definition is for each task or
It is possible to add or delete in units of registration for each grouped expression. This uses the I / F unit 8 with the user task,
The task definition, which is character string data, is realized by sending a message to the task control means 6. The task control means 6 that has received the task definition change message at its own node performs the task definition change processing, and then the task control means 6 on the other node.
Notify the change by message.

I / O with the user task to the system
When a task is submitted via the F unit 8, the local task control means 6 that has received the submission request receives the contents of the task definition unit 11, the state of the node resources to which the task can be assigned, and all the nodes. Determines the node to which the task is assigned, depending on the task being performed. When the task allocation node becomes another node, the task control unit 6 transmits a task activation request message to the task control unit 6 on the allocation node by using the inter-node communication function.

The task control means 6 which has activated the task on its own node notifies the task control means 6 on the other node of the event of task activation by sending a message. When the task ends, the task end event is notified to the task control means 6 on the other node by sending a message. This processing is to enable the task control means 6 on each node to grasp the tasks being executed on all the nodes. In this embodiment, the end of the task on the own node is detected by the signal sent by the supervisor.

The operation and details of the task control means 6 in the first embodiment will be described below with reference to the flow charts of the processing shown in FIGS. 4 and 5. First, the initialization process at the time of starting the node shown in FIG. 4 will be described. In step 121,
It shows a process of notifying other nodes that the own node has started. At this time, the newly activated node transmits the activation time of its own node as additional information. In this system, the node activated at the oldest time is the task definition distribution process of step 122 and the step 818 described in the later example when the other node is activated and when it disappears, respectively.
Responsible for the task reconfiguration processing. In step 122, a procedure for obtaining the task definition managed so as to be unique in this system from another node is shown. In the present embodiment, among the nodes that received the activation message in step 121, the task control unit 6 on the node activated at the oldest time transmits the task definition to the new node. In step 123, the task control means 6 of the newly started node inquires of another node about the state for task allocation. Step 124
Then, as a reply to step 123, processing for obtaining a state for task assignment from another node is shown. In step 125, a process of inquiring of other nodes about the information of the task being executed is shown. In step 126, a process of registering the information of the task being executed in another node in the task management mechanism in the own node is shown. Step 127 shows a process of transmitting the status for task allocation in the own node to the task control means 6 on another node.

At the end of the above initialization process, the newly started node obtains the task definition, the task being executed on the other node, and the state for allocating the task of the other node. In addition, the node that has been activated has acquired the state for task assignment of the newly activated node. As a result, the newly started node starts to function as one node in the multi-computer system.

Next, the task input processing in this system will be described with reference to the flow chart of the processing shown in FIG. In step 131, processing for newly inputting a task on the system is shown. The task is input to the system by using the I / F unit 8 for the user task. When the user task calls the I / F unit 8 with the user task,
A task activation message is sent to the task control means 6 on the local node. In step 132, task assignment processing in the task control means 6 is shown. In the task assignment process, the node to which the task is assigned is determined according to the task definition, the state of each node for assigning the task, and the information of the task being executed by all the nodes. In step 133, the process branches depending on whether the node to which the task is assigned is the own node or another node. Step 134 shows the process of requesting the task activation from the supervisor on the own node.
In step 135, the process of registering the information of the task activated on the self node in the task management mechanism of the task control means 6 is shown. In step 136, the process of transmitting the information of the task started on the own node to another node is shown. Step 137 is a process branched from step 133 and shows a process of assigning a task to another node. Here, a task activation request is issued to the task control means 6 on another node. In the task allocation process of the task control means 6 in this system, the I / F with the user task is executed on the local node.
In the task activation requested through the unit 8, an allocation node selection process is performed. On the other hand, the activation request received from the task control means 6 on the other node is immediately executed. This is because each node performs independent task allocation processing with the information necessary for task allocation. In step 138, the process branches depending on whether the number of tasks input to the system satisfies the number described in the task definition.

In step 139, a process for obtaining information on the task activated on another node is shown. This process
Since it is called after the message obtained from the task control means 6 on another node is called, it is an asynchronous process with the series of task input processes. In step 140, the process of loading the information of the task started on another node into the task management mechanism of the task control means 6 is shown.

As described above, the task control means 6 uses the task definition, the state for task assignment of all nodes, and the information of the task being executed in all nodes, and the I / F unit 8 for the user task. In response to a task activation request from a local node made using, it is possible to perform a redundant task assignment process for all nodes.

Example 2. In the second embodiment, a task execution error detection mechanism of the present invention will be described. The execution error detection means 10 in the present system is realized by using a task program, and has a function of voting the execution result of a task executed on each node to determine which task executed on which node caused an error. It is possible to judge. The execution error detecting means 10 in the present system is created according to the execution task, since it is possible to incorporate nodes having different architectures together in the system because the idea of the error differs depending on each task. Further, the execution error detection means 10 sends the result of the vote execution to the task control means 6 of its own node and the I
It has a function of notifying using the / F unit 8.

The operation and operation of the execution error detecting means will be described in detail below with reference to the flow chart of the processing shown in FIG. Step 135 shows the process in which the execution task is activated on the node by the task control means 6 and is the same as the process of step 135 of FIG. 5 described in the first embodiment. Step 201 shows a process in which the task control means 6 activates the execution error detection means 10 on the same node as the execution task input in step 135. The activation process of the execution error detection means 10 means the execution of the procedure indicated by 112 in the task definition of FIG. Step 202 shows a process in which the execution error detecting means 10 collects and compares the execution results of the tasks being executed on the system, and reports the comparison results to the task control means 6. The method of collecting the execution results depends on the method of providing the execution results by the task and is not within the scope of the present invention. The comparison result of the execution results is sent to the task control means 6 as a message by using the I / F unit 8 with the user task. Step 203 branches the processing depending on whether or not the execution task on the own node is completed. In the middle of the processing of the task, for the kind that requires the validity check of the execution result, the processing of step 202 is repeated during the execution of the task.

As described above, by activating the task and activating and executing the execution error detecting means dedicated to the executed task, it becomes easy to monitor the operation state peculiar to the task.

Example 3. In the third embodiment, handling of a task that causes an execution error will be described. As shown in the second embodiment, the result of comparing the execution results of the redundantly executed tasks is notified to the task control means 6 by the execution error detection means 10 on each node. In the task control means 6 that has received the comparison result, the task (instance) that has performed normal processing and the task (instance) that has caused an execution error
And are treated differently. The task control means 6 notifies the task (instance) that has been normally processed that the checkpoint will be saved. Checkpoint save is to secure information necessary for re-execution of a task. On the other hand, for a task (instance) that has an execution error, the redundancy execution means 6a of the task control means 6 notifies the end processing. In this embodiment, a signal registered in advance in the task definition is used as the notification means for these events. That is, the signal number 108 used for notification of checkpoint save during task definition in FIG. 4 is transmitted to the task that has performed normal processing, and the signal number used for notification when the own node fails for the task that caused the execution error. 107 is transmitted. The checkpoint data in the task is set in a secondary storage device (shared secondary storage device 5 in FIG. 1) or the like shared between the nodes as needed, and the task restarted later siphons it and continues processing of the task. I will make it possible. In addition, the task (instance) that processed the error
The task control means 6 on the faulty node that has executed the step transitions its own node to a state in which new allocation of this type of task is prohibited, and notifies other nodes of this state transition. In this state transition, even when other tasks using the device used by the faulty task are executed, there is a possibility of causing a fault on this node, and therefore assignment of those tasks is prohibited. Furthermore, the task control means 6 on the failed node has reduced the redundancy of the task, and therefore the task redundancy execution means 6a additionally inputs this task (instance) into the system. After that, for the task (instance) that was performing normal processing and the task (instance) that was newly submitted, execution is resumed from the checkpoint set by the task (instance) that was performing normal processing earlier.

The operation and operation of the execution error detection means 10 and the task redundancy execution means 6a of the task control means 6 will be described in detail below with reference to the flow charts of the processing shown in FIGS. First, with reference to the flow chart of the processing of FIG. 7, a description will be given of the stopping of a task that has caused an execution error and the task checkpoint acquisition processing in the redundancy executing means 6a and the information collecting means 6j of the task control means 6. Step 3
Reference numeral 01 denotes a process of receiving the comparison result of the task execution results from the execution error detection means 10. Step 30
In No. 2, after analyzing the message obtained from the execution error detection means 10, the process branches depending on whether or not an error has occurred in the execution result of the task. In step 303, the process branches depending on whether or not the task in which the error has occurred is being executed by the own node. Step 304 shows a process of transmitting the signal number 107 for notifying the fault of the own node registered in the task definition to the execution error task in order to stop the task that caused the execution error. In step 305, the state of the own node is changed by the assignment prohibiting means 6c of the task control means 6 so that a task performing the same process as the task in which the execution error has occurred is not newly assigned, and this state transition is performed by another node. The process of notifying the above task control means 6 is shown. Whether or not a certain task performs the same processing as the task in which the execution error has occurred is determined by the device bitmap 109 in the task definition. This is because there is a possibility that an execution error will occur on this node even for other tasks that use the device used by the task that caused the execution error. The subsequent processing is asynchronous with the series of processing performed after the notification of the comparison of the execution results. Subsequent asynchronous processing continues with the start of procedure 2 of the flow chart in FIG.

Step 306 shows the handling in the case where the task in which an execution error has occurred on another node is spatially redundantly executed on its own node and the task is normally processed. . Information collecting means 6 of task control means 6
j sends a signal number 108 used for notification of the checkpoint save described in the task definition to the task being executed on its own node that has lost the redundancy, and instructs the checkpoint save. Subsequent processing is asynchronous with the series of processing performed after the execution error is notified. Subsequent asynchronous processing continues with the start of procedure 3 in the flow chart of FIG.

The redundancy recovery processing of the task which has lost the redundancy will be described below with reference to the flow chart of the processing shown in FIG. First, the processing of procedure 2 in FIG. 8 will be described. Step 310 is a process that is called after the end of the task on the other node is known. Here, the process branches depending on whether the task that was executing on the other node has finished setting the checkpoint data. . In the present embodiment, the task that has performed the checkpoint save ends, and the event is detected by the task control means 6. Step 311 shows a process of notifying the other node of the end of the task forcibly ended in the process of step 304. Step 312 shows a process of recovering the redundancy of the task lost due to the execution error task by the task redundancy executing means 6a. In the present embodiment, since the task in which the execution error has occurred has already been terminated, the task redundancy executing means 6a notifies the task control means 6 of the node that re-executes the task of re-execution of the task. The task control means 6 that received the notification
A task is additionally input to the system by the re-execution means 6k.

Next, the processing of procedure 3 in FIG. 8 will be described. Step 320 is a process which is called in the process of the task control means 6 on the node which was executing the task which has been normally processed, after the end of the task is found to be on the node which is executing the task which caused the execution error. . Here, the process branches depending on whether the task that caused the execution error has ended. Step 321 shows processing for causing a task that has been normally processed to continue processing. In this embodiment, since the checkpoint is set for the task that has been normally processed by the own node before the task is terminated, the task is reintroduced into the system by the re-execution means 6k.

By the above processing, when an error occurs in the redundantly executed task, the normally processed task is caused to checkpoint save and is stopped, the task in which the error is caused is stopped, and the system Since the task is re-executed from the checkpoint set in the normally processed task while the task redundancy is secured, the task that caused the execution error and lost the redundancy recovers the original redundancy and continues. Processing becomes possible.

In this embodiment, the faulty task is temporarily terminated. However, if the checkpoint data of the task that has performed normal processing can be reliably passed to the execution error task, the temporary stop / reexecution processing is performed. You may finish with. Also, for the task that was performing normal processing and the newly submitted task, the task that was performing normal processing previously resumed execution from the set checkpoint, but the newly submitted task is You can restart the process from the beginning. However, in this case, the processing of the task that was performing the normal processing and the processing of the newly input task are asynchronous. Furthermore, both the task that was performing normal processing and the newly submitted task may be re-executed from the beginning. In this case, the checkpoint save is unnecessary and the result in the middle of execution must be discarded.

Example 4. The fourth embodiment will explain the processing when a running task that is not spatially redundant is continuously executed by changing the node. For example, consider a case where H / W on a node is made redundant and a failure occurs in a component of this redundant H / W. In such a node, the failure of H / W is masked, so the effect of the failure does not reach the execution of the task. However, in a node that does not have the online repair function of H / W, the H / W remains without redundancy, and it can be expected that a failure will eventually occur that affects the task being executed on the node. In this situation, if there is no spatial redundancy in the execution of the task, the process takeover data cannot be obtained from other nodes by the method described in the third embodiment, which is a problem. Therefore, it is necessary to perform processing by the task transfer means 6b in which a checkpoint is set for a task that is being executed in its own node, the task is stopped, the task is restarted on another node, and the checkpoint data is taken over.

Although it is at the discretion of the system designer that the task execution does not have the spatial redundancy, as a result of the continued degeneracy of the nodes in this system, the task control means 6 becomes spatially redundant. Such task transfer processing is essential when a task is executed without a certain degree. In this embodiment, a checkpoint save notification for a task uses a signal that can be detected by the task registered in the task definition described in the first embodiment. In this embodiment, the signal number 1 used for notification of checkpoint save in the task definition shown in FIG.
Send 08 to set a checkpoint and then stop. After that, the task is re-introduced into the system.

Below, a detailed explanation of the operation and operation of the task transfer means 6b of the task control means 6 will be given using the flow chart of the processing shown in FIG. In step 401, it is indicated that an event has occurred in which the continuous execution of a task having no spatial redundancy is at risk in the own node. For example, this is a fixed failure in a redundant processor element within a node. The occurrence status of a fixed failure or the like in the redundant H / W detected by the event notification unit 9 is transmitted to the task control means 6 by using the I / F unit 8 with the user task. In step 402, a process of transitioning the state of the own node so that a new task cannot be allocated and notifying the task control means 6 on another node of this state transition is shown. In step 403, the processing is branched depending on whether the processing form of the focused task is the forced termination / restart type. In step 404, a process in which the task transfer means 6b instructs the information collection means 6j to transmit the signal number 108 used for notification of checkpoint save described in the task definition to the task is shown. There is. The task receiving this signal sets the checkpoint in the secondary storage device 5 shared by the nodes and stops. In this embodiment, the checkpointed task ends. Since the process of step 405 is executed after the process of step 404 at the timing at which the end of the task to which the checkpoint is set is detected, the process is discontinuous. In the processing of step 405 in the present embodiment, after the task control means 6 recognizes the completion of the setting of the checkpoint by the notification of the completion of the task from the supervisor, the task transfer means 6b informs the re-input means 6i of the same task in the system. Instruct to recycle. In the task re-input process here, the task is not assigned to the own node by the process of step 402. In this way, the task transfer is performed.

The processing of step 406 is branched depending on whether the processing form of the task is forced termination. A task that is not a forced termination type is a natural termination type task, so leave it alone. The natural end type task is a task that is supposed to end naturally before a secondary failure occurs. In the processing of step 407, in order to forcibly terminate the task, the signal number 107 used for notification at the time of failure of the own node described in the task definition is transmitted.
In this embodiment, the task control unit 6 is allowed to perform the task restart processing. The own task may be re-introduced into the system by using the I / F unit 8 for the user task. Further, in the above description, the task is re-injected on another node and the re-injected task restarts the process from the checkpoint, but the process may be re-executed from the beginning. In this case, checkpoint saving is unnecessary and the execution results up to the middle are also discarded.

As described above, at a node that executes a task having no spatial redundancy, after the occurrence of a redundant H / W failure, a checkpoint save is notified to the executing task and the task is stopped. Since the same task is restarted on the node to enable execution from the checkpoint, the task can be continuously executed without interruption due to an H / W failure or the like.

Example 5. In the fifth embodiment, an example of online diagnosis in this system will be described. In this system, nodes are used as resources for securing the spatial redundancy of each task, but not all nodes in the system have the same device. Further, in order to improve the system availability, the failure of the device added to one node is prevented from the failure of the entire node. For this purpose, the function of diagnosing whether or not the devices included in the node are usable is necessary.

The operation and operation of the online diagnostic means will be described in detail below with reference to the flow chart of the processing shown in FIG. In the processing of FIG. 10, the recovery processing of steps 301 to 305 performed to secure the redundancy of the task after the execution of the task executed on the own node causes an execution error, is described in the third embodiment. Since the processing is the same as the processing, the description is omitted here. In the present embodiment, a diagnostic process executed in the faulty node after the redundancy recovery process will be described. The process of step 501 shows the process of executing the diagnostic procedure by the task control unit 6 after the occurrence of the task execution error. The execution of this diagnostic procedure means the execution of the procedure 111 used for detecting the execution error of the task described in the task definition. For example, in the process of step 501, the task (instance) in which the execution error has occurred can be re-injected into another node. After that, if it can be confirmed by the report from the execution error detection means that the normal execution is continued, the previous execution error that occurred in the task (instance) is instantaneous and the task of this kind is sent to the own node again. It makes a transition to the assigned state and notifies the other nodes of the state transition.

In the present system, after the task causes an execution error, it is considered that the device used by the task is in failure, and the assignment prohibiting means 6c of the task control means 6 causes the task using these devices to newly add a node. It is possible not to be assigned above. Further, the failure part can be specified by executing a diagnostic procedure by the online diagnostic means 6d of the task control means 6 which is executed after the task has an error. By executing the diagnostic procedure, it is possible to distinguish whether the cause of the execution error in the task is the instantaneous failure or the fixed failure of the device.

The device used by the task is registered by the system designer by assigning a value assigned by the system designer so that the device on the system is unique.
As shown in FIG. 3, it is registered by using the bit “0” and the bit “1”. In FIG. 3, bit 0 is a fixed disk, bit 1 is an expansion disk 1, bit 2 is an expansion disk 2, bit 3 is an expansion disk 3, and bit 4 is defined in advance by a printer and a system. "1" is used to represent connection or non-connection. The task control unit 6 holds a device bit map showing a device having a failure for each node, and refers to the device bit map at the time of task allocation processing.

Step 502 shows a process of notifying the task control means 6 of the diagnosis result by the diagnosis procedure executed in step 501 by using the I / F unit 8 with the user task. Step 503 branches the process depending on whether the diagnosis result indicates an instantaneous failure. Step 504 is a process when it is determined that the previous failure is an instantaneous failure, and the status of the own node that has made a transition in step 305 (bitmap data of the failed device).
Is corrected and transmitted to other nodes.

As described above, after a task has made an execution error, on the node that has made an execution error, the assignment of a task that is to use the same input / output device as the task that made the error is prohibited, and , By executing the diagnostic procedure, in the device added to the system,
It is possible to automate online diagnosis. Further, this diagnosis result can be reflected in the task allocation process.

Example 6. In a sixth embodiment, a method of replacing a task module using a task execution destination designation changing function by a task maintenance means will be described. The task control means 6 on the node has a state indicating whether or not a new task can be assigned to each node, and information for determining whether or not a task can be assigned for each individual task (the above-described first and first embodiments). The device bitmap 109) listed in No. 5 is held. In this system, the replacement work of the task modules installed on the node is performed in a state where the task to be replaced is not executed on the work node.

A detailed description of the operation and operation of the task maintenance means will be given below with reference to the flow chart of the processing shown in FIG. In step 601, the process is changed depending on whether the attribute of the task performing the replacement work is the explicit designation type of the execution destination node. Whether or not the execution destination node is an explicit specification type is determined by whether or not the priority execution destination node is specified in the task definition. If there is a specification, it is an explicit specification type. In step 602, the I / F unit 8 for the user task
Is used to indicate the process of changing the task definition. 1
The task control means 6 notifies all nodes of the change in the task definition executed on the node. Due to the processing in step 602, a new task to be replaced is not assigned to the node that performs the task module replacement work. This is useful for tasks of the type that explicitly specify the node on which the task will be executed.

In step 603, the process is branched depending on whether or not the task to be replaced has a symmetrical task in execution at the work node which replaces the task module. In step 604, the process is changed depending on whether the task to be replaced is the explicit designation type of the execution destination node. In step 605, the process of transitioning the state of the node using the function of the I / F unit 8 with the user task and leading the task executed on the own node to the end is shown. The task control means 6 notifies all nodes of the transitioned node states. By the processing in step 605, even a task of a type whose execution destination is not explicitly specified is not newly assigned to the node performing the task module replacement work. In this process, a state similar to that in the case where a failure occurs that makes it dangerous to continue executing the task on the node, and the already executed non-redundant task is transferred to another node. Note that the task transfer processing mechanism has been described in the fourth embodiment.

In step 606, it is confirmed that the task to be replaced is not operating on the node which carries out the task module replacement work, and the task maintenance means 6e carries out the task module replacement work. In step 607, the state of the node for assigning the task is returned to the original state if necessary. This is an operation required when the state is being changed in step 605. The state transition of this node is notified to all nodes. In step 608, the task definition is restored as needed. This is an operation required when the task definition is being changed in step 602. Repair of this task definition is notified to all nodes.

As in the above method, by changing the execution destination designation of a task, it is guaranteed that a specific task is not assigned to a specific node for a specific period, and a task module installed on a node can be safely and easily installed. Can be replaced with.

Example 7. The seventh embodiment will explain a procedure of maintenance work that must stop the node, such as replacement of an H / W module having no hot-swap function and replacement of a supervisor, which is performed by the online degeneration unit and the online expansion unit. . In the system to which the present invention is applied, when the node is stopped, the process being executed on the node is handed over to another node, so that the system is capable of continuous processing.

The details of the operation and operation of the online degeneracy means and online expansion means will be described below with reference to the flow chart of the processing shown in FIG. Step 7
Reference numeral 01 denotes a process for preventing a new task from being assigned to the own node and a process for leading an already assigned task to the end. The state transition here is
All the nodes are notified by the task control means 6, and a new task cannot be assigned to the node performing the maintenance work. At the same time, in the processing of step 701, a state similar to that in which a failure in which continuous execution of a task is threatened occurs on its own node, and a task having no redundancy already being executed is transferred to another node. The task transfer mechanism has been described in the fourth embodiment. In step 702, a process of removing the node performing the maintenance work from the multi-computer system by the online degeneracy means 6f is shown. In the processing here, the task control unit 6 of the own node informs the task control unit 6 of the other node that the own node is normally shut down when all the tasks being executed on the own node are completed. In step 703, maintenance work is performed. Step 70
4 shows a process of incorporating a node for which maintenance work has been completed into the original multi-computer system by the online expansion means 6g. The process of step 704 can be simplified by, for example, creating and registering an activation script so as to be a series of processes in the node reset process.

As described above, after the node is disconnected from the system, the maintenance work of the H / W module and the revision of the supervisor in the node are performed, and the node is rejoined to the system after the work is completed. ,
By using the online degeneration / expansion function of the node, it is possible to take over the processing of the node to other nodes in the system, and stop the node that is a component of the system for maintenance while the system is operating. Work can be carried out.

Example 8. In the eighth embodiment, in the processing of the task control means, a message management means for preventing loss of an inter-node message at the time of abnormal disappearance of a node, and a re-input means of a task scheduled / executed in the abnormal disappearance node explain. The message management means 6h of the task control means 6 in the present invention is characterized in the processing at the time of transmitting and receiving the message by the communication means 7. However, the message between the task control means 6 is transmitted by designating the other party. Since message exchange in this system uses highly reliable inter-node communication means provided by supervisors, it is said that supervisors will not lose data during transmission. In other words, disability is assumed to occur in circumstances where the supervisor cannot participate. The obstacles that can occur when transmitting messages between nodes in this system are: The sending node abnormally disappears while the same message is being sent to another node by the message sending node (same message sent to all nodes for status matching). 2. After the message sending node sends a specific message (a message for performing a specific process to one node), the process expected for the message is lost due to the disappearance of the receiving node. That is.

First, the format of messages exchanged between nodes is shown in FIG. In the format of the inter-node message shown in FIG. 13, the information of each field is as follows. The operation code 801 stores information indicating the command type of the message. Upon receiving the message, the task control means 6 changes the message parsing process according to the data value of this field. Message serial number 80
2 stores the serial number assigned by the task control means 6 of the message issuing node for each message. This field indicates the old and new of the message. The value assigned here is the value counted up for each message generated by the message issuing node, but when the outgoing message targets all nodes, the same number is applied to these messages. The message length 803 stores the total number of bytes of the message. The forwarding message information 804 stores information for indicating the original message sending node when the final message of the disappearing node is retransmitted (proxy transfer) by another node after the message sending node abnormally disappears. The information here is information indicating the original message sending node, the serial number attached to the original message, and the original operation code. This field is secured as much as the transfer history of multiple nodes can be stored in preparation for further proxy transfer of proxy transfer data when the proxy transfer node disappears. The operation code desired data 805 is data required by the operation code 801, and is, for example, data required when a message is analyzed based on the operation code. The identifier of the user who generated the message 806 stores information for identifying the message issuer when the system is a multi-user system. Message body 8
Reference numeral 07 is data transmitted to the task control means 6 on the other node, and the content differs depending on the operation code 801. The process of transmitting / receiving a message in the task control means 6 will be described below.

(A) At the time of message transmission A message that changes the system state, such as a new task activation request, is saved for each destination node after the transmission procedure. These stored messages are discarded after the destination node returns an ACK indicating that the message has been accepted. The ACK from the other node in response to the task activation request to the other node can be substituted by, for example, the result of the task activation process, and the content of the message indicates the information of the activated task or the reason why the task activation failed. It is something to convey. Thus, in the message between the nodes that changes the system state, ACK is returned to the requesting node after the processing at the receiving node is completed.

(B) When receiving a message The source node is distinguished and the last received message is saved. The message stored here is used for the state matching process of delivering the message transmitted last by the disappearing node to another node when the node abnormally disappears.

In the detection of the disappearance of another node in this embodiment, the event notification unit detects and notifies the task control means 6.

The detailed operation and operation of the message management means and task re-input means will be described below with reference to the flow chart of processing shown in FIG. In step 811, a process of detecting the disappearance of a node on the system is shown. The event notification unit 9 arranged on each node detects the disappearance of another node by the existence check in the inter-node communication unit, and then controls the event of the node disappearance using the I / F unit 8 with the user task. Notify the means 6. In step 811a, the task control unit 6 controls so that a new task is not assigned to the node that has abnormally disappeared. In step 812, the task control means 6 branches the processing depending on whether or not the disappeared node has ended normally. When the node is normally disconnected and shut down, the task control units are in a state-matching state, so that it is possible to determine the normal end. In step 813, the message management unit 6h performs the message transmitted last by the abnormally disappeared node to all nodes except the abnormally disappeared node (and its own node). In the process when it becomes the recipient of the message, the unprocessed message that the abnormally disappearing node was supposed to send to its own node based on the message notification number at the message issuing node and the identifier that announces the proxy transmission in the message header Only
It is possible to take out selectively. While the message transmission in the task control means 6 is performed by the highly reliable destination designation method provided by the supervisor, when the message transmission node disappears in the middle of the process of transmitting the message to all the nodes, the nodes in the system are connected. A state mismatch occurs. This process is performed to correct this mismatch. This correction process is not based on the policy of the process of the task control means 6 reversibly invalidating the process by the received message, but by adjusting the state of the message unreceived node to the same state as the received node.

In step 814, the message exchanging means 6h processes the final message of the extinct node sent by another node as if it had been sent by the extinct node. Based on the message type and the serial number of the message embedded in the information of the transfer message 804 in the message, it is determined whether or not to process the delegated message. Ignore messages that are obviously not for me and messages that are the same or older than the messages I have already processed.

In step 815, the message exchanging means 6h represents a process of reprocessing a message without an ACK after the completion of the message transfer phase by another node and delivering it to another node if necessary. In the processing here, the execution request of the task is issued to the extinction node, but the result of the above processing that there is no evidence that the task is activated in the extinction node is assigned to another node instead.

In step 816, the task re-injection means 6i determines a node for reconfiguring a task having no spatial redundancy which was being executed by the disappeared node. In the present system, the task control means 6 on each node grasps the task being executed on the system, and therefore this reconfiguration processing is possible on any node. Rather, arbitration work for deciding one node to perform the reconstruction process is required. In the present embodiment, selection is made based on the node start start time in the message issued by the node at the time of start, which is shown in the processing of step 121 of the first embodiment. For this activation start time, the time obtained from the clock local to the node may be used, and the node showing the oldest activation time among the surviving nodes is selected as the reconfiguration processing node.

In step 817, the processing is changed depending on whether or not the own node performs the reconfiguration processing of the task having no spatial redundancy which was being executed by the disappearing node.

In step 818, the task re-injection means 6i performs a process for restarting a task having no spatial redundancy executed in the disappeared node on another node, if necessary. Note that step 81 described above
1 to step 815 are processes by the message management means 6h, and steps 816 to 818 are processes by the task re-input means 6i.

As described above, the message management means for preventing the loss of inter-node messages when a node abnormally disappears, the task scheduled to be executed in the disappearing node, or the spatial redundancy that was being executed. By the re-injection means for re-injecting a task that has not been performed, it is possible to guarantee the state matching between the nodes even when the node abnormally disappears, and further it is possible to prevent the loss of the task that has already been submitted and has no spatial redundancy.

Conventionally, in a fault tolerant system, it is important to prevent data loss at the time of failure.
On the other hand, like this system, messages exchanged in a fault-tolerant system consisting of multiple nodes can be discarded if they are not delivered to the destination depending on the content, and those that need to be processed by a proxy node. It is divided into In the present invention, the above failure 1 is dealt with by proxy transmission performed by another node (step 81).
(3, 814) and 2) the transmitted message is retained, and when the message is not processed after the destination node abnormally disappears, the process is reassigned to another node, and the system is reassigned. The loss of the issued message as a whole is prevented (step 815). here,
That is, the "task that was scheduled to be executed on the disappearing node" is reassigned to the normal node. By the above process, all messages that the faulty node was trying to convey,
All messages attempting to reach the failed node are reflected throughout the system. The remaining work is system reconfiguration (task reassignment processing that was being executed in the failed node). Inheriting the process being executed by the faulty node means executing the task being executed by the faulty node by another node.
The recovery process after this will be task-specific, and if it is necessary to avoid executing duplicate instructions, it is necessary to devise a method for taking over, such as setting a checkpoint during normal execution of the task. You will need it.

Example 9. In the ninth embodiment, the operation of the checkpoint mechanism of the information collecting means of the present invention will be described with reference to the drawings. In this embodiment, the checkpoint save processing described in the third embodiment is shown. FIG. 15 is a flow chart showing processing when a task receives a checkpoint collection request. When the task receives a checkpoint collection request from the information collecting means 6j of the task control means 6, the task checkpoint processing (1000) is started.
First, when the task receives a checkpoint collection request, it is checked whether communication between the tasks was performed (10
01). When the task is not performing inter-task communication, the task processing is continued as it is until inter-task communication or input / output processing (1002). When the inter-task communication or the input / output process is reached, it is checked whether the process is inter-task communication (transmission / reception) (1003). If the process is not inter-task communication (transmission / reception), it is checked whether it is inter-task communication (rendezvous) (1004). The rendezvous indicates a state in which the task that sent the message waits for a response from the destination task. If the process is not task-to-task communication (rendezvous), the process is an input / output process, and therefore a checkpoint process (1040) for input / output is performed. In the processing of 1001, when a task is performing inter-task communication, a checkpoint process (1010) during inter-task communication is performed. In the processing of 1003, when the arrived processing is inter-task communication (transmission / reception), checkpoint processing (1020) in inter-task communication (transmission / reception) is performed. In the processing of 1004, when the arrived processing is inter-task communication (rendezvous), checkpoint processing (1030) in inter-task communication (rendezvous) is performed.

FIG. 16 shows a flow chart of the checkpoint processing (1010) during communication between tasks. In the checkpoint processing during inter-task communication, first, it is checked whether the other party of inter-task communication is on the same node (101
1). At this time, when the other party of communication is on a different node, a request is issued to the information collecting means 6j of the task managing means 6 of the node on which the other party is operating to collect the checkpoint of the other task (1012). . Next, the information collecting unit 6j of the node requested to collect the checkpoint collects the checkpoint there (1013) and stores the collected checkpoint data in the shared secondary storage device 5 (1014). The task control means 6 shares the checkpoint data with the secondary storage device 5.
Then, execution of the task is stopped after saving the
5).

Next, the procedure of the checkpoint process (1020) when reaching the intertask communication (transmission / reception) will be described. The procedure of the process is the same as the checkpoint process during the intertask communication of FIG. Therefore, a description will be given according to the flowchart of FIG. In the checkpoint processing when reaching the inter-task communication (transmission / reception), first, it is checked whether the other party of the inter-task communication is on the same node (10
11). At this time, when the other party of communication is on a different node, a request is issued to the information collecting means 6j of the task managing means 6 of the node on which the other party is operating to collect the checkpoint of the other task (1012). . Next, the information collecting unit 6j of the node requested to collect the checkpoint collects the checkpoint there (1013) and stores the collected checkpoint data in the shared secondary storage device 5 (1014). The task control means 6 shares the checkpoint data with the secondary storage device 5.
Then, execution of the task is stopped after saving the
5).

Next, the procedure of the checkpoint process (1030) when reaching the intertask communication (rendezvous) will be described. The procedure of the process is the same as the checkpoint process during the intertask communication of FIG. Therefore, FIG.
A description will be given according to the flowchart of FIG. In the checkpoint process when reaching inter-task communication (rendezvous), first,
It is checked whether the other party of the inter-task communication is on the same node (1011). At this time, when the other party of communication is on a different node, a request is issued to the information collecting means 6j of the task managing means 6 of the node on which the other party is operating to collect checkpoints of the other task (101).
2). Next, the information collection information 6j of the node requested to collect the checkpoint collects the checkpoint there (1013) and stores the collected checkpoint data in the shared secondary storage device 5 (1014).
The task control means 6 shares the checkpoint data 2
After saving in the next storage device 5, the execution of the task is stopped (1015).

FIG. 17 shows a flow chart of the checkpoint processing (1040) when the input / output processing is reached. In the checkpoint process when I / O process is reached, first,
The information collecting means 6j collects checkpoints there (1041) and stores the collected checkpoint data in the shared secondary storage device 5 (104).
2). Then, the task control means 6 saves the checkpoint data in the shared secondary storage device 5, and then stops the execution of the task (1043).

In the flow chart of FIG. 16, the checkpoint data is stored in the shared secondary storage device 5 and then the execution of the task is stopped. However, the task may be executed without being stopped. In this case, the out-of-place task becomes asynchronous with the task that was executed midway after the checkpoint save.

As described above, in this embodiment, each task continues the process until it receives the inter-task communication process after receiving the checkpoint collection request from the system or from another task. When the inter-task communication process is reached, if it is a transmission process, the checkpoint data is stored in the shared secondary storage device 5, and the task execution is stopped. Therefore, no new inter-task communication (transmission) occurs after receiving the checkpoint collection request. When the reception process of the inter-task communication is reached, the checkpoint data is stored in the shared secondary storage device 5 and the reception process is performed.
When the reception process is successful, the process is further continued. When the reception process fails, the task process is stopped. As a result, when the task receives the checkpoint collection request, it will receive the data of the inter-task communication that has already been sent without leaking, and there will be no unresolved inter-task communication. Stop at the receiving process. When the rendezvous process of inter-task communication is reached, the checkpoint data is stored in the shared secondary storage device 5 and the rendezvous process is performed. When the rendezvous process is successful, the checkpoint data is stored again in the shared secondary storage device 5. Then, the execution of the task is stopped.

As described above, according to the above-described first to ninth embodiments, in the task redundancy execution method according to the present invention, the normal processing is performed when a failure occurs in the task which is being redundantly executed. Since the checkpoint save is obtained from the task that is doing the work and the lost redundancy is restored, the process from the checkpoint is continued, which is excellent in the process of separating and recovering from the abnormality due to the failure. In the system to which the present invention is applied, control information can be given to the task control means 6 from the task definition unit 11 that stores the task attributes by using the user task I / F unit 8.
Furthermore, it is possible to control the state transition of the node that executes the task. Therefore, it is possible to control the process of removing the specific task from the node and the process of assigning the specific task to the node.
Online degeneration / expansion of nodes is possible. In the present system, the event notification unit 9 on the node can be customized according to the H / W implementation method of each node and separated from the task control means 6, thereby facilitating connection of heterogeneous nodes. Furthermore, by defining the task execution error detection means 10 for each task, it is possible to apply according to the task to be input to the system.

[0119]

As described above, according to the present invention, based on the information stored in the task definition section by the task control means,
Control the execution of tasks. For this reason, if the task definition unit stores information for realizing the redundant execution of the task, the task control means can easily perform the redundant execution of the task.

Further, according to the second invention, the task definition section stores the information to be executed by making the task redundant in time. Therefore, there is an effect that the task control unit that refers to the task definition unit and controls the execution of the task can realize the execution of the task that is temporally redundant.

Further, according to the third invention, the task definition section stores the information to be executed by making the task spatially redundant. Therefore, there is an effect that the task control means for controlling the task execution with reference to the task definition section can realize the spatially redundant task execution.

Further, in the fourth invention, the task control means receives the error notification of the task by the execution error detection means, and according to the received content and the definition content of the task definition section,
Control the execution of tasks. In addition, control at the time of error detection is defined in advance in the task definition section. Therefore, even if an error occurs during task execution, the task control means takes appropriate action and controls the execution of the task, which has the effect of increasing the diversity of the system.

According to the fifth invention, the task redundancy executing means restarts the task so as to secure the redundancy of the task. Therefore, even if an error occurs during execution of the task, the redundancy of the task is ensured, which is effective in increasing the diversity of the system.

According to the sixth invention, the redundancy executing means restarts the task in which the error has occurred in another computer. Therefore, the redundancy of the task is secured, which has the effect of increasing the diversity of the system.

According to the seventh invention, when the task transfer means fails in the redundant hardware constituting the computer, the task being normally processed by the computer is once stopped, Restart the above task on another computer. For this reason, it is possible to safely eject a task that is already being executed on the computer to the outside of the computer, which has the effect of increasing the diversity of the system.

Further, in the eighth invention, the information collecting means is
The information necessary for mid-execution of the task is collected, and the collected information is stored in the external storage unit. Then, the re-execution unit retrieves the information from the external storage unit and re-executes the task from the middle based on the retrieved information. For this reason, even if the execution of the task is forced to be interrupted due to the task error, the re-execution can be easily performed, and the redundancy of the task can be reliably realized.

Further, in the ninth invention, the allocation prohibiting means allocates another task using the input / output device to the computer when an error is detected in the input / output device constituting the computer. Ban. Therefore, it is possible to prevent the task error from occurring again due to the input / output device, which has the effect of increasing the diversity of the system.

According to the tenth aspect of the invention, the online diagnostic means diagnoses in which input / output device the error has occurred. Therefore, since the input / output device in which the error has occurred can be easily identified, the task allocation process that reflects the diagnosis result can be performed by the allocation prohibition means, and the diversity of the system can be enhanced. .

Further, in the eleventh invention, it is guaranteed that the task maintenance means does not allocate a specific task to a specific computer for a specific period of time, and the module stored in the external storage unit is taken out to execute the specific task. Replace with. For this reason, software maintenance work can be performed without stopping the system, which has the effect of increasing the diversity of the system.

In the twelfth aspect of the invention, the online degeneracy means stops the task normally processed on the computer, disconnects the computer from the system, and restarts the task on another computer. Therefore, in hardware maintenance work, maintenance can be performed without stopping the system, which has the effect of increasing the diversity of the system.

Further, in the thirteenth invention, the online expansion means re-inputs the separated computer into the system. Therefore, in the twelfth aspect of the present invention, it is possible to easily reload the hardware for which maintenance work has been completed without stopping the system, which has the effect of increasing the diversity of the system.

Further, in the fourteenth invention, the message management means receives the disappearance of the computer from the event notifying unit and prevents the message from disappearing due to the disappearance of the computer.
Therefore, the task-to-task communication is effectively secured.

In the fifteenth invention, the task re-input means prevents the task from disappearing as the computer disappears. Therefore, similar to the fourteenth invention, there is an effect of guaranteeing that the inter-task communication is safely performed.

Further, in the sixteenth invention, the information collecting means collects information necessary for mid-execution of the task. Then, the sampling timing is during communication processing of intertask communication, transmission processing of intertask communication, reception processing of intertask communication, response waiting processing in intertask communication processing, and task input / output. This is the case where any one of the processes is performed. For this reason, it is possible to collect the information necessary for the mid-execution of tasks synchronized between tasks.
There is an effect that information can be collected efficiently and safely.

Furthermore, in the seventeenth invention, even if the communication partner of the inter-task communication is on another computer, the information collecting means sends a signal to the other computer to collect information necessary for mid-execution of the task. To send. Therefore, in the case of a task that is spatially redundantly executed, there is an effect that the information necessary for intermediate execution can be easily collected.

[Brief description of drawings]

FIG. 1 is a diagram showing a first embodiment of the present invention.

FIG. 2 is a diagram showing definition data of task attributes executed / managed in this system.

FIG. 3 is a diagram showing an example of a device bitmap.

FIG. 4 is a flowchart showing a processing procedure of initialization processing at node startup.

FIG. 5 is a flowchart showing a processing procedure when a task is input to the present system.

FIG. 6 is a flowchart showing a processing procedure of a task execution error detection method in the present system.

FIG. 7 is a flowchart showing the first half of the processing procedure when a task that is being redundantly executed causes an execution error.

FIG. 8 is a flowchart showing the first half of the processing procedure when a task that is being redundantly executed causes an execution error.

FIG. 9 is a flowchart showing a processing procedure of transferring a task that is not spatially redundantly executed to another node.

FIG. 10 is a flowchart showing a processing procedure of node diagnosis.

FIG. 11 is a flowchart showing a processing procedure when replacing task modules on a node in the present system.

FIG. 12 is a flowchart showing a processing procedure when performing maintenance work on a node in the present system.

FIG. 13 is a diagram showing a format of a message exchanged in the present system.

FIG. 14 is a flowchart showing a recovery method of processing relating to an abnormally disappeared node.

FIG. 15 is a flowchart showing a procedure of processing when a task receives a checkpoint collection request in the present invention.

FIG. 16 is a flowchart showing a processing procedure when a task reaches inter-task communication processing (transmission / reception) in the present invention.

FIG. 17 is a flowchart showing a process when a task reaches an input / output process in the present invention.

FIG. 18 is a diagram showing a configuration of a conventional FT computer.

FIG. 19 is a block diagram of a conventional multiple computer architecture.

FIG. 20 is a configuration diagram showing a conventional checkpoint mechanism.

[Explanation of symbols]

1 node, 2 front end that controls I / O
Processor, 3 I / O network, 4
I / O devices, 5 shared secondary storage devices, 6 task control means, 7 internode communication means, 8 I / O with user tasks
F part, 9 event notification part, 10 execution error detecting means, 11
Task definition section.

Front Page Continuation (72) Inventor Kaoru Abe 5-1-1 Ofuna, Kamakura City Mitsubishi Electric Corporation

Claims (17)

[Claims] 1. A task redundancy execution system for a multi-computer system configured by a plurality of computers, wherein each computer has the following elements: (a) a task redundancy execution system; A task definition unit that stores in advance information for managing the task, and (b) a task control unit that controls the execution of the task based on the information stored in the task definition unit. 2. The task redundancy execution method according to claim 1, wherein the task definition unit stores information that makes a task redundant in time and executes the task. 3. The task redundancy execution method according to claim 1, wherein the task definition unit stores information that makes the task spatially redundant and executes the task. 4. The computer further comprises execution error detection means for detecting an error occurring during task execution, and notifying the task control means of the detected error, and the task definition part is provided for detecting an error. 4. The task according to claim 1, wherein control is defined in advance, and the task control means controls the execution of the task based on the notified content and according to the definition content of the task definition unit. Redundant execution method. 5. The task control means restarts the task so as to secure the redundancy of the task, when the execution error detection means notifies the error of the task being executed on a specific computer. 5. The task redundancy execution method according to claim 4, further comprising: 6. The redundancy execution means stops the execution of a task on a computer in which an error of the task being executed has occurred, and restarts the task on another computer. Redundant execution method for the listed tasks. 7. If the computer is configured by redundant hardware, and the hardware fails, and the computer is executing a task without spatial redundancy, the task control means The task redundancy means according to claim 1, 2, 3, 4 or 5, further comprising: task transfer means for once stopping a task which is being normally processed on the computer and restarting the task on another computer. Execution method. 8. The task redundancy execution method further comprises an external storage unit for storing information necessary for mid-execution of a task, and the task control means is required for execution of a task from a normal processing to a mid-execution of the task. Information collecting means for collecting various kinds of information and storing the collected information in the external storage unit, and extracting information necessary for mid-execution of the task from the external storage unit, and executing the task from the middle based on the extracted information. 8. The task redundancy execution system according to claim 1, further comprising a re-execution means for re-execution. 9. The task redundancy execution system further comprises a plurality of input / output devices, and the task control means performs input / output operation when an error is detected in a task being executed by the execution error detection means. 5. The task according to claim 1, further comprising an allocation prohibition means for prohibiting allocation of another task using an input / output device, which is scheduled to be used by the task, to the computer. Redundant execution method. 10. The task definition unit includes a diagnostic procedure for diagnosing which of the input / output devices has an error, and the task control unit executes the diagnostic procedure. 10. The task redundancy execution system according to claim 9, further comprising an online diagnostic means. 11. The external storage unit further stores a module, and the task control unit ensures that a specific task is not allocated to a specific computer for a specific period of time, and is stored in the external storage unit. 2. A task maintenance means for taking out a module and replacing the taken-out module with the specific task is provided.
~ Redundant execution method of the tasks described in 4 above. 12. The task control means comprises an online degeneration means for stopping a task which is normally processed on the computer, disconnecting the computer from the system, and re-mobilizing the task on another computer. The redundant task execution method according to claim 1, 2, 3, 4, or 5. 13. The task redundancy execution system according to claim 12, wherein said task control means further comprises an online expansion means for re-inputting the separated computer into the system. 14. The plurality of computers further comprises communication means for exchanging messages between the computers, and
An event notification unit that detects the disappearance of a computer that is communicating and notifies the task control unit of the disappearance is provided, and the task control unit is a computer based on the notification content from the event notification unit. 4. The task redundancy execution method according to claim 1, further comprising a message management unit for preventing the message from disappearing when the message disappears. 15. The task redundancy execution method according to claim 14, wherein the task control means further comprises task re-input means for preventing the task from disappearing as the computer disappears. 16. The information collecting means performs any one of processing during communication processing of inter-task communication, reception processing of inter-task communication, response waiting processing in inter-task communication, and task input / output processing. 9. When performing the above, the task redundancy execution method according to claim 8, wherein information necessary for mid-execution of the task is collected. 17. The information collecting means sends a signal to the information collecting means provided in another computer when the communication partner of the inter-task communication is on another computer, so that the information necessary for mid-execution of the task is transmitted. 17. The task redundancy execution method according to claim 16, wherein the task redundancy execution method is performed.