JPH04372017A - Computer testing system - Google Patents

Abstract

PURPOSE:To generate a test case capable of testing the execution order of instructions and the combination of instructions to be used as well as performing a test automatically and quickly. CONSTITUTION:This computer testing system is provided with an order defining means 10 equipped with an instruction execution order definition part 1 defining the execution order of the instruction of the computer system to be tested, an instruction information part 11 equipped with an instruction information file 4 storing the instruction information of each instruction of the computer to be tested, an instruction selection part 3 selecting the instruction information stored in the instruction storage part according to the attribute of the instruction defined by the order defining means, a generating means 12 equipped with a test case generating part 5 generating the test instruction string.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to a computer system testing method that focuses on the order of execution of instructions. It concerns a test method for generating . Even for the same instruction, H/
The operation of W is different. In other cases, a program may be executed by modifying its own code. The present invention relates to a test method for efficiently testing such a computer system.

0002

2. Description of the Related Art FIG. 8 is an example of a test focusing on the execution order of simple instructions. 50 is a microprocessor, 51 is a microprogram in the microprocessor, and 4
0 is the microprogram 51 of the microprocessor 50
This is a sequence of instructions to test. The instruction string 40 includes a plurality of instructions (
I1 to I5). Also, instruction string 4
0 indicates that the memory M1 is used as an operand. Also, as a test, the instruction I2 writes data to the memory M1, and the instruction I4 reads data from the memory M1. Conventional tests that take execution order into consideration have been performed by manually arranging instructions based on the function to be tested. For this reason, there was a limit to the number of test cases that could be created. Furthermore, if, for example, a store instruction is used as the instruction to allocate to the location of the instruction I2 that writes data to the memory, there is a high possibility that instructions to write data to the memory in other instruction sequences (not shown) will also use the same store instruction. The instructions used in the exam were limited.

0003

[Problem to be Solved by the Invention] In conventional test case generation methods, tests that take into account the execution order of instructions are performed using a limited number of instructions, so tests are only performed based on the execution order, and the It was not possible to sufficiently test the combinations of instructions used. For this reason, it was not possible to ensure sufficient quality.

[0004] In order to solve the above problems, the present invention is capable of testing the execution order of instructions and combinations of instructions to be used, and can automatically generate test cases for carrying out the tests in a short time. The purpose is to obtain a possible computer system testing method.

[0005]

[Means for Solving the Problems] A computer system testing method according to the present invention has the following elements. (a) Order definition means that defines the execution order of test instructions of the computer system to be tested based on the attributes of the instructions; (b) Instruction information that stores instruction information regarding each instruction of the computer to be tested based on the attributes of each instruction. (c) generation means for selecting instruction information stored in the instruction storage section according to the instruction attributes defined by the order definition means and generating a test instruction sequence for a computer to be tested;

[0006]

[Operation] The computer system test method according to this invention is
Based on the instruction execution order specified manually or randomly determined by a program, the order definition means specifies the instruction execution order with instruction attributes, and the generation means can combine from the instruction attributes. This method generates a test case that executes a test using instructions that are conscious of the instruction execution order, and then executes a test using the generated test case. By specifying the instruction execution order using the instruction attribute in this way, when testing the instruction execution order, it becomes possible to test combinations of instructions related to the instruction attribute, thereby eliminating the possibility of omitting tests regarding the instruction execution order and instruction combinations. It disappears.

[0007]

【Example】

Example 1. FIG. 1 is a schematic diagram of this test method. In the figure, 1 is an instruction execution order definition part that determines the instruction execution order, 2 is an instruction execution order file that describes the execution order of instructions to be tested, and the file contains an instruction execution order information table 21. and an instruction attribute table 22. 4 is an instruction information file in which instruction information is recorded; 3 is an instruction selection unit that selects a suitable instruction from the instruction information file 4 based on instruction attributes; and 5 is an instruction selection section that selects a suitable instruction from the instruction execution order file 2. 6 is a test case generation unit that generates test cases, and 6 is a test case file in which the generated test cases are written. 10 is an order definition means for defining the execution order of the instructions of the computer system to be tested based on the attributes of the instructions; 11 is an instruction information unit for storing instruction information regarding each instruction of the computer to be tested based on the attributes of the instructions; Reference numeral 12 denotes a generation means for selecting instruction information stored in the instruction storage section according to the instruction attributes defined by the order definition means and generating a test instruction sequence for the computer to be tested.

Next, FIG. 2 is a diagram for explaining the contents of the execution order file 2 of instructions for carrying out a test. The contents of the explanation will be based on an example of a test focusing on the execution order of instructions in the conventional method shown in FIG. 6, and the contents of the execution order information table 21 and the instruction attribute table 22 will be explained. An instruction attribute exists for each instruction in the execution order information. Instruction execution information for instruction I1 is assigned to S1 and instruction attribute A1. Here, the address where the instruction attribute A1 is recorded is stored in S1 as a "pointer to A1." In A1, an attribute "memory write" is recorded as an instruction attribute. Similarly, S2 and A2 are assigned to I2. In addition, in the case of the same instruction attribute, it is also possible to share the instruction attribute of A2, such as I2 and I5. In this way, S1 to S5 indicate the order of execution, and by recording pointers to the attributes of each instruction, they indicate the attributes of the instructions to be executed in order.

FIG. 3 is a diagram for explaining the contents of the command information file 4. In the figure, 41 is attribute information, 42 is each instruction given to the computer system to be tested, 4
3 is attached information attached to each instruction 42 such as operands and parameter conditions.

[0010] The operation of this test method will be explained. Test items related to the instruction execution order are created by an instruction execution order definition unit 1 that determines the instruction execution order. The execution order may be determined randomly by a program or manually from a terminal or the like. When determining the instruction execution procedure by a program, the operation of the computer system to be tested is
Set conditions in advance. When specifying manually, for example, the order of attribute information A1 to A4 shown in FIG. 2 is displayed on a display, and the execution order is specified using an input device such as a keyboard. Now, if the operator wants to execute "memory write" first and specifies the attribute information of A1, the instruction execution order definition unit 1 sets a pointer to A1 in S1 of the execution order information table 21. . Next, when the operator wants to "change the register", he specifies the attribute information of A2, and a pointer to A2 is set in S2. Similarly, a pointer is set in S3 as well. Next, in the case of S4, when the operator specifies that any command is acceptable without attaching any conditions to the command, "no designation" is stored and a pointer to the attribute information is not generated. Furthermore, S5 indicates the case where the pointer to A2 is set as described above. At this point, the instruction execution order definition unit 1 has set information indicating that instructions are to be executed in the following order in the instruction execution order file 2, and the state shown in FIG. 2 is created. 1. Memory writing 2. Register change 3. Reading memory 4. Not specified5. register change

Next, the test case generating section 5 generates a test case based on this instruction execution order file 2. The operation of the test case generation section 5 will be explained with reference to the flow shown in FIG. First, in F1, the instruction execution information table 21 of the specified instruction execution order file 2 is searched. Next, in F2, an instruction attribute is searched based on the instruction execution information table 21 searched in F1. For example, in this example, from S1 to A
1 and obtain attribute information A1. Next, F
In step 3, if the instruction attribute is not specified in the instruction execution information table (for example, in the case of S4), in step F5, the instruction is sent to the instruction selection unit 3 without any conditions regarding the instruction (any instruction is possible). Make a selection request. Further, if an instruction attribute is specified such as S1, S2, S3, or S5, a request for selecting an instruction is made to the instruction selection section 3 based on the instruction attribute and accompanying conditions in F4. Note that when a command selection request is made at F4, the requested command may not be available. In this case, check whether the requested instruction exists at F6, and if it does not exist, stop generating the test case for the specified instruction execution order file at F8 and finish the test case generation process. . If an instruction exists, the instruction is generated in F7 as follows. At F7, the instruction selection unit 3 obtains corresponding instruction information from the instruction information file 4 based on the instruction attributes and outputs it to the test case generation unit 5. The test case output unit 5 generates an image of an instruction, initial values of memory and registers, and expected values based on conditions attached to instruction attributes and this instruction information. In reality, it allocates memory, registers, etc., and sets values. At this time, processing such as using the same resource (memory, etc.) from multiple instructions is also performed. When generation of one instruction is completed, the next instruction execution information is searched for in F9. If there is instruction execution information at the next step F10, the process jumps to F2 and generates the next instruction. Furthermore, if the instruction execution information does not exist, the process of F11 is performed as the test case termination process. In F11, initial values and expected values of registers and memory used in the generated test case are set.

Next, the operation of the instruction selection section 3 when an instruction selection request is issued at F4 and F5 will be explained. The instruction selection section 3 selects an instruction from the instruction information file 4 based on a request from the test case generation section 5 to select an instruction that matches the instruction attributes. Further, the selected instruction is passed to the test case generation section 5 in a format that allows the test case generation section 5 to generate an image of the instruction. In this format, areas unrelated to the operation of instructions, such as registers and memory, are left in an empty format, and the test case generation unit 5 can set information in the empty format portions.

For example, if a request to select the instruction in charge of S1 is generated from the operation of F4 of the test case generation unit 5, the instruction selection unit 3 selects the attribute “memory write” because the attribute information of S1 is A1. Select one instruction from the instruction information file 4. As shown in Figure 3, “
STORE and PU for instructions with the attribute “memory write”
Assuming that there are T, etc., the instruction selection section 3 randomly passes one instruction, for example, STORE, to the test case generation section 5. At this time, the register designation and STORE destination address necessary for the STORE command are passed without being filled in.

Similarly, in the case of S2, since the attribute information of A2 is specified, a command is randomly selected from among the instruction group (ADD, SUB, AND, OR,...) in the attribute information of "register change". One instruction is selected. The same applies to S3 and S5, but in the case of S4, the attribute information is "not specified", so the instruction selection unit 3 freely selects any instruction from among the instructions recorded in the instruction information file 4. .

FIG. 5 shows an example of a test case generated in this manner. In the figure, 51 is an initial value setting command generated by the test generation unit 5, and 52 is S1 to S
An instruction string 53 is an expected value test instruction for testing the execution result of the instruction string. Each command is selected by the command selection section 3 as follows. STORE instruction 52a...randomly selected from the instruction group with attribute information "memory write" ADD instruction 52b...randomly selected from the instruction group with attribute information "register change" LOAD instruction 52c...randomly selected from the instruction group with attribute information "memory read" Selection SHIFT instruction 52d...Arbitrarily selected from a group of instructions in the instruction information file ADD instruction 52e...Randomly selected from a group of instructions with attribute information "Register change" Note that the test case 6 written by the test case generation unit 5 is In some cases, it is executed immediately in the execution section, and in other cases, multiple test cases are collectively executed at a later time. In the above embodiment, the instruction selection section 3 selects instructions at random, but there are other selection methods as follows. 1. Select commands in order from a group of commands. 2. A specific instruction that is particularly desired to be tested is selected by designation. 3. Select all instructions in the instruction group and create test cases for all cases. (However, in this case, the test case generation unit 5 also needs to have the function of generating all such cases.)

Example 2. FIG. 6 shows an example in which an operand attribute table 23 is added to the instruction execution order file 2 compared to the first embodiment. The operand here is
It refers to the memory and registers used in the instruction, and also includes the internal H/W conditions such as the instruction memory and cache in addition to the memory and registers used in the instruction. It also includes conditions for operands between instructions. For example, the operand attribute table 23 records information related to operands, such as register numbers, cache usage conditions, address conversion table status, and address duplication designations for memory to be used, divided into attributes.

FIG. 7 is a diagram showing the contents of the operand attribute table 23, and the arrows from S1 to S5 to the operand attributes P1 to P5 in the operand attribute table 23 are S1.
~S5 shows the pointer recorded. This P1~
The pointer to P5 is set by the instruction execution order definition unit 1 according to an operator's designation or a program's designation.

In this second embodiment, when generating a test case, operands are generated to match the operand attributes specified in the operand attribute table 23. Therefore, in the first embodiment, the test case generation unit 5 was able to generate operands arbitrarily, whereas in the second embodiment, an additional feature is that a test case can be created by specifying operands as well.

As described above, this embodiment provides a function to determine the instruction attributes and operand attributes of a test case based on the instruction execution order, and a function to determine the operand attributes in a computer system where H/W operation differs depending on the instruction execution order. By having the function of configuring memory and registers, the function of generating instructions based on instruction attributes, and the function of using the generated instructions and operands as test cases, the execution order of instructions can be considered. We have explained a test method that enables the generation of test cases and the testing of the target computer system.

[0020] As a technology related to the present invention, there is an ``instruction automatic generation device'' disclosed in Japanese Unexamined Patent Publication No. 1-265333. The technology described in this publication generates test cases by combining the conditions of single instructions, whereas the present invention generates test cases based on the conditions between multiple instructions. . For this reason, in the processing method of the technique disclosed in the publication, after determining an instruction, a table of corresponding instruction attributes is searched, and an instruction is generated based on the instruction attribute table. However, the present invention generates test cases focusing on instruction sequences by combining instruction attribute tables, and selects instructions that meet the conditions based on the combined instruction attribute tables. An image of the instruction is generated based on the specified instruction attribute table. In addition, even at the stage of generating an instruction image from an instruction attribute table, the technique disclosed in the publication is an instruction attribute table that focuses on individual instructions, and the generated instructions (including operands, etc.) also specify relationships between instructions. It is not possible to do so, and there is no overlapping relationship between commands. However, in the present invention, the instruction attribute table is an instruction attribute table between a plurality of instructions, and the generated instructions (including operands, etc.) also have overlapping relationships among the instructions. For example, in the technique disclosed in the publication, it is not possible to take into consideration the designation of using an operand in a previous instruction and the use of that operand in a later instruction, whereas in this invention, the same operand can be specified in a previous instruction You can specify relationships between instructions.

[0021]

[Effects of the Invention] As described above, by setting the instruction execution order based on the instruction attributes and generating instructions based on the instruction attributes, test cases can be automatically generated by a program. With regard to the instruction execution order and the combinations of instructions used therein, it is possible to generate a number of test cases that cannot be created manually. This eliminates the omission of tests regarding the instruction execution order and instruction combinations for the computer system, thereby ensuring quality.

[Brief explanation of drawings]

FIG. 1: Schematic diagram of this test method.

FIG. 2 is a diagram showing the contents of an instruction execution order file.

FIG. 3 is a diagram showing part of an instruction information section.

FIG. 4 is a schematic flow diagram of test case generation.

FIG. 5 is a diagram showing an example of a test case.

FIG. 6 is a diagram showing another example of this test method.

FIG. 7 is a diagram showing another example of an instruction execution order file.

FIG. 8 is a diagram showing an example of a test focusing on instruction execution.

[Explanation of symbols]

1 Instruction execution order definition section 2 Instruction execution order file 3 Instruction selection section 4 Command information file 5 Test case generation section 6 Test case file 10 Order definition means 11 Command information department 12 Generating means

Claims (1)

[Claims] Claim 1: A computer system testing method having the following elements: (a) an order definition means for defining the execution order of test instructions of the computer system to be tested using attributes of the instructions; (b) relating to each instruction of the computer to be tested; an instruction information section that stores instruction information based on attributes of each instruction; (c) a computer to be tested by selecting instruction information stored in the instruction storage section according to the instruction attributes defined by the order definition means; generation means for generating a test instruction sequence.