CN116363256A - Method and device for realizing visual display of code pictures - Google Patents

Abstract

The invention discloses a method and device for visually displaying a code picture, and relates to the technical field of software engineering. The method includes recognizing the code in the code picture based on an optical character recognition method, and performing error correction processing on the recognized code; based on The code formatting tool formats the code, and uses the drawing tool to make the formatted code into a code picture; performs instrumentation on the formatted code, and uses the test case generation tool to generate the instrumented code Test case; run the test case, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running stage. The invention can effectively help the user to improve the understanding of the code segment in the code picture, and simplifies the process for the user to realize the code.

Description

Method and device for implementing visual display of code pictures

technical field

The invention relates to the technical field of software engineering, in particular to a method and device for implementing a visual display of code pictures.

Background technique

In textbooks such as data structure and algorithm analysis, codes that are difficult to understand often appear. In the past, when students read such books, because the codes in the textbooks were static text, students would have doubts about how the codes in the textbooks actually run. They had to manually type out the codes in the books on the computer. to run. For example, students do not understand a sorting algorithm in textbooks, so they can only manually implement the code and design test cases into the algorithm function, and view the running results.

However, the above implementation method is not effective for students to understand the code. The main reasons are: (1) they do not understand the execution order of the code at runtime; (2) there is no way to change the parameters of the code at runtime (such as sorting) (3) The way of manually typing codes is extremely time-consuming.

Contents of the invention

In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method and device for visually displaying code pictures, which can effectively help users improve their understanding of code fragments in code pictures, and simplify the process for users to implement codes.

In order to achieve the above purpose, the present invention provides a method for realizing the visual display of code pictures, which specifically includes the following steps:

Recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code;

Format the code based on the code formatting tool, and use the drawing tool to make the formatted code into a code picture;

Perform instrumentation on the formatted code, and use the test case generation tool to generate test cases for the instrumented code;

Run the test case, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running stage.

On the basis of the above technical solution, the error correction processing of the identified code is specifically:

Codes are corrected based on heuristic rules to replace incorrect characters recognized during optical character recognition with correct characters.

On the basis of the above technical solution, in the code picture, a font of equal width is used between the codes.

On the basis of the above technical solution, the specific steps of inserting the formatted code include:

Use source code analysis and conversion tools to traverse the code, and insert the code that prints the code position number and its parameter value before and after each statement.

On the basis of the above technical solution, the code for printing the code position number and its parameter value is inserted before and after each statement, specifically:

For all statements that are not loops, insert the code that prints the line number of the statement before the statement;

For a loop statement, insert the code that prints the start and end positions of the loop before the statement;

For an assignment statement, an initial value statement, and a call function statement that inputs a specific parameter name, additionally insert a code that prints the value after the statement.

On the basis of the above technical solutions,

Obtain the location information of the current code block while traversing the code with source code analysis and conversion tools;

The position information includes code start line and end character number, and code end line and end character number.

On the basis of the above technical solution, when using the test case generation tool to generate test cases for the inserted code, limit the parameters substituted into the source code in the generated test cases.

On the basis of the above technical solution, when the test case generation tool is used to generate test cases for the inserted code, it also includes:

Use source code analysis and conversion tools to traverse test cases, judge whether the parameters of the function under test in test cases are complete, and select test cases with preset visual understanding effects.

On the basis of the above technical solution, the numerical information includes the code line number, the position number of the loop statement and the numerical value of each parameter in the code.

A device for implementing a visual display of a code image provided by the present invention includes:

A recognition module, which is used to recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code;

A first processing module, which is used to format the code based on a code formatting tool, and use a drawing tool to make the formatted code into a code picture;

The second processing module is used to insert the formatted code, and use a test case generation tool to generate a test case for the inserted code;

Execution module, which is used to run test cases, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running stage.

Compared with the prior art, the present invention has the advantages of being able to effectively help users improve their understanding of code fragments in code pictures, simplifying the process for users to implement codes, eliminating the need for users to manually input and implement codes, and simplifying the process for users to understand codes. The process automatically generates test cases and brings them into code fragments in the code picture, and automatically generates a visual display of the code's running statement sequence and changes in corresponding parameter values. It has the characteristics of automation, low cost, and easy understanding.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

FIG. 1 is a flow chart of a method for implementing a visual display of a code image in an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, but not all of them.

The present invention provides a method for implementing visual display of code pictures, which can be implemented based on smart devices, combined with smart devices to help users (such as students, programmers, etc.) quickly understand the code examples in the text, users only need to use smart devices to By intercepting the image of the target code, you can visually display the code running process by highlighting the code line, and display the real-time changes of various parameter values. It has the characteristics of automation, low cost, and easy understanding; The way of visually displaying the running process of the program improves users' (such as students, programmers, etc.) understanding of code fragments in code pictures (such as code photos on textbooks, screenshots of screen codes, etc.).

Referring to Figure 1, a method for implementing a visual display of a code image provided by an embodiment of the present invention specifically includes the following steps:

S1: Recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code;

That is, optical character recognition is performed on code pictures to obtain code strings, and error correction is performed on the recognized code strings through program analysis technology to make them compilable.

In the present invention, performing error correction processing on the recognized code is specifically: performing error correction processing on the code based on heuristic rules, so as to replace wrong characters recognized during optical character recognition with correct characters. That is, based on heuristic rules to replace common errors generated by optical character recognition, its technical form is code error correction based on static program analysis to achieve compilable results.

S2: Format the code based on the code formatting tool, and use the drawing tool to make the formatted code into a code picture; in the code picture, use a font of equal width between the codes, so as to grasp the position of each line or statement on the picture The location is convenient for subsequent highlighting and visual display.

S3: Instrument the formatted code, and use the test case generation tool to generate test cases for the instrumented code; instrument the code so that the code can be printed in the form of a string at runtime Related Information.

In the present invention, the formatted code is instrumented, and the specific steps include: using a source code analysis and conversion tool to traverse the code, inserting and printing the code position number and its parameter value before and after each statement.

In the present invention, the code that prints this code position number and its parameter value is inserted before and after each statement, specifically:

For all statements that are not loops, insert the code that prints the line number of the statement before the statement;

For a loop statement, insert the code that prints the start and end positions of the loop before the statement;

For an assignment statement, an initial value statement, and a call function statement that inputs a specific parameter name, additionally insert a code that prints the value after the statement.

Further, while traversing the code with a source code analysis and conversion tool, the location information of the current code block is obtained; the location information includes code start line and end character number, and code end line and end character number.

In the present invention, when the test case generation tool is used to generate test cases for the inserted code, the parameters of the source code are substituted into the generated test cases. That is, when using the test case generation tool, in order to make the visual understanding of the code better, the parameters brought into the source code by the generated test case are restricted.

In the present invention, after using the test case generation tool to generate the test case for the code after the post insertion, it also includes: using the source code analysis and conversion tool to traverse the test case, judging whether the parameters of the function under test in the test case are complete, and selecting to obtain Test cases with preset visual comprehension effects.

S4: Run the test case, and use a specific display method to display the code running sequence, and display the numerical information of the code running stage.

That is, when running a test case, by traversing the generated strings of related information, the code running sequence is displayed by highlighting the code, and the numerical information of the code running stage is displayed at the same time. The numerical information includes the code line number, loop statement The position number and the value of each parameter in the code.

Specifically, the implementation method for visual display in the present invention can be applied to smart devices (such as smart phones), and presented in the form of applications of smart devices. When reading text materials, if the user encounters a code segment that needs to be understood, the smart device application can be opened. The smart device application will use the camera permission of the smart device. The user only needs to point the screen at the code segment and press Click the screenshot button to see the visual display of the pop-up code.

The present invention combines an optical character recognition tool and a test case generation tool, collects code information by means of code insertion, and finally presents it in a visualized form. The whole can be divided into four stages: optical character recognition and code completion error correction stage, code traversal for instrumentation stage, test case generation and selection stage, output result collection and visual display stage.

For the optical character recognition and code completion error correction stage, first use the optical character recognition technology to convert the code text in the picture into a string, because the optical character recognition technology may have insufficient precision, it will not be able to Compile, so for code error correction, heuristic rules can be used to correct obvious and high-repetition errors.

For the instrumentation stage of traversing the code, in order to obtain the required information when running the code, use source code analysis and conversion tools to traverse the code, and insert a statement that prints the code number of the line before each line of code; in each assignment statement 1. After the initialization statement and the call function statement for inputting a specific parameter name, insert a statement that prints the parameter; before and after the loop, insert a statement that indicates the type and number of the loop and its running position.

For visual display, use the code formatting tool to format the original code to make the code style uniform. Use the drawing function to access the formatted code as a code picture, and use the picture as the base of the visual display. In order to highlight the position of the code fragment in the picture, a monospaced font is used here.

While traversing the code, since the code has been traversed, the position of the current code line or the current code fragment in the entire code can be obtained, for example, the line is in the 5th character to the 25th character of the 3rd line, And according to this position information multiplied by the height or width of the monospaced font, based on the previously created visual base image, make a picture that highlights the target code segment on the base image, and name it after the corresponding target code segment, such as height The image on the third line of code is named "3.jpg".

For the test case generation and selection phase, use the test case generation tool to generate test cases for the instrumented code. The test case generation tool will generate multiple test cases according to the set parameters. Since the test cases generated by the tool are not necessarily helpful for visual understanding of the code, it is very important to choose a better test case. Here, use source code analysis and conversion tools to check the test case code and check whether there are really non-empty value, limit the minimum length of the array, etc., if the conditions are met, access the current test case for use as a visual display, and finally, run the test cases that pass the filter.

For the output result collection and visual display stage, since the test case has been run, the code inserted at this time will print out the required information sequence (including the number of code lines, parameter values and loop status). By reading this character stream and decomposing relevant information, the previously created highlighted code snippet images can be highlighted in the running order for visual display, and the status of all parameter values when running to the code snippet is displayed.

The following describes the specific implementation process of the implementation method of the visual display of the code picture in the present invention.

(1) Use optical character recognition tools (such as ABBYY, Tesseract) to obtain the code in the code picture;

(2) Perform error correction and code completion on the recognized code, and save the code fragment as a runnable class, and use heuristic rules to replace common optical character recognition errors with correct words, such as common I, l. For the wrong problem of 1, replace words such as 1length and Ioop with the correct length and loop;

(3) Use code formatting tools (such as the PrettyPrint function of the source code analysis and conversion tool Spoon) for the code, and use the formatted code to use the drawing tool (such as java.awt.Graphics.drawString()) to make code fragments into a picture, and use the picture as the basis for visualization;

(4) Use a source code analysis and conversion tool (such as spoon) to traverse and insert the code formatted in step (3), and add the code to output the printed information to a text file (here saved as sequence.txt). For the instrumentation part, the focus is on the following three types:

A. Insert the probe before each line, print the code line number, for example out.write("!LINE!"+6+"#,#");

B. Insert a probe (including the initial of the for statement, etc.) after each assignment statement, initialization statement, and recorded parameter name is input to other functions, and print the parameter value. For example out.write("p="+p+"#,#"), if it is an array, output all the values of the array, for example out.write("a="+java.util.Arrays.toString(a) +"#,#");

C. For each cycle type, insert probes at the beginning and end to display the current status and the cycle number. For example, insert a probe outside the for loop to print "ForA1", such as out.write("ForA1#,#"), insert a probe before the start of the for loop to print "ForB1", such as out.write("ForB1#, #"), insert the probe at the end so that it prints "ForC1", for example out.write("ForC1#,#").

(5) Obtain the location information of the code block while traversing the code stub, including the code start line and end character number, end line and end character number. Based on the code picture made in step (3) during each traversal, the coordinates of the code block can be obtained by multiplying the width and height of the monospaced font. Use the coordinate information to highlight the code block on the code picture with a drawing method (such as Java graphics.fillRect()). Finally, use Java I/O to save this picture as a picture, and its file name must correspond to the characters printed by the probe in step (4).

(6) For the inserted code, use a test case generation tool (such as evosuite) to generate a test case for it. In order to improve the effect of visual understanding, add conditions for test case generation, such as increasing the maximum integer value in the evosuite command to no more than 10, and not generating assertions, and generating as many test cases as possible.

(7) For the test classes generated in step (6), one of the test classes will contain many test cases, use source code analysis and conversion tools to traverse each test case, and record assignment statements for each test case The given value and parameter name, and corresponding to the statement that the current test case calls the function under test, if the value brought in is not empty, and if the value brought in is an array with a length greater than 3, then the test case is considered to be correct Visual understanding is helpful, you can access this test case separately to a new class, and save it as test0.java, test1.java, etc. in sequence.

(8) Store the test case class instructions accessed in the running step (7) into a batch file, and run the batch file to obtain a text file (sequence.txt) for printing out information. Use Java I/O to read and traverse the text stream, and process it according to the values printed by the probes inserted in step (4). If it is a line of code or a position in a loop, convert it to the corresponding image path generated in step (5), use the scheduling method (such as timer()) to set the image to play every second, and if it is a printed parameter value, set Each parameter value is stored or updated to a data structure (Map<String, ArrayList<String>> is used here), and the data structure is displayed on the screen.

The core points of the present invention are: 1. Combining optical character recognition technology, source code analysis and conversion tools with test case automatic generation tools, and visually displaying code fragments in code pictures to help users understand; 2. Based on optical characters Compilability restoration of recognition technology and code error correction, based on heuristic rules, corrects the recognition errors of codes recognized by optical character recognition technology through static program analysis technology, so that originally incomplete or incomplete code fragments can be compiled and run; 3. Based on the visual understanding of program execution based on test case generation technology and selection technology, automatic test case generation is constructed, and test cases that are more suitable for visual understanding are selected according to the strategy of limiting the parameter range of the input code under test; 4. Combined with smart devices The application program displays the code blocks passed by the test case in a dynamic way of highlighting the execution position, combines the smart device's built-in camera to take pictures of the code, and displays the code running sequence in the way of highlighting the code blocks, simplifying Eliminates the tedious steps of running the code yourself.

The implementation method of the visual display of the code picture in the embodiment of the present invention can effectively help the user to improve the understanding of the code fragments in the code picture, simplify the process of the user implementing the code, so that the user does not need to manually input and implement the code, and simplifies the process for the user to understand the code The process automatically generates test cases and brings them into code fragments in the code picture, and automatically generates a visual display of the code's running statement sequence and changes in corresponding parameter values. It has the characteristics of automation, low cost, and easy understanding.

In a possible implementation, the embodiment of the present invention also provides a readable storage medium, the readable storage medium is located in the PLC (Programmable Logic Controller, Programmable Logic Controller) controller, and the readable storage medium stores the A computer program, when the program is executed by a processor, implements the following steps of a visual display method for a code picture:

Recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code;

Format the code based on the code formatting tool, and use the drawing tool to make the formatted code into a code picture;

Perform instrumentation on the formatted code, and use the test case generation tool to generate test cases for the instrumented code;

Run the test case, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running stage.

The storage medium may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer-readable storage medium may be, for example but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include: electrical connections with one or more leads, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a data signal carrying computer readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out the operations of the present invention may be written in one or more programming languages, or combinations thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through the Internet using an Internet service provider). connect).

An embodiment of the present invention provides an apparatus for realizing a visual display of a code picture, including an identification module, a first processing module, a second processing module, and an execution module.

The recognition module is used to recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code; the first processing module is used to format the code based on the code formatting tool, and use the drawing tool Make the formatted code into a code picture; the second processing module is used to insert the formatted code, and use the test case generation tool to generate test cases for the inserted code; the execution module is used to run the test Use cases, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running phase.

The above descriptions are only specific implementation manners of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

Claims (10)

1. A visual display implementation method of a code picture, characterized in that, specifically comprising the following steps: Recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code; Format the code based on the code formatting tool, and use the drawing tool to make the formatted code into a code picture; Perform instrumentation on the formatted code, and use the test case generation tool to generate test cases for the instrumented code; Run the test case, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running stage. 2. The method for implementing a visual display of a code picture as claimed in claim 1, wherein said error correction process is carried out on the identified code, specifically: Codes are corrected based on heuristic rules to replace incorrect characters recognized during optical character recognition with correct characters. 3. The method for implementing a visual display of a code picture according to claim 1, wherein in the code picture, a font of equal width is used between codes. 4. The method for implementing a visual display of a code picture as claimed in claim 3, wherein the specific steps of inserting the formatted code include: Use source code analysis and conversion tools to traverse the code, and insert the code that prints the code position number and its parameter value before and after each statement. 5. The visual display implementation method of a kind of code picture as claimed in claim 4, is characterized in that, before and after each statement, insert and print the code of this code position number and its parameter value, specifically: For all statements that are not loops, insert the code that prints the line number of the statement before the statement; For a loop statement, insert the code that prints the start and end positions of the loop before the statement; For an assignment statement, an initial value statement, and a call function statement that inputs a specific parameter name, additionally insert a code that prints the value after the statement. 6. The method for implementing a visual display of a code picture as claimed in claim 4, characterized in that: Obtain the location information of the current code block while traversing the code with source code analysis and conversion tools; The position information includes code start line and end character number, and code end line and end character number. 7. The method for realizing a visual display of a code picture as claimed in claim 1, characterized in that: when the test case generation tool is used to generate a test case for the code after the insertion, the source code is restricted in the generated test case parameters. 8. The method for implementing a visual display of a code image according to claim 7, wherein, after the test case generation tool is used to generate a test case for the code after the instrumentation, the method further includes: Use source code analysis and conversion tools to traverse the test cases to determine whether the parameters of the function under test in the test cases are complete, so as to select test cases with preset visual understanding effects. 9. The method for implementing a visual display of code pictures according to claim 1, wherein the numerical information includes code line numbers, loop statement position numbers, and the values of various parameters in the code. 10. A visual display implementation device for code pictures, characterized in that it comprises: A recognition module, which is used to recognize the code in the code picture based on the optical character recognition method, and perform error correction processing on the recognized code; A first processing module, which is used to format the code based on a code formatting tool, and use a drawing tool to make the formatted code into a code picture; The second processing module is used to insert the formatted code, and use a test case generation tool to generate a test case for the inserted code; Execution module, which is used to run test cases, and use a specific display method to display the code running sequence, and at the same time display the numerical information of the code running stage.

Images