CN111753624B - Examination point analysis method and device for chemical experiment - Google Patents

Abstract

The embodiment of the present application provides a test point analysis method and device for a chemical experiment. The method includes: for each target video frame in the operation video, identifying the chemical vessel from the target video frame; and/or captured by the camera above; analyze the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point judgment rule, and obtain the analysis result of the target video frame based on the test point judgment rule ; Obtain the analysis result of each test site according to the analysis result of the target video frame based on the test site determination rule. This method can conduct standardized and accurate intelligent test point analysis for chemical experiments.

Description

Analysis methods and devices for test points of chemical experiments

technical field

This application relates to the technical field of educational informationization, in particular to a test point analysis method and device for chemical experiments.

Background technique

In the current chemistry experiment exam, the scoring process of the experiment is carried out by the invigilator. The invigilator supervises multiple students who are conducting experiments at the same time in the classroom. There is no way to take into account the actual operation of all candidates, and it is easy to ignore the wrong operation of the students in the experimental operation process; It is difficult to exclude subjective factors and completely unify the examination criteria.

Contents of the invention

The application provides a test point analysis method and device for chemical experiments, which can conduct standardized and accurate intelligent test point analysis for chemical experiments.

In the first aspect, the application provides a test point analysis method for chemical experiments, including:

For each target video frame in the operation video, the chemical vessel is identified from the target video frame; the operation video is captured by an imaging device positioned in front of and/or above the chemical laboratory table;

Analyzing the image status information of the chemical utensils and/or whether the positional relationship between the chemical utensils satisfies the test site determination rules, and obtains the analysis result of the target video frame based on the test site determination rules;

The analysis result of each test site is obtained according to the analysis result of the target video frame based on the test site determination rule.

Wherein, the identification of the chemical vessel from the target video frame includes:

Detect the area where the chemical vessel is located from the target video frame;

identifying boundary keypoints of the chemical vessel from the region;

The boundary key points are connected to obtain the boundary line of the chemical vessel.

Wherein, the region where the chemical vessel is detected from the target video frame includes:

Pre-training is used to detect the model of the area where the chemical vessel is located; the model is obtained by inputting the video image of the area marked with each chemical vessel as a sample into the deep learning framework and the target detection network;

The target video frame is input into the model to obtain the area where each chemical vessel is located in the target video frame.

Wherein, the analysis of the image status information of the chemical utensils and/or whether the positional relationship between the chemical utensils satisfies the test point judgment rules is obtained to obtain the analysis results of the target video frame based on the test point judgment rules, including:

Obtain the test site determination rules that the target video frame needs to meet;

Analyzing the image status information of the chemical utensils and/or whether the positional relationship between the chemical utensils satisfies the obtained test point determination rule, and obtaining an analysis result of the target video frame based on the test point determination rule.

Wherein, the test site determination rules that the target video frame needs to satisfy include:

Obtain the analysis result of the previous frame target video frame of the target video frame based on the test point judgment rule;

If described analysis result is to meet the test site determination rule, then the test site determination rule of the next sequence of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the analysis result does not satisfy the test point determination rule, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

Wherein, the test site determination rules that the target video frame needs to satisfy include:

Obtain the analysis result of the first n frames of the target video frame of the target video frame based on the test point judgment rule; n is greater than 1;

Judgment analysis result is whether the continuous quantity that satisfies the test site determination rule is equal to n;

If the continuous quantity is greater than 0 and less than n, the test site determination rule of the test site determination rule and the test site determination rule next order is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to 0, the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to n, the test site determination rule in the next order of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy.

Wherein, the test site determination rules that the target video frame needs to satisfy include:

Obtain the test site determination rule that the continuous target video frame number does not meet the continuous frame number requirement, and use the obtained test site determination rule as the test site determination rule that the target video frame needs to meet.

In the second aspect, the embodiment of the present application provides a test point analysis device for chemical experiments, including:

The identification unit is used to identify the chemical vessel from the target video frame for each target video frame in the operation video; the operation video is captured by the camera device located in front of and/or above the chemical experiment table;

An analysis unit, configured to analyze whether the image status information of the chemical utensils and/or the positional relationship between the chemical utensils meet the test point determination rules, and obtain an analysis result of the target video frame based on the test point determination rules;

The obtaining unit is configured to obtain the analysis result of each test center according to the analysis result of the target video frame based on the test point determination rule.

In a third aspect, the embodiment of the present application provides an electronic device, including:

a display screen; one or more processors; memory; a plurality of application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the device, cause the device to perform the following steps:

For each target video frame in the operation video, the chemical vessel is identified from the target video frame; the operation video is captured by an imaging device positioned in front of and/or above the chemical laboratory table;

Analyzing the image status information of the chemical utensils and/or whether the positional relationship between the chemical utensils satisfies the test site determination rules, and obtains the analysis result of the target video frame based on the test site determination rules;

The analysis result of each test site is obtained according to the analysis result of the target video frame based on the test site determination rule.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is run on a computer, it causes the computer to execute the method described in the first aspect.

In a fifth aspect, the present application provides a computer program for executing the method described in the first aspect when the computer program is executed by a computer.

In a possible design, all or part of the program in the fifth aspect may be stored on a storage medium packaged with the processor, or part or all may be stored on a memory not packaged with the processor.

In the test point analysis method of the chemical experiment in the embodiment of the present application, for each target video frame in the operation video, the chemical vessel is identified from the target video frame, and the operation video is captured by the camera device located in front of and/or above the chemical experiment table. Obtained by shooting, analyze whether the image status information of the chemical utensils and/or the positional relationship between the chemical utensils meet the test point determination rules, and obtain the analysis results of the target video frame based on the test point determination rules, according to the target video frame based on the test point The analysis results of the judgment rules obtain the analysis results of each test site, so that standardized and accurate intelligent test site analysis can be performed on chemical experiments.

Description of drawings

Fig. 1 is an example diagram of the location of the camera device of the present application;

Fig. 2 is the flow chart of an embodiment of the examination point analysis method of the chemical experiment of the present application;

Fig. 3 is the flow chart of another embodiment of the examination point analysis method of the chemical experiment of the present application;

Fig. 4 is the flow chart of another embodiment of the examination point analysis method of the chemical experiment of the present application;

Fig. 5 is the structural diagram of an embodiment of the test point analysis device of the chemical experiment of the present application;

Fig. 6 is the structural diagram of another embodiment of the test point analysis device of the chemical experiment of the present application;

Fig. 7 is the structural diagram of another embodiment of the test point analysis device of the chemical experiment of the present application;

FIG. 8 is a schematic structural diagram of an embodiment of the electronic device of the present application.

Detailed ways

The terms used in the embodiments of the present application are only used to explain specific embodiments of the present application, and are not intended to limit the present application.

In the existing implementation plan, the scoring process of the chemical experiment test is carried out by the invigilator. Specifically, the invigilator supervises multiple students who are conducting chemical experiments in the classroom at the same time, and scores each student's experimental operation process. However, since the invigilator supervises multiple students, it is easy to ignore the wrong operation of the students during the experimental operation; moreover, it is difficult for different invigilators to exclude subjective factors and completely unify the judgment criteria for the same chemical experiment.

For this reason, this application proposes a test point analysis method for chemical experiments, which can conduct standardized and accurate intelligent test point analysis for chemical experiments.

Figure 1 is an example of the application scenario of the test point analysis method of the chemical experiment in this application. Referring to Figure 1, a camera device can be installed in front of the chemical experiment table to shoot the operation video of the students performing the chemical experiment. In order to better capture the student's operation video, the height of the camera device is preferably higher than the chemical experiment table, which can be determined based on the height of various experimental devices built in the chemical experiment. For example, in the filtration experiment, the camera device can be 20-40 cm higher than the chemical experiment bench. Referring to Fig. 1, a camera device can also be installed above the chemical experiment table to shoot down the operation video of students performing chemical experiments. There is no fixed requirement for the position of the camera device, as long as the student's operation video can be clearly captured.

The imaging device in this embodiment of the present application may be: a camera or an electronic device with an imaging function.

In the following examples of the present application, the filtration experiment will be used as an example to illustrate the test point analysis method of the chemical experiment in the embodiment of the application. Therefore, the operation process of the students in the filtration experiment will be explained in advance. In the filtration experiment, students need to follow the following process: soak the filter paper and put it on the conical funnel to assemble the filter; build the filter device in the order from bottom to top, specifically, first place the beaker on the iron stand, Then put the filter on the iron ring, so that the lower tip of the funnel in the filter is close to the inner wall of the beaker; for liquid transfer, specifically, use a glass rod close to the three-layer filter paper, and the tip of the beaker is close to the glass rod to pour the liquid , and finally require the filtered liquid to be clear and not turbid. In the above process, whether the students stick the soaked filter paper to the inner wall of the funnel, whether to build the filter device in the correct order from bottom to top, whether the tip of the funnel is close to the inner wall of the beaker, whether the glass rod is close to the three-layer filter paper of the filter paper, whether the beaker Whether the tip is close to the glass rod and whether the obtained filtrate is clear and turbid are the key issues in the filtration experiment. Based on this, the above key issues of the filtering experiment are divided into 5 key points, which are as follows:

1. Correctly make the filter, wet the filter paper with water and stick it to the inner wall of the funnel;

2. Build the filter device from bottom to top;

3. The tip of the lower end of the funnel is close to the inner wall of the beaker;

4. Correct use of glass rods during liquid transfer;

5. The obtained filtrate is clear and transparent without solid impurities.

It should be noted that the filtration experiment is only an example, and the test point analysis method of the chemical experiment in the embodiment of the present application is not limited to the filtration experiment, and can also be applied to other chemical experiments such as the taking of solid medicines, the taking of liquid medicines, etc., which are not mentioned here. Too many to list.

Fig. 2 is the flow chart of an embodiment of the test point analysis method of the chemical experiment of the present application, as shown in Fig. 2, the method may include:

Step 201: For each target video frame in the operation video, identify the chemical vessel from the target video frame; the operation video is captured by the camera device located in front of and/or above the chemical experiment table;

The target video frame is a video frame selected from the operation video and subjected to chemical vessel recognition processing, and the target video frames are preferably approximately evenly distributed in the operation video, so that the analysis results of the test points of the chemical experiment are more accurate. Preferably, the target video frames may be video frames evenly distributed in the operation video.

Step 202: Analyze whether the image state information of the chemical utensils and/or the positional relationship between the chemical utensils meet the test site determination rules, and obtain the analysis result of the target video frame based on the test site determination rules;

Step 203: Obtain the analysis result of each test site according to the analysis result of the target video frame based on the test site determination rule.

The method shown in Figure 2, for each target video frame in the operation video, identify the chemical vessel from the target video frame, the operation video is taken by the camera device positioned in front of and/or above the chemical experiment table, and the analyzed Whether the image status information of the chemical utensils and/or the positional relationship between the chemical utensils meet the test point judgment rule, obtain the analysis result of the target video frame based on the test point judgment rule, and based on the analysis result of the test point judgment rule based on the target video frame Obtain the analysis results of each test point, thereby realizing the intelligent test point analysis of chemical experiments. The above-mentioned intelligent test point analysis standards are unified, making the analysis results more standardized. The above-mentioned intelligent test point analysis is carried out for the experimental operation on a chemical experiment bench The test site analysis can more accurately analyze whether the experimental operation of the students who use the chemical experiment bench is correct.

Hereinafter, the implementation of step 201 will be described.

Based on the method shown in Figure 2, referring to Figure 3, identifying the chemical vessel from the target video frame described in step 201 may include:

Step 301: Detect the area where the chemical vessel is located from the target video frame;

Step 302: identifying the boundary key points of the chemical vessel from the region;

Step 303: Connect the boundary key points to obtain the boundary line of the chemical vessel.

Wherein, the detection of the area where the chemical vessel is located from the target video frame may include:

Input the target video frame into the pre-trained model for detecting the area where the chemical utensils are located, and obtain the area where each chemical utensil is located in the target video frame; the model inputs the video image of the area marked with each chemical utensil as a sample for deep learning The frame and the target detection network are trained.

Specifically, the model for detecting the area where the chemical vessel is located may be pre-trained, and the training method may include:

Obtain a video image of the area marked with each chemical vessel as a sample;

The sample is input into the deep learning framework and the target detection network for training to obtain a model for detecting the area where the chemical vessel is located.

Wherein, the area where the chemical vessel is located may be an area framed by a rectangle, and then for each area where the chemical vessel is located, boundary key point recognition is performed to obtain the boundary key point.

Among them, the relevant key point detection method can be used to realize the detection of the key point of the boundary of the chemical vessel.

Wherein, the deep learning framework may be: Caffe, and the target detection network may be: YOLOv2 or YOLOv3.

The implementation of step 202 will be described below.

The test sites in the embodiment of the present application correspond to the key points of the chemical experiment, and may include parameters: test site determination rules. Optionally, test sites may also include parameters: test site sequence, and/or operation video, and/or test site determination rule identification , and/or execution sequence, and/or rule sequence, and/or continuous frame number. The order of the test points is consistent with the execution order of the key points in the chemical experiment. There can be one or more test center determination rules for each test center.

The parameter of test site sequence is used to record: the sequence of test sites;

The parameter of the test site judgment rule is used to record: the operation requirements of the test site;

Test center determination rule identification This parameter is used to identify different test center determination rules;

The parameter of operation video is used to record: the video source of the target video frame that is analyzed whether it satisfies the test site judgment rules is: the camera device set in front of the chemical test bench, or the camera device set above the chemical test bench;

The parameter of execution sequence is used to record: the order of determination rules of different test sites in the same test site;

The parameter of order of rules is used to record: the order of determination rules of test sites of all test sites;

The parameter of the number of continuous frames is used to record: the minimum number of target video frames that meet the test site determination rules.

The test site judgment rules are related to the key points of the chemical experiment. For example, the 5 key points listed in the above filtering experiment, the test site judgment rules can be extracted from the key points. For example, according to key point 1, the determination rule of test site 1 can be set as: there is no gap between the filter paper and the funnel; according to key point 2, the test site determination rule of test site 2 can be set as: test site determination rule 1, the beaker is placed on the iron stand , test point judgment rule 2. The filter is placed on the iron ring; the test point 3 judgment rule can be set according to the key point 3: there is no gap between the tip of the lower end of the funnel and the inner wall of the beaker; the test point 4 can be set according to the key point 4 The judgment rules are set as follows: test point judgment rule 1, the glass rod points to the three-layer filter paper of the filter paper, test point judgment rule 2, there is no gap between the tip of the beaker and the glass rod, and there is no gap between the glass rod and the three-layer filter paper of the filter paper. Gaps; according to the key point 5, the test point judgment rule of the test point 5 can be set as: the texture and/or color difference of the image in the beaker does not exceed the preset threshold;

The parameter of execution order is an optional parameter. If there is no restriction on the execution order of the test site judgment rules of each test site in a certain chemical experiment, then the parameter of execution order may not be included in the test site; if the test site includes the execution order parameter, and a certain test site only includes one test site determination rule, or although it includes multiple test site determination rules but there is no restriction on the order of execution among the multiple test site determination rules, the parameter of the test site’s execution order can be defaulted.

The number of sustained frames is related to the operation time of the chemical experiment itself. For example, for some experimental operations that only need to complete the action, as long as there is a target video frame that detects the corresponding test point judgment rule, the number of consecutive frames can be 1 or the default, such as in the chemical experiment of taking solid medicine , it is necessary to stand the test tube upright so that the drug falls evenly to the bottom of the test tube. For this operation, the corresponding test point judgment rule is: the test tube is upright, as long as one frame of the target video frame detects that the test tube is upright, then the number of continuous frames can be is 1 or the default; and for some experimental operations that need to last for a certain period of time, the number of consecutive frames that meet the test site judgment rules can be preset, and the specific value of the number of consecutive frames can be set independently according to the duration of the experimental operation in practical applications. For example, test point 4 of the filtration experiment, the shortest time for students to pour the liquid can be counted, and the number of continuous frames without gaps between the tip of the beaker and the glass rod can be set according to the shortest time.

In addition, the number of continuous frames is related to the scoring standard of chemical experiments. For example, for test site 3 in the filtration experiment, the test site judgment rule is that there is no gap between the tip of the lower end of the funnel and the inner wall of the beaker, then the scoring standard can be that there is no gap between the tip of the lower end of the funnel and the inner wall of the beaker when the student first placed the filter. It can be judged that the operation is correct. At this time, the continuous frame number can be set to 1 frame or several frames. The scoring standard can also be that after placing the filter, there is no gap between the tip of the lower end of the funnel and the inner wall of the beaker until the end of the experiment. At this time, The number of continuous frames can be set to a large value or default.

Based on the settings of the above test sites, in the same chemical experiment, the test sites have a sequence, and there may be a sequence between the multiple test site determination rules of the same test site, therefore, the test site determination rules in the chemical experiment can be sequenced Sort. For example, for the 5 test sites of the above-mentioned filtering experiment, the order of the test site determination rules is as follows:

There is no gap between the filter paper and the funnel, the beaker is placed on the iron stand, the filter (funnel) is placed on the iron ring, there is no gap between the tip of the lower end of the funnel and the inner wall of the beaker, the glass rod points to the three-layer filter paper of the filter paper, the tip of the beaker There is no gap between the glass rod and the three layers of filter paper between the glass rod and the filter paper, and the texture and/or color difference of the image in the beaker does not exceed the preset threshold.

The role of the test site judgment rules is to judge whether the students’ experimental operations can meet the requirements of the experiment. Some test site judgment rules require the experimental operations to be judged only by the camera device located in front of the chemical experiment table, while some test site judgment rules require the experimental operations to be judged. The operation can only be captured by the shooting device located above the chemical experiment table. Therefore, the parameter of operation video is set to record the video source of the target video frame that is analyzed to meet the test site judgment rules. Taking the test site judgment rule of the above-mentioned filtration experiment as an example, for the glass rod in the test site judgment rule pointing to the three layers of filter paper of the filter paper, and the texture and/or color difference of the image in the beaker does not exceed the preset threshold, only the glass rod located above the chemical experiment table In the operation video captured by the camera device, it is possible to capture whether the students point the glass rod at the three layers of filter paper and the color of the liquid in the beaker after filtering. , which can only be captured by the camera device located in front of the chemical experiment bench.

In other chemical experiments, multiple test site determination rules may be in the same order. At this time, when analyzing whether the target video frame satisfies the test site determination rules, analyze whether the target video frame satisfies each test site determination rule, respectively, and obtain the results based on each test site. The analysis result of the determination rule is sufficient, and this step can also be completed.

Based on the above description, taking the filtering experiment as an example, information similar to the following Table 1 can be preset in the electronic device that implements the method described in this application.

Table 1

Based on the above description, referring to FIG. 4, step 202 may include:

Step 401: Obtain the test site determination rules that the target video frame needs to meet;

Step 402: Analyze whether the image state information of the chemical utensils and/or the positional relationship between the chemical utensils satisfy the test site determination rule, and obtain an analysis result of the target video frame based on the test site determination rule.

There may be one or more test site determination rules that the target video frame needs to satisfy, which is not limited in this application. If the target video frame needs to satisfy multiple test point determination rules, when analyzing the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels meets the test point determination rule, it is necessary to determine the rule for each test point respectively, Analyze whether the image status information of the chemical utensils and/or the positional relationship between the chemical utensils meet the test point determination rules.

If the target video frames in the video are analyzed in the order of the test site determination rules:

In the first possible implementation manner, if the continuous frame number of the test site determination rule is 1, then step 401 may include:

Obtain the analysis result of the previous frame target video frame of the target video frame based on the test point judgment rule;

If described analysis result is to meet the test site determination rule, then the test site determination rule of the next sequence of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the analysis result does not satisfy the test point determination rule, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

In the second possible implementation, if the number of continuous frames of the test site determination rule is n, and n is greater than 1, and the value of n of different test site determination rules can be different, then step 401 may include:

Obtain the analysis result of the first n frames of the target video frame of the target video frame based on the test point judgment rule;

Judgment analysis result is whether the continuous quantity that satisfies the test site determination rule is equal to n;

If the continuous quantity is greater than 0 and less than n, the test site determination rule of the test site determination rule and the test site determination rule next order is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to 0, the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to n, the test site determination rule in the next order of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy.

If the sequence of the test site determination rules is not limited when analyzing the target video frame in the video:

In the third possible implementation, the number of continuous frames for each test site determination rule is m, and the value of the continuous frame number m for different test site determination rules is different, and m is greater than or equal to 1. At this time, it can be set in advance to meet the test site determination rules The continuous target video frame number list, when obtaining the analysis result of a target video frame based on the test point determination rule, update the list according to the analysis result; then step 401 may include: obtaining the continuous target video frame number in the list does not meet the continuous frame number requirement The test center judgment rule is used as the test center judgment rule that the target video frame needs to satisfy.

Among them, there are two operation videos processed in step 201, including the operation video taken by the camera device located in front of the chemical experiment table and the operation video taken by the camera device above the chemical experiment table, then the previous frame of the above target video frame The target video frame, or the target video frame of the first n frames, and the test site determination rule in the next order of the test site determination rule are all for the same operation video. For example, taking Table 1 as an example, if the operation video source of the target video frame is the front , then the test site determination rule that the target video frame needs to satisfy is the test site determination rule marked as 1～4 or 6, and the test site determination rule marked as 4 is the test site determination rule of the next order is marked as the test site determination rule of 6; If the operating video source of the target video frame is above, then the test point judgment rule that the target video frame needs to satisfy is the test point judgment rule marked as 5 or 7, so the test point judgment rule of the next order of the test point judgment rule marked as 5 is The test center judgment rule marked as 7.

The implementation of step 402 is described below by way of example.

Assuming that in step 401, the test point judgment rule that the target video frame needs to satisfy is: there is no gap between the filter paper and the funnel, then this step can analyze whether there is a gap between the filter paper and the funnel according to the positional relationship between the filter paper and the funnel in the target video frame. void;

Assuming that in step 401 the target video frame needs to meet the test point judgment rule: the texture and/or color difference of the image in the beaker does not exceed the preset threshold, then this step can be analyzed according to the image state information in the beaker in the target video frame Whether it satisfies the determination rules of the test site, specifically, analyze whether the texture and/or color difference of the image in the beaker exceeds a preset threshold, so as to obtain the analysis result.

Other test center determination rules are handled in a similar manner in this step, so examples are not given here.

The implementation of step 203 will be described below.

The purpose of this step is to determine whether the students have completed the key points required by the experiment in the experimental operation. Therefore, the realization of this step is related to the test center judgment rules, the order of test centers, and the order of different test center judgment rules for the same test center. , the continuous frame number of the test site determination rules, etc. These requirements can be realized by pre-setting the order of the test sites and the different parameters in each test site. Therefore, in this step, it is only necessary to analyze whether the target video frame satisfies these preset requirements based on the analysis result of the test point determination rule.

Taking the filtering experiment as an example, assuming that the requirements are pre-stored in the electronic device in the form of Table 1, in this step, it is only necessary to sequentially judge whether the target video frame meets these preset requirements according to the analysis results of the target video frame based on the test point determination rules.

For example, judge whether there are 6 consecutive frames of the target video frame that are satisfied based on the analysis result of the test point determination rule 1. If not, the analysis result of the test point 1 is unqualified; In the frame, whether there is a target video frame whose analysis result satisfies the test point judgment rules 2~7, if yes, the analysis result of test point 1 is, the number of continuous frames is qualified, and the execution sequence is unqualified, if not, then the analysis result of test point 1 is qualified. The analysis of other test sites is similar to this and will not be repeated here.

It can be understood that some or all of the steps or operations in the foregoing embodiments are only examples, and other operations or modifications of various operations may also be performed in the embodiment of the present application. In addition, various steps may be performed in different orders presented in the above embodiments, and it may not be necessary to perform all operations in the above embodiments.

Fig. 5 is a structural diagram of an embodiment of the test point analysis device of the chemical experiment of the present application, referring to Fig. 5, the device 500 may include:

The identification unit 510 is configured to, for each target video frame in the operation video, identify the chemical vessel from the target video frame; the operation video is captured by a camera device located in front of and/or above the chemical laboratory table;

The analysis unit 520 is configured to analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test point determination rule, and obtain the analysis result of the target video frame based on the test point determination rule;

The obtaining unit 530 is configured to obtain the analysis result of each test site according to the analysis result of the target video frame based on the test site determination rule.

Wherein, as shown in FIG. 6, the identification unit 510 may include:

The detection subunit 610 is used to detect the area where the chemical vessel is located from the target video frame;

An identification subunit 620, configured to identify key points on the boundary of the chemical vessel from the area;

The connection subunit 630 is configured to connect the boundary key points to obtain the boundary line of the chemical vessel.

Wherein, the detection subunit 610 may include:

A training module for pre-training a model for detecting areas where chemical utensils are located;

The input module is configured to input the target video frame into the model to obtain the area where each chemical vessel is located in the target video frame.

Wherein, the training module can specifically be used for:

Obtain a video image of the area marked with each chemical vessel as a sample;

The sample is input into the deep learning framework and the target detection network for training to obtain a model for detecting the area where the chemical vessel is located.

Referring to FIG. 7, the analysis unit 520 may include:

Obtaining a subunit 710, used to obtain the test point determination rules that the target video frame needs to satisfy;

The analysis subunit 720 is configured to analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the obtained test point determination rule, and obtain the target video frame based on the test point determination rule analysis results.

In a possible implementation manner, the obtaining subunit 710 may be specifically configured to:

Obtain the analysis result of the previous frame target video frame of the target video frame based on the test point judgment rule;

If described analysis result is to meet the test site determination rule, then the test site determination rule of the next sequence of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the analysis result does not satisfy the test point determination rule, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

In another possible implementation manner, the obtaining subunit 710 may be specifically configured to:

Obtain the analysis result of the first n frames of the target video frame of the target video frame based on the test point judgment rule; n is greater than 1;

Judgment analysis result is whether the continuous quantity that satisfies the test site determination rule is equal to n;

If the continuous quantity is greater than 0 and less than n, the test site determination rule of the test site determination rule and the test site determination rule next order is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to 0, the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to n, the test site determination rule in the next order of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy.

In yet another possible implementation manner, the obtaining subunit 710 may be specifically configured to:

Obtain the test site determination rule that the continuous target video frame number does not meet the continuous frame number requirement, and use the obtained test site determination rule as the test site determination rule that the target video frame needs to meet.

In the device shown in Figures 5-7, for each target video frame in the operation video, the identification unit identifies the chemical vessel from the target video frame, and the analysis unit analyzes the image state information of the chemical vessel, and/or the Whether the positional relationship between the chemical utensils satisfies the test site determination rules, the analysis result of the target video frame based on the test site determination rules is obtained, and the acquisition unit obtains the analysis results of each test site based on the analysis results of the target video frame based on the test site determination rules, thus realizing The intelligent test point analysis of chemical experiments, the above-mentioned intelligent test point analysis standards are unified, making the analysis results more standardized. The above-mentioned intelligent test point analysis is for the test point analysis of the experimental operation on a chemical experiment bench, which can more accurately analyze the use of Whether the experimental operation of the students in the chemical experiment bench is correct.

The devices shown in FIGS. 5 to 7 of the present application can be set in a camera device such as a camera, or can also be set in an electronic device, and further, can be set in an electronic device with a camera function.

The devices provided by the embodiments shown in Figures 5 to 7 can be used to implement the technical solutions of the method embodiments shown in Figures 2 to 4 of this application, and its implementation principles and technical effects can further refer to the relevant descriptions in the method embodiments.

It should be understood that the division of each unit or module of the embodiment device shown in Figs. 5-7 above is only a division of logical functions, and may be fully or partially integrated into one physical entity or physically separated during actual implementation. Moreover, these units or modules can be realized in the form of software calling through processing elements; they can also be realized in the form of hardware; some units or modules can also be realized in the form of software calling through processing elements, and some units or modules can be realized in the form of hardware accomplish. For example, the identification unit may be a separately established processing element, or may be integrated into a certain chip of the electronic device for implementation. The implementation of other units or modules is similar. In addition, all or part of these units or modules can be integrated together, or implemented independently. In the implementation process, each step of the above-mentioned method or each of the above-mentioned units or modules can be completed by an integrated logic circuit of hardware in the processor element or an instruction in the form of software.

For example, the above units or modules may be one or more integrated circuits configured to implement the above method, for example: one or more specific integrated circuits (Application Specific Integrated Circuit; hereinafter referred to as ASIC), or, one or more A microprocessor (Digital Singnal Processor; hereinafter referred to as DSP), or one or more Field Programmable Gate Arrays (Field Programmable Gate Array; hereinafter referred to as FPGA), etc. For another example, these units or modules may be integrated together to be implemented in the form of a System-On-a-Chip (hereinafter referred to as SOC).

FIG. 8 is a schematic structural diagram of an embodiment of an electronic device of the present application. As shown in FIG. 8 , the above-mentioned electronic device may include: one or more processors; memory; and one or more computer programs.

Wherein, the above-mentioned display screen can include the display screen of a vehicle-mounted computer (Mobile Data Center); the above-mentioned electronic equipment can be a mobile terminal (mobile phone), a camera such as a camera, a computer, a PAD, a smart screen, a drone, an intelligent network, etc. Intelligent Connected Vehicle (hereinafter referred to as: ICV), smart/intelligent car (smart/intelligent car) or on-board equipment and other equipment.

Wherein the above-mentioned one or more computer programs are stored in the above-mentioned memory, the above-mentioned one or more computer programs include instructions, when the above-mentioned instructions are executed by the above-mentioned equipment, the above-mentioned equipment is caused to perform the following steps:

For each target video frame in the operation video, the chemical vessel is identified from the target video frame; the operation video is captured by an imaging device positioned in front of and/or above the chemical laboratory table;

Analyzing the image status information of the chemical utensils and/or whether the positional relationship between the chemical utensils satisfies the test site determination rules, and obtains the analysis result of the target video frame based on the test site determination rules;

The analysis result of each test site is obtained according to the analysis result of the target video frame based on the test site determination rule.

Wherein, when the above instruction is executed by the above device, enabling the above device to perform the step of identifying the chemical vessel from the target video frame includes:

Detect the area where the chemical vessel is located from the target video frame;

identifying boundary keypoints of the chemical vessel from the region;

The boundary key points are connected to obtain the boundary line of the chemical vessel.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, enabling the above-mentioned device to perform the step of detecting the area where the chemical vessel is located from the target video frame includes:

A model pre-trained to detect areas where chemical utensils are located;

The target video frame is input into the model to obtain the area where each chemical vessel is located in the target video frame.

Wherein, when the above instruction is executed by the above device, the step of causing the above device to execute the pre-trained model for detecting the area where the chemical vessel is located includes:

Obtain a video image of the area marked with each chemical vessel as a sample;

The sample is input into the deep learning framework and the target detection network for training to obtain a model for detecting the area where the chemical vessel is located.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the above-mentioned device is made to perform the analysis of the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point determination rule, and obtains the target video frame The steps of analyzing the results based on the test center determination rules include:

Obtain the test site determination rules that the target video frame needs to meet;

Analyzing the image status information of the chemical utensils and/or whether the positional relationship between the chemical utensils satisfies the obtained test point determination rule, and obtaining an analysis result of the target video frame based on the test point determination rule.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the test point judgment rule that needs to be satisfied for obtaining the target video frame includes:

Obtain the analysis result of the previous frame target video frame of the target video frame based on the test point judgment rule;

If described analysis result is to meet the test site determination rule, then the test site determination rule of the next sequence of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the analysis result does not satisfy the test point determination rule, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the test point judgment rule that needs to be satisfied for obtaining the target video frame includes:

Obtain the analysis result of the first n frames of the target video frame of the target video frame based on the test point judgment rule; n is greater than 1;

Judgment analysis result is whether the continuous quantity that satisfies the test site determination rule is equal to n;

If the continuous quantity is greater than 0 and less than n, the test site determination rule of the test site determination rule and the test site determination rule next order is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to 0, the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy;

If the continuous number is equal to n, the test site determination rule in the next order of the test site determination rule is used as the test site determination rule that the target video frame needs to satisfy.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the test point judgment rule that needs to be satisfied for obtaining the target video frame includes:

Obtain the test site determination rule that the continuous target video frame number does not meet the continuous frame number requirement, and use the obtained test site determination rule as the test site determination rule that the target video frame needs to meet.

The electronic device shown in FIG. 8 may be a terminal device or a circuit device built in the above-mentioned terminal device. The device may be used to execute the functions/steps in the methods provided by the embodiments shown in FIG. 2 to FIG. 4 of this application.

As shown in FIG. 8 , an electronic device 800 includes a processor 810 and a transceiver 820 . Optionally, the electronic device 800 may also include a memory 830 . Among them, the processor 810, the transceiver 820, and the memory 830 can communicate with each other through an internal connection path, and transmit control and/or data signals. call and run the computer program.

The above-mentioned memory 830 may be a read-only memory (read-only memory, ROM), other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), or other types that can store information and instructions. Type of dynamic storage device, also can be electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), read-only disc (compactdisc read-only memory, CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and Any other medium that can be accessed by a computer, etc.

Optionally, the electronic device 800 may further include an antenna 840 configured to send out the wireless signal output by the transceiver 820 .

The above-mentioned processor 810 and the memory 830 can be combined into a processing device, more commonly, they are components independent of each other, and the processor 810 is used to execute the program code stored in the memory 830 to realize the above-mentioned functions. During specific implementation, the memory 830 may also be integrated in the processor 810 , or be independent of the processor 810 .

In addition, in order to make the functions of the electronic device 800 more perfect, the electronic device 800 may also include one or more of an input unit 860, a display unit 870, an audio circuit 880, a camera 890, and a sensor 801, etc., the Audio circuitry may also include a speaker 882, a microphone 884, and the like. Wherein, the display unit 870 may include a display screen.

Optionally, the electronic device 800 may further include a power supply 850, configured to provide power to various devices or circuits in the terminal device.

It should be understood that the electronic device 800 shown in FIG. 8 can implement various processes of the methods provided in the embodiments shown in FIGS. 2 to 4 of this application. The operations and/or functions of the various modules in the electronic device 800 are respectively intended to implement the corresponding processes in the foregoing method embodiments. For details, refer to the descriptions in the method embodiments shown in FIGS. 2 to 4 of the present application. To avoid repetition, detailed descriptions are appropriately omitted here.

It should be understood that the processor 810 in the electronic device 800 shown in FIG. 8 may be a system-on-chip SOC, and the processor 810 may include a central processing unit (Central Processing Unit; hereinafter referred to as: CPU), and may further include other types of A processor, for example: an image processor (Graphics Processing Unit; hereinafter referred to as: GPU) and the like.

In short, each part of the processors or processing units inside the processor 810 can cooperate to implement the previous method flow, and the corresponding software programs of each part of the processors or processing units can be stored in the memory 830 .

The present application also provides an electronic device, the device includes a storage medium and a central processing unit, the storage medium may be a non-volatile storage medium, a computer executable program is stored in the storage medium, and the central processing unit and the The non-volatile storage medium is connected and executes the computer executable program to implement the method provided by the embodiments shown in FIGS. 1 to 5 of this application.

In each of the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and Image Signal Processing (hereinafter referred to as: ISP), the processor can also include necessary hardware accelerators or logic processing hardware circuits, such as ASIC, or one or more integrated circuits for controlling the execution of the program of the technical solution of this application circuit etc. In addition, the processor may have the function of operating one or more software programs, which may be stored in the storage medium.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When it is run on a computer, the computer executes the program provided by the embodiments shown in FIGS. 2 to 4 of the present application. method.

The embodiment of the present application also provides a computer program product, the computer program product includes a computer program, and when it is run on a computer, it causes the computer to execute the method provided in the embodiments shown in FIG. 2 to FIG. 4 of the present application.

In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three kinds of relationships, for example, A and/or B may indicate that A exists alone, A and B exist simultaneously, or B exists alone. Among them, A and B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single, or Can be multiple.

Those of ordinary skill in the art can appreciate that each unit and algorithm steps described in the embodiments disclosed herein can be realized by a combination of electronic hardware, computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In several embodiments provided in this application, if any function is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as ROM), random access memory (Random Access Memory; hereinafter referred to as RAM), magnetic disk or optical disc, etc. A medium on which program code can be stored.

The foregoing is only a specific implementation of the present application. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should be covered by the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.

Claims (10)

1. A method for analyzing a test point of a chemical experiment, comprising:

for each target video frame in the operation video, identifying a chemical vessel from the target video frame; the operation video is shot by a camera device positioned in front of and/or above the chemical experiment table;

analyzing whether the image state information of the chemical vessels and/or the position relation among the chemical vessels meet the examination point judging rule or not to obtain an analysis result of the target video frame based on the examination point judging rule;

obtaining an analysis result of each examination point according to the analysis result of the target video frame based on the examination point judgment rule;

the examination point judging rule comprises the following steps: the texture and/or color difference of the image in the beaker does not exceed a preset threshold; the analyzing whether the image state information of the chemical vessel meets the examination point judging rule comprises the following steps: analyzing whether texture and/or color differences of the image within the beaker exceed a preset threshold.

2. The method of claim 1, wherein identifying a chemical vessel from the target video frame comprises:

detecting the region of the chemical vessel from the target video frame;

identifying boundary keypoints of the chemical vessel from the region;

and connecting the boundary key points to obtain the boundary line of the chemical vessel.

3. The method of claim 2, wherein detecting the region of the chemical vessel from the target video frame comprises:

inputting the target video frame into a pre-trained model for detecting the region where the chemical vessel is located, and obtaining the region where each chemical vessel is located in the target video frame; the model is obtained by inputting video images of the areas marked with the chemical vessels as samples into a deep learning framework and a target detection network for training.

4. A method according to any one of claims 1 to 3, wherein analyzing whether the image state information of the chemical vessels and/or the positional relationship between the chemical vessels satisfies the examination point determination rule, to obtain the analysis result of the target video frame based on the examination point determination rule, comprises:

Obtaining a test point judging rule which needs to be met by a target video frame;

and analyzing whether the image state information of the chemical vessels and/or the position relation among the chemical vessels meet the obtained examination point judging rule or not to obtain an analysis result of the target video frame based on the examination point judging rule.

5. The method of claim 4, wherein the point of interest determination rule to be satisfied for obtaining the target video frame comprises:

obtaining an analysis result of a target video frame of a previous frame of the target video frame based on a test point judgment rule;

if the analysis result is that the examination point judging rule is met, taking the examination point judging rule of the next sequence of the examination point judging rule as the examination point judging rule which needs to be met by the target video frame;

and if the analysis result is that the examination point judging rule is not satisfied, taking the examination point judging rule of the target video frame of the previous frame as the examination point judging rule which is required to be satisfied by the target video frame.

6. The method of claim 4, wherein the point of interest determination rule to be satisfied for obtaining the target video frame comprises:

obtaining an analysis result of the first n frames of target video frames based on the examination point judgment rule; n is greater than 1;

Judging whether the analysis result is that the continuous number meeting the examination point judgment rule is equal to n;

if the continuous number is more than 0 and less than n, taking the examination point judgment rule and the examination point judgment rule of the next sequence of the examination point judgment rule as the examination point judgment rule which needs to be met by the target video frame;

if the continuous number is equal to 0, taking the examination point judgment rule as an examination point judgment rule which needs to be met by the target video frame;

and if the continuous number is equal to n, taking the next sequential examination point judgment rule of the examination point judgment rules as the examination point judgment rule required to be met by the target video frame.

7. The method of claim 4, wherein the point of interest determination rule to be satisfied for obtaining the target video frame comprises:

and obtaining a test point judging rule that the continuous target video frame number does not meet the continuous frame number requirement, and taking the obtained test point judging rule as the test point judging rule required to be met by the target video frame.

8. A point of care analysis device for a chemical experiment, comprising:

an identification unit for identifying a chemical vessel from the target video frames for each of the target video frames in the operation video; the operation video is shot by a camera device positioned in front of and/or above the chemical experiment table;

The analysis unit is used for analyzing whether the image state information of the chemical vessels and/or the position relation among the chemical vessels meet the examination point judgment rule or not to obtain an analysis result of the target video frame based on the examination point judgment rule;

the acquisition unit is used for acquiring an analysis result of each examination point based on the analysis result of the examination point judgment rule according to the target video frame;

the examination point judging rule comprises the following steps: the texture and/or color difference of the image in the beaker does not exceed a preset threshold; in order to realize the analysis of whether the image state information of the chemical vessel meets the examination point judging rule, the analysis unit is specifically used for: analyzing whether texture and/or color differences of the image within the beaker exceed a preset threshold.

9. An electronic device, comprising:

one or more processors; a memory; wherein one or more computer programs are stored in the memory, the one or more computer programs comprising instructions, which when executed by the processor, cause the electronic device to perform the steps of:

for each target video frame in the operation video, identifying a chemical vessel from the target video frame; the operation video is shot by a camera device positioned in front of and/or above the chemical experiment table;

Analyzing whether the image state information of the chemical vessels and/or the position relation among the chemical vessels meet the examination point judging rule or not to obtain an analysis result of the target video frame based on the examination point judging rule;

obtaining an analysis result of each examination point according to the analysis result of the target video frame based on the examination point judgment rule;

the examination point judging rule comprises the following steps: the texture and/or color difference of the image in the beaker does not exceed a preset threshold; the analyzing whether the image state information of the chemical vessel meets the examination point judging rule comprises the following steps: analyzing whether texture and/or color differences of the image within the beaker exceed a preset threshold.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when run on a computer, causes the computer to perform the method according to any of claims 1-7.