CN111753624A - Analytical methods and devices for test sites in chemical experiments - Google Patents

Abstract

The embodiments of the present application provide a test point analysis method and device for a chemical experiment. The method includes: for each target video frame in an operation video, identifying a chemical vessel from the target video frame; and/or photographed by the above camera; analyze the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point judgment rule, and obtain the analysis result of the target video frame based on the test point judgment rule ;According to the analysis result of the target video frame based on the test point judgment rule, the analysis result of each test point is obtained. This method can carry out standardized and accurate intelligent test point analysis for chemical experiments.

Description

Analytical methods and devices for test sites in chemical experiments

technical field

The present application relates to the technical field of educational informatization, and in particular, to a method and device for analyzing test sites for chemical experiments.

Background technique

In the current chemistry experiment exam, the scoring process of the experiment is carried out by the invigilator. The invigilator supervises multiple students who are conducting experiments at the same time in the classroom, and there is no way to take into account the actual operation of all candidates. It is easy to ignore the wrong operations of the students during the experimental operation; It is difficult to exclude subjective factors and be completely unified in the examination criteria.

SUMMARY OF THE INVENTION

The present application provides a test point analysis method and device for chemical experiments, which can conduct standardized, accurate and intelligent test point analysis for chemical experiments.

In the first aspect, the present application provides a test point analysis method for a chemical experiment, including:

For each target video frame in the operation video, a chemical vessel is identified from the target video frame; the operation video is captured by a camera located in front of and/or above the chemical laboratory;

Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site determination rule, and obtain the analysis result of the target video frame based on the test site determination rule;

The analysis result of each test point is obtained according to the analysis result of the target video frame based on the test point determination rule.

Wherein, identifying the chemical vessel from the target video frame includes:

Detect the area where the chemical vessel is located from the target video frame;

Identifying boundary key points of the chemical vessel from the region;

Connect the boundary key points to obtain the boundary line of the chemical vessel.

Wherein, the detection of the area where the chemical vessel is located from the target video frame includes:

Pre-training a model for detecting the area where the chemical vessel is located; the model is obtained by inputting the video image of the area of each chemical vessel as a sample into the deep learning framework and the target detection network for training;

The target video frame is input into the model to obtain the region where each chemical vessel is located in the target video frame.

Wherein, analyzing whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfies the test point determination rule, and obtaining the analysis result of the target video frame based on the test point determination rule, including:

The test point judgment rules that need to be met to obtain the target video frame;

Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the obtained test point determination rule, and obtain an analysis result of the target video frame based on the test point determination rule.

Wherein, the test point judgment rules that need to be satisfied to obtain the target video frame include:

Obtain the analysis result of the target video frame of the previous frame of the target video frame based on the test point judgment rule;

If the analysis result is to satisfy the test site judgment rule, then the test site judgment rule in the next order of the test site judgment rule is used as the test site judgment rule that the target video frame needs to satisfy;

If the analysis result is that the test point determination rule is not satisfied, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

Wherein, the test point judgment rules that need to be satisfied to obtain the target video frame include:

Obtain the analysis results of the first n frames of the target video frame based on the test point judgment rule; n is greater than 1;

Judging whether the analysis result is the continuous number that satisfies the test site judgment rule is equal to n;

If the consecutive number is greater than 0 and less than n, the test site judgment rule and the test site judgment rule in the order next to the test site judgment rule are used as the test site judgment rule that the target video frame needs to satisfy;

If the consecutive number is equal to 0, take the test point determination rule as the test point determination rule that the target video frame needs to meet;

If the consecutive number is equal to n, the test point determination rule in the order next to the test point determination rule is taken as the test point determination rule that the target video frame needs to satisfy.

Wherein, the test point judgment rules that need to be satisfied to obtain the target video frame include:

Obtain the test point judgment rule that the number of continuous target video frames does not meet the requirements of the continuous frame number, and use the obtained test point judgment rule as the test point judgment rule that the target video frame needs to meet.

In the second aspect, the embodiment of the present application provides a test point analysis device for a chemical experiment, including:

an identification unit, for identifying chemical vessels from the target video frame for each target video frame in the operation video; the operation video is captured by a camera located in front of and/or above the chemical laboratory bench;

An analysis unit, configured to analyze the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point determination rule, and obtain the analysis result of the target video frame based on the test point determination rule;

The obtaining unit is configured to obtain the analysis result of each test point according to the analysis result of the target video frame based on the test point determination rule.

In a third aspect, an embodiment of the present application provides an electronic device, including:

a display screen; one or more processors; a memory; a plurality of application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs Include instructions that, when executed by the device, cause the device to perform the following steps:

For each target video frame in the operation video, a chemical vessel is identified from the target video frame; the operation video is captured by a camera located in front of and/or above the chemical laboratory;

Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site determination rule, and obtain the analysis result of the target video frame based on the test site determination rule;

The analysis result of each test point is obtained according to the analysis result of the target video frame based on the test point determination rule.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program runs on a computer, causes the computer to execute the method described in the first aspect.

In a fifth aspect, the present application provides a computer program for executing the method of the first aspect when the computer program is executed by a computer.

In a possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged with the processor, and may also be stored in part or in part in a memory not packaged with the processor.

In the test point analysis method of the chemical experiment in the embodiment of the present application, for each target video frame in the operation video, the chemical vessel is identified from the target video frame, and the operation video is recorded by a camera located in front of and/or above the chemical experiment table. Obtained by shooting, analyzing whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfies the test site judgment rule, and obtaining the analysis result of the target video frame based on the test site judgment rule, and based on the test site according to the target video frame. The analysis results of the judgment rules can obtain the analysis results of each test site, so that the chemical experiments can be standardized and accurate intelligent test site analysis.

Description of drawings

FIG. 1 is an example diagram of the setting position of the camera device of the present application;

Fig. 2 is the flow chart of one embodiment of the test point analysis method of the chemical experiment of the application;

Fig. 3 is the flow chart of another embodiment of the test point analysis method of the chemical experiment of the application;

Fig. 4 is the flow chart of another embodiment of the test point analysis method of the chemical experiment of the application;

5 is a structural diagram of an embodiment of a test site analysis device for chemical experiments of the application;

6 is a structural diagram of another embodiment of the test site analysis device for the chemical experiment of the application;

FIG. 7 is a structural diagram of another embodiment of the test site analysis device of the chemical experiment of the application;

FIG. 8 is a schematic structural diagram of an embodiment of an electronic device of the present application.

Detailed ways

The terms used in the embodiments of the present application are only used to explain specific embodiments of the present application, and are not intended to limit the present application.

In the existing implementation scheme, the judging process of the chemical experiment test is performed by the invigilator. Specifically, the invigilator supervises multiple students in the classroom who are conducting chemical experiments at the same time, and judges the score for each student's experimental operation process. However, because the invigilator supervises multiple students, it is easy to ignore the wrong operation of the students in the process of experiment operation; moreover, it is difficult for different invigilators to exclude subjective factors and completely unify the test criteria for the same chemistry experiment.

To this end, the present application proposes a test point analysis method for chemical experiments, which can conduct standardized, accurate and intelligent test point analysis for chemical experiments.

FIG. 1 is an example of an application scenario of the test site analysis method for a chemical experiment of the present application. Referring to FIG. 1 , a camera device can be set in front of the chemical experiment bench to shoot the operation video of the students performing the chemical experiment. In order to better capture the operation video of the students, the height of the camera device is preferably higher than that of the chemical experimental bench, which can be determined based on the height of various experimental devices built in the chemical experiment. For example, in the filtering experiment, the camera device can be 20 to 40 cm higher than the chemical laboratory bench. Referring to Figure 1, a camera device can also be set above the chemical experiment table to shoot the operation video of the students conducting the chemical experiment. The position of the camera device is not fixed, as long as the students' operation video can be clearly captured.

The camera device in the embodiment of the present application may be a camera or an electronic device with a camera function.

In the following embodiments of the present application, the filter experiment will be used as an example to illustrate the test point analysis method of the chemical experiment of the embodiment of the present application. Therefore, the operation flow of the students in the filter experiment will be described in advance. In the filtration experiment, the students' operation needs to follow the following procedures: soak the filter paper and put it on the conical funnel to assemble the filter; build the filter device in the order from bottom to top. Specifically, first place the beaker on the iron stand, Then put the filter on the iron ring, so that the lower tip of the funnel in the filter is close to the inner wall of the beaker; transfer the liquid, specifically, use a glass rod to stick to the three layers of filter paper, and the beaker to close the glass rod to pour the liquid. , the final requirement is that the filtered liquid should be clear and not turbid. In the above process, whether the students put the soaked filter paper close to the inner wall of the funnel, whether to build the filter device in the correct order from bottom to top, whether the tip of the funnel is close to the inner wall of the beaker, whether the glass rod is close to the three-layer filter paper of the filter paper, and whether the beaker Whether the tip is close to the glass rod and whether the obtained filtrate is clear without turbidity are the key issues in filtration experiments. Based on this, the above key issues of the filtering experiment are divided into 5 key points, which are as follows:

1. Make the filter correctly, wet the filter paper with water and stick it to the inner wall of the funnel;

2. Build the filter device from bottom to top;

3. The tip of the lower end of the funnel is close to the inner wall of the beaker;

4. Use the glass rod correctly during the liquid transfer process;

5. The obtained filtrate is clear and transparent without solid impurities.

It should be noted that the filtration experiment is only an example, and the test point analysis method of the chemical experiment in the embodiment of the present application is not limited to the filtration experiment, and can also be applied to other chemical experiments such as the taking of solid medicines, the taking of liquid medicines, etc. More to list.

FIG. 2 is a flow chart of an embodiment of the test point analysis method of the chemical experiment of the application. As shown in FIG. 2 , the method may include:

Step 201: for each target video frame in the operation video, identify the chemical vessel from the target video frame; the operation video is captured by a camera located in front of and/or above the chemical laboratory bench;

The target video frame is a video frame selected from the operation video for chemical vessel identification processing, and the target video frame is preferably approximately uniformly distributed in the operation video, so that the analysis results of the chemical experiments are more accurate. Preferably, the target video frames may be video frames evenly distributed in the operation video.

Step 202: Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test point determination rule, and obtain the analysis result of the target video frame based on the test point determination rule;

Step 203: Obtain the analysis result of each test point according to the analysis result of the target video frame based on the test point determination rule.

In the method shown in Fig. 2, for each target video frame in the operation video, the chemical vessel is identified from the target video frame, and the operation video is captured by a camera located in front of and/or above the chemical laboratory bench, and the analysis Whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site judgment rule, obtain the analysis result of the target video frame based on the test site judgment rule, and the analysis result based on the test site judgment rule according to the target video frame Obtain the analysis results of each test site, thus realizing the intelligent test site analysis of chemistry experiments. The above-mentioned intelligent test site analysis standards are unified, which makes the analysis results more standardized. The above intelligent test site analysis is carried out for the experimental operation on a chemistry test bench. The analysis of the test center can more accurately analyze whether the experimental operation of the students who use the chemical experimental bench is correct.

Hereinafter, the implementation of step 201 will be described.

Based on the method shown in FIG. 2 and referring to FIG. 3 , the chemical vessel is identified from the target video frame in step 201, which may include:

Step 301: Detect the area where the chemical vessel is located from the target video frame;

Step 302: Identify the boundary key points of the chemical vessel from the area;

Step 303: Connect the boundary key points to obtain the boundary line of the chemical vessel.

Wherein, the detection of the area where the chemical vessel is located from the target video frame may include:

Input the target video frame into a pre-trained model for detecting the area where the chemical vessel is located, and obtain the area where each chemical vessel is located in the target video frame; the model uses the video image of the area marked with each chemical vessel as a sample input deep learning The framework and target detection network are trained.

Specifically, a model for detecting the area where the chemical vessel is located can be pre-trained, and the training method can include:

Obtain a video image of the area marked with each chemical vessel as a sample;

The sample is input into the deep learning framework and the target detection network for training to obtain a model for detecting the area where the chemical vessel is located.

Wherein, the area where the chemical vessel is located may be an area framed by a rectangle, and then for each area where the chemical vessel is located, the boundary key points are identified to obtain the boundary key points.

Among them, the relevant key point detection method can be used to realize the detection of the boundary key points of the chemical vessel.

The deep learning framework may be: Caffe, and the target detection network may be: YOLOv2 or YOLOv3.

The implementation of step 202 will be described below.

The test sites in the embodiments of the present application correspond to the key points of the chemistry experiment, and may include parameters: test site determination rules. Optionally, the test sites may also include parameters: test site sequence, and/or operation video, and/or test site determination rule identifiers , and/or the execution order, and/or the rule sequence, and/or the number of continuous frames. The order of the test points is consistent with the execution order of the key points in the chemistry experiment. Each test site can have one or more test site determination rules.

This parameter is used to record the sequence of test sites: the sequence of test sites;

The parameter of the test site determination rule is used to record: the operation requirements of the test site;

Test site judgment rule identification This parameter is used to identify different test site judgment rules;

The parameter of operation video is used to record: the video source of the target video frame being analyzed whether it satisfies the test site judgment rule is: the camera set in front of the chemical test bench, or the camera set above the chemical test bench;

The parameter of execution order is used to record: the sequence between the judgment rules of different test sites in the same test site;

The parameter of the rule sequence is used to record: the sequence between the test site judgment rules of all test sites;

The parameter of continuous frame number is used to record: the minimum number of target video frames that meet the test point judgment rules.

The test point determination rules are related to the key points of the chemistry experiment, such as the five key points listed in the above filtering experiment, and the test point determination rules can be extracted from the key points. For example, according to key point 1, the test site determination rule of test site 1 can be set as: there is no gap between the filter paper and the funnel; according to key point 2, the test site determination rule of test site 2 can be set as: test site determination rule 1. The beaker is placed on the iron stand , the test site determination rule 2. The filter is placed on the iron ring; the test site determination rule of test site 3 can be set according to key point 3: there is no gap between the tip of the funnel and the inner wall of the beaker; the test site of test site 4 can be set according to key point 4. Judgment rules are set as follows: Judgment Rule 1 of the test site: The glass rod points to the three-layer filter paper of the filter paper. Judgment rule 2 of the test site: There is no gap between the tip of the beaker and the glass rod, and there is no gap between the glass rod and the three-layer filter paper of the filter paper. Gap; according to key point 5, the test point determination rule of test point 5 can be set as: the texture and/or color difference of the image in the beaker does not exceed the preset threshold;

The parameter of execution order is an optional parameter. If there is no restriction on execution order in the test site judgment rules of each test site in a certain chemistry experiment, the test site may not include the parameter of execution order; if the test site includes the execution order parameter. parameter, and a test site only includes one test site judgment rule, or although it includes multiple test site judgment rules, but there is no restriction on the execution order of multiple test site judgment rules, the parameter of the test site execution order can be default.

The number of continuous frames is related to the operation time of the chemical experiment itself. For example, for some experimental operations that only need to complete the action, as long as one target video frame detects the corresponding test point judgment rule, the number of consecutive frames can be 1 or default, such as in the chemical experiment of taking solid medicines , you need to stand the test tube upright to let the medicine evenly fall to the bottom of the test tube. For this operation, the corresponding test site judgment rule is: the test tube is upright, as long as one target video frame detects that the test tube is upright, then the number of continuous frames can be It is 1 or default; and for some experimental operations that need to last for a certain period of time, the number of consecutive frames that meet the test point determination rules can be preset. For example, in the test site 4 of the filtration experiment, the shortest time for students to pour the liquid can be counted, and the number of continuous frames without a gap between the tip of the beaker and the glass rod can be set according to the shortest time.

In addition, the number of continuous frames is related to the scoring standard of the chemical experiment. For example, for test site 3 in the filtration experiment, the test site determination rule is that there is no gap between the tip of the funnel and the inner wall of the beaker, then the scoring standard can be that there is no gap between the tip of the funnel and the inner wall of the beaker when the students initially place the filter. It can be judged that the operation is correct. At this time, the continuous frame number can be set to 1 frame or several frames. The scoring standard can also be that after placing the filter, there is no gap between the tip of the funnel and the inner wall of the beaker until the end of the experiment. At this time, Persistent frames can be set to a large value or default.

Based on the setting of the above test sites, in the same chemistry experiment, the test sites have a sequential order, and the multiple test site judgment rules of the same test site may have a sequential order. Therefore, the test site judgment rules in the chemistry experiment can be sequenced. sort. For example, for the five test sites in the above filtering experiment, the order of the test site determination rules is as follows:

There is no gap between the filter paper and the funnel, the beaker is placed on the iron stand, the filter (funnel) is placed on the iron ring, there is no gap between the tip of the lower end of the funnel and the inner wall of the beaker, the glass rod points to the three-layer filter paper of the filter paper, and the tip of the beaker There is no gap between the glass rod and the three layers of filter paper between the glass rod and the filter paper, and the texture and/or color difference of the image in the beaker does not exceed a preset threshold.

The function of the test center judgment rules is to judge whether the students' experimental operations can meet the experimental requirements. Some test center judgment rules need to determine the experimental operations that can only be photographed by the camera located in front of the chemistry test bench, while some test center judgment rules need to determine the experiments. The operation can only be captured by the photographing device located above the chemical experiment table. Therefore, the parameter of operation video is set to record the video source of the target video frame analyzed whether it satisfies the test site judgment rules. Taking the test site determination rule of the above filtering experiment as an example, the glass rod in the test site determination rule points to the three-layer filter paper of the filter paper, and the texture and/or color difference of the image in the beaker does not exceed the preset threshold. The operation video captured by the shooting device can capture whether the students point the glass rod to the three-layer filter paper of the filter paper during the experimental operation, and the color of the liquid in the beaker after filtering. , which can only be captured by the camera device located in front of the chemical experiment bench.

In other chemistry experiments, multiple test site judgment rules may be in the same order. In this case, when analyzing whether the target video frame meets the test site judgment rules, analyze whether the target video frame satisfies each test site judgment rule separately, and obtain the results based on each test site. The analysis result of the determination rule is sufficient, and this step can also be completed.

Based on the above description, taking the filtering experiment as an example, information similar to Table 1 below can be preset in the electronic device that executes the method described in this application.

Table 1

Based on the above description, referring to FIG. 4 , step 202 may include:

Step 401: Obtain the test point judgment rules that the target video frame needs to meet;

Step 402: Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site determination rule, and obtain an analysis result of the target video frame based on the test site determination rule.

There may be one or more test point determination rules that the target video frame needs to satisfy, which is not limited in this application. If there are multiple test site determination rules to be satisfied by the target video frame, when analyzing the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels meets the test site determination rules, it is necessary to determine the rules for each test site separately. Analyze whether the image state information of the chemical vessels and/or the positional relationship between the chemical vessels satisfy the test site determination rule.

If the target video frames in the video are analyzed according to the order of the test point determination rules:

In a first possible implementation manner, if the continuous frame number of the test point determination rule is 1, step 401 may include:

Obtain the analysis result of the target video frame of the previous frame of the target video frame based on the test point judgment rule;

If the analysis result is to satisfy the test site judgment rule, then the test site judgment rule in the next order of the test site judgment rule is used as the test site judgment rule that the target video frame needs to satisfy;

If the analysis result is that the test point determination rule is not satisfied, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

In the second possible implementation manner, if the number of continuous frames of the test point determination rule is n, and n is greater than 1, and the value of n for different test point determination rules may be different, step 401 may include:

Obtain the analysis results of the first n frames of the target video frame based on the test point judgment rule;

Judging whether the analysis result is the continuous number that satisfies the test site judgment rule is equal to n;

If the consecutive number is greater than 0 and less than n, the test site judgment rule and the test site judgment rule in the order next to the test site judgment rule are used as the test site judgment rule that the target video frame needs to satisfy;

If the consecutive number is equal to 0, take the test point determination rule as the test point determination rule that the target video frame needs to meet;

If the consecutive number is equal to n, the test point determination rule in the order next to the test point determination rule is taken as the test point determination rule that the target video frame needs to satisfy.

If the target video frame in the video is analyzed, the order of the test point determination rules is not limited:

In the third possible implementation manner, the number of continuous frames for each test site determination rule is m, and the continuous frame number m of different test site determination rules has different values, and m is greater than or equal to 1. In this case, it can be preset to meet the test site determination rules. The continuous target video frame number list, when obtaining a target video frame based on the analysis result of the test point judgment rule, update the list according to the analysis result; then step 401 may include: Obtaining the continuous target video frame number in the list does not meet the continuous frame number requirement The obtained test point judgment rule is used as the test point judgment rule that the target video frame needs to meet.

Wherein, there are 2 operation videos processed in step 201, including the operation video captured by the camera located in front of the chemical experiment bench, and the operation video captured by the camera located above the chemical experiment platform, then the previous frame of the above target video frame The target video frame, or the target video frame of the first n frames, and the test point determination rule in the next order of the test point determination rule are all for the same operation video. For example, take Table 1 as an example, if the operation video source of the target video frame is the front , then the test site judgment rule that this target video frame needs to meet is the test site judgment rule marked as 1 to 4 or 6, and the test site judgment rule in the next sequence of the test site judgment rule marked as 4 is the test site judgment rule marked with 6; If the operation video source of the target video frame is above, then the test site judgment rule that the target video frame needs to meet is the test site judgment rule marked as 5 or 7, then the test site judgment rule marked as 5 is the test site judgment rule in the next sequence is The test site determination rule marked as 7.

The implementation of step 402 will be described below by way of example.

Assuming that the test point determination rule that needs to be satisfied to obtain the target video frame in step 401 is: there is no gap between the filter paper and the funnel, this step can analyze whether there is a gap between the filter paper and the funnel according to the positional relationship between the filter paper and the funnel in the target video frame. void;

Assuming that the test point determination rule that needs to be satisfied to obtain the target video frame in step 401 is: the texture and/or color difference of the image in the beaker does not exceed the preset threshold, then this step can be analyzed according to the image state information in the beaker in the target video frame. Whether it satisfies the test site determination rule, specifically, analyze whether the texture and/or color difference of the image in the beaker exceeds a preset threshold, so as to obtain the analysis result.

The processing methods of other test center judgment rules in this step are similar, and they will not be illustrated here.

Hereinafter, the implementation of step 203 will be described.

The purpose of this step is to determine whether students have completed the key points required by the experiment in the experimental operation. Therefore, the implementation of this step is related to the test site judgment rules, the sequence between test sites, and the sequence between the same test site and different test sites. , the continuous frame number of the test site determination rules, etc. These requirements can be achieved by presetting the sequence of test sites and different parameters in each test site. Therefore, in this step, it is only necessary to analyze whether the target video frame meets these preset requirements according to the analysis result of the target video frame based on the test point determination rule.

Taking the filtering experiment as an example, assuming that the requirements are pre-stored in the electronic device in the form of Table 1, in this step, it is only necessary to sequentially determine whether the target video frames meet these preset requirements according to the analysis results of the target video frame based on the test point determination rules.

For example, determine whether there are 6 consecutive target video frames based on the analysis result of the test site determination rule 1, if not, the analysis result of test site 1 is unqualified; if so, then judge the target video before the target video frame for 6 consecutive frames In the frame, whether there is a target video frame whose analysis result satisfies the test center judgment rules 2 to 7, if so, the analysis result of test center 1 is that the number of continuous frames is qualified, and the execution order is not qualified, if not, then the analysis result of test center 1 is: qualified. The analysis of other test sites is similar and will not be repeated here.

It can be understood that, some or all of the steps or operations in the foregoing embodiments are merely examples, and other operations or variations of various operations may also be performed in the embodiments of the present application. Furthermore, the various steps may be performed in a different order presented in the above-described embodiments, and may not perform all operations in the above-described embodiments.

FIG. 5 is a structural diagram of an embodiment of a test point analysis device for a chemical experiment of the present application. Referring to FIG. 5 , the device 500 may include:

The identification unit 510 is used for identifying the chemical vessel from the target video frame for each target video frame in the operation video; the operation video is captured by the camera device located in front of and/or above the chemical experiment platform;

An analysis unit 520, configured to analyze the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point determination rule, and obtain an analysis result of the target video frame based on the test point determination rule;

The obtaining unit 530 is configured to obtain the analysis result of each test point according to the analysis result of the target video frame based on the test point determination rule.

Wherein, as shown in FIG. 6 , the identifying unit 510 may include:

The detection subunit 610 is used to detect the area where the chemical vessel is located from the target video frame;

Identifying subunit 620, for identifying the boundary key points of the chemical vessel from the area;

The connecting subunit 630 is used to connect the boundary key points to obtain the boundary line of the chemical vessel.

Wherein, the detection subunit 610 may include:

A training module for pre-training a model for detecting the region where the chemical vessel is located;

The input module is used for inputting the target video frame into the model to obtain the region where each chemical vessel is located in the target video frame.

Wherein, the training module can be specifically used for:

Obtain a video image of the area marked with each chemical vessel as a sample;

The sample is input into the deep learning framework and the target detection network for training to obtain a model for detecting the area where the chemical vessel is located.

Referring to FIG. 7 , the analysis unit 520 may include:

Obtaining subunit 710, for obtaining the test point judgment rule that the target video frame needs to satisfy;

An analysis subunit 720, configured to analyze the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the obtained test point determination rule, and obtain the target video frame based on the test point determination rule analysis results.

In a possible implementation manner, the obtaining subunit 710 may be specifically used for:

Obtain the analysis result of the target video frame of the previous frame of the target video frame based on the test point judgment rule;

If the analysis result is to satisfy the test site judgment rule, then the test site judgment rule in the next order of the test site judgment rule is used as the test site judgment rule that the target video frame needs to satisfy;

If the analysis result is that the test point determination rule is not satisfied, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

In another possible implementation manner, the obtaining subunit 710 may be specifically used for:

Obtain the analysis results of the first n frames of the target video frame based on the test point judgment rule; n is greater than 1;

Judging whether the analysis result is the continuous number that satisfies the test site judgment rule is equal to n;

If the consecutive number is greater than 0 and less than n, the test site judgment rule and the test site judgment rule in the order next to the test site judgment rule are used as the test site judgment rule that the target video frame needs to satisfy;

If the consecutive number is equal to 0, take the test point determination rule as the test point determination rule that the target video frame needs to meet;

If the consecutive number is equal to n, the test point determination rule in the order next to the test point determination rule is taken as the test point determination rule that the target video frame needs to satisfy.

In yet another possible implementation manner, the obtaining subunit 710 may be specifically used for:

Obtain the test point judgment rule that the number of continuous target video frames does not meet the requirements of the continuous frame number, and use the obtained test point judgment rule as the test point judgment rule that the target video frame needs to meet.

In the apparatuses shown in FIGS. 5-7 , for each target video frame in the operation video, the identification unit identifies the chemical vessel from the target video frame, and the analysis unit analyzes the image state information of the chemical vessel, and/or the Whether the positional relationship between the chemical utensils satisfies the test site judgment rule, obtain the analysis result of the target video frame based on the test site judgment rule, and the obtaining unit obtains the analysis result of each test site according to the analysis result of the target video frame based on the test site judgment rule, thereby realizing For the intelligent test center analysis of chemical experiments, the above-mentioned intelligent test center analysis standards are unified, which makes the analysis results more standardized. The above intelligent test center analysis is based on the test point analysis of the experimental operation on a chemical laboratory bench, which can more accurately analyze and use Whether the experimental operation of the students in the chemistry laboratory is correct.

The devices shown in FIGS. 5 to 7 of the present application may be installed in a camera device such as a camera, or may be installed in an electronic device, and further, may be installed in an electronic device with a camera function.

The apparatuses provided by the embodiments shown in FIGS. 5 to 7 can be used to implement the technical solutions of the method embodiments shown in FIGS. 2 to 4 of the present application, and the implementation principles and technical effects may further refer to the relevant descriptions in the method embodiments.

It should be understood that the division of each unit or module of the apparatus in the embodiment shown in FIGS. 5 to 7 is only a division of logical functions, and may be fully or partially integrated into a physical entity or physically separated in actual implementation. And these units or modules can all be implemented in the form of software calling through processing elements; they can also all be implemented in hardware; some units or modules can also be implemented in the form of software calling through processing elements, and some units or modules can be implemented in the form of hardware. accomplish. For example, the identification unit may be a separately established processing element, or may be integrated in a certain chip of the electronic device. The implementation of other units or modules is similar. In addition, all or part of these units or modules can be integrated together, and can also be implemented independently. In the implementation process, each step of the above method or each of the above units or modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

For example, the above units or modules may be one or more integrated circuits configured to implement the above method, such as: one or more specific integrated circuits (Application Specific Integrated Circuit; hereinafter referred to as: ASIC), or, one or more A microprocessor (Digital Singnal Processor; hereinafter referred to as: DSP), or one or more Field Programmable Gate Arrays (hereinafter referred to as: FPGA), etc. For another example, these units or modules can be integrated together and implemented in the form of a system-on-a-chip (System-On-a-Chip; hereinafter referred to as: SOC).

FIG. 8 is a schematic structural diagram of an embodiment of an electronic device of the present application. As shown in FIG. 8 , the above-mentioned electronic device may include: one or more processors; a memory; and one or more computer programs.

Wherein, the above-mentioned display screen can include the display screen of the vehicle-mounted computer (Mobile Data Center); the above-mentioned electronic equipment can be a mobile terminal (mobile phone), a camera device such as a camera, a computer, a PAD, a smart screen, an unmanned aerial vehicle, an intelligent network Connected vehicle (Intelligent Connected Vehicle; hereinafter referred to as: ICV), intelligent (car) vehicle (smart/intelligent car) or on-board equipment and other equipment.

Wherein the above-mentioned one or more computer programs are stored in the above-mentioned memory, and the above-mentioned one or more computer programs include instructions that, when the above-mentioned instructions are executed by the above-mentioned equipment, cause the above-mentioned equipment to perform the following steps:

For each target video frame in the operation video, a chemical vessel is identified from the target video frame; the operation video is captured by a camera located in front of and/or above the chemical laboratory;

Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site determination rule, and obtain the analysis result of the target video frame based on the test site determination rule;

The analysis result of each test point is obtained according to the analysis result of the target video frame based on the test point determination rule.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the above-mentioned device to perform the step of identifying the chemical vessel from the target video frame includes:

Detect the area where the chemical vessel is located from the target video frame;

Identifying boundary key points of the chemical vessel from the region;

Connect the boundary key points to obtain the boundary line of the chemical vessel.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the above-mentioned step of making the above-mentioned device to perform the step of detecting the area where the chemical vessel is located from the target video frame includes:

Pre-trained models to detect regions where chemical vessels are located;

The target video frame is input into the model to obtain the region where each chemical vessel is located in the target video frame.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the pre-trained model for detecting the area where the chemical vessel is located includes:

Obtain a video image of the area marked with each chemical vessel as a sample;

The sample is input into the deep learning framework and the target detection network for training to obtain a model for detecting the area where the chemical vessel is located.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the above-mentioned device executes the analysis of the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point determination rule, and obtains the target video frame The steps of analyzing the results based on the test site determination rules include:

The test point judgment rules that need to be met to obtain the target video frame;

Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the obtained test point determination rule, and obtain an analysis result of the target video frame based on the test point determination rule.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the test point determination rule that needs to be satisfied to obtain the target video frame includes:

Obtain the analysis result of the target video frame of the previous frame of the target video frame based on the test point judgment rule;

If the analysis result is to satisfy the test site judgment rule, then the test site judgment rule in the next order of the test site judgment rule is used as the test site judgment rule that the target video frame needs to satisfy;

If the analysis result is that the test point determination rule is not satisfied, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the test point determination rule that needs to be satisfied to obtain the target video frame includes:

Obtain the analysis results of the first n frames of the target video frame based on the test point judgment rule; n is greater than 1;

Judging whether the analysis result is the continuous number that satisfies the test site judgment rule is equal to n;

If the consecutive number is greater than 0 and less than n, the test site judgment rule and the test site judgment rule in the order next to the test site judgment rule are used as the test site judgment rule that the target video frame needs to satisfy;

If the consecutive number is equal to 0, take the test point determination rule as the test point determination rule that the target video frame needs to meet;

If the consecutive number is equal to n, the test point determination rule in the order next to the test point determination rule is taken as the test point determination rule that the target video frame needs to satisfy.

Wherein, when the above-mentioned instruction is executed by the above-mentioned device, the step of causing the above-mentioned device to execute the test point determination rule that needs to be satisfied to obtain the target video frame includes:

Obtain the test point judgment rule that the number of continuous target video frames does not meet the requirements of the continuous frame number, and use the obtained test point judgment rule as the test point judgment rule that the target video frame needs to meet.

The electronic device shown in FIG. 8 may be a terminal device or a circuit device built in the above-mentioned terminal device. The device can be used to execute the functions/steps in the methods provided by the embodiments shown in FIG. 2 to FIG. 4 of the present application.

As shown in FIG. 8 , the electronic device 800 includes a processor 810 and a transceiver 820 . Optionally, the electronic device 800 may further include a memory 830 . The processor 810, the transceiver 820 and the memory 830 can communicate with each other through an internal connection path to transmit control and/or data signals. The memory 830 is used to store computer programs, and the processor 810 is used to retrieve data from the memory 830. Invoke and run the computer program.

The above-mentioned memory 830 may be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types of static storage devices that can store information and instructions. It can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and Any other medium that can be accessed by a computer, etc.

Optionally, the electronic device 800 may further include an antenna 840 for sending out the wireless signal output by the transceiver 820 .

The above-mentioned processor 810 and the memory 830 can be combined into a processing device, and more commonly they are independent components, and the processor 810 is used to execute the program codes stored in the memory 830 to realize the above-mentioned functions. During specific implementation, the memory 830 may also be integrated in the processor 810 , or be independent of the processor 810 .

Besides, in order to make the functions of the electronic device 800 more complete, the electronic device 800 may further include one or more of an input unit 860, a display unit 870, an audio circuit 880, a camera 890, a sensor 801, etc. Audio circuitry may also include speakers 882, microphones 884, and the like. Wherein, the display unit 870 may include a display screen.

Optionally, the above electronic device 800 may further include a power supply 850 for providing power to various devices or circuits in the terminal device.

It should be understood that the electronic device 800 shown in FIG. 8 can implement each process of the method provided by the embodiments shown in FIG. 2 to FIG. 4 of the present application. The operations and/or functions of each module in the electronic device 800 are respectively to implement the corresponding processes in the foregoing method embodiments. For details, reference may be made to the descriptions in the method embodiments shown in FIG. 2 to FIG. 4 of the present application. To avoid repetition, detailed descriptions are appropriately omitted here.

It should be understood that the processor 810 in the electronic device 800 shown in FIG. 8 may be a system-on-chip SOC, and the processor 810 may include a central processing unit (Central Processing Unit; hereinafter referred to as: CPU), and may further include other types of A processor, for example, a Graphics Processing Unit (Graphics Processing Unit; hereinafter referred to as: GPU) and the like.

In a word, each part of the processors or processing units inside the processor 810 can cooperate to implement the previous method process, and the corresponding software programs of each part of the processors or processing units can be stored in the memory 830 .

The present application also provides an electronic device, the device includes a storage medium and a central processing unit, the storage medium may be a non-volatile storage medium, and a computer-executable program is stored in the storage medium, and the central processing unit is connected to the central processing unit. The non-volatile storage medium is connected, and the computer-executable program is executed to implement the method provided by the embodiments shown in FIG. 1 to FIG. 5 of the present application.

In the above embodiments, the involved processors may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural-network process unit (Neural-network Process Units; hereinafter referred to as: NPU) and Image Signal Processing (hereinafter referred to as: ISP), the processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASIC, or one or more integrated circuits for controlling the execution of the programs of the technical solution of the present application circuit, etc. Furthermore, the processor may have the function of operating one or more software programs, which may be stored in a storage medium.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when it runs on a computer, causes the computer to execute the programs provided by the embodiments shown in FIG. 2 to FIG. 4 of the present application. method.

An embodiment of the present application further provides a computer program product, the computer program product includes a computer program, which, when running on a computer, enables the computer to execute the methods provided by the embodiments shown in FIG. 2 to FIG. 4 of the present application.

In the embodiments of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And/or", which describes the association relationship of the associated objects, means that there can be three kinds of relationships, for example, A and/or B, which can indicate the existence of A alone, the existence of A and B at the same time, and the existence of B alone. where A and B can be singular or plural. The character "/" generally indicates that the associated objects are an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c may represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c may be single, or Can be multiple.

Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (Read-Only Memory; hereinafter referred to as: ROM), Random Access Memory (Random Access Memory; hereinafter referred to as: RAM), magnetic disk or optical disk and other various A medium on which program code can be stored.

The above are only specific embodiments of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. a test point analysis method of chemical experiment, is characterized in that, comprises: For each target video frame in the operation video, a chemical vessel is identified from the target video frame; the operation video is captured by a camera located in front of and/or above the chemical laboratory; Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site determination rule, and obtain the analysis result of the target video frame based on the test site determination rule; The analysis result of each test point is obtained according to the analysis result of the target video frame based on the test point determination rule. 2. The method according to claim 1, wherein the identifying the chemical vessel from the target video frame comprises: Detect the area where the chemical vessel is located from the target video frame; Identifying boundary key points of the chemical vessel from the region; Connect the boundary key points to obtain the boundary line of the chemical vessel. 3. The method according to claim 2, wherein the detecting the area where the chemical vessel is located from the target video frame comprises: Input the target video frame into a pre-trained model for detecting the area where the chemical vessel is located to obtain the area where each chemical vessel is located in the target video frame; the model uses the video image of the area marked with each chemical vessel as the The samples are input into the deep learning framework and the target detection network for training. 4. The method according to any one of claims 1 to 3, wherein the analyzing the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point determination rule, Obtain the analysis results of the target video frame based on the test site determination rules, including: The test point judgment rules that need to be met to obtain the target video frame; Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the obtained test point determination rule, and obtain an analysis result of the target video frame based on the test point determination rule. 5. method according to claim 4, is characterized in that, described obtaining the test point judgment rule that the target video frame needs to satisfy, comprises: Obtain the analysis result of the target video frame of the previous frame of the target video frame based on the test point judgment rule; If the analysis result is to satisfy the test site judgment rule, then the test site judgment rule in the next order of the test site judgment rule is used as the test site judgment rule that the target video frame needs to satisfy; If the analysis result is that the test point determination rule is not satisfied, the test point determination rule of the previous target video frame is used as the test point determination rule that the target video frame needs to satisfy. 6. The method according to claim 4, characterized in that, the test point judgment rule that needs to be satisfied by the acquisition of the target video frame, comprising: Obtain the analysis results of the first n frames of the target video frame based on the test point judgment rule; n is greater than 1; Judging whether the analysis result is the continuous number that satisfies the test site judgment rule is equal to n; If the consecutive number is greater than 0 and less than n, the test site judgment rule and the test site judgment rule in the order next to the test site judgment rule are used as the test site judgment rule that the target video frame needs to satisfy; If the consecutive number is equal to 0, take the test point determination rule as the test point determination rule that the target video frame needs to meet; If the consecutive number is equal to n, the test point determination rule in the order next to the test point determination rule is taken as the test point determination rule that the target video frame needs to satisfy. 7. The method according to claim 4, characterized in that, the test point judgment rule that needs to be satisfied by the obtained target video frame, comprising: Obtain the test point judgment rule that the number of continuous target video frames does not meet the requirements of the continuous frame number, and use the obtained test point judgment rule as the test point judgment rule that the target video frame needs to meet. 8. A test site analysis device for chemical experiment, characterized in that, comprising: an identification unit, for identifying chemical vessels from the target video frame for each target video frame in the operation video; the operation video is captured by a camera located in front of and/or above the chemical laboratory bench; An analysis unit, configured to analyze the image state information of the chemical vessel and/or whether the positional relationship between the chemical vessels satisfies the test point determination rule, and obtain the analysis result of the target video frame based on the test point determination rule; The obtaining unit is configured to obtain the analysis result of each test point according to the analysis result of the target video frame based on the test point determination rule. 9. An electronic device, characterized in that, comprising: a display screen; one or more processors; a memory; a plurality of application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs Include instructions that, when executed by the device, cause the device to perform the following steps: For each target video frame in the operation video, a chemical vessel is identified from the target video frame; the operation video is captured by a camera located in front of and/or above the chemical laboratory; Analyze whether the image state information of the chemical vessel and/or the positional relationship between the chemical vessels satisfy the test site determination rule, and obtain the analysis result of the target video frame based on the test site determination rule; The analysis result of each test point is obtained according to the analysis result of the target video frame based on the test point determination rule. 10. A computer-readable storage medium, characterized in that, a computer program is stored in the computer-readable storage medium, and when it is run on a computer, the computer is made to execute the method according to any one of claims 1-7. method.

Images