WO2025020569A1 - 图像数据集处理方法、装置、设备及存储介质 - Google Patents

图像数据集处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2025020569A1
WO2025020569A1 PCT/CN2024/084979 CN2024084979W WO2025020569A1 WO 2025020569 A1 WO2025020569 A1 WO 2025020569A1 CN 2024084979 W CN2024084979 W CN 2024084979W WO 2025020569 A1 WO2025020569 A1 WO 2025020569A1
Authority
WO
WIPO (PCT)
Prior art keywords
area
score
confidence
threshold
autonomous driving
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/084979
Other languages
English (en)
French (fr)
Inventor
徐倩
杨鑫
贾楠
熊光银
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu Kawa Technology Co Ltd
Original Assignee
Chengdu Kawa Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu Kawa Technology Co Ltd filed Critical Chengdu Kawa Technology Co Ltd
Priority to JP2024558970A priority Critical patent/JP7842892B2/ja
Priority to EP24821735.8A priority patent/EP4528669A4/en
Publication of WO2025020569A1 publication Critical patent/WO2025020569A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations

Definitions

  • the embodiments of the present application relate to the field of image processing technology, and in particular, to an image data set processing method, device, equipment and storage medium.
  • the embodiments of the present application provide an image data set processing method, apparatus, device and storage medium, which can improve the accuracy of image data set processing.
  • an embodiment of the present application provides an image dataset processing method, including: obtaining an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; inputting the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; comparing the first image detection result and the second image detection result to obtain a comparison result; determining a comparison type based on the comparison result; wherein the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples, and false negative examples; determining evaluation scores of the multiple initial autonomous driving images based on the comparison type; screening out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjusting the annotations in the target autonomous driving image dataset.
  • an embodiment of the present application further provides an image dataset processing device, comprising: an initial autonomous driving image dataset acquisition module, used to acquire an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; an image detection result acquisition module, used to input the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; an image detection result comparison module, used to compare the first image detection result and the second image detection result to obtain a comparison result; a comparison type determination module, used to determine a comparison type according to the comparison result; wherein the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples, and false negative examples; an evaluation score determination module, used to determine the evaluation scores of the multiple initial autonomous driving images according to the comparison type; a target autonomous driving image dataset screening module, used to screen out a target autonomous driving image dataset from the initial autonomous driving image dataset according to the evaluation scores of the multiple
  • an embodiment of the present application further provides an electronic device, the electronic device comprising:
  • processors one or more processors
  • a storage device for storing one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image data set processing method as described in the embodiment of the present application.
  • an embodiment of the present application further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the image data set processing method as described in the embodiment of the present application.
  • the technical solution disclosed in this embodiment is to obtain an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; input the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model respectively, and obtain a first image detection result and a second image detection result respectively; compare the first image detection result and the second image detection result to obtain a comparison result; determine the comparison type according to the comparison result; wherein the first image detection result is used as a reference detection result; the comparison type includes a true positive example, a false positive example, and a false negative example; determine the evaluation scores of multiple initial autonomous driving images according to the comparison type; filter out a target autonomous driving image dataset from the initial autonomous driving image dataset according to the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image dataset.
  • the accuracy of image dataset processing can be improved.
  • FIG1 is a schematic diagram of a flow chart of an image data set processing method provided in an embodiment of the present application.
  • FIG2 is a flow chart of another method for processing an image data set provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of the effect of the intersection-and-union ratio and the intersection-and-union ratio evaluation score provided in an embodiment of the present application;
  • FIG4 is a schematic diagram of the effect between a confidence level and a second evaluation score provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of the effect between a confidence level and a first evaluation score provided in an embodiment of the present application
  • FIG6 is a schematic diagram of an effect between an area and a second evaluation score provided in an embodiment of the present application.
  • FIG7 is a schematic diagram of an effect between an area and a first evaluation score provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of the effect of the first image detection result provided in an embodiment of the present application.
  • FIG9 is a schematic diagram of the effect of the second image detection result provided in an embodiment of the present application.
  • FIG10 is a schematic diagram of the structure of an image data set processing device provided in an embodiment of the present application.
  • FIG. 11 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
  • Figure 1 is a flow chart of an image dataset processing method provided in an embodiment of the present application; this embodiment can be applied to the case of screening a target autonomous driving image dataset from an initial autonomous driving image dataset.
  • this embodiment is not limited to the scene target detection task of the autonomous driving front-view camera, but can also be other scene tasks. When switching scene tasks, the relevant configuration information is modified according to the model capabilities and project requirements.
  • the case of screening a target autonomous driving image dataset from an initial autonomous driving image dataset can also be understood as a case of data mining, in order to find the most valuable difficult samples in massive data, and give priority to labeling to accelerate model iteration.
  • the method can be executed by an image dataset processing device, and specifically includes the following steps:
  • the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations.
  • the annotations include detection box information, category, confidence, area and other information corresponding to the initial autonomous driving image.
  • the area can be a 3D target volume or a projected area.
  • the initial autonomous driving image may be acquired by a high-level autonomous driving test vehicle.
  • the autonomous driving image data set may include annotations or not, and this embodiment does not limit this.
  • the first autonomous driving detection model may be a server-side model, which has sufficient computing power and can be processed offline, and may be referred to as a large-scale model.
  • the second autonomous driving detection model may be an edge-side model, which has limited computing power and high real-time requirements, and may be referred to as a small-scale model.
  • each initial autonomous driving image includes at least one target object;
  • the first image detection result includes first detection frame information of at least one target object, a first category corresponding to at least one target object, a first confidence level of the first category, and a first area corresponding to the first detection frame information;
  • the second image detection result includes second detection frame information of at least one target object, a second category corresponding to at least one target object, a second confidence level corresponding to the second category, and a second area corresponding to the second detection frame information.
  • the target object may be a vehicle, a railing, a pedestrian, a house, etc., and this embodiment does not limit this.
  • the detection box information may include the length, width, and height of the detection box.
  • the first image detection result and the second image detection result may be compared to obtain a corresponding comparison result, and the comparison result includes a consistent comparison and an inconsistent comparison.
  • the comparison result includes a category comparison result
  • the comparison result is obtained by comparing the first image detection result and the second image detection result, including: comparing the first category and the second category to obtain a category comparison result; wherein the category comparison result includes category comparison consistency and category comparison inconsistency.
  • a category comparison result can be obtained, and the comparison type is determined based on the category comparison result.
  • S140 Determine the comparison type according to the comparison result.
  • the first image detection result is used as the benchmark detection result; the comparison types include true positive (TP), false positive (FP) and false negative (FN). For example, if the category comparison is consistent, the comparison type is a true positive; if the category comparison is inconsistent, the comparison type is a false positive or a false negative.
  • TP true positive
  • FP false positive
  • FN false negative
  • determining the comparison type according to the comparison result includes: determining the intersection-and-union (IoU) according to the first detection frame information and the second detection frame information; determining the first IoU setting value according to the first area and/or the second area; if the IoU is greater than or equal to the first IoU setting value, and the category comparison result is a consistent category comparison, determining the comparison type as a true positive example; if the IoU is less than the first IoU setting value and or the category comparison result is an inconsistent category comparison, determining the comparison type as a false positive example and a false negative example; or; if the first image detection result includes the first detection frame information of the target object, and the second image detection result does not include the second detection frame information of the target object, then the comparison type is determined as a false negative example; if the first image detection result does not include the first detection frame information of the target object, and the second image detection result includes the second detection frame information of the target object, then the comparison type is determined as a false positive example.
  • the intersection-and-union ratio can be determined based on the first detection frame information and the second detection frame information; the size of the target object is determined based on the first area and/or the second area. If the first area and/or the second area is less than 32*32, the target object is a small target, and the corresponding first intersection-and-union ratio setting value can be 0.3; if the first area and/or the second area is greater than or equal to 32*32, the target object is a normal target, and the corresponding first intersection-and-union ratio setting value can be 0.5.
  • the comparison type is determined to be a true positive example. If the IoU is less than the first IoU setting value and or the category comparison result is inconsistent, the comparison type is determined to be a false positive example and a false negative example. Or; if the first image detection result includes the first detection frame information of the target object, and the second image detection result does not include the second detection frame information of the target object, the comparison type is determined to be a false negative example, that is, the first image detection result The detection frames that are extra compared to the second image detection result are all false negative examples.
  • the comparison type is determined to be a false positive example, that is, the detection frames that are extra compared to the first image detection result are all false positive examples.
  • the comparison type is determined based on the comparison result and the intersection and union ratio, or by judging whether the first image detection result and the second image detection result include detection frame information of the same target object, so that the comparison type can be accurately determined.
  • the evaluation score of each initial autonomous driving image is determined as follows: the evaluation scores of all target objects in the initial autonomous driving image are determined according to the comparison type, the evaluation scores of the target objects include the intersection-and-union score, the confidence score, the area score and the category score, and the evaluation score of the initial autonomous driving image is obtained according to the evaluation scores of all target objects and the scene score of the initial autonomous driving image, thereby obtaining the evaluation scores of all initial autonomous driving images.
  • a target autonomous driving image dataset can be screened out from the initial autonomous driving image dataset according to the evaluation scores of all the initial autonomous driving images.
  • the target autonomous driving image dataset can be understood as a dataset that has a greater impact on the detection accuracy of the second autonomous driving detection model (or other autonomous driving detection models), and can also be understood as a dataset with a greater target value.
  • the evaluation score of an initial autonomous driving image can obtain multiple value scores, and the maximum score is taken or other fusion score algorithms are used to obtain the final value score.
  • TTA Transmission Time Augmentation
  • normal mode normal mode
  • the technical solution disclosed in this embodiment obtains an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; the initial image dataset is input into a first autonomous driving detection model and a second autonomous driving detection model respectively, and a first image detection result and a second image detection result are obtained respectively; the first image detection result and the second image detection result are compared to obtain a comparison result; a comparison type is determined according to the comparison result; wherein the first image detection result The result is used as the benchmark detection result; the comparison type includes true positive examples, false positive examples, and false negative examples; the evaluation scores of multiple initial autonomous driving images are determined according to the comparison type; the target autonomous driving image dataset is screened out from the initial autonomous driving image dataset according to the evaluation scores of the multiple initial autonomous driving images, and the annotations in the target autonomous driving image dataset are adjusted.
  • the accuracy of image dataset processing can be improved by determining the comparison type according to the comparison result between the first image detection result and the second image detection result, determining the evaluation score of the initial autonomous driving image according to the comparison type; and screening out the target autonomous driving image dataset according to the evaluation score of the initial autonomous driving image.
  • FIG2 is a flow chart of another method for processing an image data set provided in an embodiment of the present application.
  • the present application embodiment is a specific implementation of the above-mentioned invention embodiment. Referring to FIG2 , the method provided in the embodiment of the present application specifically includes the following steps:
  • the evaluation score of the target object includes the intersection-over-union score, confidence score, area score, and category score of the target object.
  • the evaluation score of the target object is different for different comparison types.
  • the evaluation scores of all target objects or some target objects in the initial autonomous driving image can be determined according to the comparison type.
  • determining an evaluation score of the target object in the initial autonomous driving image according to the comparison type includes: determining at least one of an intersection-over-union score, a confidence score, and an area score of the target object according to the comparison type; obtaining a pre-set category score corresponding to the target object; and determining the evaluation score of the target object according to at least one of the intersection-over-union score, the confidence score, the area score, and the category score.
  • the evaluation score of each target object can be obtained in the following manner: obtain the intersection-and-union score, confidence score and area score of the target object according to the comparison type, and obtain the preset category score corresponding to the target object. Multiply or weighted average the intersection-and-union score, confidence score, area score and category score, and use the multiplied result or weighted average result as the evaluation score of the target object.
  • the elements for determining the evaluation score of the target object are not limited to the intersection-and-union score, confidence score, area score and category score, and corresponding elements can also be added or reduced.
  • each target object category has a preset corresponding category weight, that is, a category score, and the category score range is [0,1].
  • the category score corresponding to the measured category.
  • category weights the distribution information of the initial autonomous driving image dataset can be integrated, and higher category weights can be set for the long tail.
  • the evaluation results of the model on the test set can also be integrated to increase the weights for categories with poor accuracy.
  • the evaluation score of the target object is determined by at least one of the intersection-over-union score, the confidence score, the area score, and the category score, so that the evaluation score of the target object can be accurately determined.
  • determining the intersection-and-union score of the target object according to the comparison type includes: determining a second intersection-and-union setting value according to the first area and/or the second area; and determining an intersection-and-union evaluation score according to the comparison type, the first intersection-and-union setting value, the second intersection-and-union setting value, and the intersection-and-union ratio.
  • the second IoU setting value is greater than the first IoU setting value; the second IoU setting value can be understood as the maximum IoU threshold, and the first IoU setting value can be understood as the minimum IoU threshold.
  • the size of the target object is determined according to the first area and/or the second area. If the first area and/or the second area is less than 32*32, the target object is a small target, and the corresponding second IoU setting value can be 0.7. If the first area and/or the second area is greater than or equal to 32*32, the target object is a normal target, and the corresponding first IoU setting value can be 0.9.
  • intersection-over-union ratio threshold formula is as follows:
  • iou_thresh represents the intersection-and-union ratio threshold value.
  • the first intersection-and-union ratio setting value may be 0.3, and the second intersection-and-union ratio setting value may be 0.7.
  • the first intersection-and-union ratio setting value may be 0.5, and the second intersection-and-union ratio setting value may be 0.9.
  • intersection-and-union ratio evaluation score can be directly obtained. If the comparison type is a true positive example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, the intersection-and-union ratio evaluation score can be directly obtained.
  • intersection-and-union ratio evaluation score can be determined based on the first intersection-and-union ratio setting value, the second intersection-and-union ratio setting value and the intersection-and-union ratio.
  • the IOR evaluation score can be accurately determined by comparing the type, the first IOR setting value, the second IOR setting value, and the IOR evaluation score determined by the IOR.
  • the intersection-and-union evaluation score is determined according to the comparison type, the first intersection-and-union setting value, the second intersection-and-union setting value, and the intersection-and-union ratio, including: if the comparison type is a false positive example or a false negative example, and the intersection-and-union ratio is less than the first intersection-and-union ratio setting value, then the intersection-and-union evaluation score is the first set intersection-and-union evaluation score; if the comparison type is a true positive example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, then the intersection-and-union evaluation score is the second set intersection-and-union evaluation score; if the comparison type is a true positive example, and the intersection-and-union ratio falls within the interval formed by the first intersection-and-union ratio setting value and the second intersection-and-union ratio setting value, then the intersection-and-union evaluation score is determined according to the second intersection-and-union ratio setting value and the intersection-and-union ratio.
  • intersection-over-union ratio evaluation score formula is as follows:
  • iou_score is the intersection-and-union ratio evaluation score
  • min_iou is the first intersection-and-union ratio setting value
  • max_iou is the second intersection-and-union ratio setting value
  • iou is the intersection-and-union ratio.
  • the first setting intersection-and-union ratio evaluation score is 1, and the second setting intersection-and-union ratio evaluation score is 0.
  • the comparison type when the comparison type is a false positive example or a false negative example, and the intersection and union ratio is less than the first intersection and union ratio setting value, the value is the highest, and the intersection and union ratio evaluation score is 1; when the comparison type is a true positive example, and the intersection and union ratio is greater than or equal to the second intersection and union ratio setting value, it can be considered that the model does not need to pay attention to small marginal differences, has no value, and the intersection and union ratio evaluation score is 0; when the comparison type is a true positive example, and the intersection and union ratio falls within the interval formed by the first intersection and union ratio setting value and the second intersection and union ratio setting value, the lower the intersection and union ratio, the higher the value, the first intersection and union ratio evaluation score is max_iou-iou, the second intersection and union ratio evaluation score is max_iou-min_iou, and the intersection and union ratio evaluation score is
  • FIG3 is a schematic diagram of the effect of the intersection-and-union ratio and the intersection-and-union ratio evaluation score provided in an embodiment of the present application.
  • the horizontal axis is the intersection-and-union ratio iou
  • the vertical axis is the intersection-and-union ratio evaluation score iou_score. It can be seen that if the target object is a small target or a normal target, the intersection-and-union ratio and the intersection-and-union ratio evaluation score are inversely proportional.
  • the first set intersection-and-union ratio evaluation score is used as the intersection-and-union ratio evaluation score; when the comparison type is a true positive example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, the second set intersection-and-union ratio evaluation score is used as the intersection-and-union ratio evaluation score; when the comparison type is a true positive example, and the intersection-and-union ratio falls within the interval formed by the first intersection-and-union ratio setting value and the second intersection-and-union ratio setting value, the intersection-and-union ratio evaluation score is determined according to the first intersection-and-union ratio evaluation score and the second intersection-and-union ratio evaluation score, so that the intersection-and-union ratio evaluation score can be accurately determined.
  • determining the confidence score of the target object according to the comparison type includes: if the comparison type is a false positive, determining a first evaluation score of a second confidence level, and using the first evaluation score of the second confidence level as is the confidence score of the target object; if the comparison type is a false negative example, a first evaluation score of a first confidence level is determined, and the first evaluation score of the first confidence level is used as the confidence score of the target object; if the comparison type is a true positive example, a second evaluation score of the first confidence level and a second evaluation score of the second confidence level are determined respectively, and the confidence score of the target object is determined according to the second evaluation score of the first confidence level and the second evaluation score of the second confidence level.
  • the formula for determining the confidence score of the target object is as follows:
  • matched_conf_score is the confidence score of the target object
  • conf_score small1 is the first evaluation score of the second confidence when the matching type is a false positive example, that is, the confidence score of the target object when the matching type is a false positive example
  • conf_score big1 is the first evaluation score of the first confidence when the matching type is a false negative example, that is, the confidence score of the target object when the matching type is a false negative example.
  • conf_score big2 is the second evaluation score of the first confidence
  • conf_score small2 is the second evaluation score of the second confidence.
  • the comparison type is a false positive example
  • the first evaluation score of the second confidence level output by the second autonomous driving detection model is determined, and the first evaluation score of the second confidence level is used as the confidence score of the target object when the comparison type is a false positive example
  • the comparison type is a false negative example
  • the first evaluation score of the first confidence level output by the first autonomous driving detection model is determined, and the first evaluation score of the first confidence level is used as the confidence score of the target object when the comparison type is a false negative example
  • the comparison type is a true positive example
  • the second evaluation score of the first confidence level and the second evaluation score of the second confidence level are respectively determined, and the confidence score of the target object when the comparison type is a true positive example is determined according to the second evaluation score of the first confidence level and the second evaluation score of the second confidence level, so that the confidence score of the target object can be accurately determined.
  • determining a first evaluation score of a first confidence level, or determining a first evaluation score of a second confidence level includes: obtaining a first confidence threshold and a second confidence threshold; wherein the second confidence threshold is greater than the first confidence threshold; if the first confidence level or the second confidence level is less than the first confidence threshold, using the first set confidence score as the first evaluation score of the first confidence level or the first evaluation score of the second confidence level; if the first confidence level is greater than or equal to the second confidence threshold, or the second confidence level is greater than if the first confidence level is less than or equal to the second confidence threshold, the second set confidence score is used as the first evaluation score of the first confidence level or the first evaluation score of the second confidence level; if the first confidence level falls within the interval formed by the first confidence threshold and the second confidence threshold, the first confidence score is determined according to the first confidence level, the second confidence threshold and the first confidence threshold; the second confidence score is determined according to the second confidence threshold and the first confidence threshold; the first evaluation score of the first confidence level is determined according to the
  • the first evaluation score formula for determining the second confidence level is as follows:
  • the second confidence threshold may be 0.5, the first confidence threshold may be 0.3, the first set confidence score may be 0, and the second set confidence score may be 1.
  • conf is the second confidence level; if the second confidence level is less than the first confidence level threshold, the first evaluation score of the second confidence level is 0, which is worthless; if the second confidence level is greater than or equal to the second confidence level threshold, the first evaluation score of the second confidence level is 1, which is of the greatest value; if the second confidence level falls within the interval formed by the first confidence level threshold and the second confidence level threshold, then is the first evaluation score of the second confidence level.
  • the first evaluation score formula for determining the first confidence is as follows:
  • conf is the first confidence level; if the first confidence level is less than the first confidence level threshold, the first evaluation score of the first confidence level is 0, which is worthless; if the first confidence level is greater than or equal to the second confidence level threshold, the first evaluation score of the first confidence level is 1, which is the most valuable; if the first confidence level falls within the interval formed by the first confidence level threshold and the second confidence level threshold, then is the first evaluation score of the first confidence level.
  • the first set confidence score is used as the first evaluation score of the first confidence; if the first confidence is greater than or equal to the second confidence threshold, the second set confidence score is used as the first evaluation score of the first confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the first evaluation score of the first confidence is determined according to the first confidence score and the second confidence score, so that the first evaluation score of the first confidence can be accurately determined.
  • determining the second evaluation score of the first confidence level, or determining the second evaluation score of the second confidence level includes: if the first confidence level or the second confidence level is less than a first confidence level threshold, using the first set confidence level score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level; if the first confidence level is greater than or equal to the first confidence level threshold, or the second confidence level is greater than or equal to the second confidence level threshold, using the first set confidence level score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level; if the first confidence level falls within the range formed by the first confidence level threshold and the second confidence level threshold, If the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, a fifth confidence score is determined according to the first confidence threshold and the second confidence threshold; a sixth confidence score is determined according to the second confidence threshold and the first confidence threshold; a second evaluation score of the first confidence is determined according to the fifth confidence score and the sixth confidence score; or; if the second confidence falls
  • the formula for determining the second evaluation score of the second confidence is as follows:
  • conf1 is the second confidence; if the second confidence is less than the first confidence threshold, the second evaluation score of the second confidence is 0, which is worthless; if the second confidence is greater than or equal to the second confidence threshold, the second evaluation score of the second confidence is 0, which is worthless; if the second confidence is greater than or equal to the second confidence threshold, the second evaluation score of the second confidence is 0, which is worthless; The confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, then is the second evaluation score of the second confidence level.
  • the first set confidence score is used as the second evaluation score of the second confidence; if the second confidence is greater than or equal to the second confidence threshold, the first set confidence score is used as the second evaluation score of the second confidence; if the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the second evaluation score of the second confidence is determined according to the seventh confidence score and the eighth confidence score, so that the second evaluation score of the second confidence can be accurately determined.
  • the formula for determining the second evaluation score of the first confidence is as follows:
  • conf2 is the first confidence; if the first confidence is less than the first confidence threshold, the second evaluation score of the first confidence is 0; if the first confidence is greater than or equal to the first confidence threshold, the second evaluation score of the first confidence is 0; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, then is the second evaluation score of the first confidence.
  • the first set confidence score is used as the second evaluation score of the first confidence; if the first confidence is greater than or equal to the first confidence threshold, the first set confidence score is used as the second evaluation score of the first confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the second evaluation score of the first confidence is determined according to the fifth confidence score and the sixth confidence score, so that the second evaluation score of the first confidence can be accurately determined.
  • FIG4 is a schematic diagram of the effect between a confidence level and a second evaluation score provided in an embodiment of the present application.
  • the horizontal axis is conf
  • conf is the first confidence level or the second confidence level
  • min_conf is the first confidence threshold
  • max_conf is the second confidence threshold.
  • the vertical axis is conf_score
  • conf_score is the second evaluation score of the first confidence level or the second confidence level.
  • the second evaluation score is 0 and has no value.
  • the second evaluation score (which can also be understood as the value) is proportional to the confidence level. Inversely proportional.
  • FIG5 is a schematic diagram of the effect between a confidence level and a first evaluation score provided in an embodiment of the present application.
  • the horizontal axis is conf
  • conf is the first confidence level or the second confidence level
  • min_conf is the first confidence level threshold
  • max_conf is the second confidence level threshold
  • the vertical axis is conf_score
  • conf_score is the first evaluation score of the first confidence level or the second confidence level.
  • the first evaluation score is all 0 and has no value.
  • the value is the largest, and the first evaluation score is all 1.
  • the first evaluation score (which can also be understood as the value) is proportional to the confidence level.
  • determining the area score of the target object according to the comparison type includes: if the comparison type is a false positive example, determining the first evaluation score of the second area, and using the first evaluation score of the second area as the area score of the target object; if the comparison type is a false negative example, determining the first evaluation score of the first area, and using the first evaluation score of the first area as the area score of the target object; if the comparison type is a true positive example, respectively determining the second evaluation score of the first area and the second evaluation score of the second area, and determining the area score of the target object based on the second evaluation score of the first area and the second evaluation score of the second area.
  • the area score formula for determining the target object is as follows:
  • matched_area_score is the area score of the target object
  • area_score small1 is the first evaluation score of the second area when the matching type is a false positive example, that is, the area score of the target object when the matching type is a false positive example.
  • area_score big1 is the first evaluation score of the first area when the matching type is a false negative example, that is, the area score of the target object when the matching type is a false negative example.
  • area_score big2 is the second evaluation score of the first area
  • area_score small2 is the second evaluation score of the second area.
  • the comparison type is a false positive example
  • the first evaluation score of the second area is determined, and the first evaluation score of the second area is used as the area score of the target object when the comparison type is a false positive example
  • the comparison type is a false negative example
  • the first evaluation score of the first area is determined, and the first evaluation score of the first area is used as the area score of the target object when the comparison type is a false negative example
  • the comparison type is a true positive example
  • the target object is determined according to the second evaluation score of the first area and the second evaluation score of the second area.
  • the area score of the target object can be accurately determined by using the area score of the target object when the comparison type is a true positive.
  • determining a first evaluation score for a first area, or determining a first evaluation score for a second area includes: obtaining a first area threshold and a second area threshold; wherein the second area threshold is greater than the first area threshold; if the first area or the second area is less than the first area threshold, using the first set area score as the first evaluation score for the first area or the first evaluation score for the second area; if the first area is greater than or equal to the second area threshold, or the second area is greater than or equal to the second area threshold, using the second set area score as the first evaluation score for the first area or the first evaluation score for the second area; if the first area falls within an interval formed by the first area threshold and the second area threshold, determining the first area score based on the first area, the second area threshold and the first area threshold; determining the second area score based on the second area threshold and the first area threshold; determining the first evaluation score for the first area based on the first area score and the second area score; or; if the second area falls within an interval
  • the first area threshold can be 30*30
  • the second area threshold can be 500*500
  • the first set area score can be 0
  • the second set area score can be 1.
  • the formula for determining the first evaluation score of the first area is as follows:
  • area is the first area. If the first area is less than the first area threshold, the first evaluation score of the first area is 0; if the first area is greater than or equal to the second area threshold, the first evaluation score of the first area is 1; if the first area falls within the interval formed by the first area threshold and the second area threshold, then A first assessment score is given for the first area.
  • the first set area score is used as the first evaluation score of the first area; if the first area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the first area; if the first area falls within the interval formed by the first area threshold and the second area threshold, the first evaluation score of the first area is determined according to the first area score and the second area score, so that the first evaluation score of the first area can be accurately determined.
  • the formula for determining the first evaluation score of the second area is as follows:
  • area is the second area; if the second area is less than the first area threshold, the first evaluation score of the second area is 0; if the second area is greater than or equal to the second area threshold, the first evaluation score of the second area is 1; if the second area falls within the interval formed by the first area threshold and the second area threshold, then A score is given for the first assessment of the second area.
  • the first set area score is used as the first evaluation score of the second area; if the second area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the second area; if the second area falls within the interval formed by the first area threshold and the second area threshold, the first evaluation score of the second area is determined according to the third area score and the fourth area score, so that the first evaluation score of the second area can be accurately determined.
  • determining the second evaluation score of the first area, or determining the second evaluation score of the second area includes: if the first area or the second area is less than the first area threshold, using the first set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area is greater than or equal to the first area threshold, or if the second area is greater than or equal to the first area threshold, using the second set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area value, determining a fifth area score based on the first area and the first area threshold; determining a sixth area score based on the second area threshold and the first area threshold; determining the second evaluation score of the first area based on the fifth area score and the sixth area score; or; if the second area falls within the interval formed by the first area threshold and the second area value, determining a seventh area score based on the second area and the first area threshold; determining
  • the formula for determining the second evaluation score of the first area is as follows:
  • conf2 is the first area; if the first area is less than the first area threshold, the second evaluation score of the first area is 0; if the first area is greater than or equal to the first area threshold, the second evaluation score of the first area is 1; if the first area falls within the interval formed by the first area threshold and the second area value, then A second evaluation score is given for the first area.
  • the first set area score is used as is the second evaluation score of the first area; if the first area is greater than or equal to the first area threshold, the second set area score is used as the second evaluation score of the first area; if the first area falls within the interval formed by the first area threshold and the second area value, the second evaluation score of the first area is determined according to the fifth area score and the sixth area score, so that the second evaluation score of the first area can be accurately determined.
  • the formula for determining the second evaluation score of the second area is as follows:
  • area2 is the second area; if the second area is less than the first area threshold, the second evaluation score of the second area is 0; if the second area is greater than or equal to the first area threshold, the second evaluation score of the second area is 1; if the second area falls within the interval formed by the first area threshold and the second area value, then A second assessment score is provided for the second area.
  • the first set area score is used as the second evaluation score of the second area; if the second area is greater than or equal to the first area threshold, the second set area score is used as the second evaluation score of the second area; if the second area falls within the interval formed by the first area threshold and the second area value, the second evaluation score of the second area is determined according to the seventh area score and the eighth area score, so that the second evaluation score of the second area can be accurately determined.
  • FIG6 is a schematic diagram of the effect between an area and a second evaluation score provided in an embodiment of the present application.
  • the horizontal axis is area
  • area is the first area or the second area
  • min_area is the first area threshold
  • max_conf is the second area threshold.
  • the vertical axis is area_score
  • area_score is the second evaluation score.
  • the first area or the second area is less than the first area threshold
  • the second evaluation score is 0 and has no value.
  • the first area or the second area is greater than or equal to the second area threshold, the value is the largest, and the second evaluation scores are all 1.
  • the second evaluation score (which can also be understood as the value) is proportional to the area.
  • FIG7 is a schematic diagram of the effect between an area and a first evaluation score provided in an embodiment of the present application.
  • the horizontal axis is area
  • area is the first area or the second area
  • min_area is the first area threshold
  • max_area is the second area threshold
  • the vertical axis is area_score
  • area_score is the first evaluation score of the area.
  • the first evaluation score is all 0 and has no value.
  • the first area or the second area is greater than or equal to the second area threshold, the value is the largest, and the first evaluation score is all 1.
  • the first evaluation score (which can also be understood as the value) is The areas are directly proportional.
  • the scene of the initial autonomous driving image there is no restriction on the scene of the initial autonomous driving image, such as rainy scenes, snowy scenes, traffic jam scenes, off-road scenes, etc.
  • actual scene information can be obtained based on the initial autonomous driving image, the initial autonomous driving image is input into the autonomous driving scene model, predicted scene information is output, and a scene score is obtained based on the similarity between the predicted scene information and the actual scene information.
  • determining a scene score of an initial autonomous driving image includes: inputting the initial autonomous driving image into an autonomous driving scene model and outputting predicted scene information; obtaining set scene information corresponding to the initial autonomous driving image; determining scene similarity between the predicted scene information and the set scene information; and determining a scene score of the initial autonomous driving image based on the scene similarity.
  • the autonomous driving scene model may be a scene model based on any deep learning algorithm.
  • the set scene information may be the real scene information corresponding to the initial autonomous driving image.
  • the initial autonomous driving image is input into the autonomous driving scene model, and the corresponding predicted scene information is output; the set scene information corresponding to the initial autonomous driving image is obtained, and the scene similarity between the predicted scene information and the set scene information is calculated based on any similarity algorithm; and the scene score of the initial autonomous driving image is determined according to the scene similarity.
  • the range of scene similarity is [0, 1], that is, the range of scene score is [0, 1].
  • the scene score of the initial autonomous driving image is determined by predicting the scene similarity between the scene information and the set scene information, so that the scene score of the initial autonomous driving image can be accurately determined.
  • S207 Determine an evaluation score of the initial autonomous driving image according to the evaluation score of the target object and the scene score.
  • the evaluation score of any initial autonomous driving image is calculated as follows: the evaluation scores of all target objects in the current initial autonomous driving image are accumulated to obtain an accumulated result, and then the accumulated result and the scene score are multiplied to obtain the evaluation score of the current initial autonomous driving image.
  • the formula for determining the evaluation score of the initial autonomous driving image is as follows:
  • img_value_score represents the evaluation score of the initial autonomous driving image
  • scene_score represents the scene score
  • bbox_value_score represents the evaluation score of the target object.
  • the initial autonomous driving images in the initial autonomous driving image data set can be sorted from high to low according to the evaluation scores of each initial autonomous driving image, and a set number of initial autonomous driving images with the highest sorting scores are extracted as a set number of target autonomous driving images, and the annotations corresponding to the set number of target autonomous driving images are adjusted.
  • a target autonomous driving image dataset is screened out from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, including: sorting the multiple initial autonomous driving images according to the evaluation scores of the multiple initial autonomous driving images to obtain a sorted initial autonomous driving image dataset; and screening out a set number of target autonomous driving image datasets from the sorted initial autonomous driving image dataset.
  • multiple initial autonomous driving images are sorted from high to low according to their evaluation scores to obtain a sorted initial autonomous driving image dataset; a set number of target autonomous driving image datasets ranked at the top are selected from the sorted initial autonomous driving image dataset, that is, a set number of initial autonomous driving images are extracted as the set number of target autonomous driving images.
  • multiple initial autonomous driving images are sorted from low to high according to their evaluation scores to obtain a sorted initial autonomous driving image dataset; a set number of target autonomous driving image datasets ranked at the bottom are selected from the sorted initial autonomous driving image dataset, that is, a set number of initial autonomous driving images are extracted as the set number of target autonomous driving images.
  • the target autonomous driving images can be accurately screened out.
  • FIG8 is a schematic diagram of the effect of the first image detection result provided in an embodiment of the present application
  • FIG9 is a schematic diagram of the effect of the second image detection result provided in an embodiment of the present application.
  • FIG8 is the first image detection result obtained by the first autonomous driving detection model
  • FIG9 is the second image detection result obtained by the second autonomous driving detection model.
  • a group of bicycles detected by the first autonomous driving detection model only has one frame, while each bicycle in the second autonomous driving detection model has a separate frame, so the frames of all bicycles are not matched, and the comparison type is FN or FP; the last column in FIG8 or FIG9 is detected by the first autonomous driving detection model, but not by the second autonomous driving detection model, and the comparison type is FN, with a high value score; therefore, most of the evaluation scores of the target objects are the evaluation scores of FN and FP, and the scores are very high.
  • FIG10 is a schematic diagram of the structure of an image data set processing device provided in an embodiment of the present application.
  • the device includes: an initial autonomous driving image data set acquisition module 1001, an image detection result acquisition module 1002, an image detection result comparison module 1003, a comparison type determination module 1004, and an evaluation score module 1005.
  • An initial autonomous driving image data set acquisition module 1001 is used to acquire an initial autonomous driving image data set; wherein the initial autonomous driving image data set includes a plurality of initial autonomous driving images and corresponding annotations;
  • An image detection result obtaining module 1002 is used to input the initial image data set into the first autonomous driving detection model and the second autonomous driving detection model respectively to obtain a first image detection result and a second image detection result respectively;
  • An image detection result comparison module 1003 is used to compare the first image detection result and the second image detection result to obtain a comparison result
  • a comparison type determination module 1004 is used to determine a comparison type according to the comparison result; wherein the first image detection result is used as a reference detection result; and the comparison type includes true positive examples, false positive examples, and false negative examples;
  • An evaluation score determination module 1005, configured to determine evaluation scores of the plurality of initial autonomous driving images according to the comparison type
  • the target autonomous driving image data set screening module 1006 is used to screen out a target autonomous driving image data set from the initial autonomous driving image data set according to the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image data set.
  • the technical solution disclosed in this embodiment is to obtain an initial autonomous driving image dataset through an initial autonomous driving image dataset acquisition module; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; through an image detection result acquisition module, the initial image dataset is respectively input into a first autonomous driving detection model and a second autonomous driving detection model to obtain a first image detection result and a second image detection result respectively; through an image detection result comparison module, the first image detection result and the second image detection result are compared to obtain a comparison result; through a comparison type determination module, a comparison type is determined according to the comparison result; wherein the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples and false negative examples; through an evaluation score determination module, evaluation scores of multiple initial autonomous driving images are determined according to the comparison type; through a target autonomous driving image dataset screening module, a target autonomous driving image dataset is screened out from the initial autonomous driving image dataset according to the evaluation scores of multiple initial autonomous driving images, and the annotations in the target autonomous driving image dataset are adjusted.
  • a comparison type is determined based on a comparison result between a first image detection result and a second image detection result, and an evaluation score of an initial autonomous driving image is determined based on the comparison type; and a target autonomous driving image data set is screened out based on the evaluation score of the initial autonomous driving image. This can improve the accuracy of image data set processing.
  • each initial autonomous driving image includes at least one target object;
  • the image detection result includes first detection frame information of at least one target object, a first category corresponding to the at least one target object, a first confidence level of the first category, and a first area corresponding to the first detection frame information;
  • the second image detection result includes second detection frame information of at least one target object, a second category corresponding to the at least one target object, a second confidence level corresponding to the second category, and a second area corresponding to the second detection frame information;
  • the comparison result includes a category comparison result.
  • an image detection result comparison module is specifically used to: compare the first category and the second category to obtain a category comparison result; wherein the category comparison result includes category comparison consistency and category comparison inconsistency.
  • the comparison type determination module is specifically used to: determine the intersection-and-union (IoU) according to the first detection frame information and the second detection frame information; determine the first IoU setting value according to the first area and/or the second area; if the IoU is greater than or equal to the first IoU setting value, and the category comparison result is a consistent category comparison, determine the comparison type as a true positive example; if the IoU is less than the first IoU setting value and or the category comparison result is an inconsistent category comparison, determine the comparison type as a false positive example and a false negative example; or; if the first image detection result includes the first detection frame information of the target object, and the second image detection result does not include the second detection frame information of the target object, determine the comparison type as a false negative example; if the first image detection result does not include the first detection frame information of the target object, and the second image detection result includes the second detection frame information of the target object, determine the comparison type as a false positive example.
  • IoU intersection-and-union
  • the evaluation score determination module is specifically used to: for the evaluation score of any initial autonomous driving image, determine the evaluation score of the target object in the initial autonomous driving image according to the comparison type; determine the scene score of the initial autonomous driving image; and determine the evaluation score of the initial autonomous driving image according to the evaluation score of the target object and the scene score.
  • the evaluation score determination module is also used to: determine at least one of the intersection-and-union score, confidence score and area score of the target object based on the comparison type; obtain a preset category score corresponding to the target object; and determine the evaluation score of the target object based on at least one of the intersection-and-union score, the confidence score, the area score and the category score.
  • the evaluation score determination module is also used to: determine a second intersection-and-union ratio setting value based on the first area and/or the second area; wherein the second intersection-and-union ratio setting value is greater than the first intersection-and-union ratio setting value; and determine the intersection-and-union ratio evaluation score based on the comparison type, the first intersection-and-union ratio setting value, the second intersection-and-union ratio setting value, and the intersection-and-union ratio.
  • the evaluation score determination module is also used to: if the comparison type is a false positive example or a false negative example, and the intersection-and-union ratio is less than the first intersection-and-union ratio setting value, then the intersection-and-union ratio evaluation score is the first setting intersection-and-union ratio evaluation score; if the comparison type is a true positive example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, then the intersection-and-union ratio evaluation score is the second ...
  • the first I/O ratio evaluation score is determined based on the second I/O ratio setting value and the I/O ratio; the second I/O ratio evaluation score is determined based on the second I/O ratio setting value and the first I/O ratio setting value; and the I/O ratio evaluation score is determined based on the first I/O ratio evaluation score and the second I/O ratio evaluation score.
  • the evaluation score determination module is also used to: if the comparison type is a false positive example, determine the first evaluation score of the second confidence level, and use the first evaluation score of the second confidence level as the confidence score of the target object; if the comparison type is a false negative example, determine the first evaluation score of the first confidence level, and use the first evaluation score of the first confidence level as the confidence score of the target object; if the comparison type is a true positive example, respectively determine the second evaluation score of the first confidence level and the second evaluation score of the second confidence level, and determine the confidence score of the target object based on the second evaluation score of the first confidence level and the second evaluation score of the second confidence level.
  • the evaluation score determination module is also used to: obtain a first confidence threshold and a second confidence threshold; wherein the second confidence threshold is greater than the first confidence threshold; if the first confidence or the second confidence is less than the first confidence threshold, the first set confidence score is used as the first evaluation score of the first confidence or the first evaluation score of the second confidence; if the first confidence is greater than or equal to the second confidence threshold, or the second confidence is greater than or equal to the second confidence threshold, the second set confidence score is used as the first evaluation score of the first confidence or the first evaluation score of the second confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the second set confidence score is used as the first evaluation score of the first confidence or the first evaluation score of the second confidence according to the The first confidence score is determined by the first confidence, the second confidence threshold and the first confidence threshold; the second confidence score is determined according to the second confidence threshold and the first confidence threshold; the first evaluation score of the first confidence is determined according to the first confidence score and the second confidence score; or; if the
  • the evaluation score determination module is also used to: if the first confidence or the second confidence is less than a first confidence threshold, use the first set confidence score as the second evaluation score of the first confidence or the second evaluation score of the second confidence; if the first confidence is greater than or equal to the first confidence threshold, or the second confidence is greater than or equal to the second confidence threshold, use the first set confidence score as the second evaluation score of the first confidence or the second evaluation score of the second confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, determine a fifth confidence score according to the first confidence and the second confidence threshold; determine a sixth confidence score according to the second confidence threshold and the first confidence threshold; determine the second evaluation score of the first confidence according to the fifth confidence score and the sixth confidence score; or; if the second confidence If it falls within the interval formed by the first confidence threshold and the second confidence threshold, a seventh confidence score is determined according to the second confidence threshold and the second confidence threshold; an eighth confidence score is determined according to the second confidence threshold and the first confidence threshold; and a second confidence threshold
  • the evaluation score determination module is also used to: if the comparison type is a false positive example, determine the first evaluation score of the second area, and use the first evaluation score of the second area as the area score of the target object; if the comparison type is a false negative example, determine the first evaluation score of the first area, and use the first evaluation score of the first area as the area score of the target object; if the comparison type is a true positive example, respectively determine the second evaluation score of the first area and the second evaluation score of the second area, and determine the area score of the target object based on the second evaluation score of the first area and the second evaluation score of the second area.
  • the evaluation score determination module is also used to: obtain a first area threshold and a second area threshold; wherein the second area threshold is greater than the first area threshold; if the first area or the second area is less than the first area threshold, the first set area score is used as the first evaluation score of the first area or the first evaluation score of the second area; if the first area is greater than or equal to the second area threshold, or the second area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the first area or the first evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area threshold, the first area score is determined according to the first area, the second area threshold and the first area threshold; the second area score is determined according to the second area threshold and the first area threshold; the first evaluation score of the first area is determined according to the first area score and the second area score; or; if the second area falls within the interval formed by the first area threshold and the second area threshold, the third area score is determined according to the
  • the evaluation score determination module is also used to: if the first area or the second area is less than a first area threshold, use the first set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area is greater than or equal to the first area threshold, or, if the second area is greater than or equal to the first area threshold, use the second set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area value, determine the fifth area score according to the first area and the first area threshold; determine the sixth area score according to the second area threshold and the first area threshold; determine the second evaluation score of the first area according to the fifth area score and the sixth area score; or; if the second area falls within the interval formed by the first area threshold and the second area value, determine the seventh area score according to the second area and the first area threshold; determine the eighth area score according to the second area threshold and the first area threshold; determine the second evaluation score of the second area
  • the evaluation score determination module is also used to: input the initial autonomous driving image into an autonomous driving scene model and output predicted scene information; obtain set scene information corresponding to the initial autonomous driving image; determine the scene similarity between the predicted scene information and the set scene information; and determine the scene score of the initial autonomous driving image based on the scene similarity.
  • the target autonomous driving image dataset screening module is specifically used to: sort the multiple initial autonomous driving images according to their evaluation scores to obtain a sorted initial autonomous driving image dataset; and screen out a set number of target autonomous driving image datasets from the sorted initial autonomous driving image dataset.
  • the image data set processing device provided in the embodiments of the present application can execute the image data set processing method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.
  • FIG11 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Referring to FIG11 below, it shows a schematic diagram of the structure of an electronic device (e.g., a terminal device or server in FIG11) 1100 suitable for implementing an embodiment of the present application.
  • the terminal device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG11 is merely an example and should not impose any restrictions on the functions and scope of use of the embodiments of the present application.
  • the electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage device 1108 to a random access memory (RAM) 1103.
  • a processing device 1101 e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • Various programs and data required for the operation of the electronic device 1100 are also stored in the RAM 1103.
  • the processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104.
  • An edit/output (I/O) interface 1105 is also connected to the bus 1104.
  • the following devices may be connected to the I/O interface 1105: input devices 1106 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1108 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1109.
  • the communication devices 1109 may allow the electronic device 1100 to communicate wirelessly or wired with other devices to exchange data.
  • FIG. 11 shows an electronic device 1100 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from the network through the communication device 1109, or installed from the storage device 1108, or installed from the ROM 1102.
  • the processing device 1101 the above-mentioned functions defined in the method of the embodiment of the present application are executed.
  • the electronic device provided in the embodiment of the present application and the image data set processing method provided in the above embodiment belong to the same inventive concept.
  • the technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
  • An embodiment of the present application provides a computer storage medium on which a computer program is stored.
  • the program is executed by a processor, the image data set processing method provided in the above embodiment is implemented.
  • the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code.
  • This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network (“WAN”), an internetwork (eg, the Internet), and a peer-to-peer network (eg, an ad hoc peer-to-peer network), as well as any network currently known or developed in the future.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs.
  • the electronic device obtains an initial autonomous driving image data set; wherein the initial autonomous driving image data set includes multiple initial autonomous driving images and corresponding annotations; inputs the initial image data set into a first autonomous driving detection model and a second autonomous driving detection model respectively, and obtains a first image detection result and a second image detection result respectively; compares the first image detection result with the second image detection result to obtain a comparison result; determines a comparison type according to the comparison result; wherein the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples, and false negative examples; determines the evaluation scores of the multiple initial autonomous driving images according to the comparison type; screens out a target autonomous driving image data set from the initial autonomous driving image data set according to the evaluation scores of the multiple initial autonomous driving images, and adjusts the annotations in the target autonomous driving image data set.
  • Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., via the Internet using an Internet service provider
  • each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing the specified logical functions.
  • the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flowchart, and the combination of boxes in the block diagram and/or flowchart can be implemented with a dedicated hardware-based system that performs the specified functions or operations, or can be implemented with a dedicated It is implemented by a combination of hardware and computer instructions.
  • the units involved in the embodiments described in the present application may be implemented by software or hardware.
  • the name of the unit does not limit the unit itself in some cases.
  • the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chips
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开一种图像数据集处理方法、装置、设备及存储介质，该方法包括：获取初始自动驾驶图像数据集；将初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；比对第一图像检测结果和第二图像检测结果，获得比对结果；根据比对结果确定比对类型；根据比对类型确定多个初始自动驾驶图像的评估得分；根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对目标自动驾驶图像数据集中的标注进行调整。本申请实施例，可以提高图像数据集处理的准确率。

Description

图像数据集处理方法、装置、设备及存储介质
本申请要求在2023年07月26日提交中国专利局、申请号为202310922726.X的中国专利申请的优先权，以上申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及图像处理技术领域，尤其涉及一种图像数据集处理方法、装置、设备及存储介质。
背景技术
众所周知，自动驾驶的发展和人工智能技术进步密不可分，数据、算力和算法作为人工智能的三要素显著影响着自动驾驶技术的成熟度，而在这三要素中，数据有着举足轻重的作用。人工智能只有经过大量数据的训练，才能总结出规律。人工智能在实际应用中，如果出现了训练集中从未有过的场景，则人工智能基本处于盲猜状态，从而出现预测错误的情况，因此可靠并且高质量的数据在自动驾驶开发过程中的作用就显得尤为重要。
虽然高等级自动驾驶测试车每天采集的数据量是太字节(TB)级别的容量，且需要拍字节(PB)级的存储空间，但这些数据中，可用于训练自动驾驶的价值数据大约小于或等于5％。如何在海量的数据中找到当前最有价值的困难样本，并优先对这些困难样本进行标注，加速自动驾驶模型迭代及算法量产化，是亟需待解决的问题。
发明内容
本申请实施例提供一种图像数据集处理方法、装置、设备及存储介质，可以提高图像数据集处理的准确率。
第一方面，本申请实施例提供了一种图像数据集处理方法，包括：获取初始自动驾驶图像数据集；其中，所述初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；将所述初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果；根据所述比对结果确定比对类型；其中，所述第一图像检测结果作为基准检测结果；所述比对类型包括真正例、假正例及假反例；根据所述比对类型确定所述多个初始自动驾驶图像的评估得分；根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对所述目标自动驾驶图像数据集中的标注进行调整。
第二方面，本申请实施例还提供了一种图像数据集处理装置，包括：初始自动驾驶图像数据集获取模块，用于获取初始自动驾驶图像数据集；其中，所述初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；图像检测结果获得模块，用于将所述初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；图像检测结果比对模块，用于比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果；比对类型确定模块，用于根据所述比对结果确定比对类型；其中，所述第一图像检测结果作为基准检测结果；所述比对类型包括真正例、假正例及假反例；评估得分确定模块，用于根据所述比对类型确定所述多个初始自动驾驶图像的评估得分；目标自动驾驶图像数据集筛选模块，用于根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对所述目标自动驾驶图像数据集中的标注进行调整。
第三方面，本申请实施例还提供了一种电子设备，所述电子设备包括：
一个或多个处理器；
存储装置，用于存储一个或多个程序，
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本申请实施例所述的图像数据集处理方法。
第四方面，本申请实施例还提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如本申请实施例所述的图像数据集处理方法。
本实施例公开的技术方案，获取初始自动驾驶图像数据集；其中，初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；将初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；比对第一图像检测结果和第二图像检测结果，获得比对结果；根据比对结果确定比对类型；其中，第一图像检测结果作为基准检测结果；比对类型包括真正例、假正例及假反例；根据比对类型确定多个初始自动驾驶图像的评估得分；根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对目标自动驾驶图像数据集中的标注进行调整。本申请实施例，通过根据第一图像检测结果和第二图像检测结果之间的比对结果确定比对类型，根据比对类型确定初始自动驾驶图像的评估得分；根据初始自动驾驶图像的评估得分筛选出目标自动驾驶图像数据集的方式，可以提高图像数据集处理的准确率。
附图说明
结合附图并参考以下具体实施方式，本申请各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。
图1为本申请实施例提供的一种图像数据集处理方法流程示意图；
图2为本申请实施例提供的另一种图像数据集处理方法的流程图；
图3为本申请实施例提供的交并比与交并比评估得分的效果示意图；
图4为本申请实施例提供的一种置信度与第二评估得分之间的效果示意图；
图5为本申请实施例提供的一种置信度与第一评估得分之间的效果示意图；
图6为本申请实施例提供的一种面积与第二评估得分之间的效果示意图；
图7为本申请实施例提供的一种面积与第一评估得分之间的效果示意图；
图8为本申请实施例提供的第一图像检测结果的效果示意图；
图9为本申请实施例提供的第二图像检测结果的效果示意图；
图10为本申请实施例所提供的一种图像数据集处理装置结构示意图；
图11为本申请实施例所提供的一种电子设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本申请的实施例。虽然附图中显示了本申请的某些实施例，然而应当理解的是，本申请可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本申请。应当理解的是，本申请的附图及实施例仅用于示例性作用，并非用于限制本申请的保护范围。
应当理解，本申请的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本申请的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本申请中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能 的顺序或者相互依存关系。
需要注意，本申请中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。
图1为本申请实施例提供的一种图像数据集处理方法流程示意图；本实施例可适用于从初始自动驾驶图像数据集中筛选目标自动驾驶图像数据集的情况，需要说明的是，本实施例不限于自动驾驶前视摄像头场景目标检测任务，也可以是其他场景任务，当切换场景任务时，根据模型能力及项目需求，修改相关配置信息。本实施例，从初始自动驾驶图像数据集中筛选目标自动驾驶图像数据集的情况还可以理解为数据挖掘的情况，以在海量的数据中找到最有价值的困难样本，并优先进行标注，加速模型迭代。该方法可以由图像数据集处理装置来执行，具体包括如下步骤：
S110、获取初始自动驾驶图像数据集。
其中，初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注。其中，标注包括初始自动驾驶图像对应的检测框信息、类别、置信度、面积等信息。当本实施例用于3维场景任务时，面积可以为3维目标体积或投影面积。
其中，初始自动驾驶图像可以通过高等级自动驾驶测试车采集得到。自动驾驶图像数据集中可以包括标注也可以不包括标注，本实施例，对此不作限制。
S120、将初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果。
其中，第一自动驾驶检测模型可以为服务端模型，该服务端模型算力充足且可离线处理，可称为大规模模型。第二自动驾驶检测模型可以为边缘端模型，该模型算力有限且实时性要求高，可称为小规模模型。
可选的，其中，每个初始自动驾驶图像包括至少一个目标对象；第一图像检测结果包括至少一个目标对象的第一检测框信息、至少一个目标对象分别对应的第一类别、第一类别的第一置信度及第一检测框信息对应的第一面积；第二图像检测结果包括至少一个目标对象的第二检测框信息、至少一个目标对象分别对应的第二类别、第二类别对应的第二置信度、第二检测框信息对应的第二面积。
其中，目标对象可以为车辆、栏杆、行人、房屋等，本实施例对此不作限 制。检测框信息可以包括检测框的长度、宽度和高度。
S130、比对第一图像检测结果和第二图像检测结果，获得比对结果。
本实施例，可以将第一图像检测结果和第二图像检测结果进行对应比对，得到对应的比对结果，比对结果包括比对一致和比对不一致。
可选的，比对结果包括类别比对结果，比对第一图像检测结果和第二图像检测结果，获得比对结果，包括：比对第一类别和第二类别，获得类别比对结果；其中，类别比对结果包括类别比对一致和类别比对不一致。
本实施例中，通过比对第一类别和第二类别，可以得到类别比对结果，通过类别比对结果确定比对类型。
S140、根据比对结果确定比对类型。
其中，第一图像检测结果作为基准检测结果；比对类型包括真正例(True Positive，TP)、假正例(False Positive，FP)及假反例(False Negtive，FN)。示例性的，若类别比对一致，则比对类型为真正例，若类别比对不一致，则比对类型为假正例或假反例。
可选的，根据比对结果确定比对类型，包括：根据第一检测框信息和第二检测框信息确定交并比；根据第一面积和/或第二面积确定第一交并比设定值；若交并比大于或等于第一交并比设定值，且类别比对结果为类别比对一致，则将比对类型确定为真正例；若交并比小于第一交并比设定值和或类别比对结果为类别比对不一致，则将比对类型确定为假正例及假反例；或者；第一图像检测结果中包括目标对象的第一检测框信息，第二图像检测结果中不包括目标对象的第二检测框信息，则将比对类型确定为假反例；第一图像检测结果中不包括目标对象的第一检测框信息，第二图像检测结果中包括目标对象的第二检测框信息，则将比对类型确定为假正例。
本实施例中，可以根据第一检测框信息和第二检测框信息确定交并比；根据第一面积和/或第二面积确定目标对象的大小，若第一面积和/或第二面积小于32*32，则目标对象为小目标，对应的第一交并比设定值可以为0.3，若第一面积和/或第二面积大于或等于32*32，则目标对象为正常目标，对应的第一交并比设定值可以为0.5。
若交并比大于或等于第一交并比设定值，且类别比对结果为类别比对一致，则将比对类型确定为真正例。若交并比小于第一交并比设定值和或类别比对结果为类别比对不一致，则将比对类型确定为假正例及假反例。或者；第一图像检测结果中包括目标对象的第一检测框信息，第二图像检测结果中不包括目标对象的第二检测框信息，则将比对类型确定为假反例，也即第一图像检测结果 相对第二图像检测结果多出的检测框，对应的比对类型均为假反例。第一图像检测结果中不包括目标对象的第一检测框信息，第二图像检测结果中包括目标对象的第二检测框信息，则将比对类型确定为假正例，也即第二图像检测结果相对第一图像检测结果多出的检测框，对应的比对类型均为假正例。
本实施例，通过根据比对结果以及交并比确定比对类型的方式，或者通过判断第一图像检测结果和第二图像检测结果中是否包括相同目标对象的检测框信息确定比对类型的方式，可以准确确定出比对类型。
S150、根据比对类型确定多个初始自动驾驶图像的评估得分。
本实施例中，每一个初始自动驾驶图像的评估得分确定方式为：根据比对类型确定初始自动驾驶图像中所有目标对象的评估得分，目标对象的评估得分包括交并比得分、置信度得分、面积得分以及类别得分，根据所有目标对象的评估得分和初始自动驾驶图像的场景得分得到该初始自动驾驶图像的评估得分，从而可以得到所有的初始自动驾驶图像的评估得分。
S160、根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对目标自动驾驶图像数据集中的标注进行调整。
本实施例，可以根据所有的初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集。其中，目标自动驾驶图像数据集可以理解为对第二自动驾驶检测模型(也可以为其他自动驾驶检测模型)的检测准确度具有较大影响的数据集，也可理解为目标价值较大的数据集。通过对目标自动驾驶图像数据集中的标注进行调整，可以提高自动驾驶检测模型的检测准确率。
本实施例中，不限于两个模型之间的比对，也可以是多个模型之间的两两对比，一个初始自动驾驶图像的评估得分可以得到多个价值得分，取最大的得分或通过其他融合得分算法得到最终价值得分。
本实施例中，也可以只使用一个模型，但需进行两种不同模式的前向计算，比如TTA(Test Time Augmentation)模式和正常模式，得到两组不同的检测结果。
本实施例公开的技术方案，获取初始自动驾驶图像数据集；其中，初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；将初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；比对第一图像检测结果和第二图像检测结果，获得比对结果；根据比对结果确定比对类型；其中，第一图像检测结 果作为基准检测结果；比对类型包括真正例、假正例及假反例；根据比对类型确定多个初始自动驾驶图像的评估得分；根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对目标自动驾驶图像数据集中的标注进行调整。本申请实施例，通过根据第一图像检测结果和第二图像检测结果之间的比对结果确定比对类型，根据比对类型确定初始自动驾驶图像的评估得分；根据初始自动驾驶图像的评估得分筛选出目标自动驾驶图像数据集的方式，可以提高图像数据集处理的准确率。
图2为本申请实施例提供的另一种图像数据集处理方法的流程图。本申请实施例是在上述发明实施例基础上的具体化，参见图2，本申请实施例提供的方法具体包括如下步骤：
S201、获取初始自动驾驶图像数据集。
S202、将初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果。
S203、比对第一图像检测结果和第二图像检测结果，获得比对结果。
S204、根据比对结果确定比对类型。
S205、对于任意一个初始自动驾驶图像的评估得分，根据比对类型确定初始自动驾驶图像中目标对象的评估得分。
其中，目标对象的评估得分包括目标对象的交并比得分、置信度得分、面积得分以及类别得分等。本实施例中，不同的比对类型，目标对象的评估得分不同。根据比对类型可以确定出初始自动驾驶图像中全部目标对象的评估得分或者部分目标对象的评估得分。
可选的，根据比对类型确定初始自动驾驶图像中目标对象的评估得分，包括：根据比对类型确定目标对象的交并比得分、置信度得分及面积得分中的至少一个；获取目标对象对应的预先设置的类别得分；根据交并比得分、置信度得分、面积得分以及类别得分中的至少一项确定目标对象的评估得分。
本实施例中，每一个目标对象的评估得分可以通过如下方式得到：分别根据比对类型得到目标对象的交并比得分、置信度得分及面积得分，获取目标对象所对应的预先设置的类别得分。将交并比得分、置信度得分、面积得分以及类别得分进行相乘或加权平均，将相乘后的结果或加权平均后的结果作为目标对象的评估得分。本实施例中，确定目标对象的评估得分的元素不限于交并比得分、置信度得分、面积得分以及类别得分，还可以增加或减少相应的元素。
需要说明的，每个目标对象的类别均预先设置了对应的类别权重，也即类别得分，类别得分的范围为[0,1]。当预测出目标对象所属的类别之后，可获取预 测的类别所对应的类别得分。对于类别权重，可将初始自动驾驶图像数据集分布信息融合进来，针对长尾部分，设置更高的类别权重。还可将模型在测试集上的评估结果融合进来，针对精度差的类别增加权重。
本实施例，通过交并比得分、置信度得分、面积得分以及类别得分中的至少一项确定目标对象的评估得分的方式，可以准确确定出目标对象的评估得分。
可选的，根据比对类型确定目标对象的交并比得分，包括：根据第一面积和/或第二面积确定第二交并比设定值；根据比对类型、第一交并比设定值、第二交并比设定值以及交并比确定交并比评估得分。
其中，第二交并比设定值大于第一交并比设定值；第二交并比设定值可以理解为最大交并比阈值，第一交并比设定值可以理解为最小交并比阈值。本实施例中，根据第一面积和/或第二面积确定目标对象的大小，若第一面积和/或第二面积小于32*32，则目标对象为小目标，对应的第二交并比设定值可以为0.7，若第一面积和/或第二面积大于或等于32*32，则目标对象为正常目标，对应的第一交并比设定值可以为0.9。
示例性的，交并比阈值公式如下：
其中，iou_thresh表示交并比阈值，当目标对象为小目标时，第一交并比设定值可以为0.3，第二交并比设定值可以为0.7。当目标对象为正常目标时，第一交并比设定值可以为0.5，第二交并比设定值可以为0.9。
具体的，若比对类型为假正例或假反例，且交并比小于第一交并比设定值，则可以直接得到交并比评估得分。若比对类型为真正例，且交并比大于或等于第二交并比设定值，则可以直接得到交并比评估得分。若比对类型为真正例，且交并比落入第一交并比设定值和第二交并比设定值构成的区间之内，则可以根据第一交并比设定值、第二交并比设定值以及交并比确定交并比评估得分。
本实施例，通过比对类型、第一交并比设定值、第二交并比设定值以及交并比确定交并比评估得分的方式，可以准确确定出交并比评估得分。
可选的，根据比对类型、第一交并比设定值、第二交并比设定值以及交并比确定交并比评估得分，包括：若比对类型为假正例或假反例，且交并比小于第一交并比设定值，则交并比评估得分为第一设定交并比评估得分；若比对类型为真正例，且交并比大于或等于第二交并比设定值，则交并比评估得分为第二设定交并比评估得分；若比对类型为真正例，且交并比落入第一交并比设定值和第二交并比设定值构成的区间之内，则根据第二交并比设定值和交并比确 定第一交并比评估得分；根据第二交并比设定值和第一交并比设定值确定第二交并比评估得分；根据第一交并比评估得分和第二交并比评估得分确定交并比评估得分。
示例性的，交并比评估得分公式如下：
其中，iou_score为交并比评估得分，min_iou为第一交并比设定值，max_iou为第二交并比设定值，iou为交并比。其中，第一设定交并比评估得分为1，第二设定交并比评估得分为0。
其中，在比对类型为假正例或假反例，且交并比小于第一交并比设定值的情况下，价值最高，交并比评估得分为1；在比对类型为真正例，且交并比大于或等于第二交并比设定值的情况下，可认为模型无需关注较小边缘差异，无价值，交并比评估得分为0；在比对类型为真正例，且交并比落入第一交并比设定值和第二交并比设定值构成的区间之内的情况下，交并比越低价值越高，第一交并比评估得分为max_iou-iou，第二交并比评估得分为max_iou-min_iou，交并比评估得分为
示例性的，图3为本申请实施例提供的交并比与交并比评估得分的效果示意图。如图3所示，横坐标为交并比iou，纵坐标为交并比评估得分iou_score。可以看出目标对象为小目标或正常目标，交并比与交并比评估得分之间呈反比。
需要说明的是，将目标对象按照目标大小进行区别对待是因为：目标位置相同的绝对坐标偏差(以像素为计量单位)，在配准时对小目标的iou值影响更大，因此普遍小目标的iou值不会很高，会使得更多的小目标的评估得分较高，这与“大目标价值更大”的理念不符。
本实施例，通过在比对类型为假正例或假反例，且交并比小于第一交并比设定值的情况下，将第一设定交并比评估得分作为交并比评估得分；在比对类型为真正例，且交并比大于或等于第二交并比设定值的情况下，将第二设定交并比评估得分作为交并比评估得分；在比对类型为真正例，且交并比落入第一交并比设定值和第二交并比设定值构成的区间之内的情况下，根据第一交并比评估得分和第二交并比评估得分确定交并比评估得分的方式，可以准确确定出交并比评估得分。
可选的，根据比对类型确定目标对象的置信度得分，包括：若比对类型为假正例，则确定第二置信度的第一评估得分，将第二置信度的第一评估得分作 为目标对象的置信度得分；若比对类型为假反例，则确定第一置信度的第一评估得分，将第一置信度的第一评估得分作为目标对象的置信度得分；若比对类型为真正例，则分别确定第一置信度的第二评估得分和第二置信度的第二评估得分，根据第一置信度的第二评估得分和第二置信度的第二评估得分确定目标对象的置信度得分。
示例性的，确定目标对象的置信度得分公式如下：
其中，matched_conf_score为目标对象的置信度得分，conf_scoresmall1为比对类型为假正例情况下的第二置信度的第一评估分，也即比对类型为假正例情况下的目标对象的置信度得分。conf_scorebig1为比对类型为假反例情况下的第一置信度的第一评估得分，也即比对类型为假反例情况下的目标对象的置信度得分。为比对类型为真正例情况下的目标对象的置信度得分。conf_scorebig2为第一置信度的第二评估得分，conf_scoresmall2为第二置信度的第二评估得分。
本实施例，通过若比对类型为假正例，则确定第二自动驾驶检测模型所输出的第二置信度的第一评估得分，将第二置信度的第一评估得分作为目标对象在比对类型为假正例情况下的置信度得分的方式；通过若比对类型为假反例，则确定第一自动驾驶检测模型所输出的第一置信度的第一评估得分，将第一置信度的第一评估得分作为目标对象在比对类型为假反例情况下的置信度得分的方式；通过若比对类型为真正例，则分别确定第一置信度的第二评估得分和第二置信度的第二评估得分，根据第一置信度的第二评估得分和第二置信度的第二评估得分确定目标对象在比对类型为真正例情况下的置信度得分的方式，可以准确确定出目标对象的置信度得分。
可选的，确定第一置信度的第一评估得分，或者，确定第二置信度的第一评估得分，包括：获取第一置信度阈值和第二置信度阈值；其中，第二置信度阈值大于第一置信度阈值；若第一置信度或第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第一评估得分或者第二置信度的第一评估得分；若第一置信度大于或等于第二置信度阈值，或者，第二置信度大 于或等于第二置信度阈，则将第二设定置信度得分作为第一置信度的第一评估得分或者第二置信度的第一评估得分；若第一置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第一置信度、第二置信度阈值和第一置信度阈值确定第一置信度得分；根据第二置信度阈值和第一置信度阈值确定第二置信度得分；根据第一置信度得分和第二置信度得分确定第一置信度的第一评估得分；或者；若第二置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第二置信度、第二置信度阈值和第一置信度阈值确定第三置信度得分；根据第二置信度阈值和第一置信度阈值确定第四置信度得分；根据第三置信度得分和第四置信度得分确定第二置信度的第一评估得分。
示例性的，在比对类型为FP情况下，确定第二置信度的第一评估得分公式如下：
其中，第二置信度阈值可以为0.5，第一置信度阈值可以为0.3。第一设定置信度得分可以为0，第二设定置信度得分可以为1。
其中，当比对类型为假正例时，conf为第二置信度；若第二置信度小于第一置信度阈值，则第二置信度的第一评估得分为0，无价值；若第二置信度大于或等于第二置信度阈，则第二置信度的第一评估得分为1，价值最大；若第二置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则为第二置信度的第一评估得分。
本实施例，通过若第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第二置信度的第一评估得分；若第二置信度大于或等于第二置信度阈，则将第二设定置信度得分作为第二置信度的第一评估得分；若第二置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第三置信度得分和第四置信度得分确定第二置信度的第一评估得分的方式，可以准确确定出第二置信度的第一评估得分。
示例性的，在比对类型为FN情况下，确定第一置信度的第一评估得分公式如下：
其中，当比对类型为假反例时，conf为第一置信度；若第一置信度小于第一置信度阈值，则第一置信度的第一评估得分为0，无价值；若第一置信度大于或等于第二置信度阈值，则第一置信度的第一评估得分为1，价值最大；若第一置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则为第一置信度的第一评估得分。
本实施例，通过若第一置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第一评估得分；若第一置信度大于或等于第二置信度阈值，则将第二设定置信度得分作为第一置信度的第一评估得分；若第一置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第一置信度得分和第二置信度得分确定第一置信度的第一评估得分的方式，可以准确确定出第一置信度的第一评估得分。
可选的，确定第一置信度的第二评估得分，或者，确定第二置信度的第二评估得分，包括：若第一置信度或第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第二评估得分或者第二置信度的第二评估得分；若第一置信度大于或等于第一置信度阈值，或者，第二置信度大于或等于第二置信度阈，则将第一设定置信度得分作为第一置信度的第二评估得分或者第二置信度的第二评估得分；若第一置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第一置信度和第二置信度阈值确定第五置信度得分；根据第二置信度阈值和第一置信度阈值确定第六置信度得分；根据第五置信度得分和第六置信度得分确定第一置信度的第二评估得分；或者；若第二置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第二置信度和第二置信度阈值确定第七置信度得分；根据第二置信度阈值和第一置信度阈值确定第八置信度得分；根据第七置信度得分和第八置信度得分确定第二置信度的第二评估得分。
示例性的，在比对类型为真正例情况下，确定第二置信度的第二评估得分的公式如下：
其中，在比对类型为真正例情况下，conf1为第二置信度；若第二置信度小于第一置信度阈值，则第二置信度的第二评估得分为0，无价值；若第二置信度大于或等于第二置信度阈，则第二置信度的第二评估得分为0，无价值；若第二 置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则为第二置信度的第二评估得分。
本实施例，通过若第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第二置信度的第二评估得分；若第二置信度大于或等于第二置信度阈，则将第一设定置信度得分作为第二置信度的第二评估得分；若第二置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第七置信度得分和第八置信度得分确定第二置信度的第二评估得分的方式，可以准确确定出第二置信度的第二评估得分。
示例性的，在比对类型为真正例情况下，确定第一置信度的第二评估得分的公式如下：
其中，在比对类型为真正例情况下，conf2为第一置信度；若第一置信度小于第一置信度阈值，则第一置信度的第二评估得分为0；若第一置信度大于或等于第一置信度阈值，则第一置信度的第二评估得分为0；若第一置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则为第一置信度的第二评估得分。
本实施例，通过若第一置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第二评估得分；若第一置信度大于或等于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第二评估得分；若第一置信度落入第一置信度阈值和第二置信度阈值构成的区间之内，则根据第五置信度得分和第六置信度得分确定第一置信度的第二评估得分的方式，可以准确确定出第一置信度的第二评估得分。
示例性的，图4为本申请实施例提供的一种置信度与第二评估得分之间的效果示意图。如图4所示，在比对类型为真正例情况下，横坐标为conf，conf为第一置信度或第二置信度，min_conf为第一置信度阈值，max_conf为第二置信度阈值。纵坐标为conf_score，conf_score为第一置信度或第二置信度的第二评估得分。当第一置信度或第二置信度小于第一置信度阈值，或者，大于等于第二置信度阈值时，第二评估得分均为0，无价值。conf位于第一置信度阈值和第二置信度阈值之间时，第二评估得分(也可以理解为价值)与置信度之间呈 反比。
示例性的，图5为本申请实施例提供的一种置信度与第一评估得分之间的效果示意图。如图5所示，在比对类型为假反例或假正例情况下，横坐标为conf，conf为第一置信度或第二置信度，min_conf为第一置信度阈值，max_conf为第二置信度阈值，纵坐标为conf_score，conf_score为第一置信度或第二置信度的第一评估得分。当第一置信度或第二置信度小于第一置信度阈值，第一评估得分全部为0，无价值。当第一置信度或第二置信度大于等于第二置信度阈值时，价值最大，第一评估得分全部为1。conf位于第一置信度阈值和第二置信度阈值之间时，第一评估得分(也可以理解为价值)与置信度之间呈正比。
可选的，根据比对类型确定目标对象的面积得分，包括：若比对类型为假正例，则确定第二面积的第一评估得分，将第二面积的第一评估得分作为目标对象的面积得分；若比对类型为假反例，则确定第一面积的第一评估得分，将第一面积的第一评估得分作为目标对象的面积得分；若比对类型为真正例，则分别确定第一面积的第二评估得分和第二面积的第二评估得分，根据第一面积的第二评估得分和第二面积的第二评估得分确定目标对象的面积得分。
示例性的，确定目标对象的面积得分公式如下：
其中，matched_area_score为目标对象的面积得分，area_scoresmall1为比对类型为假正例情况下的第二面积的第一评估得分，也即比对类型为假正例情况下的目标对象的面积得分。area_scorebig1为比对类型为假反例情况下的第一面积的第一评估得分，也即比对类型为假反例情况下的目标对象的面积得分。为比对类型为真正例情况下的目标对象的面积得分。area_scorebig2为第一面积的第二评估得分，area_scoresmall2为第二面积的第二评估得分。
本实施例，通过若比对类型为假正例，则确定第二面积的第一评估得分，将第二面积的第一评估得分作为目标对象在比对类型为假正例情况下的面积得分；若比对类型为假反例，则确定第一面积的第一评估得分，将第一面积的第一评估得分作为目标对象在比对类型为假反例情况下的面积得分；若比对类型为真正例，则根据第一面积的第二评估得分和第二面积的第二评估得分确定目 标对象在比对类型为真正例情况下的面积得分的方式，可以准确确定出目标对象的面积得分。
可选的，确定第一面积的第一评估得分，或者，确定第二面积的第一评估得分，包括：获取第一面积阈值和第二面积阈值；其中，第二面积阈值大于第一面积阈值；若第一面积或第二面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第一评估得分或者第二面积的第一评估得分；若第一面积大于或等于第二面积阈值，或者，第二面积大于或等于第二面积阈值，则将第二设定面积得分作为第一面积的第一评估得分或者第二面积的第一评估得分；若第一面积落入第一面积阈值和第二面积阈值构成的区间之内，则根据第一面积、第二面积阈值和第一面积阈值确定第一面积得分；根据第二面积阈值和第一面积阈值确定第二面积得分；根据第一面积得分和第二面积得分确定第一面积的第一评估得分；或者；若第二面积落入第一面积阈值和第二面积阈值构成的区间之内，则根据第二面积、第二面积阈值和第一面积阈值确定第三面积得分；根据第二面积阈值和第一面积阈值确定第四面积得分；根据第三面积得分和第四面积得分确定第二面积的第一评估得分。
其中，第一面积阈值可以为30*30，第二面积阈值为500*500；第一设定面积得分可以为0；第二设定面积得分可以为1。
示例性的，在比对类型为FN情况下，确定第一面积的第一评估得分的公式如下：
其中，当比对类型为假反例时，area为第一面积，若第一面积小于第一面积阈值，则第一面积的第一评估得分为0；若第一面积大于或等于第二面积阈值，则第一面积的第一评估得分为1；若第一面积落入第一面积阈值和第二面积阈值构成的区间之内，则为第一面积的第一评估得分。
本实施例，通过若第一面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第一评估得分；若第一面积大于或等于第二面积阈值，则将第二设定面积得分作为第一面积的第一评估得分；若第一面积落入第一面积阈值和第二面积阈值构成的区间之内，则根据第一面积得分和第二面积得分确定第一面积的第一评估得分的方式，可以准确确定出第一面积的第一评估得分。
示例性的，在比对类型为FP情况下，确定第二面积的第一评估得分的公式如下：
其中，当比对类型为假正例时，area为第二面积；若第二面积小于第一面积阈值，则第二面积的第一评估得分为0；若第二面积大于或等于第二面积阈值，则第二面积的第一评估得分为1；若第二面积落入第一面积阈值和第二面积阈值构成的区间之内，则为第二面积的第一评估得分。
本实施例，通过若第二面积小于第一面积阈值，则将第一设定面积得分作为第二面积的第一评估得分；若第二面积大于或等于第二面积阈值，则将第二设定面积得分作为第二面积的第一评估得分；若第二面积落入第一面积阈值和第二面积阈值构成的区间之内，则根据第三面积得分和第四面积得分确定第二面积的第一评估得分的方式，可以准确确定出第二面积的第一评估得分。
可选的，确定第一面积的第二评估得分，或者，确定第二面积的第二评估得分，包括：若第一面积或第二面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第二评估得分或者第二面积的第二评估得分；若第一面积大于或等于第一面积阈值，或者，若第二面积大于或等于第一面积阈值，则将第二设定面积得分作为第一面积的第二评估得分或者第二面积的第二评估得分；若第一面积落入第一面积阈值和第二面积值构成的区间之内，则根据第一面积和第一面积阈值确定第五面积得分；根据第二面积阈值和第一面积阈值确定第六面积得分；根据第五面积得分和第六面积得分确定第一面积的第二评估得分；或者；若第二面积落入第一面积阈值和第二面积值构成的区间之内，则根据第二面积和第一面积阈值确定第七面积得分；根据第二面积阈值和第一面积阈值确定第八面积得分；根据第七面积得分和第八面积得分确定第二面积的第二评估得分。
示例性的，在比对类型为真正例情况下，确定第一面积的第二评估得分的公式如下：
其中，在比对类型为真正例情况下，conf2为第一面积；若第一面积小于第一面积阈值，则第一面积的第二评估得分为0；若第一面积大于或等于第一面积阈值，则第一面积的第二评估得分为1；若第一面积落入第一面积阈值和第二面积值构成的区间之内，则为第一面积的第二评估得分。
本实施例，通过若第一面积小于第一面积阈值，则将第一设定面积得分作 为第一面积的第二评估得分；若第一面积大于或等于第一面积阈值，则将第二设定面积得分作为第一面积的第二评估得分；若第一面积落入第一面积阈值和第二面积值构成的区间之内，则根据第五面积得分和第六面积得分确定第一面积的第二评估得分的方式，可以准确确定出第一面积的第二评估得分。
示例性的，在比对类型为真正例情况下，确定第二面积的第二评估得分的公式如下：
其中，在比对类型为真正例情况下，area2为第二面积；若第二面积小于第一面积阈值，则第二面积的第二评估得分0；若第二面积大于或等于第一面积阈值，则第二面积的第二评估得分1；若第二面积落入第一面积阈值和第二面积值构成的区间之内，则为第二面积的第二评估得分。
本实施例，通过若第二面积小于第一面积阈值，则将第一设定面积得分作为第二面积的第二评估得分；若第二面积大于或等于第一面积阈值，则将第二设定面积得分作为第二面积的第二评估得分；若第二面积落入第一面积阈值和第二面积值构成的区间之内，则根据第七面积得分和第八面积得分确定第二面积的第二评估得分的方式，可以准确确定出第二面积的第二评估得分。
示例性的，图6为本申请实施例提供的一种面积与第二评估得分之间的效果示意图。如图6所示，在比对类型为真正例情况下，横坐标为area，area为第一面积或第二面积，min_area为第一面积阈值，max_conf为第二面积阈值。纵坐标为area_score，area_score为第二评估得分。当第一面积或第二面积小于第一面积阈值，第二评估得分均为0，无价值。当第一面积或第二面积大于等于第二面积阈值时，价值最大，第二评估得分全部为1。area位于第一面积阈值和第二面积阈值之间时，第二评估得分(也可以理解为价值)与面积之间呈正比。
示例性的，图7为本申请实施例提供的一种面积与第一评估得分之间的效果示意图。如图7所示，在比对类型为假反例或假正例情况下，横坐标为area，area为第一面积或第二面积，min_area为第一面积阈值，max_area为第二面积阈值，纵坐标为area_score，area_score为面积的第一评估得分。当第一面积或第二面积小于第一面积阈值，第一评估得分全部为0，无价值。当第一面积或第二面积大于等于第二面积阈值时，价值最大，第一评估得分全部为1。area位于第一面积阈值和第二面积阈值之间时，第一评估得分(也可以理解为价值)与 面积之间呈正比。
S206、确定初始自动驾驶图像的场景得分。
本实施例中，对初始自动驾驶图像的场景不作限制，例如下雨场景、下雪场景、交通堵塞场景、越野场景等。
本实施例中，可以根据初始自动驾驶图像获得实际场景信息，将初始自动驾驶图像输入至自动驾驶场景模型中，输出预测场景信息，根据预测场景信息和实际场景信息的相似度得到场景得分。
可选的，确定初始自动驾驶图像的场景得分，包括：将初始自动驾驶图像输入至自动驾驶场景模型中，输出预测场景信息；获取初始自动驾驶图像对应的设定场景信息；确定预测场景信息和设定场景信息确定场景相似度；根据场景相似度确定初始自动驾驶图像的场景得分。
其中，自动驾驶场景模型可以是基于任意深度学习算法的场景模型。设定场景信息可以是初始自动驾驶图像对应的真实场景信息。本实施例，将初始自动驾驶图像输入至自动驾驶场景模型中，输出对应的预测场景信息；获取初始自动驾驶图像对应的设定场景信息，基于任意相似度算法计算预测场景信息和设定场景信息之间的场景相似度；根据场景相似度确定初始自动驾驶图像的场景得分。其中，场景相似度的范围为[0，1]，也即场景得分的范围为[0，1]。
本实施例，通过预测场景信息和设定场景信息之间的场景相似度确定初始自动驾驶图像的场景得分的方式，可以准确确定出初始自动驾驶图像的场景得分。
S207、根据目标对象的评估得分和场景得分确定初始自动驾驶图像的评估得分。
本实施例中，任意一张初始自动驾驶图像的评估得分计算方式如下：将当前初始自动驾驶图像中的所有目标对象的评估得分进行累加，得到累加结果，再将累加结果和场景得分进行相乘，得到当前初始自动驾驶图像的评估得分。
示例性的，确定初始自动驾驶图像的评估得分的公式如下：
其中，img_value_score表示初始自动驾驶图像的评估得分，scene_score表示场景得分，bbox_value_score表示目标对象的评估得分。
S208、根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集 中筛选出目标自动驾驶图像数据集，并对目标自动驾驶图像数据集中的标注进行调整。
本实施例中，可以按照各个初始自动驾驶图像的评估得分，对初始自动驾驶图像数据集中的初始自动驾驶图像进行由高到低排序，提取出排序靠前的设定数量的初始自动驾驶图像，作为设定数量的目标自动驾驶图像，并将设定数量的目标自动驾驶图像对应的标注均进行调整。
可选的，根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，包括：根据多个初始自动驾驶图像的评估得分对多个初始自动驾驶图像进行排序，获得排序后的初始自动驾驶图像数据集；从排序后的初始自动驾驶图像数据集中筛选出设定数量的目标自动驾驶图像数据集。
具体的，根据多个初始自动驾驶图像的评估得分对多个初始自动驾驶图像进行由高到低的排序，获得排序后的初始自动驾驶图像数据集；从排序后的初始自动驾驶图像数据集中筛选出排序靠前的设定数量的目标自动驾驶图像数据集，也即提取出前设定数量的初始自动驾驶图像，作为设定数量的目标自动驾驶图像。或者，根据多个初始自动驾驶图像的评估得分对多个初始自动驾驶图像进行由低到高的排序，获得排序后的初始自动驾驶图像数据集；从排序后的初始自动驾驶图像数据集中筛选出排序靠后的设定数量的目标自动驾驶图像数据集，也即提取出后设定数量的初始自动驾驶图像，作为设定数量的目标自动驾驶图像。
本实施例，通过在排序后的初始自动驾驶图像数据集中筛选出设定数量的目标自动驾驶图像数据集，可以准确筛选出目标自动驾驶图像。
示例性的，图8为本申请实施例提供的第一图像检测结果的效果示意图；图9为本申请实施例提供的第二图像检测结果的效果示意图。图8为第一自动驾驶检测模型检测得到的第一图像检测结果，图9为第二自动驾驶检测模型检测得到的第二图像检测结果。从图8和图9中可以看出，第一自动驾驶检测模型检测到的一群自行车只出一个框，而第二自动驾驶检测模型每个自行车都单独出框，因此所有自行车的框都没配上，比对类型为FN或FP；图8或图9中最后面的柱子，第一自动驾驶检测模型检测到，第二自动驾驶检测模型未检测到，比对类型为FN，价值得分高；因此目标对象的评估得分大部分为FN和FP的评估得分，得分很高。
图10为本申请实施例所提供的一种图像数据集处理装置结构示意图，如图10所示，装置包括：初始自动驾驶图像数据集获取模块1001、图像检测结果获得模块1002、图像检测结果比对模块1003、比对类型确定模块1004、评估得分 确定模块1005及目标自动驾驶图像数据集筛选模块1006；
初始自动驾驶图像数据集获取模块1001，用于获取初始自动驾驶图像数据集；其中，所述初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；
图像检测结果获得模块1002，用于将所述初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；
图像检测结果比对模块1003，用于比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果；
比对类型确定模块1004，用于根据所述比对结果确定比对类型；其中，所述第一图像检测结果作为基准检测结果；所述比对类型包括真正例、假正例及假反例；
评估得分确定模块1005，用于根据所述比对类型确定所述多个初始自动驾驶图像的评估得分；
目标自动驾驶图像数据集筛选模块1006，用于根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对所述目标自动驾驶图像数据集中的标注进行调整。
本实施例公开的技术方案，通过初始自动驾驶图像数据集获取模块获取初始自动驾驶图像数据集；其中，初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；通过图像检测结果获得模块将初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；通过图像检测结果比对模块比对第一图像检测结果和第二图像检测结果，获得比对结果；通过比对类型确定模块根据比对结果确定比对类型；其中，第一图像检测结果作为基准检测结果；比对类型包括真正例、假正例及假反例；通过评估得分确定模块根据比对类型确定多个初始自动驾驶图像的评估得分；通过目标自动驾驶图像数据集筛选模块根据多个初始自动驾驶图像的评估得分从初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对目标自动驾驶图像数据集中的标注进行调整。本申请实施例，通过根据第一图像检测结果和第二图像检测结果之间的比对结果确定比对类型，根据比对类型确定初始自动驾驶图像的评估得分；根据初始自动驾驶图像的评估得分筛选出目标自动驾驶图像数据集的方式，可以提高图像数据集处理的准确率。
可选的，其中，每个初始自动驾驶图像包括至少一个目标对象；所述第一 图像检测结果包括至少一个目标对象的第一检测框信息、所述至少一个目标对象分别对应的第一类别、所述第一类别的第一置信度及所述第一检测框信息对应的第一面积；所述第二图像检测结果包括至少一个目标对象的第二检测框信息、所述至少一个目标对象分别对应的第二类别、所述第二类别对应的第二置信度、所述第二检测框信息对应的第二面积；所述比对结果包括类别比对结果，可选的，图像检测结果比对模块，具体用于：比对所述第一类别和所述第二类别，获得类别比对结果；其中，所述类别比对结果包括类别比对一致和类别比对不一致。
可选的，比对类型确定模块具体用于：根据第一检测框信息和所述第二检测框信息确定交并比；根据所述第一面积和/或第二面积确定第一交并比设定值；若所述交并比大于或等于第一交并比设定值，且所述类别比对结果为类别比对一致，则将所述比对类型确定为真正例；若所述交并比小于第一交并比设定值和或所述类别比对结果为类别比对不一致，则将所述比对类型确定为假正例及假反例；或者；所述第一图像检测结果中包括目标对象的第一检测框信息，所述第二图像检测结果中不包括所述目标对象的第二检测框信息，则将所述比对类型确定为假反例；所述第一图像检测结果中不包括目标对象的第一检测框信息，所述第二图像检测结果中包括所述目标对象的第二检测框信息，则将所述比对类型确定为假正例。
可选的，评估得分确定模块具体用于：对于任意一个初始自动驾驶图像的评估得分，根据比对类型确定初始自动驾驶图像中目标对象的评估得分；确定所述初始自动驾驶图像的场景得分；根据所述目标对象的评估得分和所述场景得分确定初始自动驾驶图像的评估得分。
可选的，评估得分确定模块还用于：根据比对类型确定目标对象的交并比得分、置信度得分及面积得分中的至少一个；获取目标对象对应的预先设置的类别得分；根据所述交并比得分、所述置信度得分、所述面积得分以及所述类别得分中的至少一项确定目标对象的评估得分。
可选的，评估得分确定模块还用于：根据所述第一面积和/或第二面积确定第二交并比设定值；其中，所述第二交并比设定值大于所述第一交并比设定值；根据所述比对类型、所述第一交并比设定值、第二交并比设定值以及所述交并比确定交并比评估得分。
可选的，评估得分确定模块还用于：若所述比对类型为假正例或假反例，且所述交并比小于所述第一交并比设定值，则交并比评估得分为第一设定交并比评估得分；若所述比对类型为真正例，且所述交并比大于或等于所述第二交并比设定值，则交并比评估得分为第二设定交并比评估得分；若所述比对类型 为真正例，且所述交并比落入所述第一交并比设定值和第二交并比设定值构成的区间之内，则根据第二交并比设定值和所述交并比确定第一交并比评估得分；根据所述第二交并比设定值和所述第一交并比设定值确定第二交并比评估得分；根据所述第一交并比评估得分和所述第二交并比评估得分确定交并比评估得分。
可选的，评估得分确定模块还用于：若所述比对类型为假正例，则确定第二置信度的第一评估得分，将所述第二置信度的第一评估得分作为目标对象的置信度得分；若所述比对类型为假反例，则确定第一置信度的第一评估得分，将所述第一置信度的第一评估得分作为目标对象的置信度得分；若所述比对类型为真正例，则分别确定第一置信度的第二评估得分和第二置信度的第二评估得分，根据第一置信度的第二评估得分和第二置信度的第二评估得分确定目标对象的置信度得分。
可选的，评估得分确定模块还用于：获取第一置信度阈值和第二置信度阈值；其中，所述第二置信度阈值大于所述第一置信度阈值；若所述第一置信度或所述第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第一评估得分或者第二置信度的第一评估得分；若所述第一置信度大于或等于第二置信度阈值，或者，所述第二置信度大于或等于第二置信度阈，则将第二设定置信度得分作为第一置信度的第一评估得分或者第二置信度的第一评估得分；若所述第一置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第一置信度、所述第二置信度阈值和所述第一置信度阈值确定第一置信度得分；根据所述第二置信度阈值和所述第一置信度阈值确定第二置信度得分；根据所述第一置信度得分和所述第二置信度得分确定第一置信度的第一评估得分；或者；若所述第二置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第二置信度、所述第二置信度阈值和所述第一置信度阈值确定第三置信度得分；根据所述第二置信度阈值和所述第一置信度阈值确定第四置信度得分；根据所述第三置信度得分和所述第四置信度得分确定第二置信度的第一评估得分。
可选的，评估得分确定模块还用于：若所述第一置信度或所述第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第二评估得分或者第二置信度的第二评估得分；若所述第一置信度大于或等于第一置信度阈值，或者，所述第二置信度大于或等于第二置信度阈，则将第一设定置信度得分作为第一置信度的第二评估得分或者第二置信度的第二评估得分；若所述第一置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第一置信度和第二置信度阈值确定第五置信度得分；根据第二置信度阈值和第一置信度阈值确定第六置信度得分；根据所述第五置信度得分和所述第六置信度得分确定第一置信度的第二评估得分；或者；若所述第二置信度 落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第二置信度和第二置信度阈值确定第七置信度得分；根据第二置信度阈值和第一置信度阈值确定第八置信度得分；根据所述第七置信度得分和所述第八置信度得分确定第二置信度的第二评估得分。
可选的，评估得分确定模块还用于：若所述比对类型为假正例，则确定第二面积的第一评估得分，将所述第二面积的第一评估得分作为目标对象的面积得分；若所述比对类型为假反例，则确定第一面积的第一评估得分，将所述第一面积的第一评估得分作为目标对象的面积得分；若所述比对类型为真正例，则分别确定第一面积的第二评估得分和第二面积的第二评估得分，根据第一面积的第二评估得分和第二面积的第二评估得分确定目标对象的面积得分。
可选的，评估得分确定模块还用于：获取第一面积阈值和第二面积阈值；其中，所述第二面积阈值大于第一面积阈值；若所述第一面积或所述第二面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第一评估得分或者第二面积的第一评估得分；若所述第一面积大于或等于第二面积阈值，或者，所述第二面积大于或等于第二面积阈值，则将第二设定面积得分作为第一面积的第一评估得分或者第二面积的第一评估得分；若所述第一面积落入所述第一面积阈值和所述第二面积阈值构成的区间之内，则根据所述第一面积、第二面积阈值和第一面积阈值确定第一面积得分；根据第二面积阈值和第一面积阈值确定第二面积得分；根据所述第一面积得分和所述第二面积得分确定第一面积的第一评估得分；或者；若所述第二面积落入所述第一面积阈值和所述第二面积阈值构成的区间之内，则根据所述第二面积、第二面积阈值和第一面积阈值确定第三面积得分；根据第二面积阈值和第一面积阈值确定第四面积得分；根据所述第三面积得分和所述第四面积得分确定第二面积的第一评估得分。
可选的，评估得分确定模块还用于：若所述第一面积或所述第二面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第二评估得分或者第二面积的第二评估得分；若所述第一面积大于或等于第一面积阈值，或者，若所述第二面积大于或等于第一面积阈值，则将第二设定面积得分作为第一面积的第二评估得分或者第二面积的第二评估得分；若所述第一面积落入所述第一面积阈值和所述第二面积值构成的区间之内，则根据所述第一面积和第一面积阈值确定第五面积得分；根据第二面积阈值和第一面积阈值确定第六面积得分；根据所述第五面积得分和所述第六面积得分确定第一面积的第二评估得分；或者；若所述第二面积落入所述第一面积阈值和所述第二面积值构成的区间之内，则根据所述第二面积和第一面积阈值确定第七面积得分；根据第二面积阈值和第一面积阈值确定第八面积得分；根据所述第七面积得分和所述第八面积得分确定第二面积的第二评估得分。
可选的，评估得分确定模块还用于：将所述初始自动驾驶图像输入至自动驾驶场景模型中，输出预测场景信息；获取所述初始自动驾驶图像对应的设定场景信息；确定所述预测场景信息和所述设定场景信息确定场景相似度；根据所述场景相似度确定初始自动驾驶图像的场景得分。
可选的，目标自动驾驶图像数据集筛选模块具体用于：根据所述多个初始自动驾驶图像的评估得分对所述多个初始自动驾驶图像进行排序，获得排序后的初始自动驾驶图像数据集；从所述排序后的初始自动驾驶图像数据集中筛选出设定数量的目标自动驾驶图像数据集。
本申请实施例所提供的图像数据集处理装置可执行本申请任意实施例所提供的图像数据集处理方法，具备执行方法相应的功能模块和有益效果。
值得注意的是，上述装置所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本申请实施例的保护范围。
图11为本申请实施例所提供的一种电子设备的结构示意图。下面参考图11，其示出了适于用来实现本申请实施例的电子设备(例如图11中的终端设备或服务器)1100的结构示意图。本申请实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图11示出的电子设备仅仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图11所示，电子设备1100可以包括处理装置(例如中央处理器、图形处理器等)1101，其可以根据存储在只读存储器(ROM)1102中的程序或者从存储装置1108加载到随机访问存储器(RAM)1103中的程序而执行各种适当的动作和处理。在RAM 1103中，还存储有电子设备1100操作所需的各种程序和数据。处理装置1101、ROM 1102以及RAM 1103通过总线1104彼此相连。编辑/输出(I/O)接口1105也连接至总线1104。
通常，以下装置可以连接至I/O接口1105：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1106；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置1107；包括例如磁带、硬盘等的存储装置1108；以及通信装置1109。通信装置1109可以允许电子设备1100与其他设备进行无线或有线通信以交换数据。虽然图11示出了具有各种装置的电子设备1100，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本申请的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置1109从网络上被下载和安装，或者从存储装置1108被安装，或者从ROM 1102被安装。在该计算机程序被处理装置1101执行时，执行本申请实施例的方法中限定的上述功能。
本申请实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
本申请实施例提供的电子设备与上述实施例提供的图像数据集处理方法属于同一发明构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的有益效果。
本申请实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的图像数据集处理方法。
需要说明的是，本申请上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通 信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：获取初始自动驾驶图像数据集；其中，所述初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；将所述初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果；根据所述比对结果确定比对类型；其中，所述第一图像检测结果作为基准检测结果；所述比对类型包括真正例、假正例及假反例；根据所述比对类型确定所述多个初始自动驾驶图像的评估得分；根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对所述目标自动驾驶图像数据集中的标注进行调整。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用 硬件与计算机指令的组合来实现。
描述于本申请实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本申请的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
以上描述仅为本申请的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本申请中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本申请中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本申请的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (18)

  1. 一种图像数据集处理方法，包括：
    获取初始自动驾驶图像数据集；其中，所述初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；
    将所述初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；
    比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果；
    根据所述比对结果确定比对类型；其中，所述第一图像检测结果作为基准检测结果；所述比对类型包括真正例、假正例及假反例；
    根据所述比对类型确定所述多个初始自动驾驶图像的评估得分；
    根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对所述目标自动驾驶图像数据集中的标注进行调整。
  2. 根据权利要求1所述的方法，其中，每个初始自动驾驶图像包括至少一个目标对象；所述第一图像检测结果包括至少一个目标对象的第一检测框信息、所述至少一个目标对象分别对应的第一类别、所述第一类别的第一置信度及所述第一检测框信息对应的第一面积；所述第二图像检测结果包括至少一个目标对象的第二检测框信息、所述至少一个目标对象分别对应的第二类别、所述第二类别对应的第二置信度、所述第二检测框信息对应的第二面积；所述比对结果包括类别比对结果，比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果，包括：
    比对所述第一类别和所述第二类别，获得类别比对结果；其中，所述类别比对结果包括类别比对一致和类别比对不一致。
  3. 根据权利要求2所述的方法，其中，根据所述比对结果确定比对类型，包括：
    根据第一检测框信息和所述第二检测框信息确定交并比；
    根据所述第一面积和/或第二面积确定第一交并比设定值；
    若所述交并比大于或等于第一交并比设定值，且所述类别比对结果为类别比对一致，则将所述比对类型确定为真正例；
    若所述交并比小于第一交并比设定值和或所述类别比对结果为类别比对不一致，则将所述比对类型确定为假正例及假反例；
    或者；
    所述第一图像检测结果中包括目标对象的第一检测框信息，所述第二图像检测结果中不包括所述目标对象的第二检测框信息，则将所述比对类型确定为假反例；
    所述第一图像检测结果中不包括目标对象的第一检测框信息，所述第二图像检测结果中包括所述目标对象的第二检测框信息，则将所述比对类型确定为假正例。
  4. 根据权利要求3所述的方法，其中，根据所述比对类型确定所述多个初始自动驾驶图像的评估得分，包括：
    对于任意一个初始自动驾驶图像的评估得分，根据比对类型确定初始自动驾驶图像中目标对象的评估得分；
    确定所述初始自动驾驶图像的场景得分；
    根据所述目标对象的评估得分和所述场景得分确定初始自动驾驶图像的评估得分。
  5. 根据权利要求4所述的方法，其中，根据比对类型确定初始自动驾驶图像中目标对象的评估得分，包括：
    根据比对类型确定目标对象的交并比得分、置信度得分及面积得分中的至少一个；
    获取目标对象对应的预先设置的类别得分；
    根据所述交并比得分、所述置信度得分、所述面积得分以及所述类别得分中的至少一项确定目标对象的评估得分。
  6. 根据权利要求5所述的方法，其中，根据比对类型确定目标对象的交并比得分，包括：
    根据所述第一面积和/或第二面积确定第二交并比设定值；其中，所述第二交并比设定值大于所述第一交并比设定值；
    根据所述比对类型、所述第一交并比设定值、第二交并比设定值以及所述交并比确定交并比评估得分。
  7. 根据权利要求6所述的方法，其中，根据所述比对类型、所述第一交并比设定值、第二交并比设定值以及所述交并比确定交并比评估得分，包括：
    若所述比对类型为假正例或假反例，且所述交并比小于所述第一交并比设定值，则交并比评估得分为第一设定交并比评估得分；
    若所述比对类型为真正例，且所述交并比大于或等于所述第二交并比设定 值，则交并比评估得分为第二设定交并比评估得分；
    若所述比对类型为真正例，且所述交并比落入所述第一交并比设定值和第二交并比设定值构成的区间之内，则根据第二交并比设定值和所述交并比确定第一交并比评估得分；
    根据所述第二交并比设定值和所述第一交并比设定值确定第二交并比评估得分；
    根据所述第一交并比评估得分和所述第二交并比评估得分确定交并比评估得分。
  8. 根据权利要求5所述的方法，其中，根据比对类型确定目标对象的置信度得分，包括：
    若所述比对类型为假正例，则确定第二置信度的第一评估得分，将所述第二置信度的第一评估得分作为目标对象的置信度得分；
    若所述比对类型为假反例，则确定第一置信度的第一评估得分，将所述第一置信度的第一评估得分作为目标对象的置信度得分；
    若所述比对类型为真正例，则分别确定第一置信度的第二评估得分和第二置信度的第二评估得分，根据第一置信度的第二评估得分和第二置信度的第二评估得分确定目标对象的置信度得分。
  9. 根据权利要求8所述的方法，其中，确定第一置信度的第一评估得分，或者，确定第二置信度的第一评估得分，包括：
    获取第一置信度阈值和第二置信度阈值；其中，所述第二置信度阈值大于所述第一置信度阈值；
    若所述第一置信度或所述第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第一评估得分或者第二置信度的第一评估得分；
    若所述第一置信度大于或等于第二置信度阈值，或者，所述第二置信度大于或等于第二置信度阈，则将第二设定置信度得分作为第一置信度的第一评估得分或者第二置信度的第一评估得分；
    若所述第一置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第一置信度、所述第二置信度阈值和所述第一置信度阈值确定第一置信度得分；
    根据所述第二置信度阈值和所述第一置信度阈值确定第二置信度得分；
    根据所述第一置信度得分和所述第二置信度得分确定第一置信度的第一评估得分；
    或者；
    若所述第二置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第二置信度、所述第二置信度阈值和所述第一置信度阈值确定第三置信度得分；
    根据所述第二置信度阈值和所述第一置信度阈值确定第四置信度得分；
    根据所述第三置信度得分和所述第四置信度得分确定第二置信度的第一评估得分。
  10. 根据权利要求9所述的方法，其中，确定第一置信度的第二评估得分，或者，确定第二置信度的第二评估得分，包括：
    若所述第一置信度或所述第二置信度小于第一置信度阈值，则将第一设定置信度得分作为第一置信度的第二评估得分或者第二置信度的第二评估得分；
    若所述第一置信度大于或等于第一置信度阈值，或者，所述第二置信度大于或等于第二置信度阈，则将第一设定置信度得分作为第一置信度的第二评估得分或者第二置信度的第二评估得分；
    若所述第一置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第一置信度和第二置信度阈值确定第五置信度得分；
    根据第二置信度阈值和第一置信度阈值确定第六置信度得分；
    根据所述第五置信度得分和所述第六置信度得分确定第一置信度的第二评估得分；
    或者；
    若所述第二置信度落入所述第一置信度阈值和所述第二置信度阈值构成的区间之内，则根据所述第二置信度和第二置信度阈值确定第七置信度得分；
    根据第二置信度阈值和第一置信度阈值确定第八置信度得分；
    根据所述第七置信度得分和所述第八置信度得分确定第二置信度的第二评估得分。
  11. 根据权利要求5所述的方法，其中，根据比对类型确定目标对象的面积得分，包括：
    若所述比对类型为假正例，则确定第二面积的第一评估得分，将所述第二面积的第一评估得分作为目标对象的面积得分；
    若所述比对类型为假反例，则确定第一面积的第一评估得分，将所述第一面积的第一评估得分作为目标对象的面积得分；
    若所述比对类型为真正例，则分别确定第一面积的第二评估得分和第二面积的第二评估得分，根据第一面积的第二评估得分和第二面积的第二评估得分确定目标对象的面积得分。
  12. 根据权利要求11所述的方法，其中，确定第一面积的第一评估得分，或者，确定第二面积的第一评估得分，包括：
    获取第一面积阈值和第二面积阈值；其中，所述第二面积阈值大于第一面积阈值；
    若所述第一面积或所述第二面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第一评估得分或者第二面积的第一评估得分；
    若所述第一面积大于或等于第二面积阈值，或者，所述第二面积大于或等于第二面积阈值，则将第二设定面积得分作为第一面积的第一评估得分或者第二面积的第一评估得分；
    若所述第一面积落入所述第一面积阈值和所述第二面积阈值构成的区间之内，则根据所述第一面积、第二面积阈值和第一面积阈值确定第一面积得分；
    根据第二面积阈值和第一面积阈值确定第二面积得分；
    根据所述第一面积得分和所述第二面积得分确定第一面积的第一评估得分；
    或者；
    若所述第二面积落入所述第一面积阈值和所述第二面积阈值构成的区间之内，则根据所述第二面积、第二面积阈值和第一面积阈值确定第三面积得分；
    根据第二面积阈值和第一面积阈值确定第四面积得分；
    根据所述第三面积得分和所述第四面积得分确定第二面积的第一评估得分。
  13. 根据权利要求12所述的方法，其中，确定第一面积的第二评估得分，或者，确定第二面积的第二评估得分，包括：
    若所述第一面积或所述第二面积小于第一面积阈值，则将第一设定面积得分作为第一面积的第二评估得分或者第二面积的第二评估得分；
    若所述第一面积大于或等于第一面积阈值，或者，若所述第二面积大于或等于第一面积阈值，则将第二设定面积得分作为第一面积的第二评估得分或者第二面积的第二评估得分；
    若所述第一面积落入所述第一面积阈值和所述第二面积值构成的区间之内，则根据所述第一面积和第一面积阈值确定第五面积得分；
    根据第二面积阈值和第一面积阈值确定第六面积得分；
    根据所述第五面积得分和所述第六面积得分确定第一面积的第二评估得分；
    或者；
    若所述第二面积落入所述第一面积阈值和所述第二面积值构成的区间之内，则根据所述第二面积和第一面积阈值确定第七面积得分；
    根据第二面积阈值和第一面积阈值确定第八面积得分；
    根据所述第七面积得分和所述第八面积得分确定第二面积的第二评估得分。
  14. 根据权利要求4所述的方法，其中，确定所述初始自动驾驶图像的场景得分，包括：
    将所述初始自动驾驶图像输入至自动驾驶场景模型中，输出预测场景信息；
    获取所述初始自动驾驶图像对应的设定场景信息；
    确定所述预测场景信息和所述设定场景信息确定场景相似度；
    根据所述场景相似度确定初始自动驾驶图像的场景得分。
  15. 根据权利要求1所述的方法，其中，根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，包括：
    根据所述多个初始自动驾驶图像的评估得分对所述多个初始自动驾驶图像进行排序，获得排序后的初始自动驾驶图像数据集；
    从所述排序后的初始自动驾驶图像数据集中筛选出设定数量的目标自动驾驶图像数据集。
  16. 一种图像数据集处理装置，包括：
    初始自动驾驶图像数据集获取模块，设置为获取初始自动驾驶图像数据集；其中，所述初始自动驾驶图像数据集包括多个初始自动驾驶图像以及对应的标注；
    图像检测结果获得模块，设置为将所述初始图像数据集分别输入至第一自动驾驶检测模型和第二自动驾驶检测模型中，分别获得第一图像检测结果和第二图像检测结果；
    图像检测结果比对模块，设置为比对所述第一图像检测结果和所述第二图像检测结果，获得比对结果；
    比对类型确定模块，设置为根据所述比对结果确定比对类型；其中，所述第一图像检测结果作为基准检测结果；所述比对类型包括真正例、假正例及假反例；
    评估得分确定模块，设置为根据所述比对类型确定所述多个初始自动驾驶图像的评估得分；
    目标自动驾驶图像数据集筛选模块，设置为根据所述多个初始自动驾驶图像的评估得分从所述初始自动驾驶图像数据集中筛选出目标自动驾驶图像数据集，并对所述目标自动驾驶图像数据集中的标注进行调整。
  17. 一种电子设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序，
    当所述至少一个程序被所述至少一个处理器执行，使得所述一个或多个处理器实现如权利要求1-15中任一所述的图像数据集处理方法。
  18. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-15中任一所述的图像数据集处理方法。
PCT/CN2024/084979 2023-07-26 2024-03-29 图像数据集处理方法、装置、设备及存储介质 Pending WO2025020569A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024558970A JP7842892B2 (ja) 2023-07-26 2024-03-29 画像データセット処理方法、装置、機器および記憶媒体
EP24821735.8A EP4528669A4 (en) 2023-07-26 2024-03-29 APPARATUS AND METHOD FOR PROCESSING ASSEMBLY OF IMAGE DATA, DEVICE AND STORAGE MEDIA

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310922726.XA CN117115580B (zh) 2023-07-26 2023-07-26 图像数据集处理方法、装置、设备及存储介质
CN202310922726.X 2023-07-26

Publications (1)

Publication Number Publication Date
WO2025020569A1 true WO2025020569A1 (zh) 2025-01-30

Family

ID=88795550

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/084979 Pending WO2025020569A1 (zh) 2023-07-26 2024-03-29 图像数据集处理方法、装置、设备及存储介质

Country Status (4)

Country Link
EP (1) EP4528669A4 (zh)
JP (1) JP7842892B2 (zh)
CN (1) CN117115580B (zh)
WO (1) WO2025020569A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117115580B (zh) * 2023-07-26 2026-02-06 成都市德赛西威卡蛙科技有限公司 图像数据集处理方法、装置、设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114022858A (zh) * 2021-10-18 2022-02-08 西南大学 一种针对自动驾驶的语义分割方法、系统、电子设备及介质
CN114511715A (zh) * 2022-01-05 2022-05-17 惠州市德赛西威汽车电子股份有限公司 一种驾驶场景数据挖掘方法
CN114742145A (zh) * 2022-04-02 2022-07-12 中国科学院软件研究所 目标检测模型的性能测试方法、装置、设备及存储介质
CN115439828A (zh) * 2022-09-22 2022-12-06 东风汽车股份有限公司 自动驾驶数据增广方法、装置、设备及存储介质
CN115713749A (zh) * 2022-11-30 2023-02-24 北京百度网讯科技有限公司 自动驾驶模型的训练方法、装置、电子设备和存储介质
US20230138650A1 (en) * 2021-12-28 2023-05-04 Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd. Test method for automatic driving, and electronic device
CN117115580A (zh) * 2023-07-26 2023-11-24 成都市卡蛙科技有限公司 图像数据集处理方法、装置、设备及存储介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2892036B1 (en) * 2014-01-06 2017-12-06 Harman International Industries, Incorporated Alert generation correlating between head mounted imaging data and external device
US20190294999A1 (en) * 2018-06-16 2019-09-26 Moshe Guttmann Selecting hyper parameters for machine learning algorithms based on past training results
US11164399B2 (en) * 2019-06-29 2021-11-02 Gm Cruise Holdings Llc Automatic detection of data for annotation for autonomous vehicle perception
JP7396040B2 (ja) * 2019-12-27 2023-12-12 日本電気株式会社 画像分析装置、画像分析方法及びコンピュータプログラム
CN112200218B (zh) * 2020-09-10 2023-06-20 浙江大华技术股份有限公司 一种模型训练方法、装置及电子设备
CN115661502A (zh) * 2021-07-07 2023-01-31 华为技术有限公司 图像处理方法、电子设备及存储介质
WO2023034665A1 (en) * 2021-09-02 2023-03-09 Canoo Technologies Inc. Metamorphic labeling using aligned sensor data
CN114299037B (zh) * 2021-12-30 2023-09-01 广州极飞科技股份有限公司 一种对象检测结果的质量评估方法、装置、电子设备及计算机可读存储介质
CN114527675B (zh) * 2021-12-31 2025-05-09 广州文远知行科技有限公司 自动驾驶仿真测试方法、装置、设备及存储介质
CN115527083B (zh) * 2022-09-27 2023-04-11 中电金信软件有限公司 图像标注方法、装置和电子设备
CN115797701A (zh) * 2022-12-22 2023-03-14 重庆长安汽车股份有限公司 目标分类方法、装置、电子设备及存储介质
CN116091869A (zh) * 2023-02-23 2023-05-09 长城汽车股份有限公司 模型训练方法、图像标注方法、装置、电子设备及介质
CN116452957B (zh) * 2023-06-20 2023-09-26 小米汽车科技有限公司 图像标注数据的质量检测方法、装置及电子设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114022858A (zh) * 2021-10-18 2022-02-08 西南大学 一种针对自动驾驶的语义分割方法、系统、电子设备及介质
US20230138650A1 (en) * 2021-12-28 2023-05-04 Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd. Test method for automatic driving, and electronic device
CN114511715A (zh) * 2022-01-05 2022-05-17 惠州市德赛西威汽车电子股份有限公司 一种驾驶场景数据挖掘方法
CN114742145A (zh) * 2022-04-02 2022-07-12 中国科学院软件研究所 目标检测模型的性能测试方法、装置、设备及存储介质
CN115439828A (zh) * 2022-09-22 2022-12-06 东风汽车股份有限公司 自动驾驶数据增广方法、装置、设备及存储介质
CN115713749A (zh) * 2022-11-30 2023-02-24 北京百度网讯科技有限公司 自动驾驶模型的训练方法、装置、电子设备和存储介质
CN117115580A (zh) * 2023-07-26 2023-11-24 成都市卡蛙科技有限公司 图像数据集处理方法、装置、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4528669A4

Also Published As

Publication number Publication date
EP4528669A1 (en) 2025-03-26
JP2025528640A (ja) 2025-09-02
JP7842892B2 (ja) 2026-04-08
CN117115580A (zh) 2023-11-24
CN117115580B (zh) 2026-02-06
EP4528669A4 (en) 2026-01-21

Similar Documents

Publication Publication Date Title
US12608813B2 (en) Image segmentation method for a target object, device, and storage medium
CN110704751B (zh) 数据处理方法、装置、电子设备及存储介质
US20190279088A1 (en) Training method, apparatus, chip, and system for neural network model
CN111401516A (zh) 一种神经网络通道参数的搜索方法及相关设备
WO2020186887A1 (zh) 一种连续小样本图像的目标检测方法、装置及设备
CN114416260B (zh) 图像处理方法、装置、电子设备及存储介质
CN113033682B (zh) 视频分类方法、装置、可读介质、电子设备
WO2020007177A1 (zh) 计算机执行的报价方法、报价装置、电子设备及存储介质
CN112364829B (zh) 一种人脸识别方法、装置、设备及存储介质
WO2023116138A1 (zh) 多任务模型的建模方法、推广内容处理方法及相关装置
CN115146478B (zh) 基于优化算法的行驶工况构建方法、装置及相关设备
CN118736343A (zh) 目标检测模型的训练方法、目标检测方法、装置及设备
WO2025020569A1 (zh) 图像数据集处理方法、装置、设备及存储介质
CN113033707B (zh) 视频分类方法、装置、可读介质及电子设备
CN112418233B (zh) 图像处理方法、装置、可读介质及电子设备
CN111291715B (zh) 基于多尺度卷积神经网络的车型识别方法、电子设备及存储介质
CN114648713B (zh) 视频分类方法、装置、电子设备及计算机可读存储介质
WO2025130671A1 (zh) 一种目标检测的方法和装置
WO2024007938A1 (zh) 一种多任务预测方法、装置、电子设备及存储介质
CN116152597A (zh) 一种用于目标检测的模型的训练方法及装置
CN115690724A (zh) 行人骑行人目标过滤方法、装置、设备、存储介质及车辆
CN114429631A (zh) 三维对象检测方法、装置、设备以及存储介质
CN114648629A (zh) 感兴趣区域生成方法、装置、设备和计算机可读介质
CN117077036B (zh) 一种模型稳定性检测方法、装置及设备
WO2025213845A1 (zh) 一种视频生成方法、装置、设备、介质、产品

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2024558970

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 2024821735

Country of ref document: EP

Effective date: 20241220

NENP Non-entry into the national phase

Ref country code: DE