CN111753625A - A pedestrian detection method, device, equipment and medium - Google Patents

Abstract

Embodiments of the present invention disclose a pedestrian detection method, device, equipment and medium. The method includes: acquiring an image to be detected; inputting the to-be-detected image into a trained single-shot target detector to obtain a single-shot target detector According to the output information of the single-shot object detector, the pedestrian detection result of the image to be detected is determined; wherein, the single-shot object detector is the original detection of the single-shot object detector and the head detection network including the initial construction in advance. model, obtained by training. The pedestrian detection method provided by the embodiment of the present invention performs pedestrian detection on the single-shot target detector obtained by training the original detection model including the initially constructed single-shot target detector and the head detection network in advance. On the basis of the pedestrian detection speed of the object detector, the pedestrian detection accuracy is improved.

Description

A pedestrian detection method, device, equipment and medium

technical field

Embodiments of the present invention relate to the field of target detection, and in particular, to a pedestrian detection method, device, device, and medium.

Background technique

Pedestrian detection has many application scenarios in the field of computer vision, such as security monitoring, autonomous driving, robotics, etc. Most of the current mainstream pedestrian detection methods are based on deep learning, such as the target detector Faster RCNN based on candidate regions, or the single-shot target detector SSD, YOLO, etc. The object detector based on candidate regions is divided into two parts, one is the region candidate Region Proposal Networks, RPN) network, and the other is the region-based convolution (FastR-CNN) network. When used, the candidate region of the foreground frame is roughly extracted by RPN, and then the candidate region is fine-tuned by Fast R-CNN, and the final object coordinates and object classification results are returned. The single-shot object detector does not have an RPN network, and directly returns the object coordinates and object classification results.

In the process of realizing the present invention, the inventor found that there are at least the following technical problems in the prior art: the above method has achieved good results on standard pedestrian detection data, but in occlusion scenes (including intra-class occlusion, people occlusion with people, as well as occlusion between classes, occlusion between people and objects, etc.), has not achieved satisfactory results. At present, in order to improve the accuracy of pedestrian recognition in occluded scenes, some optimization methods for target detectors based on candidate regions have been proposed. Therefore, how to improve the pedestrian detection accuracy on the basis of ensuring the pedestrian detection speed of the single-shot target detector is an urgent technical problem to be solved.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a pedestrian detection method, device, equipment and medium, so as to improve the pedestrian detection accuracy on the basis of ensuring the pedestrian detection speed of the single-shot target detector.

In a first aspect, an embodiment of the present invention provides a pedestrian detection method, including:

Obtain the image to be detected;

Input the image to be detected into the trained single-shot target detector, and obtain the output information of the single-shot target detector;

According to the output information of the single-shot target detector, determine the pedestrian detection result of the image to be detected;

The single-shot target detector is obtained by pre-training an original detection model including an initially constructed single-shot target detector and a head detection network.

In a second aspect, an embodiment of the present invention further provides a pedestrian detection device, including:

a to-be-detected image acquisition module for acquiring the to-be-detected image;

The image pedestrian detection module is used to input the image to be detected into the trained single-shot target detector, and obtain the output information of the single-shot target detector, wherein the single-shot target detector The initially constructed single-shot target detector and the original detection model of the head detection network are obtained by training;

The detection result determination module is configured to determine the pedestrian detection result on the image to be detected according to the output information of the single-shot target detector.

In a third aspect, an embodiment of the present invention further provides a computer device, the device comprising:

one or more processors;

a storage device for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the pedestrian detection method provided by any embodiment of the present invention.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the pedestrian detection method provided by any embodiment of the present invention.

In the embodiment of the present invention, the image to be detected is acquired; the image to be detected is input into the trained single-shot target detector, and the output information of the single-shot target detector is obtained; according to the output information of the single-shot target detector, the image to be detected is determined The pedestrian detection results of The original detection model of the detector and the head detection network, the single-shot target detector obtained by training is used for pedestrian detection, and the pedestrian detection accuracy is improved on the basis of ensuring the pedestrian detection speed of the single-shot target detector.

Description of drawings

1 is a flowchart of a pedestrian detection method provided in Embodiment 1 of the present invention;

2 is a flowchart of a pedestrian detection method provided in Embodiment 2 of the present invention;

3a is a flowchart of a pedestrian detection method provided in Embodiment 3 of the present invention;

3b is a schematic diagram of a network architecture of an original detection model provided in Embodiment 3 of the present invention;

4 is a schematic structural diagram of a pedestrian detection device according to Embodiment 4 of the present invention;

FIG. 5 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Example 1

FIG. 1 is a flowchart of a pedestrian detection method provided by Embodiment 1 of the present invention. This embodiment is applicable to the situation when pedestrian detection is performed. The method may be performed by a pedestrian detection apparatus, which may be implemented in software and/or hardware, for example, the pedestrian detection apparatus may be configured in computer equipment. As shown in Figure 1, the method includes:

S110. Acquire an image to be detected.

In this embodiment, the image to be detected may be an image that needs to be detected for pedestrians. The acquisition method of the image to be detected is not limited herein. Optionally, a video frame captured by a camera may be directly acquired as an image to be detected, or an existing video may be processed to obtain an image to be detected for pedestrian detection.

S120: Input the image to be detected into the trained single-shot target detector, and obtain the output information of the single-shot target detector, wherein the single-shot target detector is a pre-built single-shot target detector including the initial construction and the head The original detection model of the detection network is obtained by training.

After acquiring the image to be detected, the pre-trained single-shot target detector is used to detect the to-be-detected image, and the output information of the single-shot target detector is obtained, so as to determine the pedestrian detection result according to the output information of the single-shot target detector. Among them, the single-shot target detector may be a single-shot multi-box detector (Single Shot MultiBox Detector, SSD), YOLO (You Only Live Once), RetinaNet and other detectors. Optionally, the output information of the single-shot target detector may be the identified pedestrian frames in the image to be detected, and the score of each pedestrian frame.

It should be noted that the single-shot target detector is obtained by pre-training the original detection model including the initially constructed single-shot target detector and the head detection network. In order to solve the technical problem of inaccurate detection results of the single-shot target detector in the occlusion scene in the prior art, when training the single-shot target detector, the head detection network and the single-shot target detector are combined for training. The detection results of the partial detection network and the detection results of the single-shot target detector are adjusted, and the setting parameters in the single-shot target detector are adjusted to realize the optimization of the parameters in the single-shot target detector, so that the trained single-shot target detector recognizes the results. more precise.

S130: Determine the pedestrian detection result of the image to be detected according to the output information of the single-shot target detector.

In this embodiment, after obtaining the output information after the single-shot target detector detects the to-be-detected image, the pedestrian detection result of the to-be-detected image is determined according to the output information of the single-shot target detector. Optionally, the pedestrian detection result of the image to be detected may be set according to detection requirements. Exemplarily, if the detection requirement is to detect the number of pedestrians included in the image to be detected, the number of pedestrian frames output by the single target detector is counted as the pedestrian detection result of the image to be detected; if the detection requirement is to detect pedestrians in the image to be detected position, the position of the pedestrian in the image to be detected is determined according to the position of the pedestrian frame output by the single target detector.

In the embodiment of the present invention, the image to be detected is acquired; the image to be detected is input into the trained single-shot target detector, and the output information of the single-shot target detector is obtained; according to the output information of the single-shot target detector, the image to be detected is determined Among them, the single-shot object detector is obtained by pre-training the original detection model including the initially constructed single-shot object detector and the head detection network. By using the original detection model including the initially constructed single-shot object detector and the head detection network in advance, the single-shot object detector obtained by training is used to detect pedestrians, so as to ensure the pedestrian detection speed of the single-shot object detector. improve pedestrian detection accuracy.

Embodiment 2

FIG. 2 is a flowchart of a pedestrian detection method provided by Embodiment 2 of the present invention. On the basis of the above-mentioned embodiments, the present embodiment implements the training of the single-shot target detector. As shown in Figure 2, the method includes:

S210. Obtain a sample image, a pedestrian frame annotation result corresponding to the sample image, and a pedestrian head annotation result corresponding to the sample image.

In this embodiment, the sample image may be an image including a pedestrian, preferably, an image including a occluded pedestrian. The sample image is manually annotated, the pedestrian frame and pedestrian head in the sample image are marked, and the sample image and the pedestrian frame annotation result corresponding to the sample image and the pedestrian head annotation result corresponding to the sample image are obtained.

S220 , generating a training sample pair based on the sample image, the pedestrian frame labeling result corresponding to the sample image, and the pedestrian head labeling result corresponding to the sample image, and using the training sample pair to train the pre-built original detection model to obtain a trained original detection model .

After the sample image is annotated, a training sample pair is generated based on the sample image, the pedestrian frame annotation result corresponding to the sample image, and the pedestrian head annotation result corresponding to the sample image, and the training sample pair is used to train the pre-built original detection model to obtain training. Good original detection model. The pre-built original detection model may include a single-shot target detector and at least one head detection network. The output of each original feature network layer in the single-shot object detector is connected to the input of the head detection network.

In an embodiment of the present invention, using training samples to train a pre-built original detection model to obtain a trained original detection model includes: inputting a sample image into an initially constructed single-shot target detector, obtaining The original feature map output by the single target detector, the pedestrian frame detection result, and the detection score corresponding to the pedestrian frame detection result; the pedestrian frame detection results are sorted according to the detection score, and a preset number of target pedestrian frame detections are obtained according to the sorting results. Result; input the original feature map and the target pedestrian frame detection result into the head detection network, and obtain the pedestrian head detection result output by the head detection network; determine the first loss value according to the pedestrian frame detection result and the pedestrian frame labeling result, according to The pedestrian head detection result and the pedestrian head labeling result determine the second loss value, and the target loss value is determined according to the first loss value and the second loss value; the original detection model is trained with the goal that the target loss value reaches the convergence condition.

Specifically, using the training samples to train the pre-built original detection model can be as follows: input the sample image into the initially constructed single-shot target detector, and obtain the pedestrian frame detection result output by the single-shot target detector, and each pedestrian The detection score corresponding to the frame detection result, the pedestrian frame detection results are sorted in reverse order according to the detection score, and the preset number of pedestrian frame detection results before sorting is taken as the target pedestrian frame detection result; the target pedestrian frame detection result and each original feature The original feature map output by the network layer is input into the head detection network, and the pedestrian head detection result output by the head detection network is obtained. Then, the first loss value corresponding to the pedestrian frame detection result is calculated based on the set pedestrian frame loss function, the pedestrian frame detection result and the pedestrian frame labeling result, and the first loss value corresponding to the pedestrian frame detection result is calculated based on the set pedestrian head loss function, pedestrian head detection result and pedestrian head. The labeling result calculates the second loss value corresponding to the pedestrian head detection result, and calculates the target loss value according to the first loss value and the second loss value. When the target loss value does not meet the convergence condition, adjust the parameters in the head detection network and the parameters in the single-shot target detector, and predict the sample image again based on the adjusted parameters until the target loss value meets the convergence condition, and the training is completed. the original detection model. Among them, the pedestrian frame loss function and the pedestrian head loss function may be the same or different.

Optionally, calculating the target loss value according to the first loss value and the second loss value may be: taking the sum of the first loss value and the second loss value as the target loss value. The target loss value satisfies the convergence condition may be: the number of iterations satisfies the set number of times or the difference between the two adjacent target loss values is less than the set threshold.

Optionally, the head detection network may include a region extraction module and a head labeling module, the region extraction module is used to extract the head feature map corresponding to the detection result of the target pedestrian frame from the original feature map, and the head labeling module is used to The head feature map is marked to obtain the pedestrian head detection result.

On the basis of the above solution, the single-shot target detector includes a plurality of original feature network layers, an upsampling module is further included between the target feature network layer in the original detection model and the head detection network, and the method further includes: according to The image size of the original feature map output by each original feature network layer, select at least one original feature network layer as the target feature network layer; add an upsampling module after the target feature network layer, and add a head detection network after the upsampling module.

Optionally, considering that the original feature map output by some of the original feature network layers in the single-shot target detector is small, the head detection feature extraction based on the original feature map will lead to the loss of pedestrian information and affect the pedestrian head detection result. In this embodiment, according to the image size of the original feature map, the target feature network layer is selected from the original feature network layer, an upsampling module is added after the target original feature network layer, and a head detection network is added after the upsampling module, using The upsampled original feature map is used for head detection to improve pedestrian head detection results.

Optionally, selecting at least one original feature network layer as the target feature network layer according to the image size of the original feature map output by each original feature network layer, including: selecting the original feature network corresponding to the original feature map whose image size is smaller than the set threshold. layer as the target feature network layer. In one embodiment, an image size threshold may be preset, and the original feature network layer corresponding to the original feature map whose image size is smaller than the image size threshold may be used as the target feature network layer. In this embodiment, the up-sampling module is not limited, as long as the original feature map can be up-sampled into an up-sampling feature map whose image size is not smaller than the set image size threshold. Exemplarily, the upsampling module may be a transposed convolution module.

In an embodiment of the present invention, before inputting the original feature map and the detection result of the target pedestrian frame into the head detection network, and obtaining the pedestrian head detection result output by the head detection network, the method further includes: acquiring the target feature network The original feature map of the target output by the layer, input the original feature map of the target into the up-sampling module, and obtain the up-sampling feature map output by the up-sampling module; correspondingly, input the original feature map and the detection result of the target pedestrian frame into the head detection network. , to obtain the pedestrian head detection results output by the head detection network, including: the original feature map, the upsampling feature map and the target pedestrian frame detection results output by other original feature network layers in the original feature network layer except the target feature network layer Input to the head detection network to obtain the pedestrian head detection result output by the head detection network.

After adding an upsampling module between the target network feature layer and the head detection network, correspondingly, when predicting the pedestrian head, the upsampling module first upsamples the target original feature map output by the target network feature layer. The upsampling feature map is obtained, and then the upsampling feature map and the original feature map output from the original feature network layer except the target feature network layer in the original feature network layer are used as the input of the head detection network, and the head detection network is used. Detect the pedestrian head, and obtain the pedestrian head detection result output by the head detection network. Upsampling the original feature map of the target and then detecting it through the head detection network can retain the pedestrian information in the original feature map and avoid the impact of pedestrian information loss on pedestrian head detection.

S230. Use the single-shot target detector in the trained original detection model as the trained single-shot target detector.

In this embodiment, after the trained original detection model is obtained, the single-shot target detector in the trained original detection model is used as the trained single-shot target detector, and the trained single-shot target detector is used for pedestrian detection. detection. In the detection process, only the single-shot target detector is used for detection, which improves the detection accuracy of the single-shot target detector on the basis of ensuring the detection speed of the single-shot target detector.

S240. Acquire an image to be detected.

S250: Input the image to be detected into the trained single-shot target detector, and obtain output information of the single-shot target detector.

S260: Determine the pedestrian detection result of the image to be detected according to the output information of the single target detector.

The embodiment of the present invention concretizes the training of the single-shot target detector, and obtains the sample image, the pedestrian frame labeling result corresponding to the sample image, and the pedestrian head labeling result corresponding to the sample image; The frame annotation results and the pedestrian head annotation results corresponding to the sample images generate training sample pairs, and use the training samples to train the pre-built original detection model to obtain the trained original detection model; The secondary target detector is used as a trained single-shot target detector. By adding pedestrian head features as training features, the training accuracy of the single-shot target detector is improved, thereby improving the accuracy of the pedestrian detection results of the single-shot target detector.

Embodiment 3

FIG. 3 a is a flowchart of a pedestrian detection method according to Embodiment 3 of the present invention. This embodiment provides a preferred embodiment on the basis of the above-mentioned embodiment. As shown in Figure 3a, the method includes:

S310. Build an original detection model to be trained based on the single-shot target detector.

In this embodiment, on the basis of the single-shot target detector, the prediction of the head is added to obtain the constructed original detection model. The entire original detection model predicts the pedestrian frame and the mark of the pedestrian head during the training process, and uses the head detection task to assist in improving the accuracy of pedestrian detection. Among them, the single-shot target detector can be a detector such as SSD, YOLO, and RetinaNet.

Fig. 3b is a schematic diagram of a network architecture of an original detection model according to Embodiment 3 of the present invention, and Fig. 3b schematically shows an original detection network model based on an SSD network. As shown in FIG. 3b , the original detection model includes an SSD network 310 , an upsampling module 320 and a head detection module 330 .

The SSD network 310 part includes an image input layer, a basic network layer, a feature network layer and a detection layer, and the feature network layer includes a feature layer 1, a feature layer 2 and a feature layer 3. The input of the SSD network 310 is the image to be detected, and the output is the detected pedestrian frame and three feature maps. After the input image passes through the basic network layer and the feature network layer, feature maps of different scales are obtained, and the obtained feature map passes through the detection layer to obtain the prediction result of the pedestrian frame, including the coordinates of the pedestrian frame and the score of the pedestrian frame.

The upsampling module 320 includes a target feature network layer and an upsampling layer, wherein the target feature network layer is a feature network layer corresponding to a feature map whose image size is smaller than a set image size threshold. The input of the upsampling module 320 is the feature map of small scale in SSD, and the output is the feature map after upsampling. The feature map of small scale (the image size is smaller than the set threshold) (that is, the feature map output by feature layer 3 in Figure 3b) is upsampled to a large feature map by transposed convolution, to avoid when the original feature map is small, from the original feature map. The information loss caused by the extraction of the head feature map. Exemplarily, assuming that the size of the original feature map is 8*8, the size of the up-sampled feature map obtained after the transposed convolution is 64*64.

The head detection module 330 partially includes the ROIAlign and marker layers. The input of the head detection module 330 is the feature map 1 output by the feature layer 1 in the SSD, the feature map 2 output by the feature layer 2, the pedestrian detection result and the obtained upsampling feature map 3 output by the upsampling layer, and the output is the pedestrian head. Test results. This part first sorts the pedestrian frames according to the score, then extracts the top 100 pedestrian frames, and then uses ROIAlign the head feature map of the corresponding position of the above-mentioned top 100 pedestrian frames, and adjusts and zooms through the size, and the unified output is the set size. (such as 28*28) head feature map, and finally perform pedestrian head detection on the head feature map.

S320. Obtain sample data, and train an original detection model based on the obtained sample data to obtain a trained original detection model.

Specifically, after obtaining the sample data (sample image), the sample data is marked to obtain a training sample pair, wherein the training sample pair data needs to include the sample image, the annotation of the pedestrian frame, the annotation of the pedestrian head, and the annotation of the pedestrian frame. As the standard data for pedestrian detection, the annotation of pedestrian head is used as the standard data for pedestrian head annotation. When training the original detection model, input the sample image into the single-shot target detector, obtain the prediction result of the pedestrian frame output by the single-shot target detector, and calculate the pedestrian frame detection loss value at the same time. Then, the pedestrian frames are sorted according to the scores of each pedestrian frame, and the first 100 pedestrian frame detection results and the feature maps corresponding to the above pedestrian frame detection results are obtained. Then use the up-sampling module to enlarge the small-scale feature map to obtain the up-sampled feature map. Through ROIAlign, the head feature maps corresponding to the first 100 pedestrian frames are extracted from the original feature maps whose scales meet the set size requirements and the up-sampled feature maps, and marked on the extracted head feature maps to obtain pedestrian head detection. As a result, the pedestrian head detection result is compared with the pedestrian head annotation to obtain the pedestrian head detection loss value. Then, the pedestrian frame detection loss value and the pedestrian head detection loss value are added as the overall loss, and the original detection model is trained with the overall loss convergence as the goal, and the trained original detection model is obtained. By introducing head prediction into the single-shot object detector, the pedestrian detection accuracy in occluded scenes is improved.

S330. Use the single-shot target detector in the trained original detection model as the single-shot target detector to be tested, and test the single-shot target detector to be tested.

After obtaining the trained original detection model, extract the single-shot target detector in the trained original detection model, use it as the single-shot target detector to be tested, and use the test data to test the single-shot target detector to be tested, and obtain Test Results.

S340 , after the single-shot target detector to be tested passes the test, use the single-shot target detector that has passed the test to perform pedestrian detection.

When the single-shot target detector to be tested passes the test, the single-shot target detector that has passed the test can be directly used for pedestrian detection. When the single target detector to be tested fails the test, the training data is re-acquired to train the original detection model, until the single target detector in the trained original detection model passes the test.

In the embodiment of the present invention, a pedestrian head mark detection branch is added to the single-shot target detector, and an up-sampling module is added at the same time to increase the small-scale feature map, so that it can also participate in the pedestrian head mark detection. Pedestrian detection accuracy in occluded scenes with secondary object detectors.

Embodiment 4

FIG. 4 is a schematic structural diagram of a pedestrian detection device according to Embodiment 4 of the present invention. The pedestrian detection apparatus may be implemented in software and/or hardware, for example, the pedestrian detection apparatus may be configured in a computer device. As shown in FIG. 4 , the device includes an image acquisition module 410 to be detected, an image pedestrian detection module 420 and a detection result determination module 430, wherein:

a to-be-detected image acquisition module 410, configured to acquire the to-be-detected image;

The image pedestrian detection module 420 is configured to input the to-be-detected image into the trained single-shot target detector, and obtain the output information of the single-shot target detector, wherein the single-shot target detector is pre-tested. The original detection model including the initially constructed single-shot target detector and the head detection network is obtained by training;

The detection result determination module 430 is configured to determine the pedestrian detection result of the to-be-detected image according to the output information of the single-shot target detector.

In the embodiment of the present invention, an image to be detected is acquired through an image acquisition module to be detected; the image pedestrian detection module inputs the to-be-detected image into the trained single-shot target detector, and obtains the output information of the single-shot target detector; detection The result determination module determines the pedestrian detection result of the image to be detected according to the output information of the single-shot target detector; wherein, the single-shot target detector is a pre-determined single-shot target detector including the initial construction and the head The original detection model of the partial detection network is obtained by training. By using the original detection model including the initially constructed single-shot object detector and the head detection network in advance, the single-shot object detector obtained by training is used to detect pedestrians, so as to ensure the pedestrian detection speed of the single-shot object detector. improve pedestrian detection accuracy.

Optionally, on the basis of the above solution, the device further includes a single-shot target detector determination module for:

obtaining a sample image, a pedestrian frame annotation result corresponding to the sample image, and a pedestrian head annotation result corresponding to the sample image;

A training sample pair is generated based on the sample image, the pedestrian frame labeling result corresponding to the sample image, and the pedestrian head labeling result corresponding to the sample image, and the training sample pair is used to train the pre-built original detection model to obtain a trained The original detection model of ;

Take the one-shot object detector in the trained original detection model as the trained one-shot object detector.

Optionally, on the basis of the above solution, the single-shot target detector determination module includes:

The pedestrian frame detection unit is used to input the sample image into the initially constructed single-shot target detector, and obtain the original feature map output by the single-shot target detector, the pedestrian frame detection result, and the corresponding pedestrian frame detection result. test score;

a target pedestrian frame determination unit, configured to sort the pedestrian frame detection results according to the detection score, and obtain a preset number of target pedestrian frame detection results according to the sorting results;

a pedestrian head detection unit, configured to input the original feature map and the target pedestrian frame detection result into the head detection network, and obtain the pedestrian head detection result output by the head detection network;

A loss value determination unit, configured to determine a first loss value according to the pedestrian frame detection result and the pedestrian frame labeling result, determine a second loss value according to the pedestrian head detection result and the pedestrian head labeling result, and determine a second loss value according to the pedestrian head detection result and the pedestrian head labeling result. the first loss value and the second loss value determine a target loss value;

The original detection model training unit is configured to train the original detection model with the target loss value reaching a convergence condition as the goal.

Optionally, on the basis of the above solution, the single-shot target detector includes a plurality of original feature network layers, and the target feature network layer in the original detection model and the head detection network also includes an upper layer. Sampling module, the device also includes an original detection model building module for:

According to the image size of the original feature map output by each of the original feature network layers, at least one of the original feature network layers is selected as the target feature network layer;

An upsampling module is added after the target feature network layer, and the head detection network is added after the upsampling module.

Optionally, on the basis of the above solution, the original detection model building module is specifically used for:

The original feature network layer corresponding to the original feature map whose image size is smaller than the set threshold is used as the target feature network layer.

Optionally, on the basis of the above solution, the single-shot target detector determination module further includes an up-sampling unit for:

Obtain the target original feature map output by the target feature network layer, input the target original feature map into the upsampling module, and obtain the upsampling feature map output by the upsampling module;

Correspondingly, the pedestrian head detection unit is specifically used for:

Inputting the original feature map, the up-sampling feature map and the target pedestrian frame detection result output by other original feature network layers except the target feature network layer in the original feature network layer into the head detection network, Obtain the pedestrian head detection result output by the head detection network.

Optionally, based on the above solution, the upsampling module is a transposed convolution module.

The pedestrian detection device provided by the embodiment of the present invention can execute the pedestrian detection method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

Embodiment 5

FIG. 5 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention. Figure 5 shows a block diagram of an exemplary computer device 512 suitable for use in implementing embodiments of the present invention. The computer device 512 shown in FIG. 5 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention.

As shown in FIG. 5, computer device 512 takes the form of a general-purpose computing device. Components of computer device 512 may include, but are not limited to, one or more processors 516, system memory 528, and a bus 518 connecting various system components, including system memory 528 and processor 516.

Bus 518 represents one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor 516, or a local bus using any of a variety of bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.

Computer device 512 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 512, including both volatile and nonvolatile media, removable and non-removable media.

System memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and/or cache memory 532 . Computer device 512 may further include other removable/non-removable, volatile/non-volatile computer system storage media. For example only, storage device 534 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 5, commonly referred to as a "hard disk drive"). Although not shown in Figure 5, a disk drive may be provided for reading and writing to removable non-volatile magnetic disks (eg "floppy disks"), as well as removable non-volatile optical disks (eg CD-ROM, DVD-ROM) or other optical media) to read and write optical drives. In these cases, each drive may be connected to bus 518 through one or more data media interfaces. Memory 528 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present invention.

A program/utility 540 having a set (at least one) of program modules 542, which may be stored, for example, in memory 528, such program modules 542 including, but not limited to, an operating system, one or more application programs, other program modules, and program data , each or some combination of these examples may include an implementation of a network environment. Program modules 542 generally perform the functions and/or methods of the described embodiments of the present invention.

Computer device 512 may also communicate with one or more external devices 514 (eg, keyboard, pointing device, display 524, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 512, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 512 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 522 . Also, the computer device 512 may communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network such as the Internet) through a network adapter 520 . As shown, network adapter 520 communicates with other modules of computer device 512 via bus 518 . It should be appreciated that, although not shown, other hardware and/or software modules may be used in conjunction with computer device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives and data backup storage systems.

The processor 516 executes various functional applications and data processing by running the program stored in the system memory 528, for example, to implement the pedestrian detection method provided by the embodiment of the present invention, and the method includes:

Obtain the image to be detected;

Input the image to be detected into the trained single-shot target detector, and obtain the output information of the single-shot target detector;

According to the output information of the single-shot target detector, determine the pedestrian detection result of the image to be detected;

The single-shot target detector is obtained by pre-training an original detection model including an initially constructed single-shot target detector and a head detection network.

Of course, those skilled in the art can understand that the processor can also implement the technical solution of the pedestrian detection method provided by any embodiment of the present invention.

Embodiment 6

Embodiment 6 of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the pedestrian detection method provided by the embodiment of the present invention, and the method includes:

Obtain the image to be detected;

Input the image to be detected into the trained single-shot target detector, and obtain the output information of the single-shot target detector;

According to the output information of the single-shot target detector, determine the pedestrian detection result of the image to be detected;

The single-shot target detector is obtained by pre-training an original detection model including an initially constructed single-shot target detector and a head detection network.

Certainly, the computer program stored on the computer-readable storage medium provided by the embodiment of the present invention is not limited to the above-mentioned method operations, and can also perform related operations of the pedestrian detection method provided by any embodiment of the present invention.

The computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .

Program code embodied on a computer readable medium may be transmitted using any suitable medium, including - but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (10)

1. A pedestrian detection method, characterized by comprising:

acquiring an image to be detected;

inputting the image to be detected into a trained single target detector, and acquiring output information of the single target detector;

determining a pedestrian detection result of the image to be detected according to the output information of the single target detector;

the single-time target detector is obtained by training an original detection model which comprises an initially constructed single-time target detector and a head detection network in advance.

2. The method of claim 1, further comprising, before inputting the image to be detected into a trained single-pass object detector:

acquiring a sample image, a pedestrian frame marking result corresponding to the sample image and a pedestrian head marking result corresponding to the sample image;

generating a training sample pair based on the sample image, the pedestrian frame labeling result corresponding to the sample image and the pedestrian head labeling result corresponding to the sample image, and training a pre-constructed original detection model by using the training sample pair to obtain a trained original detection model;

and taking the single target detector in the trained original detection model as the trained single target detector.

3. The method of claim 2, wherein the training of the pre-constructed raw detection model using the training sample pairs to obtain a trained raw detection model comprises:

inputting the sample image into an initially constructed single target detector to obtain an original feature map, a pedestrian frame detection result and a detection score corresponding to the pedestrian frame detection result, which are output by the single target detector;

sorting the pedestrian frame detection results according to the detection scores, and acquiring a preset number of target pedestrian frame detection results according to the sorting results;

inputting the original feature map and the target pedestrian frame detection result into the head detection network to obtain a pedestrian head detection result output by the head detection network;

determining a first loss value according to the pedestrian frame detection result and the pedestrian frame marking result, determining a second loss value according to the pedestrian head detection result and the pedestrian head marking result, and determining a target loss value according to the first loss value and the second loss value;

and training the original detection model by taking the goal that the target loss value reaches a convergence condition as a goal.

4. The method of claim 3, wherein the single-pass object detector comprises a plurality of primitive feature network layers, and wherein an upsampling module is further included between the object feature network layer in the primitive detection model and the header detection network, and wherein the method further comprises:

selecting at least one original characteristic network layer as a target characteristic network layer according to the image size of the original characteristic graph output by each original characteristic network layer;

and adding an up-sampling module behind the target feature network layer, and adding the head detection network behind the up-sampling module.

5. The method according to claim 4, wherein the selecting at least one of the original feature network layers as a target feature network layer according to an image size of an original feature map output by each of the original feature network layers comprises:

and taking the original characteristic network layer corresponding to the original characteristic graph with the image size smaller than the set threshold value as the target characteristic network layer.

6. The method according to claim 5, before inputting the original feature map and the target pedestrian frame detection result into the head detection network and obtaining the pedestrian head detection result output by the head detection network, further comprising:

acquiring a target original characteristic diagram output by the target characteristic network layer, and inputting the target original characteristic diagram into the up-sampling module to obtain an up-sampling characteristic diagram output by the up-sampling module;

correspondingly, the inputting the original feature map and the target pedestrian frame detection result into the head detection network to obtain the pedestrian head detection result output by the head detection network includes:

and inputting the original feature map, the up-sampling feature map and the target pedestrian frame detection result output by other original feature network layers except the target feature network layer in the original feature network layer into the head detection network to obtain the pedestrian head detection result output by the head detection network.

7. The method of claim 4, wherein the upsampling module is a transposed convolution module.

8. A pedestrian detection device, characterized by comprising:

the image acquisition module to be detected is used for acquiring an image to be detected;

the image pedestrian detection module is used for inputting the image to be detected into a trained single target detector and acquiring output information of the single target detector, wherein the single target detector is obtained by training an original detection model which comprises the initially constructed single target detector and a head detection network in advance;

and the detection result determining module is used for determining the pedestrian detection result of the image to be detected according to the output information of the single target detector.

9. A computer device, the device comprising:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the pedestrian detection method of any one of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the pedestrian detection method according to any one of claims 1 to 7.

Images