CN117711076A - Speckle living body detection method, system, equipment and storage medium based on time sequence - Google Patents

Abstract

The present invention provides a time-series-based speckle living body detection method, which includes: obtaining a video to be detected, performing target detection, obtaining an initial frame containing the target object, and using a target tracking algorithm to perform target tracking. For each frame of image tracked , obtain the area of interest, construct a brightness change vector of N pixels for the area of interest in multiple image frames, use the brightness change vector as the input of the living body detection classifier, and determine the M image frames represented by the feature vector. Whether the target object is a living body. The invention has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

Description

A time-series-based speckle living body detection method, system, equipment and storage medium

Technical field

The present invention relates to the technical field of human biometric identification, and specifically to a time-series-based speckle living body detection method, system, equipment and storage medium.

Background technique

Using speckles for living body recognition is an advanced technology. It is based on the characteristics of speckles and achieves the identification and classification of living organisms by acquiring, processing and analyzing speckle images. This technology has the characteristics of non-contact, high precision and high efficiency, and has been widely used in medical, security, biometric authentication and other fields.

In the process of acquiring speckle images, it is necessary to use appropriate light sources and optical systems, and ensure that the light sources and systems are clean and stable. Through the processing and analysis of speckle images, the characteristics of organisms can be extracted, classified and identified. The advantages of speckle technology are that it is safe, difficult to replicate and non-contact. Because speckle characteristics are closely related to individual physiological structures, they are difficult to copy and forge. At the same time, speckle technology is a non-contact biometric technology that will not cause any harm to the human body.

At present, speckle technology has been widely used in many fields such as finance, security, and medical care. For example, in the financial field, speckle technology can be used for identity verification to improve transaction security; in the security field, speckle technology can be used to identify people and improve security efficiency; in the medical field, speckle technology can be used to diagnose diseases. , improve the accuracy of diagnosis.

However, there are also some challenges in using speckles for living body recognition, such as lighting conditions, noise interference, feature extraction and classifier design, and the diversity of application scenarios.

The disclosure of the above background technology content is only used to assist in understanding the inventive concepts and technical solutions of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, In this case, the above background technology should not be used to evaluate the novelty and inventiveness of this application.

Contents of the invention

To this end, the present invention detects target objects in multi-frame speckle images. After detecting target objects in consecutive multiple frames, it provides a group of pixels for the speckle spots in the area of interest, constructs a vector in which the brightness of the pixels changes with time, and based on The brightness change vector determines whether the target object is a living body. It has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

In a first aspect, the present invention provides a time-series-based speckle living body detection method, which is characterized by including:

Step S1: Obtain the video to be detected;

Step S2: Use the first frame image in the video to be detected as the frame to be detected;

Step S3: Perform target detection on the frame to be detected, and determine whether the frame to be detected contains a target object; if so, locate the target area and execute step S4; otherwise, execute step S5;

Step S4: Use the frame to be detected as an initial frame, use a feature point detection algorithm to extract feature points that should be selected in the initial frame, use the target area located in the initial frame as a tracking area, and use the target area to be detected as the tracking area. The next frame of the initial frame in the video is used as the frame to be tracked, and step S6 is executed;

Step S5: Use the next frame of the image in the video to be detected as the frame to be detected, and execute step S3;

Step S6: Use the tracking algorithm to obtain candidate feature points of the frame to be tracked, and determine whether the number of feature points that should be selected in the frame to be tracked meets the first preset condition, wherein the feature points that should be selected are the Feature points among candidate feature points that satisfy a second preset condition, where the second preset condition is that the distance to the corresponding feature point that should be selected in the previous frame of the frame to be tracked is less than or equal to S, if The number of feature points that should be selected in the frame to be tracked satisfies the first preset condition, it is determined that the target object is tracked in the frame to be tracked, and the target object to be tracked is determined based on the position of the feature points that should be selected in the frame to be tracked. The target area on the frame, and execute step S7, otherwise, determine that the target object is not tracked in the frame to be tracked, use the next frame of the frame to be tracked as the frame to be detected, and execute step S3;

Step S7: Obtain the area of interest in the target area on the frame to be tracked, and execute step S8;

Step S8: Determine whether the number of image frames tracked to the target is equal to M. If so, execute step S9; otherwise, execute step S10;

Step S9: Extract a group of pixels from each speckle spot in the area of interest on M image frames in the same direction, and construct a vector of brightness changes with time for each pixel, and obtain the brightness change vector of N pixels , and execute step S11;

Step S10, obtain the next frame of the frame to be tracked, use the next frame of the frame to be tracked as a new frame to be tracked, and execute step S6;

Step S11, use the brightness change vector as the input of a living body detection classifier to determine whether the target object in the M image frames represented by the feature vector is a living body, wherein the living body detection classifier uses pre-obtained from It is trained with feature vectors extracted from real target objects and/or attack target objects.

Optionally, the described speckle living body detection method based on time series is characterized in that, when extracting pixel points in step S9, M image frames are extracted at fixed positions.

Optionally, the described speckle living body detection method based on time series is characterized in that, when extracting pixel points in step S9, M image frames are extracted with the center of the speckle spot as the axis.

Optionally, the described speckle living body detection method based on timing is characterized in that step S9 includes:

Step S91: According to the distribution of speckle spots in the area of interest, identify the straight line direction connecting the most speckle spots as the first direction;

Step S92: Extract the speckle spots in the region of interest in the M image frames in the first direction, and obtain the brightness change vector of the speckle spots;

Step S93: Extract all speckle spots in the region of interest, obtain N groups of brightness change vectors, and execute step S11.

Optionally, the time-series-based speckle life detection method is characterized in that the neural network of the life detection classifier during training includes: a first reverse residual module, a first pooling layer, The second reverse residual module, the second pooling layer, the third reverse residual module, the first overlay layer, the third pooling layer, the fourth reverse residual module, the second overlay layer, and the fourth pooling layer, the fifth reverse residual module, the third convolution layer, the fifth pooling layer, the random dropout layer, the classifier, the first convolution layer, the second convolution layer; wherein, the first reverse residual The difference module outputs to the first pooling layer and the first convolution layer, the first convolution layer and the third inverse residual module input the first overlay layer, and the second inverse residual module The residual module is output to the second convolution layer and the second pooling layer, and the second convolution layer and the fourth inverse residual module input the second overlay layer; the classification The device outputs the probability of living body and the probability of prosthetic body.

Optionally, the described speckle life detection method based on time series is characterized in that, in step S11, the positional relationship of multiple pixels and the brightness change vector are also used as the input of the life detection classifier. .

Optionally, the time-series-based speckle living body detection method is characterized in that, in step S11, different weight values are assigned to the brightness change vectors according to the positions of the pixel points.

In a second aspect, the present invention provides a timing-based speckle living body detection system for implementing any of the aforementioned timing-based speckle living body detection methods, which is characterized in that it includes:

Video module, used to obtain the video to be detected;

A frame detection module, configured to use the first frame image in the video to be detected as the frame to be detected;

An object detection module is used to perform target detection on the frame to be detected, and determine whether the frame to be detected contains a target object; if so, locate the target area and execute the area module; otherwise, execute the first movement module;

A region module, configured to use the frame to be detected as an initial frame, use a feature point detection algorithm to extract feature points that should be selected in the initial frame, use the target area located in the initial frame as a tracking area, and use the feature point detection algorithm to extract the selected feature points in the initial frame. The next frame of the initial frame in the video to be detected is used as the frame to be tracked, and the filtering module is executed;

The first moving module is used to use the next frame of the image in the video to be detected as the frame to be detected, and execute the object detection module;

A screening module, configured to use a tracking algorithm to obtain candidate feature points of the frame to be tracked, and determine whether the number of feature points that should be selected in the frame to be tracked satisfies the first preset condition, wherein the feature points that should be selected are Feature points among the candidate feature points that satisfy a second preset condition, wherein the second preset condition is that the distance to the corresponding feature point that should be selected in the previous frame of the frame to be tracked is less than or equal to S, If the number of feature points that should be selected in the frame to be tracked meets the first preset condition, it is determined that the target object is tracked in the frame to be tracked, and the target object is determined according to the position of the feature points that should be selected in the frame to be tracked. The target area on the frame to be tracked, and execute the interception module; otherwise, determine that the target object is not tracked in the frame to be tracked, use the next frame of the frame to be tracked as the frame to be detected, and execute the object detection module;

An interception module, used to obtain the area of interest in the target area on the frame to be tracked, and execute the confirmation module;

The confirmation module is used to determine whether the number of image frames tracked to the target is equal to M. If so, execute the pixel module; otherwise, execute the second movement module;

The pixel module is used to extract a set of pixels from each speckle spot in the area of interest on M image frames in the same direction, and construct a vector of changes in brightness over time for each pixel to obtain the brightness of N pixels. Change the vector and execute the judgment module;

The second movement module is used to obtain the next frame of the frame to be tracked, use the next frame of the frame to be tracked as a new frame to be tracked, and execute the filtering module;

A judgment module used to use the brightness change vector as an input to a living body detection classifier to judge whether the target object in the M image frames represented by the feature vector is a living body, wherein the living body detection classifier uses pre-obtained It is trained with feature vectors extracted from real target objects and/or attack target objects.

In a third aspect, the present invention provides a time-series-based speckle living body detection device, which is characterized in that it includes:

processor;

A memory in which executable instructions of the processor are stored;

Wherein, the processor is configured to execute any one of the foregoing steps of the timing-based speckle life detection method by executing the executable instructions.

In a fourth aspect, the present invention provides a computer-readable storage medium for storing a program, which is characterized in that when the program is executed, the steps of any one of the aforementioned timing-based speckle life detection methods are implemented.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention uses the changes between multi-frame speckle images for living body detection. It takes advantage of the different effects of speckles at different depths and different material surfaces, and the different characteristics that change with time. It is better to determine whether the target object is a living body than by using a single speckle pattern. Speckle characteristics or single-frame speckle patterns have better results in living body recognition. At the same time, due to the comparison between multi-frame speckles, it is effective for high-imitation three-dimensional prostheses (such as high-simulation silicone hand models, rubber hand models, etc.) Better recognition effect.

The invention can make prosthetic judgments on three-dimensional silicone hand models, three-dimensional plastic hand models, three-dimensional rubber hand models, three-dimensional silicone sleeves, and two-dimensional planar prosthetic hands, and can effectively grasp the essential characteristics of living real hands and attack the prostheses. It has a wide range of resistance to various types of attack prostheses.

The invention can effectively reduce the impact of ambient lighting on the algorithm, and can work stably without being affected during the day and night and under various extreme lighting conditions.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without exerting creative efforts. Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

Figure 1 is a step flow chart of a timing-based speckle living body detection method in an embodiment of the present invention;

Figure 2 is a speckle pattern in an embodiment of the present invention;

Figure 3 is a schematic diagram of extracting a speckle pattern in an embodiment of the present invention;

Figure 4 is a schematic diagram of the change in speckle brightness reflected by a real person's skin over time in an embodiment of the present invention;

Figure 5 is a schematic diagram of the speckle brightness reflected by a silicone hand mold prosthesis changing with time in an embodiment of the present invention;

Figure 6 is a schematic structural diagram of a neural network in an embodiment of the present invention;

Figure 7 is a schematic structural diagram of a reverse residual module in an embodiment of the present invention;

Figure 8 is a flow chart of steps for obtaining the brightness change vectors of N pixels in an embodiment of the present invention;

Figure 9 is a schematic structural diagram of a timing-based speckle living body detection system in an embodiment of the present invention;

Figure 10 is a schematic structural diagram of a timing-based speckle life detection device in an embodiment of the present invention; and

Figure 11 is a schematic structural diagram of a computer-readable storage medium in an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects without necessarily using Used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein, for example, can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "include" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

The embodiment of the present invention provides a time-series-based speckle living body detection depth module method, aiming to solve the problems existing in the existing technology.

The technical solution of the present invention and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will be described below with reference to the accompanying drawings.

The present invention detects target objects in multi-frame speckle patterns. After detecting target objects in consecutive multi-frames, it provides a group of pixels for the speckle spots in the area of interest, constructs a vector in which the brightness of the pixels changes with time, and calculates the brightness change vector according to the brightness change vector. It determines whether the target object is a living body, has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

Figure 1 is a step flow chart of a time-series-based speckle living body detection method in an embodiment of the present invention. As shown in Figure 1, the steps of a timing-based speckle living body detection method in the embodiment of the present invention include:

Step S1: Obtain the video to be detected.

In this step, the video to be detected is extracted from the data source. The video can be a live video stream or a pre-recorded video. The video can be a video obtained locally in real time, or a video collected and sent by other terminals. The video to be detected can be a video of any length, as long as part of it meets the requirements. The video to be detected can be a complete video or an edited video. The video to be detected is a video composed of speckle patterns. The speckle image is generated by the infrared spot projection signal and contains information about the target object. The target object can be any human body part such as face, palm, pupil, etc. As shown in Figure 2, the speckle image is composed of multiple discontinuous speckle patterns, which is obviously different from continuous infrared images, RGB images, etc. At the same time, the shape of the speckle contains the depth information of the target object, which can be better used for live detection.

The depth value of the target object needs to be within a preset range. If it exceeds the preset range, the signal of the target object may be weak, making it difficult to perform effective signal extraction. For speckle images, the depth value of each speckle object can be calculated directly using the parallax principle, so that it can be evaluated whether it meets the requirements.

Step S2: Use the first frame image in the video to be detected as the frame to be detected.

In this step, according to the time sequence of the image frames in the video to be detected, or the specified starting time point, the earliest first frame of the image is selected as the frame to be detected.

Step S3: Perform target detection on the frame to be detected, and determine whether the frame to be detected contains a target object; if so, locate the target area and execute step S4; otherwise, execute step S5.

In this step, object detection is performed on the selected frames. This involves using image processing or machine learning techniques to identify and locate target objects in videos. If the target object is included in the frame to be detected, the algorithm will locate the target area and enter the next step S4; otherwise, it will return to step S5 and continue to detect the next frame.

When performing target detection, select an appropriate target detection model according to the application scenario. For example, when detecting faces, the target detection model is used; when detecting palms, the palm detection model is used; when detecting pupils, the pupil detection model is used. It should be noted that since the speckle pattern is used in this embodiment, the target detection model needs to be a model that can be applied to the speckle pattern, rather than the traditional target detection model based on infrared images or RGB images.

In some embodiments, infrared images or RGB images are also obtained simultaneously. The infrared image or RGB image is aligned with the speckle image. By detecting the target object on the infrared image or RGB image, it is judged whether the target object is included in the speckle image. If the target object is detected on the infrared image or RGB image, the target area can be obtained, and then the corresponding relationship is used to obtain the target area on the speckle image.

Since the target detection algorithm takes a long time, if target detection is performed on every frame of the video to be detected, it will take a lot of time to complete the living body detection, resulting in a very low efficiency of living body detection. After using the target detection algorithm to detect the target object on a frame to be detected, the target tracking algorithm is used to track the target object in subsequent image frames, which can significantly shorten the time of live detection, thereby improving the efficiency of live detection.

Step S4: Use the frame to be detected as an initial frame, use a feature point detection algorithm to extract feature points that should be selected in the initial frame, use the target area located in the initial frame as a tracking area, and use the target area to be detected as the tracking area. The next frame of the initial frame in the video is used as the frame to be tracked, and step S6 is performed.

In this step, the frame to be detected detected in step S3 is used as the initial frame for target tracking. The target area in the initial frame is used as the tracking area, and the next frame of the initial frame is tracked based on the tracking area, thereby obtaining the target object in the next frame of the initial frame. When performing target tracking, similar to the previous steps, a target tracking algorithm suitable for speckle patterns can be used. In some embodiments, when performing target tracking, a target tracking algorithm suitable for RGB images or infrared images can also be used, and then the selected feature points can be obtained based on the correspondence between the RGB image or infrared image and the speckle image.

Step S5: Use the next frame of the image in the video to be detected as the frame to be detected, and execute step S3.

In this step, we use the next frame of the image in the video to be detected as the new frame to be detected, and then return to step S3 to continue target detection.

Step S6: Use the tracking algorithm to obtain the candidate feature points of the frame to be tracked, and determine whether the number of feature points that should be selected in the frame to be tracked meets the first preset condition, wherein the feature points that should be selected are the Feature points among candidate feature points that satisfy a second preset condition, wherein the second preset condition is that the distance to the corresponding feature point that should be selected in the previous frame of the frame to be tracked is less than or equal to S, if The number of feature points that should be selected in the frame to be tracked meets the first preset condition, it is determined that the target object is tracked in the frame to be tracked, and the target object to be tracked is determined based on the position of the feature points that should be selected in the frame to be tracked. target area on the frame, and execute step S7. Otherwise, it is determined that the target object is not tracked in the frame to be tracked, and the next frame of the frame to be tracked is used as the frame to be detected, and step S3 is executed.

In this step, a tracking algorithm is used to obtain candidate feature points of the frame to be tracked, and then it is determined whether the number of feature points that should be selected in the frame to be tracked meets the first preset condition. If it is satisfied, we determine that the target object is tracked in the frame to be tracked, and determine the target area on the frame to be tracked based on the location of the selected feature points; if not, we determine that the target object is not tracked in the frame to be tracked, and then Use the next frame of the frame to be tracked as the new frame to be detected, and return to step S3.

Step S7: Obtain the area of interest in the target area on the frame to be tracked, and execute step S8.

In this step, the region of interest is part of the target area. Since the target area usually includes the target object and part of the background, the background and part of the target object have no practical significance for the living body determination in this embodiment, but will interfere with the subsequent proportion judgment, so it needs to be used in the area of interest. Feature points identify smaller regions of interest. For example, for faces, the area between multiple keys is identified as the area of interest; for palms, the key points of the palm are identified as the area of interest. The region of interest is the part of the target area that changes most over time. The extent of the region of interest varies with the type of target object.

Step S8: Determine whether the number of image frames tracked to the target is equal to M. If so, execute step S9. Otherwise, execute step S10.

In this step, every time an image frame containing the target object is tracked, it is judged whether the number of image frames that have tracked the target object so far reaches the preset M, where the specific value of M can be based on the living body detection. Settings are required, and the specific value of M is not limited in this embodiment. When the number of image frames tracking the target object reaches the preset M value, steps S9 and S10 are executed to extract M image frames. When the number of frames in which the target object is tracked has not reached the preset M value, the target object tracking is continued on the next frame of the current frame until the number of image frames in which the target object is tracked reaches the preset M value.

Step S9: Extract a group of pixels from each speckle spot in the area of interest on M image frames in the same direction, and construct a vector of brightness changes with time for each pixel, and obtain the brightness change vector of N pixels. , and execute step S11.

In this step, pixel points are extracted in the same direction from multiple speckles on the speckle map. As shown in Figure 3, when extracting pixels, a straight line passes through the speckle spots to obtain a set of pixels.

A set of images was taken of the real palm and the silicone hand model, and a set of pixels was extracted from each frame of the two sets of images. For each image, extract the speckle pattern brightness of this group of pixels on the horizontal line along the drawn horizontal line as the value of the ordinate, use the horizontal position of the horizontal line (ie, the X-axis direction) as the abscissa, and The brightness of each group of multi-frame images is plotted on the same coordinate axis, and the time-varying characteristics of the speckle brightness of the real person and the attack prosthesis are obtained, as shown in Figures 4 and 5 respectively. As can be seen from Figure 4, after speckles are irradiated onto the skin of a real person, the capillaries in the skin are in a dynamic state of blood flow, resulting in dynamic changes in optical properties. Therefore, the brightness of the speckles reflected by the real person is is in a dynamic change; in contrast, as can be seen from Figure 5, the 3D silicone hand prosthesis does not have this characteristic of real skin, so the reflection brightness of laser speckles is always at a stable level. state.

When constructing a vector of changes in brightness over time for each pixel, take each pixel as an object and record the relationship between changes in brightness over time to obtain a vector of brightness over time. A total of N pixels are extracted from the area of interest of the image frame, and the change vectors of the N pixels can be obtained. The pixels extracted in this step represent the characteristics of each speckle in the area of interest. Compared with the entire speckle data, the calculation amount is smaller, and the results are comparable.

In some embodiments, when extracting pixel points, M image frames are extracted at fixed positions. In the initial frame, the pixels of each speckle spot are obtained according to the aforementioned method, and the position of the pixel is fixed. The position is directly used to extract in subsequent images, thereby finally obtaining the brightness change vector of N pixels. This embodiment can take advantage of the fast frame rate feature to reduce the amount of calculation and increase the calculation speed.

In some embodiments, when extracting pixel points, M image frames are extracted with the center of the speckle spot as the axis. The position of the pixel is re-determined for each frame, and the correspondence between multiple pixels is determined based on the correspondence between speckles.

In some embodiments, if the number of pixels of the same speckle on different image frames is different, the minimum number of pixels corresponding to the speckle in the M image frames is regarded as the effective pixel, and the number of pixels is determined from the centermost pixel of each group. The pixels are corresponding, and corresponding are performed on both sides in order to obtain an accurate correspondence, thereby obtaining a unique brightness change vector.

Step S10: Obtain the next frame of the frame to be tracked, use the frame next to the frame to be tracked as a new frame to be tracked, and execute step S6.

In this step, when the number of frames in which the face is tracked has not reached the preset M value, face tracking continues on the next frame of the current frame to be tracked.

Step S11, use the brightness change vector as the input of a living body detection classifier to determine whether the target object in the M image frames represented by the feature vector is a living body, wherein the living body detection classifier uses pre-obtained from It is trained with feature vectors extracted from real target objects and/or attack target objects.

In this step, the brightness change vector is used as the input of the living body detection classifier to determine whether the target object in these M image frames is a living body. This living body detection classifier is trained using feature vectors extracted from real target objects and/or attack target objects in advance. Living body detection can be treated as a two-classification problem. For any input feature vector, the feature can be judged The vector represents whether the target object in the M image frames is a living body.

In some embodiments, the positional relationship of multiple pixels and the brightness change vector are also used as the input of the living body detection classifier. Compared with the previous embodiment, the brightness change vector in this embodiment not only represents the brightness change of a single pixel over time, but can also horizontally represent the brightness relationship between pixels at different positions, thereby forming a more three-dimensional brightness change space. , greatly improving the recognition effect of prostheses.

In some embodiments, in step S11, different weight values are assigned to the brightness change vectors according to the positions of the pixels. For the human body, the changes caused by the flow of blood vessels vary from different individuals and different parts. When the weight is set, weight analysis is performed based on the location of the blood vessels. The thicker the blood vessels, the greater the weight of nearby pixels; the thinner the blood vessels, the smaller the weight of nearby pixels. The blood vessels in the face and palms are distributed differently, so the weight settings are also completely different. It should be noted that the distribution of blood vessels on the face or palms of different people, especially the distribution positions of capillaries, is not exactly the same. Therefore, when setting the weights in this embodiment, the area near the blood vessels is used to assign weights to several adjacent pixels. The same weight value. For example, pixels are divided into three categories according to their locations: close to thick blood vessels, close to thin blood vessels, and without surrounding blood vessels, and given different weight values respectively. In this embodiment, by assigning different weight values to the brightness change vectors of different pixel points, positions with obvious changes can be more easily identified, the prosthesis can be better identified, and the identification efficiency can be improved.

Figure 6 is a schematic structural diagram of a neural network in an embodiment of the present invention. As shown in Figure 6, a neural network in the embodiment of the present invention is used to train a deep learning model, including: a first reverse residual module, a first pooling layer, a second reverse residual module, and a second pooling layer. layer, the third reverse residual module, the first overlay layer, the third pooling layer, the fourth reverse residual module, the second overlay layer, the fourth pooling layer, the fifth reverse residual module, the third Convolutional layer, fifth pooling layer, random discarding layer, classifier, first convolutional layer, second convolutional layer; wherein the first reverse residual module outputs to the first pooling layer and The first convolution layer, the first convolution layer and the third inverse residual module input the first overlay layer, and the second inverse residual module outputs to the second convolution layer and the second pooling layer, the second convolution layer and the fourth inverse residual module input the second overlay layer; the classifier outputs the probability of living body and the probability of prosthesis.

It should be noted that those skilled in the art can adjust the number of pooling layers according to the image size, thereby striking a balance between training speed and training effect.

The neural network design during training of deep learning models has the following significant advantages:

Layer-by-layer pre-training: The deep learning model adopts a layer-by-layer pre-training training mechanism, which enables it to overcome the problems of easy overfitting and slow training speed of traditional neural networks.

Automatic feature learning: Through layer-by-layer data pre-training, the deep learning model can automatically learn the primary features of the data, thus overcoming the time-consuming and laborious traditional method of manually designing features.

Distributed data learning: Deep learning models can learn more effectively on distributed data, making learning efficiency exponential.

Deep modeling capabilities: Compared with shallow modeling methods, deep modeling can represent actual complex nonlinear problems in more detail and efficiently. This means that deep learning models have stronger performance when dealing with complex real-world problems, such as image recognition, speech recognition, etc.

This embodiment enables the deep learning model to have higher efficiency and accuracy when processing complex tasks.

Figure 7 is a schematic structural diagram of a reverse residual module in an embodiment of the present invention. As shown in Figure 7, a reverse residual module in the embodiment of the present invention includes: a first sub-convolution layer, a first batch normalization layer, a first non-linear activation layer, a second sub-convolution layer, two batch normalization layers, a second nonlinear activation layer, a third sub-convolution layer, a third batch normalization layer, and a first sub-overlay layer; wherein the first sub-convolution layer is output to the A batch normalization layer, the first sub-convolutional layer and the third batch normalization layer are input to the first overlay layer.

The reverse residual module in this embodiment may be the first reverse residual module, the second reverse residual module, the third reverse residual module, the fourth reverse residual module in the previous embodiment. Any one or more of the residual module and the fifth reverse residual module.

This embodiment has at least the following obvious advantages:

Reduce the number of model parameters: The inverse residual module can effectively reduce the number of model parameters by performing a reversible transformation on the residual connection. Traditional residual modules require a large number of convolutional layers, which require a large number of parameters. The inverse residual module effectively reduces the number of parameters by inverting the residual structure, that is, first performing projected convolution to increase the dimension, then using depth convolution, and finally using projected convolution to reduce the dimension.

Improve model performance: The design of the inverse residual module prevents the network from disappearing or exploding gradients during training, allowing the training of very deep networks. This helps improve model performance.

Low implementation complexity: The implementation of the inverse residual module is mainly completed through two layers of the same or different convolutional layers and a skip connection. Its implementation is relatively simple and the computational complexity is low.

Therefore, the embodiment effectively solves the gradient problem and the problem of too many parameters in deep network training, improves the performance of the model, and is relatively simple to implement, which makes it of high practical value when processing complex tasks.

FIG. 8 is a flow chart of steps for obtaining brightness change vectors of N pixel points in an embodiment of the present invention. As shown in Figure 8, in the embodiment of the present invention, a step of obtaining the brightness change vector of N pixel points includes:

Step S91: According to the distribution of speckle spots in the area of interest, identify the straight line direction connecting the most speckle spots as the first direction.

In this step, you first need to count and analyze the speckle spots in the area of interest. The distribution of speckle spots can be identified by calculating the coordinate value of each speckle spot and then using a clustering algorithm (such as K-means) to group these coordinate values. Next, you can calculate the center point of each set of speckle spots and find the direction of the straight line connecting these center points. Finally, the straight line direction connecting the most speckle spots is selected as the first direction.

Because speckle is designed, it will be distributed in a certain pattern within the region of interest in an image frame. By analyzing this pattern, the direction of the straight lines connecting the most speckle spots can be identified.

Step S92: Extract the speckle spots in the region of interest in the M image frames in the first direction to obtain the brightness change vector of the speckle spots.

In this step, the speckle spots of the area of interest in a series of image frames (M image frames) are extracted using the first direction identified in the previous step. This process will obtain the brightness change vector of each speckle spot. This vector can represent the brightness change of the speckle spot between different frames.

Step S93: Extract all speckle spots in the region of interest, obtain N groups of brightness change vectors, and execute step S11.

In this step, all speckle spots in the area of interest need to be extracted to obtain N sets of brightness change vectors. These brightness change vectors are stored in a matrix for subsequent processing in step S11.

This embodiment identifies the first direction connecting the most speckle spots, extracts some pixels from the speckle spots, and obtains N sets of brightness change vectors, which can more accurately extract speckle spots in the area of interest, thus improving the accuracy of subsequent processing. The distribution and changes of speckle spots can be better analyzed, the amount of calculation can be reduced, and the calculation efficiency can be improved.

FIG. 9 is a schematic structural diagram of a timing-based speckle life detection system in an embodiment of the present invention. As shown in Figure 9, a timing-based speckle living body detection system in the embodiment of the present invention includes:

Video module, used to obtain the video to be detected;

A frame detection module, configured to use the first frame image in the video to be detected as the frame to be detected;

An object detection module is used to perform target detection on the frame to be detected, and determine whether the frame to be detected contains a target object; if so, locate the target area and execute the area module; otherwise, execute the first movement module;

A region module, configured to use the frame to be detected as an initial frame, use a feature point detection algorithm to extract feature points that should be selected in the initial frame, use the target area located in the initial frame as a tracking area, and use the feature point detection algorithm to extract the selected feature points in the initial frame. The next frame of the initial frame in the video to be detected is used as the frame to be tracked, and the filtering module is executed;

The first moving module is used to use the next frame of the image in the video to be detected as the frame to be detected, and execute the object detection module;

A screening module, configured to use a tracking algorithm to obtain candidate feature points of the frame to be tracked, and determine whether the number of feature points that should be selected in the frame to be tracked satisfies the first preset condition, wherein the feature points that should be selected are Feature points among the candidate feature points that satisfy a second preset condition, wherein the second preset condition is that the distance to the corresponding feature point that should be selected in the previous frame of the frame to be tracked is less than or equal to S, If the number of feature points that should be selected in the frame to be tracked meets the first preset condition, it is determined that the target object is tracked in the frame to be tracked, and the target object is determined according to the position of the feature points that should be selected in the frame to be tracked. The target area on the frame to be tracked, and execute the interception module; otherwise, determine that the target object is not tracked in the frame to be tracked, use the next frame of the frame to be tracked as the frame to be detected, and execute the object detection module;

An interception module, used to obtain the area of interest in the target area on the frame to be tracked, and execute the confirmation module;

The confirmation module is used to determine whether the number of image frames tracked to the target is equal to M. If so, execute the pixel module; otherwise, execute the second movement module;

The pixel module is used to extract a set of pixels from each speckle spot in the area of interest on M image frames in the same direction, and construct a vector of changes in brightness over time for each pixel to obtain the brightness of N pixels. Change the vector and execute the judgment module;

The second movement module is used to obtain the next frame of the frame to be tracked, use the next frame of the frame to be tracked as a new frame to be tracked, and execute the filtering module;

A judgment module used to use the brightness change vector as an input to a living body detection classifier to judge whether the target object in the M image frames represented by the feature vector is a living body, wherein the living body detection classifier uses pre-obtained It is trained with feature vectors extracted from real target objects and/or attack target objects.

This embodiment detects the target object in multi-frame speckle images. After detecting the target object in consecutive multiple frames, a group of pixels is provided for the speckle spots in the area of interest, and a vector of the brightness of the pixels changing with time is constructed, and a vector is constructed according to the brightness change. The vector determines whether the target object is a living body. It has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

An embodiment of the present invention also provides a timing-based speckle life detection device, including a processor. Memory, which stores the executable instructions of the processor. Wherein, the processor is configured to execute the steps of a timing-based speckle life detection method by executing executable instructions.

As above, this embodiment detects the target object in multi-frame speckle images. After detecting the target object in consecutive multiple frames, it provides a set of pixels for the speckle spots in the area of interest, constructs a vector of changes in the brightness of the pixels over time, and based on The brightness change vector determines whether the target object is a living body. It has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

Those skilled in the art will understand that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention can be implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "Circuit", "Module" or "Platform".

Figure 10 is a schematic structural diagram of a timing-based speckle life detection device in an embodiment of the present invention. An electronic device 600 according to this embodiment of the present invention is described below with reference to FIG. 10 . The electronic device 600 shown in FIG. 10 is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present invention.

As shown in Figure 10, electronic device 600 is embodied in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, and the like.

Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the various exemplary embodiments of the present invention described in the above-mentioned timing-based speckle life detection method part of this specification. step. For example, processing unit 610 may perform steps as shown in FIG. 1 .

The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and/or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

Storage unit 620 may also include a program/utility 6204 having a set of (at least one) program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples, or some combination, may include the implementation of a network environment.

Bus 630 may be a local area representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or using any of a variety of bus structures. bus.

Electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 600, and/or with Any device (eg, router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. This communication may occur through input/output (I/O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 660. Network adapter 660 may communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in Figure 10, other hardware and/or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes Drives and data backup storage platforms, etc.

Embodiments of the present invention also provide a computer-readable storage medium for storing a program, and when the program is executed, the steps of a timing-based speckle living body detection method are implemented. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to execute the above described instructions. The steps according to various exemplary embodiments of the present invention are described in the section of a time-series-based speckle living body detection method.

As shown above, this embodiment detects target objects in multi-frame speckle patterns. After detecting target objects in consecutive multiple frames, a group of pixels are provided for the speckle spots in the area of interest to construct a vector in which the brightness of the pixels changes with time. It also determines whether the target object is a living body based on the brightness change vector. It has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

Figure 11 is a schematic structural diagram of a computer-readable storage medium in an embodiment of the present invention. Referring to FIG. 11 , a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be used on a terminal device, For example, run on a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus or device.

The Program Product may take the form of one or more readable media in any combination. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

A computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave carrying the readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural programming. Language—such as "C" or a similar programming language. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In situations involving remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., provided by an Internet service). (business comes via Internet connection).

This embodiment detects the target object in multi-frame speckle images. After detecting the target object in consecutive multiple frames, a group of pixels is provided for the speckle spots in the area of interest, and a vector of the brightness of the pixels changing with time is constructed, and a vector is constructed according to the brightness change. The vector determines whether the target object is a living body. It has the advantages of strong anti-interference ability and good recognition effect on various types of prostheses.

Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. The above description of the disclosed embodiments enables those skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above. Those skilled in the art can make various variations or modifications within the scope of the claims, which does not affect the essence of the present invention.

Claims (10)

1. A timing-based speckle biopsy method, comprising:

step S1: acquiring a video to be detected;

Step S2: taking a first frame image in the video to be detected as a frame to be detected;

step S3: performing target detection on the frame to be detected, and judging whether the frame to be detected contains a target object or not; if yes, positioning the target area and executing a step S4, otherwise, executing a step S5;

step S4: taking the frame to be detected as an initial frame, extracting feature points to be selected in the initial frame by utilizing a feature point detection algorithm, taking a target area positioned in the initial frame as a tracking area, taking the next frame of the initial frame in the video to be detected as the frame to be tracked, and executing a step S6;

step S5: taking the next frame image in the video to be detected as a frame to be detected, and executing a step S3;

step S6: obtaining candidate feature points of the frame to be tracked by using a tracking algorithm, judging whether the number of the feature points to be selected in the frame to be tracked meets a first preset condition, wherein the feature points to be selected are feature points meeting a second preset condition in the candidate feature points, the second preset condition is that the distance between the feature points to be selected and the corresponding feature points to be selected in the previous frame of the frame to be tracked is smaller than or equal to S, if the number of the feature points to be selected in the frame to be tracked meets the first preset condition, determining that a target object is tracked in the frame to be tracked, determining a target area on the frame to be tracked according to the position of the feature points to be selected in the frame to be tracked, executing a step S7, otherwise, determining that the target object is not tracked in the frame to be tracked, taking the next frame of the frame to be tracked as a frame to be detected, and executing a step S3;

Step S7: acquiring a region of interest in a target region on the frame to be tracked, and executing a step S8;

step S8: judging whether the number of the image frames tracked to the target is equal to M, if so, executing a step S9, otherwise, executing a step S10;

step S9: extracting a group of pixel points from each scattered spot of the interested area on M image frames in the same direction, constructing a vector of brightness change along with time for each pixel point, obtaining brightness change vectors of N pixel points, and executing step S11;

step S10, obtaining the next frame of the frame to be tracked, taking the next frame of the frame to be tracked as a new frame to be tracked, and executing step S6;

and S11, taking the brightness change vector as the input of a living body detection classifier, and judging whether the target object in M image frames characterized by the feature vector is a living body or not, wherein the living body detection classifier is obtained by training the feature vector extracted from a real target object and/or an attack target object, which is acquired in advance.

2. The method according to claim 1, wherein M image frames are extracted at fixed positions when the pixel points are extracted in step S9.

3. The method according to claim 1, wherein M image frames are extracted with the center of the speckle as the axis when extracting the pixel point in step S9.

4. The timing-based speckle biopsy method of claim 1, wherein step S9 comprises:

step S91: according to the scattered spot distribution in the interested area, recognizing the straight line direction connecting the most scattered spots as a first direction;

step S92: extracting scattered spots of the region of interest in M image frames in the first direction to obtain brightness change vectors of the scattered spots;

step S93: and extracting all scattered spots of the region of interest to obtain N groups of brightness change vectors, and executing step S11.

5. The timing-based speckle biopsy method of claim 1, wherein the neural network of the biopsy classifier when trained comprises: the device comprises a first reverse residual error module, a first pooling layer, a second reverse residual error module, a second pooling layer, a third reverse residual error module, a first superposition layer, a third pooling layer, a fourth reverse residual error module, a second superposition layer, a fourth pooling layer, a fifth reverse residual error module, a third convolution layer, a fifth pooling layer, a random discarding layer, a classifier, a first convolution layer and a second convolution layer; the first reverse residual error module is output to the first pooling layer and the first convolution layer, the first convolution layer and the third reverse residual error module input the first superposition layer, the second reverse residual error module is output to the second convolution layer and the second pooling layer, and the second convolution layer and the fourth reverse residual error module input the second superposition layer; the classifier outputs a living probability and a prosthesis probability.

6. The method according to claim 1, wherein in step S11, the positional relationship of the plurality of pixels is also used as an input of a living body detection classifier together with the luminance change vector.

7. The method according to claim 1, wherein in step S11, different weight values are respectively assigned to the luminance change vectors according to the positions of the pixels.

8. A timing-based speckle biopsy system for implementing the timing-based speckle biopsy method of any one of claims 1 to 7, comprising:

the video module is used for acquiring a video to be detected;

the detection frame module is used for taking a first frame image in the video to be detected as a frame to be detected;

the object detection module is used for carrying out target detection on the frame to be detected and judging whether the frame to be detected contains a target object or not; if yes, positioning the target area and executing an area module, otherwise, executing a first mobile module;

the region module is used for taking the frame to be detected as an initial frame, extracting feature points to be selected in the initial frame by utilizing a feature point detection algorithm, taking a target region positioned in the initial frame as a tracking region, taking the next frame of the initial frame in the video to be detected as the frame to be tracked, and executing a screening module;

The first moving module is used for taking the next frame image in the video to be detected as a frame to be detected and executing an object detection module;

the screening module is used for obtaining candidate feature points of the frame to be tracked by utilizing a tracking algorithm, judging whether the number of the feature points to be selected in the frame to be tracked meets a first preset condition, wherein the feature points to be selected are feature points meeting a second preset condition in the candidate feature points, the second preset condition is that the distance between the second preset condition and the feature points to be selected corresponding to the previous frame of the frame to be tracked is smaller than or equal to S, if the number of the feature points to be selected in the frame to be tracked meets the first preset condition, determining that a target object is tracked in the frame to be tracked, determining a target area on the frame to be tracked according to the position of the feature points to be selected in the frame to be tracked, executing the intercepting module, otherwise, determining that the target object is not tracked in the frame to be tracked, taking the next frame of the frame to be tracked as a frame to be detected, and executing the object detecting module;

the intercepting module is used for acquiring an interested region in a target region on the frame to be tracked and executing the confirmation module;

The confirming module is used for judging whether the number of the image frames tracked to the target is equal to M, if so, executing the pixel module, and if not, executing the second moving module;

the pixel module is used for extracting a group of pixel points from each scattered spot of the interested area on M image frames in the same direction, constructing a vector of brightness change along with time for each pixel point, obtaining brightness change vectors of N pixel points, and executing the judging module;

the second moving module is used for acquiring the next frame of the frame to be tracked, taking the next frame of the frame to be tracked as a new frame to be tracked, and executing the screening module;

and the judging module is used for taking the brightness change vector as the input of a living body detection classifier and judging whether the target object in the M image frames characterized by the feature vector is a living body or not, wherein the living body detection classifier is obtained by training the feature vector extracted from the real target object and/or the attack target object, which is acquired in advance.

9. A timing-based speckle biopsy device, comprising:

a processor;

a memory having stored therein executable instructions of the processor;

Wherein the processor is configured to perform the steps of the timing-based speckle biopsy method of any one of claims 1 to 7 via execution of the executable instructions.

10. A computer-readable storage medium storing a program, wherein the program when executed implements the steps of the time-series-based speckle live detection method of any one of claims 1 to 7.