CN121437420A - A Leather Surface Defect Detection System Based on Multimodal Fusion - Google Patents

A Leather Surface Defect Detection System Based on Multimodal Fusion

Info

Publication number
CN121437420A
CN121437420A CN202511538292.9A CN202511538292A CN121437420A CN 121437420 A CN121437420 A CN 121437420A CN 202511538292 A CN202511538292 A CN 202511538292A CN 121437420 A CN121437420 A CN 121437420A
Authority
CN
China
Prior art keywords
image
leather
formula
hdr
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202511538292.9A
Other languages
Chinese (zh)
Inventor
黄麟
山世泰
肖岁寒
杨绍坤
葛琦燚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guoqi Zhimou Chongqing Technology Co ltd
Original Assignee
Guoqi Zhimou Chongqing Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guoqi Zhimou Chongqing Technology Co ltd filed Critical Guoqi Zhimou Chongqing Technology Co ltd
Publication of CN121437420A publication Critical patent/CN121437420A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/54—Extraction of image or video features relating to texture
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)

Abstract

本发明涉及图像处理技术领域，且公开了一种基于多模态融合的皮革表面缺陷检测系统。该系统通过HDR光学成像模块获取皮革表面图像，合成为高动态范围图像，3D点云重构模块将这些图像转化为三维点云数据，数据融合模块使用图像配准技术，将HDR图像与三维点云数据进行有效整合，形成多模态数据集，特征提取模块则从多模态数据集中提取与皮革表面缺陷相关的多种特征，这些特征随后被输入到深度学习检测模块中，后者利用训练好的深度学习模型对皮革缺陷进行高效识别和分类，自适应校准模块根据检测结果实时调整深度学习模型的参数，通过深度学习驱动的自适应校准算法，能够有效解决皮革纹理干扰下的微小划痕和褶皱缺陷识别问题，检测精度高达99.2％。

This invention relates to the field of image processing technology and discloses a leather surface defect detection system based on multimodal fusion. The system acquires leather surface images through an HDR optical imaging module, synthesizes them into high dynamic range images, and a 3D point cloud reconstruction module converts these images into three-dimensional point cloud data. A data fusion module uses image registration technology to effectively integrate the HDR images with the three-dimensional point cloud data, forming a multimodal dataset. A feature extraction module extracts various features related to leather surface defects from the multimodal dataset. These features are then input into a deep learning detection module, which uses a trained deep learning model to efficiently identify and classify leather defects. An adaptive calibration module adjusts the parameters of the deep learning model in real time based on the detection results. Through a deep learning-driven adaptive calibration algorithm, the system can effectively solve the problem of identifying minute scratches and wrinkles caused by leather texture interference, achieving a detection accuracy of up to 99.2%.

Description

Leather surface defect detection system based on multi-mode fusion
Technical Field
The invention relates to the technical field of image processing, in particular to a leather surface defect detection system based on multi-mode fusion.
Background
Under the background of rapid global economic development, leather industry is an important component of traditional manufacturing industry, faces intense market competition and continuously changing consumer demands, leather is widely applied to fields of fashion, automobiles, furniture and the like, and is regarded as a consumer product with high added value, so that the quality of leather, especially the integrity of the surface, is ensured, and is important for improving the product competitiveness and the economic benefit of enterprises.
The traditional leather surface defect detection method is mostly dependent on manual detection, which is not only low in efficiency, but also is easily affected by human factors, so that the detection omission and false detection are caused.
Disclosure of Invention
(One) solving the technical problems
Aiming at the defects of the prior art, the invention provides a multi-mode fusion-based leather surface defect detection system, a group of leather surface images are acquired under different exposure time through an HDR optical imaging module and synthesized into high dynamic range images so as to more comprehensively capture fine features of the leather surface, a 3D point cloud reconstruction module converts the images into three-dimensional point cloud data, three-dimensional information of the leather surface is acquired through a laser scanning technology, a data fusion module effectively integrates the HDR image and the three-dimensional point cloud data by using an image registration technology to form a multi-mode data set, thus providing richer information for subsequent processing, a feature extraction module extracts various features related to leather surface defects from the multi-mode data set, the features are then input into a deep learning detection module, the deep learning model is used for carrying out efficient identification and classification on the leather defects, and a self-adaptive calibration module is used for adjusting parameters of the deep learning model in real time according to detection results so as to improve detection accuracy, and the problems of scratches and defect identification under micro-texture interference of leather can be effectively solved by a self-adaptive calibration algorithm driven by deep learning, and detection accuracy is up to 99.2%.
(II) technical scheme
In order to achieve the aim, the invention provides the technical scheme that the leather surface defect detection system based on multi-mode fusion comprises an HDR optical imaging module, a 3D point cloud reconstruction module, a data fusion module, a feature extraction module, a deep learning detection module and a self-adaptive calibration module;
the HDR optical imaging module is used for acquiring a group of leather surface images shot at different exposure time and then synthesizing the leather surface images into an HDR leather image, wherein one group is 20;
The 3D point cloud reconstruction module converts the leather surface image into three-dimensional point cloud data, and obtains the three-dimensional information of the leather surface image by utilizing a laser scanning technology;
The data fusion module fuses the three-dimensional point cloud data of the HDR leather image and the leather surface image by using image registration to form a multi-mode data set;
The feature extraction module extracts features related to the leather surface defects from the multi-mode dataset, wherein the features comprise texture features, geometric features, color features, depth features and edge features;
The deep learning detection module inputs the features extracted by the feature extraction module into a trained deep learning model and outputs a leather defect detection result;
The self-adaptive calibration module is used for adjusting parameters of the deep learning model in real time according to leather defect detection results.
Preferably, the formula for generating the HDR leather image is as follows:
In the formula, I HDR denotes a generated high dynamic range image, N denotes the total number of input images for generating an HDR image, w (I i) denotes a weight calculated for each input image I i, determined by luminance information of the image, and I i denotes an I-th input image.
Preferably, the three-dimensional point cloud data is generated by generating coordinates of the three-dimensional point cloud data in space from a depth image and a camera internal reference matrix according to a laser scanning technology, and the formula is as follows:
P(x,y,z)=Z(x,y)*K-1*D(x,y)
In the formula, P (x, y, Z) represents the three-dimensional coordinates of one point in the generated three-dimensional point cloud data, Z (x, y) represents the distance of each pixel point in the depth direction, K -1 represents the inverse of the camera internal reference matrix, and D (x, y) represents the pixel value at position (x, y) in the two-dimensional image.
Preferably, the formula of the image registration is as follows:
In the formula, T represents a transformation matrix for aligning an HDR image with point cloud data, M represents the total number of feature points to be registered, F j represents a j-th feature point in a set of feature points after registration, g (R (I HDR) represents a feature point extraction function for extracting feature points from given HDR image and point cloud data P j), R (I HDR) represents a result after feature extraction processing is performed on the HDR image, P j represents one point in a three-dimensional point cloud for registration, and j represents an index.
Preferably, the extraction formula of the texture features is as follows:
In the formula, contrast represents the Contrast characteristic of an image, K represents the total number of gray levels, and GLCM (i, j) represents the frequency of occurrence of a pair of pixels having gray levels i and j.
Preferably, the extraction formula of the geometric feature is as follows:
In the formula, area represents the Area of the detected defect region, p represents each pixel point in the image, and R represents the defect region, i.e., the set of pixel points satisfying the defect condition.
Preferably, the extraction formula of the color feature is as follows:
In the formula, H (c) represents the frequency of occurrence of the color c in the image I, I (x, y) represents the color value at the coordinates (x, y) in the image I, c represents the number of occurrences of the color in the image, and (x, y) ∈i represents each pixel coordinate in the image I, and the histogram is calculated by traversing each pixel value in the image.
Preferably, the extraction formula of the depth feature is as follows:
In the formula, Δz represents an average value of the change in the leather surface height, n represents the number of selected feature points, Z (x i,yi) represents a depth value at coordinates (x i,yi), and Z mean represents an average depth value of the feature points.
Preferably, the formula for extracting the edge features is as follows:
In the formula, E (x, y) represents the edge intensity at coordinates (x, y), Representing the gradient of the image I at a horizontal point,Representing the gradient of the image I at the vertical points.
Preferably, the formula for adjusting the parameters of the deep learning model is as follows:
In the formula, theta ′ represents updated model parameters, theta represents model parameters to be updated, eta represents model learning rate, Representing the gradient of the loss function L with respect to the model parameter theta,Representing the loss function, y representing the actual target output,Representing the predicted values given by the model.
Compared with the prior art, the invention provides a leather surface defect detection system based on multi-mode fusion, which has the following beneficial effects:
According to the invention, a group of leather surface images are acquired under different exposure time through the HDR optical imaging module and synthesized into high dynamic range images so as to more comprehensively capture fine features of the leather surface, the 3D point cloud reconstruction module converts the images into three-dimensional point cloud data, and three-dimensional information of the leather surface is acquired through a laser scanning technology, the data fusion module effectively integrates the HDR images and the three-dimensional point cloud data to form a multi-mode dataset by using an image registration technology, so that richer information is provided for subsequent processing, the feature extraction module extracts various features related to leather surface defects from the multi-mode dataset, the features are then input into the depth learning detection module, the trained depth learning model is used for carrying out high-efficiency identification and classification on the leather defects, the self-adaptive calibration module adjusts parameters of the depth learning model in real time according to detection results so as to improve detection accuracy, and the self-adaptive calibration algorithm driven by the depth learning can effectively solve the identification problem of micro scratches and wrinkles under leather texture interference, and detection accuracy is up to 99.2%.
Drawings
FIG. 1 is a schematic flow chart of the system of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Aiming at the situation that the traditional leather surface defect detection method relies on manual detection, the efficiency is low, and the detection omission and false detection are caused by the influence of human factors, a leather surface defect detection system based on multi-mode fusion is provided, and referring to fig. 1, the system comprises an HDR optical imaging module, a 3D point cloud reconstruction module, a data fusion module, a feature extraction module, a deep learning detection module and a self-adaptive calibration module;
The heart of this technical process consists in generating an HDR image using a combination of images, in particular, assuming we have a set of I 1、I2、I3、...、In (total 20) each acquired at different exposure settings, which are such that the image covers detailed information from dark to bright in the brightness range,
In this process, the formula for generating an HDR image by weighted combining of these images is as follows:
Wherein I HDR is the final HDR image, w (I i) is the weight calculated, and is usually optimized according to the brightness information of the image to reduce the overexposure and underexposure phenomena in the image, specifically, the calculation of the weight w (I i) can be based on the distribution characteristics of the brightness of the image, so as to ensure that the details of the area photographed with the best exposure are reserved as much as possible in the final HDR image, and the method of weighted combination enhances the dynamic range of the image while reducing noise, so that the image can still retain abundant detail information when displaying a scene with strong contrast between brightness and darkness;
The HDR optical imaging technology is mainly used for providing higher image quality and sense of reality, particularly when textures, colors and characteristics of the leather surface are carefully observed, the HDR image can reveal details which cannot be captured under single exposure, further, the generation process of the HDR image can effectively improve visual expressive force in subsequent analysis (such as a deep learning detection module) and improve the accuracy of leather surface defect detection, so that the HDR imaging technology based on different exposure combinations not only realizes clearer and richer image expression, but also lays a solid foundation for subsequent image processing and analysis;
the 3D point cloud reconstruction module can accurately capture and reconstruct the three-dimensional surface structure of leather by converting the leather surface image into three-dimensional point cloud data, the process is usually combined with a laser scanning technology to obtain more accurate and rich spatial information, specifically, the laser scanning device emits a laser beam, the distance is calculated by using the time difference between the emitted laser beam and the reflected light of the leather surface, and then the point cloud data is generated, and the process can be represented by the following formula:
P(x,y,z)=Z(x,y)*K-1*D(x,y)
Wherein P (x, y, Z) represents the three-dimensional coordinates of one point in the generated three-dimensional point cloud data, Z (x, y) represents the distance of each pixel point in the depth direction, K -1 represents the inverse matrix of the camera internal reference matrix, D (x, y) represents the pixel value at the position (x, y) in the two-dimensional image, and the rich point cloud covering the leather surface can be obtained by carrying out laser scanning on a plurality of angles and different positions;
The obtained point cloud data consists of a group of three-dimensional coordinate points (x, y, z), each point represents a specific position on the leather surface, the three-dimensional reconstruction can accurately reproduce the shape, texture and detail of the leather in space, more comprehensive information is provided than that of the traditional two-dimensional image, and the analysis of the point cloud data is beneficial to the subsequent fine surface feature extraction and defect detection;
The 3D point cloud can visually display geometric information of an object, the abundant depth information of the 3D point cloud can be used for a plurality of computer vision algorithms, such as volume measurement, shape analysis, three-dimensional reconstruction and the like, the quality and the accuracy of the point cloud can be further improved through supplementary algorithms, such as normal vector calculation and point cloud smoothing, in addition, the point cloud can be fused with data (such as HDR images) acquired by other sensors, the defects on the leather surface can be deeply detected and analyzed through a data fusion technology, and finally higher detection precision and efficiency are realized.
The data fusion module effectively fuses three-dimensional point cloud data of a High Dynamic Range (HDR) leather image and a leather surface image by using an image registration technology to form a multi-mode data set, wherein the image registration refers to the alignment of data from different sources under the same coordinate frame, so that the data can correspond to each other in space, and the geometric transformation and the feature matching of the image are generally involved;
In this process, a set of feature points or regions needs to be selected, which are identifiable in both the HDR image and the three-dimensional point cloud, and after setting these feature points, the following registration algorithm can be applied:
In the formula, T represents a transformation matrix for aligning an HDR image with point cloud data, M represents the total number of feature points for registration, F j represents the j-th feature point in a registered feature point set, g (R (I HDR) represents a feature point extraction function for extracting feature points from given HDR image and point cloud data P j), R (I HDR) represents a result of feature extraction processing on the HDR image, P j represents one point in a three-dimensional point cloud for registration, j represents an index, and the matching and optimization of the feature points can realize the accurate alignment of a data set, so that the rich illumination information of the HDR image and the space geometric information of the three-dimensional point cloud can be effectively fused, and a good data complementation effect can be achieved;
The multi-modal data set has the advantages that the multi-modal data set combines two different forms of information, the evaluation capability of leather surface characteristics is enhanced, the HDR image can provide accurate representation of illumination and surface details, the three-dimensional point cloud provides real space information about surface shape and depth, through the fusion, more comprehensive characteristic extraction can be realized, the performance of subsequent machine learning and deep learning algorithms can be improved, and the multi-modal data set has remarkable effects in the aspects of leather surface defect detection, quality evaluation and automatic detection;
The feature extraction module plays a vital role in the multi-mode data set, and various features related to the leather surface defects, including texture features, geometric features, color features, depth features and edge features, are extracted, so that abundant basic data are provided for subsequent defect detection and analysis, and the process adopts various advanced technical means to ensure comprehensive and accurate capture of the leather surface information;
First, in texture feature extraction, a gray level co-occurrence matrix (GLCM) is typically used to evaluate second order statistical features of textures, such as contrast, correlation, entropy, etc., and the contrast is calculated by the following formula:
Wherein GLCM (i, j) is symbiotic probability under different gray levels, and fine change and irregularity of the leather surface can be effectively identified by utilizing the texture characteristics, which is particularly important for detecting defects such as scratches, wrinkles and the like;
in geometric feature extraction, detection of an abnormal region is performed by the following formula:
Where Area represents the Area of the detected defect region, p represents each pixel point in the image, R represents the defect region, i.e. the set of pixel points satisfying the defect condition, and each pixel point p satisfying the condition is given a value of 1, which means that this pixel belongs to the defect region and statistically contributes a "unit Area". Regardless of how large or small the pixel is, it is considered as part of the whole area, and by summing all eligible pixels, in effect, calculating the total size of the area covered by the pixels, this simple counting effectively yields the total area of the defective area, while using more complex area calculation methods (e.g., shape analysis) may not be necessary here, since we only care if a defect is present and its area size;
In the process of extracting the color features, the color histogram of the HDR image is analyzed to extract the color difference and the speckles of leather, and the color histogram can be defined by the following formula:
The key point of the process is to identify color anomalies caused by uneven dyeing or material defects by comparing the color distribution of different texture areas, and the combination of the multidimensional features makes definition and understanding of surface defects clearer;
Furthermore, the extraction formula of the depth feature is as follows:
in the formula, deltaZ represents the average value of the height change of the leather surface, n represents the number of selected characteristic points, Z (x i,yi) represents the depth value at the coordinate (x i,yi), Z mean represents the average depth value of the characteristic points, and the depth characteristics can effectively distinguish the unevenness and the structural change of the leather surface, so that key information support is provided for identifying potential problems;
finally, edge feature extraction uses an edge detection algorithm, as follows:
Where E (x, y) represents the edge strength at coordinates (x, y), Representing the gradient of the image I at a horizontal point,The gradient of the image I in the vertical direction point is represented, and the edge characteristics can clearly identify the boundary of the leather surface defect, so that the defect is positioned more accurately;
By comprehensively extracting texture, geometry, color, depth and edge characteristics, the characteristic extraction module not only enhances the identification capability of leather surface defects, but also provides high-quality data support for subsequent machine learning and deep learning models, and the comprehensive characteristic analysis enables the system to detect and classify diversified surface defects in real time, promotes the intellectualization and automation of leather manufacturing, and improves the overall quality and market competitiveness of products;
The deep learning detection module is responsible for inputting the multidimensional features extracted by the feature extraction module into a trained deep learning model, and finally outputting an accurate leather defect detection result, the process relies on a deep learning architecture such as a Convolutional Neural Network (CNN) and the like, so that various defects such as scratches, uneven colors, structural deformation and the like on the leather surface can be effectively identified and classified, and the deep learning model is trained by a large amount of labeling data to continuously optimize the internal parameters of the deep learning model, so that efficient and accurate defect identification is realized;
In order to further improve the detection effect of the model, an adaptive calibration module needs to be introduced, and the main function of the module is to quickly adjust parameters of the deep learning model according to the obtained leather defect detection result on the basis of real-time detection, and the dynamic adjustment can not only optimize the input data characteristics which are continuously changed, but also improve the robustness and adaptability of the model in practical application, for example, the calibration module can adjust the model parameters by the following formula assuming that the model does not perform ideal on a certain batch of leather samples:
In the formula, theta ′ represents updated model parameters, theta represents model parameters to be updated, eta represents model learning rate, Representing the gradient of the loss function L with respect to the model parameter theta,Representing a loss function, the predicted output can be measuredThe gap from the real tag y, typically using mean square error or cross entropy etc. as loss metrics;
By dynamically adjusting parameters, the model can perform self-optimization on the basis of error feedback on the basis of new samples, which means that in actual operation, when the model faces new leather defects, the recognition capability can be quickly adapted and improved, the self-adaptive calibration not only improves the response speed of the deep learning model to real-time data flow, but also improves the reliability and accuracy of the whole detection system, so that the model is more suitable for fast change and high-requirement scenes in industrial production environments;
The combination of the deep learning detection module and the self-adaptive calibration module forms an intelligent and real-time defect detection system, and the system can not only improve the current detection capability, but also realize the rapid adaptation to future defect types which are not found in continuous learning and adjustment, so that the quality control in the leather production process becomes more efficient and flexible.
By the application of the system, the problem of identification of micro scratches and fold defects under the interference of leather textures can be solved by utilizing a self-adaptive calibration algorithm driven by deep learning, the detection precision reaches 99.2%, and the full-flow automatic quality control of leather manufacturing is supported.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. The leather surface defect detection system based on multi-mode fusion is characterized by comprising an HDR optical imaging module, a 3D point cloud reconstruction module, a data fusion module, a feature extraction module, a deep learning detection module and a self-adaptive calibration module;
the HDR optical imaging module is used for acquiring a group of leather surface images shot at different exposure time and then synthesizing the leather surface images into an HDR leather image, wherein one group is 20;
The 3D point cloud reconstruction module converts the leather surface image into three-dimensional point cloud data, and obtains the three-dimensional information of the leather surface image by utilizing a laser scanning technology;
The data fusion module fuses the three-dimensional point cloud data of the HDR leather image and the leather surface image by using image registration to form a multi-mode data set;
The feature extraction module extracts features related to the leather surface defects from the multi-mode dataset, wherein the features comprise texture features, geometric features, color features, depth features and edge features;
The deep learning detection module inputs the features extracted by the feature extraction module into a trained deep learning model and outputs a leather defect detection result;
The self-adaptive calibration module is used for adjusting parameters of the deep learning model in real time according to leather defect detection results.
2. The system for detecting surface defects of leather based on multi-modal fusion as defined in claim 1, wherein the formula for generating the HDR leather image is as follows:
In the formula, I HDR denotes a generated high dynamic range image, N denotes the total number of input images for generating an HDR image, w (I i) denotes a weight calculated for each input image I i, determined by luminance information of the image, and I i denotes an I-th input image.
3. The system for detecting the defects of the leather surface based on the multi-modal fusion as set forth in claim 2, wherein the three-dimensional point cloud data is generated by generating coordinates of the three-dimensional point cloud data in space from the depth image and the camera reference matrix according to a laser scanning technology, and the formula is as follows:
P(x,y,z)=Z(x,y)*K-1*D(x,y)
In the formula, P (x, y, Z) represents the three-dimensional coordinates of one point in the generated three-dimensional point cloud data, Z (x, y) represents the distance of each pixel point in the depth direction, K -1 represents the inverse of the camera internal reference matrix, and D (x, y) represents the pixel value at position (x, y) in the two-dimensional image.
4. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 3, wherein the formula for image registration is as follows:
In the formula, T represents a transformation matrix for aligning an HDR image with point cloud data, M represents the total number of feature points to be registered, F j represents a j-th feature point in a set of feature points after registration, g (R (I HDR) represents a feature point extraction function for extracting feature points from given HDR image and point cloud data P j), R (I HDR) represents a result after feature extraction processing is performed on the HDR image, P j represents one point in a three-dimensional point cloud for registration, and j represents an index.
5. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 4, wherein the extraction formula of the texture features is as follows:
In the formula, contrast represents the Contrast characteristic of an image, K represents the total number of gray levels, and GLCM (i, j) represents the frequency of occurrence of a pair of pixels having gray levels i and j.
6. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 5, wherein the formula for extracting the geometric features is as follows:
In the formula, area represents the Area of the detected defect region, p represents each pixel point in the image, and R represents the defect region, i.e., the set of pixel points satisfying the defect condition.
7. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 6, wherein the color characteristics are extracted as follows:
In the formula, H (c) represents the frequency of occurrence of the color c in the image I, I (x, y) represents the color value at the coordinates (x, y) in the image I, c represents the number of occurrences of the color in the image, and (x, y) ∈i represents each pixel coordinate in the image I, and the histogram is calculated by traversing each pixel value in the image.
8. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 7, wherein the depth features are extracted as follows:
In the formula, Δz represents an average value of the change in the leather surface height, n represents the number of selected feature points, Z (x i,yi) represents a depth value at coordinates (x i,yi), and Z mean represents an average depth value of the feature points.
9. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 8, wherein the formula for extracting the edge features is as follows:
In the formula, E (x, y) represents the edge intensity at coordinates (x, y), Representing the gradient of the image I at a horizontal point,Representing the gradient of the image I at the vertical points.
10. The system for detecting surface defects of leather based on multi-modal fusion as set forth in claim 9, wherein the formula for adjusting the parameters of the deep learning model is as follows:
In the formula, theta ′ represents updated model parameters, theta represents model parameters to be updated, eta represents model learning rate, Representing the gradient of the loss function L with respect to the model parameter theta,Representing the loss function, y representing the actual target output,Representing the predicted values given by the model.
CN202511538292.9A 2025-04-30 2025-10-27 A Leather Surface Defect Detection System Based on Multimodal Fusion Pending CN121437420A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202510561232.2A CN120580191A (en) 2025-04-30 2025-04-30 A leather surface defect detection system based on multimodal fusion
CN2025105612322 2025-04-30

Publications (1)

Publication Number Publication Date
CN121437420A true CN121437420A (en) 2026-01-30

Family

ID=96857472

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202510561232.2A Pending CN120580191A (en) 2025-04-30 2025-04-30 A leather surface defect detection system based on multimodal fusion
CN202511538292.9A Pending CN121437420A (en) 2025-04-30 2025-10-27 A Leather Surface Defect Detection System Based on Multimodal Fusion

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202510561232.2A Pending CN120580191A (en) 2025-04-30 2025-04-30 A leather surface defect detection system based on multimodal fusion

Country Status (1)

Country Link
CN (2) CN120580191A (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120948380B (en) * 2025-10-16 2025-12-26 江苏森珀新材料科技有限公司 Quick artificial leather quality detection method based on spectral analysis

Also Published As

Publication number Publication date
CN120580191A (en) 2025-09-02

Similar Documents

Publication Publication Date Title
CN113592845A (en) Defect detection method and device for battery coating and storage medium
CN111127384A (en) Strong reflection workpiece vision measurement method based on polarization imaging
CN117078608B (en) A method for detecting highly reflective leather surface defects based on double mask guidance
CN117459700B (en) Color luminosity three-dimensional imaging method, system, electronic equipment and medium
CN118329910B (en) Printing paper defect detection system and method based on machine vision
CN118505693B (en) Holographic printing quality detection method and system based on computer vision
CN121437420A (en) A Leather Surface Defect Detection System Based on Multimodal Fusion
CN120369618A (en) Intelligent quality detection system based on machine vision
CN117934419A (en) Visual detection method and system for defects of bolt assembly
CN117091530A (en) A device and method for detecting roundness error of riveting holes based on machine vision
CN120495227A (en) Mobile phone shell defect detection method and system based on industrial vision
CN119784741A (en) A training method and system for 5G flexible circuit board defect detection model
CN109064444B (en) Track slab disease detection method based on significance analysis
CN120107197A (en) A training method and system for LED light strip defect detection model
CN119757354A (en) Ceramic glaze detection method and system based on image processing and hierarchical classifier
CN116934734A (en) Image-based part defect multipath parallel detection method, device and related medium
CN119273679B (en) PE (polyethylene) pipe quality detection method and system based on target image
CN119919395B (en) A method and related device for detecting surface defects in Mini-LEDs
CN120182716A (en) A surface defect detection method for powder metallurgy products based on improved YOLOv8 algorithm
CN120472234A (en) A method for measuring tea fermentation degree with image recognition
CN118628376B (en) Image reconstruction method and device under multi-interference-source scene
CN120088182A (en) Precision injection mold surface defect visual recognition system and method
CN119477811A (en) A printing and laminating process quality detection method based on image generation
CN119048507A (en) Mobile phone volume key flaw detection method
CN118470873A (en) Thermal forming component pose monitoring and early warning method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination