WO2019052561A1 - 检查方法和检查设备以及计算机可读介质 - Google Patents

检查方法和检查设备以及计算机可读介质 Download PDF

Info

Publication number
WO2019052561A1
WO2019052561A1 PCT/CN2018/106021 CN2018106021W WO2019052561A1 WO 2019052561 A1 WO2019052561 A1 WO 2019052561A1 CN 2018106021 W CN2018106021 W CN 2018106021W WO 2019052561 A1 WO2019052561 A1 WO 2019052561A1
Authority
WO
WIPO (PCT)
Prior art keywords
inspected
neural network
image
semantic
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/106021
Other languages
English (en)
French (fr)
Inventor
顾建平
刘涛
赵自然
刘耀红
李强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuctech Co Ltd
Original Assignee
Nuctech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nuctech Co Ltd filed Critical Nuctech Co Ltd
Priority to KR1020197034843A priority Critical patent/KR102240058B1/ko
Priority to EP18856136.9A priority patent/EP3699579B1/en
Priority to JP2019565493A priority patent/JP7678661B2/ja
Publication of WO2019052561A1 publication Critical patent/WO2019052561A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N23/00—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00
    • G01N23/02—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material
    • G01N23/04—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
    • G01V5/00—Prospecting or detecting by the use of ionising radiation, e.g. of natural or induced radioactivity
    • G01V5/20—Detecting prohibited goods, e.g. weapons, explosives, hazardous substances, contraband or smuggled objects
    • G01V5/22—Active interrogation, i.e. by irradiating objects or goods using external radiation sources, e.g. using gamma rays or cosmic rays
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/768—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using context analysis, e.g. recognition aided by known co-occurring patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/695—Preprocessing, e.g. image segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/19173—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/26—Techniques for post-processing, e.g. correcting the recognition result
    • G06V30/262—Techniques for post-processing, e.g. correcting the recognition result using context analysis, e.g. lexical, syntactic or semantic context
    • G06V30/274—Syntactic or semantic context, e.g. balancing
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2223/00—Investigating materials by wave or particle radiation
    • G01N2223/03—Investigating materials by wave or particle radiation by transmission
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2223/00—Investigating materials by wave or particle radiation
    • G01N2223/10—Different kinds of radiation or particles
    • G01N2223/101—Different kinds of radiation or particles electromagnetic radiation
    • G01N2223/1016—X-ray
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2223/00—Investigating materials by wave or particle radiation
    • G01N2223/40—Imaging
    • G01N2223/401—Imaging image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10—Recognition assisted with metadata

Definitions

  • the present application relates to security inspections, and in particular to a radiographic imaging based inspection method and inspection apparatus and computer readable medium.
  • Article 85 of China's Anti-Terrorism Law stipulates that “the railway, highway, water, air cargo and postal, express delivery and other logistics operating units have not implemented a safety inspection system, and have checked the identity of customers, or have not transported or delivered according to regulations. The article shall be punished for security inspection or unpacking inspection, non-transportation, delivery of customer identity and item information registration system.” This requires express delivery companies to implement 100% prior inspection and post-sealing, 100% delivery real-name system, 100% X-ray machine security and other system guarantees.
  • embodiments of the present disclosure provide an inspection method, an inspection apparatus, and a computer readable medium, which are capable of automatically checking an object to be inspected, while ensuring the accuracy of inspection, greatly Improve the speed of inspection.
  • an inspection method comprising the steps of: scanning an object to be inspected with X-rays to obtain an image of the object to be inspected; processing the image with the first neural network to obtain a semantic description of the object to be inspected; Reading the text information of the manifest of the object to be inspected; processing the text information of the manifest of the object to be inspected by using the second neural network to obtain a semantic feature of the object to be inspected; based on the semantic description and the semantic feature To determine whether the object to be inspected is allowed to pass.
  • the first neural network is a convolutional neural network or a candidate region-based convolutional neural network or a fast candidate region-based convolutional neural network
  • the second neural network being a cyclic neural network or a bidirectional Circulating neural network.
  • the first neural network is trained using a pre-established image-semantic pair set.
  • the method before the processing the image by using the first neural network, the method further includes the steps of: binarizing the image of the object to be inspected; calculating an average value of the binarized image; using the binary value Each pixel value of the image is subtracted from the average.
  • the step of determining whether the object to be inspected is allowed to pass based on the semantic description and the semantic feature comprises: calculating a first vector representing the semantic description and a second representation representing the semantic feature The distance between the vectors; allowing the object to be inspected to pass if the calculated distance is less than the threshold.
  • a plurality of regional features included in the sample image are associated with a plurality of words included in manifest information of the sample picture during training of the first neural network.
  • a dot product between a feature vector representing the region feature and a semantic vector representing the word is used as a similarity between the region feature and the word, and a plurality of regions of the sample image are utilized
  • the weighted sum of the similarity between the feature and the plurality of words included in the manifest information is used as the similarity between the sample image and its manifest information.
  • an inspection apparatus comprising: a scanning device that scans an object to be inspected with X-rays to obtain a scanned image; an input device that inputs manifest information of the object to be inspected; And configured to: process the image by using the first neural network to obtain a semantic description of the object to be inspected; and process the text information of the manifest of the object to be inspected by using the second neural network to obtain a semantic feature of the object to be inspected; The semantic description and the semantic feature are used to determine whether the object to be inspected is allowed to pass.
  • a computer readable medium stored with a computer program that, when executed by a processor, implements the following steps:
  • the inspection speed can be ensured, and the inspection speed is greatly improved, so that the efficiency of the safety inspection is greatly improved.
  • FIG. 1 shows a schematic diagram of an inspection apparatus according to an embodiment of the present disclosure
  • Figure 2 is a diagram showing the internal structure of a computer for image processing in the embodiment shown in Figure 1;
  • FIG. 3 is a schematic diagram depicting an artificial neural network used in an inspection apparatus and an inspection method of an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram showing another artificial neural network used in the inspection apparatus and the inspection method of the embodiment of the present disclosure
  • FIG. 5 is a schematic diagram depicting a process of aligning images and semantics in accordance with an embodiment of the present disclosure
  • FIG. 6 is a flow chart describing establishing an image-semantic model in an inspection apparatus and an inspection method according to an embodiment of the present invention
  • FIG. 7 is a flowchart describing a process of performing a security check on an object to be inspected according to an inspection method according to an embodiment of the present disclosure.
  • the embodiment of the present disclosure proposes an inspection technique based on deep learning, which can intelligently complete the comparison of the article machine drawings.
  • an area in the image of the goods of the goods machine that is inconsistent with the customs declaration data of the item declaration can be found. This area may be a false report or a false report.
  • False newspapers are generally prohibited items in logistics goods or dangerous goods that pose a threat to the safety of logistics and transportation. In order to evade detection, goods are described as safe goods on the customs declaration form.
  • the newspapers are generally small quantities and small items, also called entrainment, which are common means of smuggling and contraband.
  • the problem of one box of multiple items in the logistics goods is a problem that cannot be solved under the conventional image processing conditions. To be precise, it is affected by equipment inconsistency, and the ambiguous complex segmentation problem under the supervision of customs declaration data. For example, the algorithm under different devices will inevitably have different performances.
  • the data form of the customs declaration sheet gives multiple supervised values (such as how many kinds of goods, each type and unit weight, etc.), and each pixel on the image may belong to Multiple goods, etc.
  • the method of the present disclosure solves the problem by using a deep learning-based method.
  • FIG. 1 shows a schematic structural view of an inspection apparatus according to an embodiment of the present disclosure.
  • the inspection apparatus 10 shown in Fig. 1 includes an X-ray source 11, a detector module 15, an acquisition circuit 16, a controller 17, a data processing computer 18, and the like.
  • the source 11 includes one or more X-ray generators that can perform a single-energy transmission scan or a dual-energy transmission scan.
  • an object 14 to be inspected such as a baggage item, is placed on the conveyor belt 13 through a scanning area between the radiation source 11 and the detector module 15.
  • the detector module 15 and the acquisition circuit 16 are, for example, detectors and data collectors having an integral modular structure, such as multiple rows of detectors, for detecting radiation transmitted through the object under test 14, obtaining an analog signal, and The analog signal is converted into a digital signal, thereby outputting a transmission image of the object 14 to be inspected for X-rays.
  • one row of detectors can be provided for high energy rays, another row of detectors for low energy rays, or the same row of detectors for high energy and low energy rays.
  • the controller 17 is used to control the various parts of the entire system to work synchronously.
  • the data processing computer 18 is used to process the data collected by the data acquisition circuit 16, process the image data, and output the results.
  • the data processing computer 18 runs an image processing program, analyzes and learns the scanned image, obtains a semantic description of the image, and then compares the obtained semantic description with the semantic features included in the manifest information of the baggage item, and determines Whether the declaration information is consistent with the object in the baggage item. In the case of uniformity, the baggage item is allowed to pass, otherwise an alarm is issued to remind the security personnel that the baggage item may have a problem.
  • the detector module 15 and the acquisition circuit 16 are used to acquire transmission data of the object 14 to be inspected.
  • the acquisition circuit 16 includes a data amplification shaping circuit that operates in either (current) integration mode or pulse (count) mode.
  • the data output cable of the acquisition circuit 16 is coupled to the controller 17 and the data processing computer 18, and the acquired data is stored in the data processing computer 18 in accordance with a trigger command.
  • the detector module 15 includes a plurality of detection units that receive X-rays that penetrate the object under inspection 14.
  • the acquisition circuit 16 is coupled to the detector module 15 to convert the signals generated by the detector module 16 into probe data.
  • the controller 17 is connected to the radiation source 11 via a control line CTRL1, to the detector module 15 via a control line CTRL2, and to the acquisition circuit 16, for controlling one or more X-ray generators in the radiation source 11 to the object 14 to be inspected.
  • a single-energy scan is performed, or a dual-energy scan is performed on the object 14 to be inspected, so that X-rays are transmitted through the object 14 to be inspected as the object 14 to be inspected moves.
  • controller 17 controls detector module 15 and acquisition circuit 16 to obtain corresponding transmission data, such as single energy transmission data or dual energy transmission data.
  • the data processing computer 18 obtains an image of the object 14 to be inspected based on the transmission data, processes the image, and determines whether the two are consistent based on the manifest information of the object 14 to be inspected.
  • FIG. 2 shows a block diagram of the structure of the data processing computer shown in FIG. 1.
  • the data processing calculation 20 includes a storage device 21, a read only memory (ROM) 22, a random access memory (RAM) 23, an input device 24, a processor 25, a display device 26 and an interface unit 27, and a bus 28. Wait.
  • ROM read only memory
  • RAM random access memory
  • the data collected by the acquisition circuit 16 is stored in the storage device 21 via the interface unit 27 and the bus 28.
  • Configuration information and a program of the computer data processor are stored in the read only memory (ROM) 22.
  • a random access memory (RAM) 23 is used to temporarily store various data during the operation of the processor 25.
  • a computer program for performing data processing is also stored in the storage device 21.
  • the internal bus 28 is connected to the above-described storage device 21, read only memory 22, random access memory 23, input device 24, processor 25, display device 28, and interface unit 27.
  • the instruction code of the computer program instructs the processor 25 to execute the data processing algorithm, and after obtaining the data processing result, displays it on an LCD display or the like.
  • the processing result is output on the display device 27, or directly in the form of a hard copy such as printing.
  • the radiation source 11 may be a radioactive isotope (e.g., cobalt-60), or may be a low energy X-ray machine or a high energy X-ray accelerator or the like.
  • a radioactive isotope e.g., cobalt-60
  • the radiation source 11 may be a radioactive isotope (e.g., cobalt-60), or may be a low energy X-ray machine or a high energy X-ray accelerator or the like.
  • the detector module 15 is divided from materials, which may be gas detectors, scintillator detectors or solid detectors, etc., which are divided into arrays, which may be single row, double row or multiple rows, and single layer detectors. Or double-layer high and low energy detectors.
  • the object 14 to be inspected such as a baggage item, is moved through the inspection area by the conveyor belt 13, but it will be appreciated by those skilled in the art that the object 14 to be inspected may be stationary and the source of radiation and the array of detectors may be moved to complete the scanning process.
  • FIG. 3 is a schematic diagram showing a convolutional neural network 30 in accordance with an embodiment of the present disclosure.
  • the convolutional neural network 30, as shown in Figure 3 can generally comprise a plurality of convolutional layers 32 and 34, which are generally small neurons that are partially overlapping each other (which is also referred to mathematically as A collection of convolution kernels, which are used interchangeably unless otherwise stated.
  • layers of input data or input layers, such as input layer 31 of FIG.
  • a layer referred to as “before” or “below” and another layer closer to the output data (or output layer, such as output layer 37 of Figure 3) is referred to as a "behind” or “on” layer.
  • the direction from the input layer (eg, input layer 31 of FIG. 3) to the output layer (eg, output layer 37 of FIG. 3) during training, verification, and/or use is referred to as forward or forward (forward)
  • the direction from the output layer (eg, output layer 37 of FIG. 3) to the input layer (eg, input layer 31 of FIG. 3) is referred to as backward or backward.
  • these small neurons can process various parts of the input image.
  • the outputs of these small neurons are then combined into an output (referred to as a feature map, such as a square in the first convolutional layer 32) to obtain an output image that better represents certain features in the original image.
  • the partially overlapping arrangement between adjacent neurons also causes the convolutional neural network 30 to have a degree of translational tolerance for features in the original image.
  • the convolutional neural network 30 can correctly identify the feature even if the feature in the original image changes its position in a translational manner within a certain tolerance.
  • the next layer is the optional pooling layer, the first pooling layer 33, which is mainly used to downsample the output data of the previous convolution layer 32 while maintaining the features, reducing the calculation. Quantity and prevent overfitting.
  • the next layer is also a convolutional layer, and the second convolutional layer 34 can perform further feature sampling on the output data generated by the first convolutional layer 32 and downsampled via the pooling layer 33.
  • the features learned are globally larger than those learned by the first convolutional layer.
  • subsequent convolutional layers are global to the characteristics of the previous convolutional layer.
  • the convolutional layer (eg, the first and second convolutional layers 32 and 34) is the core building block of the CNN (eg, convolutional neural network 30).
  • the parameters of this layer consist of a collection of learnable convolution kernels (or simply convolution kernels), each with a small receptive field, but extending over the entire depth of the input data.
  • each convolution kernel is convolved along the width and height of the input data, the dot product between the elements of the convolution kernel and the input data is computed, and a two-dimensional activation map of the convolution kernel is generated.
  • the network is able to learn the convolution kernel that can be activated when a particular type of feature is seen at a spatial location of the input.
  • the activation maps of all convolution kernels are stacked in the depth direction to form the full output data of the convolutional layer.
  • each element in the output data can be interpreted as an output of a convolution kernel that sees small regions in the input and shares parameters with other convolution kernels in the same activation map.
  • the depth of the output data controls the number of convolution kernels in the same area of the layer that are connected to the input data. For example, as shown in FIG. 3, the depth of the first convoluted layer 32 is 4, and the depth of the second convolutional layer 34 is 6. All of these convolution kernels will learn to activate for different features in the input. For example, if the first convolutional layer 32 is input with the original image, then different convolution kernels along the depth dimension (ie, different squares in FIG. 3) may have various directional edges, or grayscales, appearing in the input data. Activated when the block.
  • the training process is a very important part of deep learning.
  • a stochastic gradient descent method can be used.
  • the Nesterov optimization algorithm can be used to solve.
  • the initial learning rate can be set to start at 0.01 and gradually decrease until an optimal value is found.
  • a Gaussian random process with a smaller variance can be used to initialize the weight values of the respective convolution kernels.
  • the image training set may employ a tagged item image that is each labeled with a feature location in the image.
  • a dense picture description is generated on the picture using CNN (Convolution Neural Networks) as shown in FIG. 3 to characterize information of items in the image of the item machine.
  • CNN Convolution Neural Networks
  • a candidate region-based convolutional neural network or a fast candidate region convolutional neural network (Faster-RCNN) extraction method may be employed.
  • CNN Convolution Neural Networks
  • the RCNN Regular Convolutional Neural Network
  • RNN Recurrent Neural Networks
  • the RNN contains input units, the input set is labeled ⁇ x 0 , x 1 ,..., x t , x t+1 ,... ⁇ , and the output unit (Output units)
  • the output set of ) is labeled ⁇ y 0 , y 1 ,..., y t , y t+1 .,.. ⁇ .
  • the RNN also contains Hidden units, marking its output set as ⁇ s 0 , s 1 ,..., s t , s t+1 ,... ⁇ , and these hidden units do the most important work.
  • the circulating neural network is expanded into a full neural network.
  • the expanded network is a five-layer neural network, with each layer representing a word.
  • the word vector refers to the use of a real-number vector v of a specified length to represent a word. You can use the One-hot vector to represent a word, that is, to generate a vector of
  • each input step each layer shares the parameters U, V, W, which reflects that each step in the RNN is doing the same thing, but the input is different, thus greatly reducing the need to learn in the network. parameter.
  • the improvement of the RNN by the Bidirectional RNN is that the current output (the output of the t-th step) is not only related to the previous sequence, but also related to the following sequence. For example, predicting missing words in a statement requires prediction based on context.
  • the Bidirectional RNN is a relatively simple RNN consisting of two RNNs stacked one on top of the other. The output is determined by the state of the hidden layers of the two RNNs.
  • an RNN Recurrent Neural Networks
  • a method of bidirectional RNN is employed in an embodiment of the present disclosure.
  • the BRNN enters a sequence of n words, each encoded with one-hot, converting each word into a fixed h-dimensional vector.
  • the expression of the words is enriched because the context of the change in length around the words is used.
  • Those skilled in the art will appreciate that other artificial neural networks may be employed to identify and learn features in the manifest information. Embodiments of the present disclosure are not limited thereto.
  • the inspection process involves three parts: 1) establishing a scanned image and a database corresponding to the semantic information of the customs declaration; 2) establishing an image model of the luggage item and a corresponding semantic model, and 3) establishing a comparison model of the chart. Smart chart verification.
  • the scanned image and the database corresponding to the semantic information of the customs declaration include three parts: image acquisition, image preprocessing and sample preprocessing.
  • the database for establishing images and corresponding semantic information of customs declaration is mainly divided into three parts: image acquisition, image preprocessing and sample preprocessing.
  • the establishment process is: (1) image acquisition. A considerable number of images of the items scanned by the item machine are collected, so that the image database contains images of various items. Note that the image at this time includes normal items and prohibited items. (2) Image preprocessing. The scanned image is pre-processed with noise information attached to the scanned image. Since the binary image has a uniform physical resolution and can be used in combination with a variety of algorithms, this patent uses a binary image. Note that to ensure the generalization of the model, you can subtract the mean of all the images for each image.
  • an image of an item scanned by a substantial number of item machines is acquired such that the image database contains various item images.
  • the image at this time includes normal items and prohibited items.
  • noise information is attached to the image acquired by the scan, and the scanned image needs to be preprocessed.
  • the binary image has a uniform physical resolution and can be conveniently used in combination with a plurality of algorithms
  • the present application uses a binary image. After obtaining the binary image, the gray average of the image is calculated. In order to ensure the generalization performance of the model, the average of all the images is subtracted from each image, and the obtained result is used as the image input of the model.
  • samples are annotated using customs declaration semantic information.
  • Image and semantic information form a complete pair of information to facilitate network training.
  • the semantic information here also includes the item information described in the express delivery note.
  • the sample is annotated with the semantic information of the customs declaration.
  • Image and semantic information form a complete pair of information to facilitate network training.
  • non-key words in the item description such as deleting non-item keywords such as "one, one type”.
  • the process of establishing a graph comparison model is shown in Fig. 5.
  • the intrinsic pattern correspondence between the language and the visual data is learned on the data set composed of the pictures and their corresponding sentence descriptions.
  • the method of the present disclosure is based on a new model combination approach, while also aligning the two mode inputs through a multi-mode coding model based on a structural purpose.
  • the image features and semantic features are aligned by a multi-mode coding model, and the graph comparison model is finally obtained by random gradient descent training, that is, RCNN 52 and BRNN 53 as shown in FIG.
  • the result of the calculation is that the image corresponds to the semantic information of the item it declares
  • the item automatically passes through the item machine.
  • the result of the calculation is that the image does not correspond to the semantic information of the item it declares
  • the item machine issues a warning, prompting the staff to have an abnormality and performing corresponding processing.
  • models need to be independent of assumptions, such as special hard-coded models, rules, or categories, but only learn from training corpora.
  • the language description of the image is treated as a weak annotation.
  • the consecutively segmented words in these statements correspond to some special but unknown locations in the image.
  • These "alignments” are inferred using neural networks 52 and 53 and applied to the learning description generation model.
  • the present disclosure employs a deep neural network model that can infer potential alignment relationships between fragmentation statements and their corresponding descriptive picture regions. Such models link the two models together through a common, multimodal coding space, and a structured target.
  • a multi-mode recursive neural network architecture is adopted, inputting an image, generating a corresponding text description, generating a picture description and image annotation information for keyword matching, and determining the similarity between the generated statement description and the annotation information.
  • the generated text description statement is significantly better than the retrieval-based method.
  • the model was trained and tested with a new local annotated data set.
  • the image features and semantic features are aligned by a multi-mode coding model, and the graph comparison model is finally obtained by random gradient descent training.
  • each of the small item machine scan pictures and corresponding description statements can be converted into a common h-dimensional vector set by the established RCNN 52 and BRNN 53, respectively.
  • the supervised corpus is granular across the entire picture and the entire statement, but (picture-statement) can be seen as a function of (region-word) scoring.
  • a (picture-statement) combination if one of its words can find sufficient object or attribute support in the picture, then they should get a higher matching score.
  • a plurality of regional features included in a sample image are associated with a plurality of words included in manifest information of the sample picture. For example, a dot product between a feature vector representing a region feature and a semantic vector representing a word is used as a similarity between the region feature and the word, and a plurality of region features of the sample image and a plurality of the manifest information thereof are utilized. The weighted sum of similarities between words is used as the similarity between the sample image and its manifest information.
  • step S61 the tagged image prepared in advance is input to the convolutional neural network 30 for training.
  • a trained image-semantic model is obtained in step S62, that is, a convolutional neural network after preliminary training.
  • the test picture is input to test the network in step S63, and the prediction score and the difference from the label are calculated in step S64, for example, the test picture is input into the neural network, and the semantic description of the prediction and the semantics are obtained. Describe the difference between the label and the label.
  • step S65 it is judged whether the difference is smaller than the threshold, and if it is larger than the network parameter is updated in step S66, the network is adjusted. If the difference is less than the threshold, the network model is established in step S67, that is, the training of the network ends.
  • FIG. 7 is a flowchart describing a process of performing a security check on an object to be inspected according to an inspection method according to an embodiment of the present disclosure.
  • step S71 the inspection object 14 is scanned using the inspection apparatus shown in Fig. 1 to obtain a transmission image of the inspection object.
  • step S74 the manifest information of the object to be inspected is input to the data processing computer 18 by manual entry or a barcode scanner or the like.
  • the transmission image is processed using a first neural network, such as a convolutional neural network or RCNN, etc., in step S72, and a semantic description of the object to be inspected is obtained in step S73.
  • step S75 the text information of the manifest is processed by the bidirectional cyclic neural network to obtain the semantic features of the object to be inspected.
  • step S76 it is judged whether the semantic feature is consistent with the semantic description obtained from the image, and if not, an alarm is issued in step S77. If they are identical, the object to be inspected is allowed to pass in step S78.
  • a distance between a first vector representing a semantic description and a second vector representing a semantic feature may be calculated, and then the checked object is allowed to pass if the calculated distance is less than a threshold.
  • the distance between the two vectors here can be expressed as the sum of the absolute values of the differences between the elements of the two vectors, or by the Euclidean distance of the two vectors. Embodiments of the present disclosure are not limited thereto.
  • the present disclosure realizes whether the intelligent inspection of the declared items is consistent with the actual items, on the one hand, the work efficiency can be greatly improved, and the “pass-through” can be realized, and on the other hand, the side effects of various subjective factors can be reduced. Achieving "management" is an important means of intelligent inspection of security inspections and has huge market potential.
  • aspects of the embodiments disclosed herein may be implemented in an integrated circuit as a whole or in part, as one or more of one or more computers running on one or more computers.
  • a computer program eg, implemented as one or more programs running on one or more computer systems
  • implemented as one or more programs running on one or more processors eg, implemented as one or One or more programs running on a plurality of microprocessors, implemented as firmware, or substantially in any combination of the above, and those skilled in the art, in accordance with the present disclosure, will be provided with design circuitry and/or write software and / or firmware code capabilities.
  • signal bearing media include, but are not limited to, recordable media such as floppy disks, hard drives, compact disks (CDs), digital versatile disks (DVDs), digital tapes, computer memories, and the like; and transmission-type media such as digital and / or analog communication media (eg, fiber optic cable, waveguide, wired communication link, wireless communication link, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Analytical Chemistry (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • General Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Geophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Image Analysis (AREA)
  • Analysing Materials By The Use Of Radiation (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种检查方法和检查设备以及计算机可读介质。该方法包括：用X射线扫描被检查物体，得到被检查物体的图像；利用第一神经网络处理所述图像，得到被检查物体的语义描述；读取所述被检查物体的舱单的文字信息；利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。该方法能够在确保检查准确率的同时，大大提高查验的速度，使得安全检查的效率得到大幅度提升。

Description

检查方法和检查设备以及计算机可读介质 技术领域
本申请涉及安全检查，具体涉及一种基于射线成像的检查方法和检查设备以及计算机可读介质。
背景技术
目前，物流行业扮演着愈发重要的角色。无论是“全民网购”思潮的兴起，还是国家提出的“一带一路”政策，物流都需要在其中发挥至关重要的作用。然而，有不法分子利用物流的便利性伪报寄送物品，实际邮寄毒品、爆炸物、枪支等违禁品。这给社会安全造成巨大威胁。
中国的《反恐怖主义法》第85条规定“铁路、公路、水上、航空的货运和邮政、快递等物流运营单位未实行安全查验制度，对客户身份进行查验，或者未依照规定对运输、寄递物品进行安全检查或者开封验视、未实行运输、寄递客户身份和物品信息登记制度的进行处罚”。这要求快递企业须落实100％先验视后封箱、100％寄递实名制、100％X光机安检等制度保障。
在这种情况下，如果采用人工查验的方法，确认申报信息与真实物品是否一致，势必造成大量的资源浪费和效率低下等问题。人工查验也会造成查验人员极大的工作压力。
发明内容
针对现有技术的一个或多个问题，本公开的实施例提出了一种检查方法、检查设备以及计算机可读介质，能够对被检查物体进行自动检查，在保证查验准确率的同时，极大地提高查验速度。
在本公开的一个方面，提出了一种检查方法，包括步骤：用X射线扫描被检查物体，得到被检查物体的图像；利用第一神经网络处理所述图像，得到被检查物体的语义描述；读取所述被检查物体的舱单的文字信息；利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。
根据本公开的实施例，所述第一神经网络是卷积神经网络或者基于候选区域的卷积神经网络或者基于快速候选区域的卷积神经网络，所述第二神经网络是循环神 经网络或者双向循环神经网络。
根据本公开的实施例，利用事先建立的图像-语义对集合来训练所述第一神经网络。
根据本公开的实施例，在利用所述第一神经网络对图像进行处理前还包括步骤：对所述被检查物体的图像进行二值化；将二值化的图像计算平均值；用二值化的图像的每个像素值减去所述平均值。
根据本公开的实施例，基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过的步骤包括：计算表示所述语义描述的第一向量与表示所述语义特征的第二向量之间的距离；在计算的距离小于阈值的情况下允许所述被检查物体通过。
根据本公开的实施例，在第一神经网络的训练过程中在样本图像中包含的多个区域特征与所述样本图片的舱单信息中包括的多个词语之间建立对应关系。
根据本公开的实施例，将表示所述区域特征的特征矢量与表示所述词语的语义矢量之间的点积作为区域特征与词语之间的相似度，并且利用所述样本图像的多个区域特征与其舱单信息包括的多个词语之间的相似度的加权和作为所述样本图像与其舱单信息之间的相似度。
在本公开的另一方面，提出了一种检查设备，包括：扫描装置，用X射线对被检查物体进行扫描，得到扫描图像；输入装置，输入所述被检查物体的舱单信息；处理器，配置为：利用第一神经网络处理所述图像，得到被检查物体的语义描述；利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。
在本公开的再一方面，提出了一种计算机可读介质，存储有计算机程序，所述计算机程序被处理器执行时实现如下步骤：
利用第一神经网络处理被检查物体的X射线图像，得到被检查物体的语义描述；
利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；
基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。
利用上述实施例的方案，能够在确保检查准确率的同时，大大提高查验的速度，使得安全检查的效率得到大幅度提升。
附图说明
为了更好地理解本公开，将根据以下附图对本公开进行详细描述：
图1示出了根据本公开实施例的检查设备的示意图；
图2示出了在图1所示的实施例中用于图像处理的计算机的内部结构的示意图；
图3是描述在本公开实施例的检查设备和检查方法中使用的人工神经网络的示意图；
图4示出了在本公开实施例的检查设备和检查方法中使用的另一人工神经网络的示意图；
图5是描述根据本公开实施例的将图像和语义对齐的过程的示意图；
图6是描述根据本发明实施例的检查设备和检查方法中建立图像-语义模型的流程图；以及
图7是描述根据本公开实施例的检查方法对被检查物体进行安全检查的过程的流程图。
具体实施方式
下面将详细描述本发明的具体实施例，应当注意，这里描述的实施例只用于举例说明，并不用于限制本发明。在以下描述中，为了提供对本发明的透彻理解，阐述了大量特定细节。然而，对于本领域普通技术人员显而易见的是：不必采用这些特定细节来实行本发明。在其他实例中，为了避免混淆本发明，未具体描述公知的结构、材料或方法。
针对现有技术中的检查效率低的问题，本公开的实施例提出一种基于深度学习的检查技术，能够智能地完成物品机图单对比。利用本公开实施例的方案，可以发现物品机货物图像中，与物品申报的报关单数据不一致的区域。这个区域可能是伪报或瞒报。伪报一般是物流货品中的违禁物品或对物流运输安全造成威胁的危险物品，为了逃避检测，在报关单上把货物描述为运输安全的货物。瞒报一般是数量和体积较小的物品，也叫夹带，是走私、违禁品常用的手段。
此外，在物品机扫描图像中，物流货品中一箱多物的问题是传统图像处理条件下无法解决的难题。准确的说，它是受到设备不一致性影响的，在报关单数据监督下的多义性复杂分割问题。比如，不同的设备下算法必然会有不同的表现，报关单的数据形式给出了多个监督值(比如货物有多少种，每种类型与单位重量等)，图像上的每个像素可能属于多个货物等。本公开的方法采用基于深度学习的方法解决该问 题，无论一箱有几种物品，不用手动选取特征，通过卷积神经网络和大规模的训练数据得到训练好的图像模型以及语义模型，进而可以对测试图片生成对应的图像语义描述，从而准确完成一箱多货的图单对比。
图1示出了根据本公开实施例的检查设备的结构示意图。如图1所示的检查设备10包括X射线源11，探测器模块15、采集电路16、控制器17和数据处理计算机18等。射线源11包括一个或多个X射线发生器，可以进行单能透射扫描，也可以进行双能透射扫描。
如图1所示，例如行李物品之类的被检查物体14放在传送带13上穿过射线源11与探测器模块15之间的扫描区域。在一些实施例中，探测器模块15和采集电路16例如是具有整体模块结构的探测器及数据采集器，例如多排探测器，用于探测透射被检物体14的射线，获得模拟信号，并且将模拟信号转换成数字信号，从而输出被检查物体14针对X射线的透射图像。在双能的情况下，可以例如针对高能射线设置一排探测器，针对低能射线设置另一排探测器，或者高能射线和低能射线分时使用同一排探测器。控制器17用于控制整个系统的各个部分同步工作。数据处理计算机18用来处理由数据采集电路16采集的数据，对图像数据进行处理，输出结果。例如数据处理计算机18运行图像处理程序，对扫描得到的图像进行分析和学习，得到该图像的语义描述，然后将得到的语义描述与给行李物品的舱单信息中包括的语义特征进行对比，判断申报信息与行李物品中的物体是否一致。在一致的情况下，允许给行李物品通过，否则发出警报，提醒安检人员该行李物品可能有问题。
根据该实施例，探测器模块15和采集电路16用于获取被检查物体14的透射数据。采集电路16中包括数据放大成形电路，它可工作于(电流)积分方式或脉冲(计数)方式。采集电路16的数据输出电缆与控制器17和数据处理计算机18连接，根据触发命令将采集的数据存储在数据处理计算机18中。
在一些实施例中，探测器模块15包括多个探测单元，接收穿透被检查物体14的X射线。采集电路16与探测器模块15耦接，将探测器模块16产生的信号转换为探测数据。控制器17通过控制线路CTRL1与射线源11连接，通过控制线路CTRL2与探测器模块连接15，并且与采集电路16连接，控制射线源11中的一个或多个X射线发生器对被检查物体14进行单能扫描，或者对被检查物体14进行双能扫描，从而随着被检查物体14的移动而发出X射线穿透被检查物体14。此外，控制器17控制探测器模块15和采集电路16，获得相应的透射数据，例如单能透射数据或者 双能透射数据。数据处理计算机18基于透射数据得到被检查物体14的图像，对该图像进行处理并且基于被检查物体14的舱单信息判断二者是否一致。
图2示出了如图1所示的数据处理计算机的结构框图。如图2所示，数据处理计算20包括存储设备21、只读存储器(ROM)22、随机存取存储器(RAM)23、输入装置24、处理器25、显示设备26和接口单元27以及总线28等。
采集电路16所采集的数据通过接口单元27和总线28存储在存储设备21中。只读存储器(ROM)22中存储有计算机数据处理器的配置信息以及程序。随机存取存储器(RAM)23用于在处理器25工作过程中暂存各种数据。另外，存储设备21中还存储有用于进行数据处理的计算机程序。内部总线28连接上述的存储设备21、只读存储器22、随机存取存储器23、输入装置24、处理器25、显示设备28和接口单元27。
在用户通过诸如键盘和鼠标之类的输入装置24输入的操作命令后，计算机程序的指令代码命令处理器25执行数据处理算法，在得到数据处理结果之后，将其显示在诸如LCD显示器之类的显示装置27上，或者直接以诸如打印之类硬拷贝的形式输出处理结果。
例如，射线源11可以是放射性同位素(例如钴-60)，也可以是低能的X光机或高能的X射线加速器等。
例如，探测器模块15从材料上划分，可以是气体探测器、闪烁体探测器或固体探测器等，从阵列排布上划分，可以是单排、双排或者多排，以及单层探测器或双层高低能探测器等。
以上描述的是被检查物体14，例如行李物品，通过传送带13移动通过检查区域，但是本领域的技术人员应该想到，也可以是被检查物体14静止而射线源和探测器阵列移动完成扫描过程。
为了识别透射图像中的特征，本公开的实施例提出使用卷积神经网络CNN来对图像中的特征进行识别。以下结合图3来详细说明根据本公开实施例的卷积神经网络30。图3是示出了根据本公开实施例的卷积神经网络30的示意图。如图3所示卷积神经网络30通常可以包含多个卷积层32和34，这些卷积层32和34一般是由彼此部分重叠的小型神经元(其在数学的意义上也被称为卷积核，以下如无特别声明，这两个术语可以互换使用)的集合。此外，在本公开的上下文中，除非另有明确声明，否则对于卷积神经网络30中的任何两层而言，更接近输入数据(或输入层， 例如图3的输入层31)的层被称为“在前”或“在下”的层，而另一个更接近输出数据(或输出层，例如图3的输出层37)的层被称为“在后”或“在上”的层。此外，在训练、验证和/或使用期间，从输入层(例如，图3的输入层31)到输出层(例如，图3的输出层37)的方向被称为前向或正向(forward)，而从输出层(例如，图3的输出层37)到输入层(例如，图3的输入层31)的方向被称为后向或反向(backward)。
以图3所示的第一卷积层32为例，这些小型神经元可以对输入图像的各个局部进行处理。然后这些小型神经元的输出被合并排列为一个输出(称为特征映射，例如第一卷积层32中的方形)，以获得对原始图像中某些特征进行更好表示的输出图像。同时，相邻神经元之间部分重叠的排列也使得卷积神经网络30对于原始图像中的特征具备一定程度的平移容忍度。换言之，即使原始图像中的特征在某个容忍度内以平移方式改变了其位置，该卷积神经网络30也可以正确地识别出该特征。关于卷积层的详细描述将在后文中给出，此处不再详细讨论。
接下来的一层是可选的池化(pooling)层，即第一池化层33，其主要用于在保持特征的情况下对前一卷积层32的输出数据进行下采样，减少计算量并防止过拟合。
接下来的一层同样是一个卷积层，第二卷积层34，可以对于第一卷积层32所产生的、并经由池化层33下采样的输出数据进行进一步的特征采样。从直观上看，其所学习到的特征在全局性上大于第一卷积层所学习到的特征。同样地，后续的卷积层都是对前一卷积层的特征的全局化。
卷积层(例如，第一和第二卷积层32和34)是CNN(例如，卷积神经网络30)的核心构造单元。该层的参数由可学习卷积核(或简称为卷积核)的集合来构成，每个卷积核具有很小的感受野，但是在输入数据的整个深度上延伸。在前向过程中，将每个卷积核沿输入数据的宽度和高度进行卷积，计算卷积核的元素与输入数据之间的点积，并产生该卷积核的二维激活映射。作为结果，网络能够学习到在输入的某个空间位置上看到某个具体类型的特征时才可以激活的卷积核。
将所有卷积核的激活映射沿深度方向进行堆叠，形成了卷积层的全输出数据。因此，输出数据中的每个元素可以被解释为看到输入中的小区域并与相同激活映射中的其他卷积核共享参数的卷积核的输出。
输出数据的深度控制了层中连接到输入数据的相同区域的卷积核的数量。例如，如图3所示，第一卷积层32的深度为4，第二卷积层34的深度为6。所有这些卷积核将学习到针对输入中的不同特征来激活。例如，如果第一卷积层32以原始图像为 输入，则沿着深度维度的不同卷积核(即，图3中的不同方形)可以在输入数据中出现各种定向的边、或灰度块时激活。
训练过程在深度学习中是一个非常重要的部分。为了保证网络能够有效收敛，可以采用随机梯度下降法。例如，可以采用Nesterov优化算法来求解。在一些实施例中，初始的学习速率可以设置为从0.01开始，并逐渐减小，直至找到一个最优值。此外，在一些实施例中，对于权重的初始值，可以使用具有较小方差的Gaussian随机过程来初始化各卷积核的权重值。在一些实施例中，图像训练集可以采用标记的物品图像，其均标记有图像中的特征位置。
根据本公开的实施例，在图片上利用如图3所示的CNN(Convolution Neural Networks)，生成稠密的图片描述来表征物品机图像中物品的信息。特别的，利用卷积神经网络模型提取图像特征时，可以采用基于候选区域的卷积神经网络或者基于快速候选区域卷积神经网络(Faster-RCNN)的提取方法。例如，在图片上利用CNN(Convolution Neural Networks)生成稠密的图片描述来表征小型物品机图像中物品的信息。由于图片的语句描述常常与图片中的物体以及它们的属性有关系，因此采用RCNN(Region Convolutional Neural Network)在每张图片中检测物体。在整个图片中检测，得到一系列物品及其置信度，采用置信度最高的10个检测位置，并根据每个边界框内的全部像素来计算图片表示。本领域的技术人员应该理解，可以采用其他的人工神经网络来识别和学习透射图像中的特征。本公开的实施例不限于此。
为了学习舱单信息或者说确定舱单中包括的信息，本公开的实施例采用了适用于自然语言学习的循环神经网络RNN(Recurrent Neural Networks)。图4示出了在本公开实施例的检查设备和检查方法中使用的人工神经网络的示意图。
如图4所示，通常RNN包含输入单元(Input units)，输入集标记为{x 0,x 1,...,x t,x t+1,...}，而输出单元(Output units)的输出集则被标记为{y 0,y 1,...,y t,y t+1.,..}。RNN还包含隐藏单元(Hidden units)，将其输出集标记为{s 0,s 1,...,s t,s t+1,...}，这些隐藏单元完成了最为主要的工作。在图4中，有一条单向流动的信息流是从输入单元到达隐藏单元的，与此同时另一条单向流动的信息流从隐藏单元到达输出单元。在某些情况下，RNN会打破后者的限制，引导信息从输出单元返回隐藏单元，这些被称为“Back Projections”，并且隐藏层的输入还包括上一隐藏层的状态，即隐藏层内的节点可以自连也可以互连。
如图4所示，将循环神经网络展开成一个全神经网络。例如，对一个包含5个单词的语句，那么展开的网络便是一个五层的神经网络，每一层代表一个单词。
这样将自然语言转换成机器能够识别的符号，并且方便在机器学习过程中，对其进行数值化。由于词是自然语言理解与处理的基础，因此需要对词进行数值化，本公开的实施例提出使用词向量(Word Representation)。词向量是指使用一个指定长度的实数向量v来表示一个词。可以使用One-hot vector表示单词，即根据单词的数量|V|生成一个|V|*1的向量，当某一位为一的时候其他位都为零，然后这个向量就代表一个单词。
在RNN中，每输入一步，每一层各自都共享参数U,V,W，其反映了RNN中的每一步都在做相同的事，只是输入不同，因此大大地降低了网络中需要学习的参数。
Bidirectional RNN(双向循环神经网络)对RNN的改进之处是，当前的输出(第t步的输出)不仅仅与前面的序列有关，并且还与后面的序列有关。例如：预测一个语句中缺失的词语那么就需要根据上下文来进行预测。Bidirectional RNN是一个相对较简单的RNN，是由两个RNN上下叠加在一起组成的。输出由这两个RNN的隐藏层的状态决定的。
在本公开的实施例中，针对图像对应的报关单信息或者快递单物品信息，采用RNN(Recurrent Neural Networks)模型，如图4所示。例如，本公开的实施例中采用双向RNN(BRNN)的方法。BRNN输入n个词的序列，每个用one-hot编码，将每个词语转换成一个固定的h维向量。因为使用了词语周围长度变化的上下文，词语的表示得到了丰富。本领域的技术人员应该理解，可以采用其他的人工神经网络来识别和学习舱单信息中的特征。本公开的实施例不限于此。
根据本公开的实施例，检查过程涉及到三部分：1)建立已扫描图像以及对应报关单语义信息的数据库；2)建立行李物品图像模型与对应的语义模型，3)建立图单对比模型完成智能图单核查。已扫描图像以及对应报关单语义信息的数据库包括图像采集、图像预处理、样本预处理三个部分。
建立图像以及对应报关单语义信息的数据库主要分为图像采集、图像预处理、样本预处理三个部分。建立过程为：(1)图像采集。采集相当数量的物品机扫描的物品图像，使图像数据库包含各种物品图像。注意，此时图像包括正常物品和违禁物品。(2)图像预处理。在扫描采集得到的图像中附带噪声信息，可以对扫描图像进行预处理。由于二值图像具备统一的物理分辨率，而且可以方便多种算法结合使用，因此本专利采用二值图像。注意，为了保证模型的泛化性能，可以将每张图片都减去所有图片的均值。
例如，采集相当数量的物品机扫描的物品图像，使图像数据库包含各种物品图 像。注意，此时图像包括正常物品和违禁物品。然后，在扫描采集得到的图像中附带噪声信息，需要对扫描图像进行预处理。由于二值图像具备统一的物理分辨率，而且可以方便多种算法结合使用，因此本申请采用二值图像。得到二值图像之后，计算图像的灰度平均值，为了保证模型的泛化性能，将每张图片都减去所有图片的均值，得到的结果当作模型的图像输入。
根据本公开的实施例，采用报关单语义信息对样本进行标注。图像和语义信息形成一个完整的信息对，以便于网络的训练。注意，此处语义信息还包括快递运输单等所述的物品信息。例如，采用报关单语义信息对样本进行标注。图像和语义信息形成一个完整的信息对，以便于网络的训练。为了对使模型对关键词保持敏感，需要把物品描述里面的非关键词语进行处理，比如删除“一种、一类”之类的非物品关键词。
建立图单对比模型的过程如图5所示，在图片和它们对应的语句描述构成的数据集合上，学习语言和视觉数据内在的模式对应关系。如图5所示，本公开的方法基于一个新模型组合的方式，同时也基于一个结构化目的，通过一个多模编码模型来对齐两种模式输入。
将图像特征与语义特征通过多模编码模型来对齐，通过随机梯度下降训练最终得到图单对比模型，也就是如图5所示的RCNN 52和BRNN 53。例如，若计算的结果是图像与其申报的物品语义信息对应的，物品自动通过物品机。若计算的结果是图像与其申报的物品语义信息不对应的，物品机发出警告，提示工作人员存在异常，进行相应的处理。在本公开中，模型需要不依赖于假设，例如特殊的硬编码模型，规则或者是类别，而仅仅是从训练语料中学习它们。
这样，利用这些大量的(图片-语句)数据集，将图片的语言描述看成弱标注。这些语句中连续分割的词语对应图片中一些特殊但未知的位置。利用神经网络52和53推断这些“对齐”并且将他们应用到学习描述生成模型。具体的，如上所述本公开采用了一个深度神经网络模型，能够推断分片语句和它们对应的描述图片区域间的潜在对齐关系。这样的模型通过一个共同的、多模的编码空间，以及一个结构化目标，将两种模式联系在一起。采用了一个多模的递归神经网络架构，输入一副图像，生成对应的文本描述，生成的图片描述与图像标注信息进行关键词匹配，判断生成的语句描述与标注信息的相似性。实验显示，生成的文本描述语句明显优于基于检索的方法。在推断的对应关系上，训练这个模型，并且用一个新的局部注释的数据集来测试了它的效果。
如图5所示，将图像特征与语义特征通过多模编码模型来对齐，通过随机梯度下降训练最终得到图单对比模型。
例如通过已建立的RCNN 52和BRNN 53，可以将每个小型物品机扫描图片和对应的描述语句转化成一个公共的h维向量集合。因为有监督语料是在整个图片和整个语句粒度的，但是可以把(图片-语句)看成是(区域-词语)打分的函数。直觉地来看，一个(图片-语句)组合，如果它的某个词语在图片中能找到充足的物体或属性支持，那么它们应该得到较高的匹配得分。将第i个图片区域的矢量v i和第t个词语矢量的s t之间的点积解释成相似度的度量，并且使用它来进一步定义图片和语句之间的得分，得分越高，表明图像与报关单对应程度越高。
这样可以训练出一个完整的(图片-语义)模型。另外，在实际检查过程中，可以把生成的语义描述与真实物品语义描述的差别也作为损失函数的一部分。检查过程中根据训练模型输入一张处理的图片之后，会得到与图片对应的语义描述，之后与真实的报关单进行匹配，得到一个置信度，据此判断图单是否一致。
根据本公开的实施例，在神经网络的训练过程中，在样本图像中包含的多个区域特征与所述样本图片的舱单信息中包括的多个词语之间建立对应关系。例如，将表示区域特征的特征矢量与表示词语的语义矢量之间的点积作为区域特征与词语之间的相似度，并且利用所述样本图像的多个区域特征与其舱单信息包括的多个词语之间的相似度的加权和作为所述样本图像与其舱单信息之间的相似度。
图6是描述根据本发明实施例的检查设备和检查方法中建立图像-语义模型的流程图。如图6所示，在步骤S61，将事先准备的带标签的图像输入到卷积神经网络30中进行训练。在步骤S62得到训练的图像-语义模型，也就是初步训练后的卷积神经网络。为了提高模型的准确性，在步骤S63输入测试图片对网络进行测试，在步骤S64计算预测得分以及与标签的差值，例如将测试图片输入到神经网络中，得到了预测的语义描述以及该语义描述与标签之间的差值。例如分别用两个向量表示语义表述和标签，则用两个向量之间的差表示二者之间的差值。在步骤S65，判断差值是否小于阈值，如果大于在步骤S66更新网络参数对网络进行调整。如果差值小于阈值，则在步骤S67建立网络模型，也就是对网络的训练结束。
图7是描述根据本公开实施例的检查方法对被检查物体进行安全检查的过程的流程图。
在实际的安全检查过程中，在步骤S71，利用图1所示的检查设备对被检查物体14进行扫描，得到被检查物体的透射图像。在步骤S74，通过手动录入或者条码 扫描仪或者其他方式将被检查物体的舱单信息输入到数据处理计算机18中。在数据处理计算机18中，在步骤S72利用第一神经网络，例如卷积神经网络或者RCNN等，对透射图像进行处理，在步骤S73得到被检查物体的语义描述。在步骤S75利用双向循环神经网络对舱单的文字信息进行处理，得到被检查物体的语义特征。然后在步骤S76，判断语义特征是否与根据图像得到的语义描述一致，如果不一致，则在步骤S77发出警报。如果一致，则在步骤S78，允许被检查物体通过。根据一些实施例，可以计算表示语义描述的第一向量与表示语义特征的第二向量之间的距离，然后在计算的距离小于阈值的情况下允许所述被检查物体通过。这里两个向量之间的距离可以用两个向量的元素之间的差值的绝对值之和表示，也可以用两个向量的欧氏距离来表示。本公开的实施例不局限于此。
本公开通过图像处理、深度学习的方式，实现智能查验申报物品与实际物品是否一致，一方面可以大大提高工作效率，实现“通得快”，另一方面也可以减少各类主观因素的副作用，实现“管得住”，是当前安检智能化查验的重要手段，具有巨大的市场潜力。
以上的详细描述通过使用示意图、流程图和/或示例，已经阐述了检查方法和检查设备的众多实施例。在这种示意图、流程图和/或示例包含一个或多个功能和/或操作的情况下，本领域技术人员应理解，这种示意图、流程图或示例中的每一功能和/或操作可以通过各种结构、硬件、软件、固件或实质上它们的任意组合来单独和/或共同实现。在一个实施例中，本发明的实施例所述主题的若干部分可以通过专用集成电路(ASIC)、现场可编程门阵列(FPGA)、数字信号处理器(DSP)、或其他集成格式来实现。然而，本领域技术人员应认识到，这里所公开的实施例的一些方面在整体上或部分地可以等同地实现在集成电路中，实现为在一台或多台计算机上运行的一个或多个计算机程序(例如，实现为在一台或多台计算机系统上运行的一个或多个程序)，实现为在一个或多个处理器上运行的一个或多个程序(例如，实现为在一个或多个微处理器上运行的一个或多个程序)，实现为固件，或者实质上实现为上述方式的任意组合，并且本领域技术人员根据本公开，将具备设计电路和/或写入软件和/或固件代码的能力。此外，本领域技术人员将认识到，本公开所述主题的机制能够作为多种形式的程序产品进行分发，并且无论实际用来执行分发的信号承载介质的具体类型如何，本公开所述主题的示例性实施例均适用。信号承载介质的示例包括但不限于：可记录型介质，如软盘、硬盘驱动器、紧致盘(CD)、数字通用 盘(DVD)、数字磁带、计算机存储器等；以及传输型介质，如数字和/或模拟通信介质(例如，光纤光缆、波导、有线通信链路、无线通信链路等)。
虽然已参照几个典型实施例描述了本发明，但应当理解，所用的术语是说明和示例性、而非限制性的术语。由于本发明能够以多种形式具体实施而不脱离发明的精神或实质，所以应当理解，上述实施例不限于任何前述的细节，而应在随附权利要求所限定的精神和范围内广泛地解释，因此落入权利要求或其等效范围内的全部变化和改型都应为随附权利要求所涵盖。

Claims (15)

  1. 一种检查方法，包括步骤：
    用X射线扫描被检查物体，得到被检查物体的图像；
    利用第一神经网络处理所述图像，得到被检查物体的语义描述；
    读取所述被检查物体的舱单的文字信息；
    利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；
    基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。
  2. 如权利要求1所述的检查方法，其中所述第一神经网络是卷积神经网络或者基于候选区域的卷积神经网络或者基于快速候选区域的卷积神经网络，所述第二神经网络是循环神经网络或者双向循环神经网络。
  3. 如权利要求1所述的检查方法，其中利用事先建立的图像-语义对集合来训练所述第一神经网络。
  4. 如权利要求1所述的检查方法，其中，在利用所述第一神经网络对图像进行处理前还包括步骤：
    对所述被检查物体的图像进行二值化；
    将二值化的图像计算平均值；
    用二值化的图像的每个像素值减去所述平均值。
  5. 如权利要求1所述的检查方法，其中基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过的步骤包括：
    计算表示所述语义描述的第一向量与表示所述语义特征的第二向量之间的距离；
    在计算的距离小于阈值的情况下允许所述被检查物体通过。
  6. 如权利要求1所述的检查方法，其中在第一神经网络的训练过程中在样本图像中包含的多个区域特征与所述样本图片的舱单信息中包括的多个词语之间建立对应关系。
  7. 如权利要求6所述的检查方法，其中将表示所述区域特征的特征矢量与表示所述词语的语义矢量之间的点积作为区域特征与词语之间的相似度，并且利用所述样本图像的多个区域特征与其舱单信息包括的多个词语之间的相似度的加权和作为所述样本图像与其舱单信息之间的相似度。
  8. 一种检查设备，包括：
    扫描装置，用X射线对被检查物体进行扫描，得到扫描图像；
    输入装置，输入所述被检查物体的舱单信息；
    处理器，配置为：
    利用第一神经网络处理所述图像，得到被检查物体的语义描述；
    利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；
    基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。
  9. 如权利要求8所述的检查设备，其中所述第一神经网络是卷积神经网络或者基于候选区域的卷积神经网络或者基于快速候选区域的卷积神经网络，所述第二神经网络是循环神经网络或者双向循环神经网络。
  10. 如权利要求8所述的检查设备，其中利用事先建立的图像-语义对集合来训练所述第一神经网络。
  11. 如权利要求8所述的检查设备，其中，所述处理器还被配置为在利用所述第一神经网络对图像进行处理前：
    对所述被检查物体的图像进行二值化；
    将二值化的图像计算平均值；
    用二值化的图像的每个像素值减去所述平均值。
  12. 如权利要求8所述的检查设备，其中所述处理器还被配置为：
    计算表示所述语义描述的第一向量与表示所述语义特征的第二向量之间的距离；
    在计算的距离小于阈值的情况下允许所述被检查物体通过。
  13. 如权利要求8所述的检查设备，其中所述处理器被配置为在第一神经网络的训练过程中在样本图像中包含的多个区域特征与所述样本图片的舱单信息中包括的多个词语之间建立对应关系。
  14. 如权利要求13所述的检查设备，其中将表示所述区域特征的特征矢量与表示所述词语的语义矢量之间的点积作为区域特征与词语之间的相似度，并且利用所述样本图像的多个区域特征与其舱单信息包括的多个词语之间的相似度的加权和作为所述样本图像与其舱单信息之间的相似度。
  15. 一种计算机可读介质，存储有计算机程序，所述计算机程序被处理器执行时实现如下步骤：
    利用第一神经网络处理被检查物体的X射线图像，得到被检查物体的语义描述；
    利用第二神经网络对被检查物体的舱单的文字信息进行处理，得到被检查物体的语义特征；
    基于所述语义描述和所述语义特征来判断所述被检查物体是否允许通过。
PCT/CN2018/106021 2017-09-18 2018-09-17 检查方法和检查设备以及计算机可读介质 Ceased WO2019052561A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
KR1020197034843A KR102240058B1 (ko) 2017-09-18 2018-09-17 검사 방법과 검사 장비 및 컴퓨터 판독 가능한 매체
EP18856136.9A EP3699579B1 (en) 2017-09-18 2018-09-17 Inspection method and inspection device and computer-readable medium
JP2019565493A JP7678661B2 (ja) 2017-09-18 2018-09-17 検査方法、検査設備及びコンピューター読み取り可能な媒体

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710845577.6 2017-09-18
CN201710845577.6A CN109522913B (zh) 2017-09-18 2017-09-18 检查方法和检查设备以及计算机可读介质

Publications (1)

Publication Number Publication Date
WO2019052561A1 true WO2019052561A1 (zh) 2019-03-21

Family

ID=65722436

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/106021 Ceased WO2019052561A1 (zh) 2017-09-18 2018-09-17 检查方法和检查设备以及计算机可读介质

Country Status (5)

Country Link
EP (1) EP3699579B1 (zh)
JP (1) JP7678661B2 (zh)
KR (1) KR102240058B1 (zh)
CN (1) CN109522913B (zh)
WO (1) WO2019052561A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110674292A (zh) * 2019-08-27 2020-01-10 腾讯科技（深圳）有限公司 一种人机交互方法、装置、设备及介质
CN111192252A (zh) * 2019-12-30 2020-05-22 深圳大学 一种图像分割结果优化方法、装置、智能终端及存储介质
CN112633652A (zh) * 2020-12-15 2021-04-09 北京交通大学 基于语义风险自适应识别的物流安检方法
CN112860889A (zh) * 2021-01-29 2021-05-28 太原理工大学 一种基于bert的多标签分类方法
CN113093308A (zh) * 2021-05-08 2021-07-09 佳都科技集团股份有限公司 X射线行李检查设备的校正方法、装置、设备及存储介质
CN116046810A (zh) * 2023-04-03 2023-05-02 云南通衢工程检测有限公司 基于rpc盖板破坏荷载的无损检测方法
CN117150097A (zh) * 2023-08-31 2023-12-01 应急管理部大数据中心 一种执法检查清单自动匹配方法
CN118887609A (zh) * 2024-07-09 2024-11-01 内蒙古电力(集团)有限责任公司蒙电项目建管分公司 一种视频流实时处理的园区建设物资安全监控方法及系统

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113836998A (zh) * 2021-08-16 2021-12-24 北京澎湃信用管理有限公司 针对待查验物品进行分析的信息处理装置及方法
CN116092096A (zh) * 2021-11-05 2023-05-09 同方威视技术股份有限公司 用于检验申报信息真实性的方法、系统、设备及介质
CN115730878B (zh) * 2022-12-15 2024-01-12 广东省电子口岸管理有限公司 基于数据识别的货物进出口查验管理方法
CN118279578B (zh) * 2022-12-30 2025-05-27 同方威视科技江苏有限公司 Ct图像处理方法及装置、以及国际快件的查验方法及装置
JP2024118842A (ja) * 2023-02-21 2024-09-02 株式会社東芝 情報処理装置及びプログラム
KR102808878B1 (ko) * 2023-11-27 2025-05-19 주식회사 딥노이드 대형멀티모달모델을 이용한 화물의 방사선 이미지와 텍스트 정보를 매치하기 위한 장치 및 이를 위한 방법

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110206240A1 (en) * 2008-10-30 2011-08-25 Baoming Hong Detecting concealed threats
CN103917862A (zh) * 2011-09-07 2014-07-09 拉皮斯坎系统股份有限公司 整合舱单数据和成像/检测处理的x射线检查系统
CN104165896A (zh) * 2014-08-18 2014-11-26 公安部第一研究所 一种液态物品安全检查的方法与装置
CN106706677A (zh) * 2015-11-18 2017-05-24 同方威视技术股份有限公司 检查货物的方法和系统
CN107145910A (zh) * 2017-05-08 2017-09-08 京东方科技集团股份有限公司 医学影像的表现生成系统、其训练方法及表现生成方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07160665A (ja) * 1993-12-13 1995-06-23 Fujitsu Ltd 物体識別装置
US6765981B2 (en) 2002-07-31 2004-07-20 Agilent Technologies, Inc. Computed tomography
JP2009075744A (ja) * 2007-09-19 2009-04-09 Spirit21:Kk 基板実装部品の管理システム
PT2639749T (pt) 2012-03-15 2017-01-18 Cortical Io Gmbh Métodos, aparelhos e produtos para processamento semântico de texto
CN103901489B (zh) * 2012-12-27 2017-07-21 清华大学 检查物体的方法、显示方法和设备
CN105808555B (zh) 2014-12-30 2019-07-26 清华大学 检查货物的方法和系统
US10423874B2 (en) * 2015-10-02 2019-09-24 Baidu Usa Llc Intelligent image captioning
CN106845499A (zh) * 2017-01-19 2017-06-13 清华大学 一种基于自然语言语义的图像目标检测方法
CN108734183A (zh) * 2017-04-14 2018-11-02 清华大学 检查方法和检查设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110206240A1 (en) * 2008-10-30 2011-08-25 Baoming Hong Detecting concealed threats
CN103917862A (zh) * 2011-09-07 2014-07-09 拉皮斯坎系统股份有限公司 整合舱单数据和成像/检测处理的x射线检查系统
CN104165896A (zh) * 2014-08-18 2014-11-26 公安部第一研究所 一种液态物品安全检查的方法与装置
CN106706677A (zh) * 2015-11-18 2017-05-24 同方威视技术股份有限公司 检查货物的方法和系统
CN107145910A (zh) * 2017-05-08 2017-09-08 京东方科技集团股份有限公司 医学影像的表现生成系统、其训练方法及表现生成方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3699579A4 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110674292A (zh) * 2019-08-27 2020-01-10 腾讯科技（深圳）有限公司 一种人机交互方法、装置、设备及介质
CN111192252A (zh) * 2019-12-30 2020-05-22 深圳大学 一种图像分割结果优化方法、装置、智能终端及存储介质
CN111192252B (zh) * 2019-12-30 2023-03-31 深圳大学 一种图像分割结果优化方法、装置、智能终端及存储介质
CN112633652A (zh) * 2020-12-15 2021-04-09 北京交通大学 基于语义风险自适应识别的物流安检方法
CN112633652B (zh) * 2020-12-15 2023-09-29 北京交通大学 基于语义风险自适应识别的物流安检方法
CN112860889A (zh) * 2021-01-29 2021-05-28 太原理工大学 一种基于bert的多标签分类方法
CN113093308A (zh) * 2021-05-08 2021-07-09 佳都科技集团股份有限公司 X射线行李检查设备的校正方法、装置、设备及存储介质
CN113093308B (zh) * 2021-05-08 2024-04-16 佳都科技集团股份有限公司 X射线行李检查设备的校正方法、装置、设备及存储介质
CN116046810A (zh) * 2023-04-03 2023-05-02 云南通衢工程检测有限公司 基于rpc盖板破坏荷载的无损检测方法
CN117150097A (zh) * 2023-08-31 2023-12-01 应急管理部大数据中心 一种执法检查清单自动匹配方法
CN117150097B (zh) * 2023-08-31 2024-03-01 应急管理部大数据中心 一种执法检查清单自动匹配方法
CN118887609A (zh) * 2024-07-09 2024-11-01 内蒙古电力(集团)有限责任公司蒙电项目建管分公司 一种视频流实时处理的园区建设物资安全监控方法及系统

Also Published As

Publication number Publication date
CN109522913B (zh) 2022-07-19
EP3699579A4 (en) 2021-05-19
JP7678661B2 (ja) 2025-05-16
EP3699579B1 (en) 2023-08-02
JP2020534508A (ja) 2020-11-26
KR20200003011A (ko) 2020-01-08
EP3699579A1 (en) 2020-08-26
CN109522913A (zh) 2019-03-26
KR102240058B1 (ko) 2021-04-15

Similar Documents

Publication Publication Date Title
CN109522913B (zh) 检查方法和检查设备以及计算机可读介质
US10013615B2 (en) Inspection methods and devices
Zhong et al. Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery
Shi et al. Point-gnn: Graph neural network for 3d object detection in a point cloud
EP3349050B1 (en) Inspection devices and methods for detecting a firearm
CN105808555B (zh) 检查货物的方法和系统
CN105809091B (zh) 检查方法和系统
JP2017097853A (ja) 貨物の検査方法及びそのシステム
CN108303748A (zh) 检查设备和检测行李物品中的枪支的方法
Li et al. Tunnel crack detection using coarse‐to‐fine region localization and edge detection
EP3349049B1 (en) Inspection devices and methods for inspecting a container
KR102283197B1 (ko) 상품의 유형을 결정하는 방법 및 디바이스
CN109557114B (zh) 检查方法和检查设备以及计算机可读介质
CN114462487A (zh) 目标检测网络训练及检测方法、装置、终端及存储介质
Jha E-commerce product image classification using transfer learning
CN113762029A (zh) 危险品识别方法、装置、设备及存储介质
Liu et al. A Lightweight Dangerous Liquid Detection Method Based on Depthwise Separable Convolution for X‐Ray Security Inspection
NL2034690B1 (en) Method and apparatus of training radiation image recognition model online, and method and apparatus of recognizing radiation image
HK40004246A (zh) 检查方法和检查设备以及计算机可读介质
HK40004246B (zh) 检查方法和检查设备以及计算机可读介质
Chopade et al. Single shot detector application for image disease localization
Paranjape et al. Segmentation of handguns in dual energy X-ray imagery of passenger carry-on baggage
KR102934312B1 (ko) 통관 시스템에서 이종의 정보를 통합하여 제공하기 위한 장치 및 이를 위한 방법
Ansary Transfer Learning With CLIP for Intelligent Concrete Crack Detection in Structural Health Monitoring
Cao Research on The Vehicle Detection Technology Based on The Yolo Model

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18856136

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20197034843

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2019565493

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018856136

Country of ref document: EP

Effective date: 20200420