EP2240887B1 - Merkmalbasierte signaturen für bildidentifizierung - Google Patents

Merkmalbasierte signaturen für bildidentifizierung Download PDF

Info

Publication number
EP2240887B1
EP2240887B1 EP08775812.4A EP08775812A EP2240887B1 EP 2240887 B1 EP2240887 B1 EP 2240887B1 EP 08775812 A EP08775812 A EP 08775812A EP 2240887 B1 EP2240887 B1 EP 2240887B1
Authority
EP
European Patent Office
Prior art keywords
image
representation
feature
region
regions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP08775812.4A
Other languages
English (en)
French (fr)
Other versions
EP2240887A1 (de
Inventor
Miroslaw Bober
Paul Brasnett
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Mitsubishi Electric R&D Centre Europe BV Netherlands
Original Assignee
Mitsubishi Electric Corp
Mitsubishi Electric R&D Centre Europe BV Netherlands
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp, Mitsubishi Electric R&D Centre Europe BV Netherlands filed Critical Mitsubishi Electric Corp
Publication of EP2240887A1 publication Critical patent/EP2240887A1/de
Application granted granted Critical
Publication of EP2240887B1 publication Critical patent/EP2240887B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/48—Extraction of image or video features by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features

Definitions

  • the present invention relates to a method and apparatus for representing an image, and, in addition, a method and apparatus for comparing or matching images, for example, for the purposes of searching or validation.
  • the present invention relates to improvements upon the image identification techniques disclosed in earlier, co-pending European patent application No: EP 06255239.3 , and UK patent application Nos: GB 0700468.2 , GB 0712388.8 and GB 0719833.6 , the contents of which are incorporated herein by reference. Details of the inventions and embodiments disclosed in these earlier, co-pending patent applications apply analogously to the present invention and embodiments.
  • Mikolajczyk K et al "A performance evaluation of local descriptors" IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Service Center, Los Alamitos, CA, US, vol. 27, no. 10, 1 October 2005 (2005-10-01), pages 1615-1630 relates to local descriptors.
  • the present invention provides a method of deriving a representation of an image as defined in accompanying claim 1.
  • the present invention concerns a new approach to representing an image based on a small number of regions of interest described by region-based Trace-transform descriptors.
  • a representative small set of interest or feature points are selected from an image. Constraints are used in the selection procedure such as an exclusion zone around previously selected features.
  • a binary "feature signature" is extracted from each selected region of interest, using a modified Trace-transform based method. The feature signature is used to characterise the local neighbourhood of the feature.
  • An image is represented by the plurality of local feature signatures and their geometric relations.
  • an image matching procedure is performed as a two stage process that involves rapidly identifying candidate sets of matching pairs of feature points using the cumulative Hamming distance before applying geometric constraints to increase the accuracy of the method.
  • the information provided by the geometry of the feature points selected from the image helps to significantly reduce the false alarm rate.
  • the geometric information may also be used to determine the class of image transformation and the transformation parameters.
  • Embodiments of the present invention are designed to be complementary to the image identification apparatuses and methods described by EP 06255239.3 , GB 0700468.2 , GB 0712388.8 and GB 0719833.6 .
  • the present invention advantageously provides additional robustness to certain image modifications.
  • independent use of the technique of the present invention may not offer the same performance, in terms of search speed, as the techniques proposed in the above-referenced earlier, co-pending patent applications.
  • the process of identifying potential pairs of features according to embodiments of the present invention may be at least an order of magnitude slower than the previously proposed methods.
  • an "image identifier” (also referred to simply as “identifier”, “signature” or “image signature”) is an example of a representation of an image and the term is used merely to denote a representation of an image, or descriptor.
  • the proposed design may typically achieve a low false-alarm rate of less than 10 parts per million (ppm) on a broad class of images and typical detection rates above 95.0%.
  • Fig. 1 shows an example of an image and modified versions of the image. More specifically, Fig. 1a is an original image, Fig. 1b is a reduced (scaled) version of the image of Fig. 1a, Fig. 1c is a rotated version of the image of Fig. 1a, Fig. 1d is a blurred version of the image of Fig. 1a, Fig. 1e is a flipped version of the image of Fig. 1a, Fig. 1f is a compressed version of the image of Fig. 1a and Fig. 1g is a cropped version of the image of Fig. 1a .
  • An embodiment of the invention derives a representation of an image, and more specifically, an image identifier, by processing signals and/or image data corresponding to the image.
  • the image in the initial stage of extraction of the image identifier the image is optionally pre-processed by resizing and filtering.
  • the resizing step is used to normalise the images before processing.
  • the filtering step can comprise of filtering to remove effects such as aliasing it can also include region selection and tapering.
  • the image is resized to a resolution of 192xN or Nx192, where N ⁇ 192 and preserving the aspect ratio.
  • the image is resized to a square of 192x192.
  • the image is then low pass filtered with a 3x3 Gaussian kernel. A circular region is extracted from the centre of the image for further processing.
  • the pre-processing steps are optional and can include any combination of the above.
  • One embodiment uses a feature detector to detect potential features in an image and their strength based on measuring image gradients.
  • One suitable image gradient feature detector is a Harris feature detector.
  • a selection procedure is applied to choose a representative set of features. In an embodiment, up to 16 features are chosen.
  • a boundary is set around the edge of the image from which no features are chosen, in the embodiment the boundary is 16 pixels wide. The strongest feature from the allowable features amongst the detected features is selected first. The subsequent features are selected by order of strength with the restriction that a feature is not selected if it is within an exclusion zone around the previously selected features.
  • a circular region with a radius of 10 pixels is used as the exclusion zone.
  • the selection process terminates once a predetermined number of representative features (e.g. in the preferred embodiment 16 pixels) has been selected or no more features remain.
  • a set of features selected from those detected in Fig. 3 is shown in Fig. 4 .
  • each feature signature is a binary representation of an image region associated with a detected feature.
  • a circular image region having a radius of 32 pixels, centred on a feature point is selected and the binary signature is computed for the image region.
  • the circular region is preferably extracted to sub-pixel level accuracy, using cubic interpolation, for example.
  • the extracted region is then processed to derive a corresponding image identifier, using the method according to GB 0719833.6 .
  • a Trace transform T(d, ⁇ ) of the image region is performed by projecting all possible lines, parameterised by d, ⁇ over the image and applying a first functional T over these lines.
  • the result of the Trace transform may be decomposed to reduce its resolution in either or both of its dimensions d, ⁇ in accordance with the method disclosed in GB 0700468.2 .
  • a second functional P may be applied to the columns of the Trace transform to give a vector of real numbers (i.e. a one dimensional function), as in the methods of EP 06255239.3 and GB 0700468.2 .
  • This second functional P is known as the diametrical functional and the resulting vector is known as the circus function.
  • a third functional, the circus functional, may be applied to the circus function to give a single number, although this step is not used in the preferred embodiment.
  • a so-called "band-circus" function may be obtained, as an alternative to the circus function discussed above, which restricts the Trace transform to a subset of lines of the image, by selecting and processing values from only a part of the Trace transform.
  • the properties of the result can be controlled by appropriate choices of the first, second and/or third functional (Trace, diametrical and circus).
  • a binary identifier for the image region is extracted from the circus function (or the band-circus function) via a frequency representation.
  • a function c ( ⁇ ) may be defined on the magnitude coefficients of the Fourier transform.
  • B ⁇ b 0 ,...,b n ⁇ .
  • a further six, "multi resolution" identifiers may be extracted from circus functions obtained by decomposing (or sub-sampling) the distance (d) parameter of the Trace transform by a factor of 8, 16 and 32 and applying both (1) and (2), in accordance with the method disclosed in GB 0700468.2 .
  • a further three functions are selected from bands in the Trace transform, in accordance with the method disclosed in GB 0712388.8 , and two functions are extracted from the trace-annulus functions and one function is extracted from the trace-circular function, in accordance with the method disclosed in GB 0719833.6 .
  • Each of these 14 component identifiers is 8 bits, giving a total of 112 bits for each feature identifier.
  • the coordinates (in the pre-processed image) of the feature points are stored along with the identifier, as geometric information about the corresponding feature/image region.
  • a subset of the 112 bits could be selected or derived.
  • the complete set of feature points may be represented in a number of ways.
  • the image descriptor is a bit stream containing a number of fields of information, the first field is the number of features. Then each feature is represented in the bitstream as the feature point coordinates (x and y), followed by the identifier.
  • padding could be used with a flag set to represent an unused field.
  • a first step involves comparing all features from one image with all features from the second image.
  • the three unique pairs with the minimum Hamming distance H are found and the feature-based cumulative Hamming distance between two images is the sum of the three individual Hamming distances.
  • a second step involves applying a geometric constraint to determine if the images are a genuine or false match.
  • the application of the geometric constraint is relatively computationally expensive, compared to the Hamming distance. To minimise complexity only image pairs with a cumulative Hamming distance below a predefined threshold for potential matches are compared using the geometric method.
  • a set of three feature points (a, b and c) are chosen. Using these three points a ratio of line lengths between the features can be used as a constraint, as shown in Fig. 5 .
  • the ratio of line lengths between defined points in an image are invariant to transformations such as rotation, translation and scaling, such that the above geometric constraint is suitable for matching an image that has undergone such geometric transformations.
  • transformations such as rotation, translation and scaling
  • other suitable geometric constraints may be used, consistent with design requirements.
  • two thresholds are applied to the distance measures, one to constrain the magnitude of the individual measures and the other to constrain the sum of the distances.
  • multiple candidate sets of three feature point pairs can be preselected using a cumulative (and/or pairwise) Hamming distance constraint.
  • the geometric constraint can then be applied to each of the (possibly ordered by for example cumulative Hamming distance) candidate sets in order until a geometric match is found or no more candidate sets remain.
  • the image matching technique of the present invention may be combined with other image matching techniques, such as the method disclosed in GB 0719833.6 .
  • a descriptor is generated for each image that contains all of the individual identifiers.
  • the identifier bitstream contains two identifiers based on GB 0719833.6 , and the descriptor for the present invention.
  • the first is generated by pre-processing the image to maintain the image aspect ratio and the other where it is processed to a square.
  • the matching is performed first with the faster algorithm of GB 0719833.6 at a very low false alarm threshold. Any image pairs below the threshold are considered to be a match, and pairs above the threshold (that is not considered a match by GB 0719833.6 ) are then processed in accordance with the present invention. Firstly, the cumulative Hamming distance is applied and sets of three pairs above the threshold are considered to be different (no match). A geometric constraint is applied for sets of three pairs below the cumulative Hamming distance threshold. If the geometric measure for the set of three pairs is below the threshold level for the geometric measure the image pair are considered to be a match; otherwise they are considered to be different.
  • FIG. 6 An apparatus for implementing the present invention, according to an embodiment, for carrying the above methods is shown in Fig. 6 .
  • the implementation involves building a database of identifiers 240 for images stored in an image database 230.
  • the two databases 230 and 240 may be the same or separated databases.
  • the apparatus allows searching for an identifier 260 extracted from a query image 250 to find matches in the identifier database 140.
  • A, possibly ordered, list of images is returned to the user 290 or a query application.
  • the described embodiments identify image regions, from which the image signature is derived, based on feature detection, other techniques may be used to identify image regions that are representative of, or significant within, the image.
  • the described embodiment uses a feature detector based on an image gradient method, many other suitable methods may be used for the detection of interest points or regions in an image.
  • other techniques for selecting the best or strongest detected feature or interest points are possible.
  • any number of features points can be used in conjunction with distance measures.
  • other geometric constraints may be based on angles between feature points.
  • an affine model may be used to define the geometric constraint.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Library & Information Science (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Claims (7)

  1. Verfahren zum Ableiten einer Darstellung eines Bildes durch die Verarbeitung von Signalen, die dem Bild entsprechen, wobei das Verfahren umfasst:
    Identifizieren mehrerer Bereiche des Bildes, wobei jeder Bereich für das Bild kennzeichnend oder bezeichnend ist; und
    für jeden erkannten Bildbereich:
    Berechnen von mindestens einer Funktion von mindestens einem Teil des Bildbereichs; und
    Verwenden der Funktion zum Erhalten einer intermediären Darstellung von mindestens dem Teil des Bildbereichs, und
    Ableiten der Darstellung des Bildes von den intermediären Darstellungen von mindestens zwei der mehreren Bildbereiche,
    wobei das verfahren das Identifizieren mehrerer Bereiche des Bildes durch Folgendes umfasst:
    Identifizieren von Eigenschaftspunkten des Bildes, und
    Auswählen von Bildbereichen, die erkannten Eigenschaftspunkten des Bildes entsprechen,
    basierend auf einem vordefinierten Bildbereich um einen Eigenschaftspunkt eines Bildes, Anwenden einer Trace-Transformation mithilfe der Funktion auf jeden der Bildbereiche, um mehrere Darstellungen abzuleiten, die von Kreisfunktionen (circus functions) der Trace-Transformation mithilfe unterschiedlicher Funktionen extrahiert werden, um dadurch die Darstellung des Bildes zu erhalten.
  2. Verfahren zum Identifizieren eines Bildes umfassend:
    Ableiten einer Darstellung des Bildes mithilfe des Verfahrens nach einem vorangegangenen Anspruch, und
    zuordnen der Darstellung zum Bild.
  3. Verwendung für die Übermittlung oder den Empfang einer Darstellung, die mithilfe des Verfahrens nach einem der Ansprüche 1 bis 3 abgeleitet wird.
  4. Vorrichtung, die zum Ausführen des Verfahrens nach einem der Ansprüche 1 bis 3 eingerichtet ist.
  5. Vorrichtung nach Anspruch 4, die ein Steuergerät zum Steuern des Betriebs des Geräts zum Durchführen des Verfahrens nach einem der Ansprüche 1 bis 3 umfasst.
  6. Vorrichtung nach Anspruch 4, die ferner eines oder mehrere von Speichermitteln zum Speichern von Bildern und/oder Darstellungen von Bildern, beispielsweise einer Bilddatenbank und/oder einer Deskriptordatenbank, Anzeigemitteln und Bildauswahlmitteln umfasst.
  7. Computerspeichermedium, das Anweisungen umfasst, die, wenn sie von einem Computer ausgeführt werden, das verfahren nach einem der Ansprüche 1 bis 3 ausführen.
EP08775812.4A 2008-01-09 2008-06-30 Merkmalbasierte signaturen für bildidentifizierung Active EP2240887B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GBGB0800364.2A GB0800364D0 (en) 2008-01-09 2008-01-09 Feature-based signatures for image identification
PCT/GB2008/002262 WO2009087340A1 (en) 2008-01-09 2008-06-30 Feature-based signatures for image identification

Publications (2)

Publication Number Publication Date
EP2240887A1 EP2240887A1 (de) 2010-10-20
EP2240887B1 true EP2240887B1 (de) 2014-01-01

Family

ID=39144682

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08775812.4A Active EP2240887B1 (de) 2008-01-09 2008-06-30 Merkmalbasierte signaturen für bildidentifizierung

Country Status (7)

Country Link
US (1) US8660385B2 (de)
EP (1) EP2240887B1 (de)
JP (1) JP5431362B2 (de)
KR (1) KR101506060B1 (de)
CN (1) CN101911100B (de)
GB (1) GB0800364D0 (de)
WO (1) WO2009087340A1 (de)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0807411D0 (en) 2008-04-23 2008-05-28 Mitsubishi Electric Inf Tech Scale robust feature-based indentfiers for image identification
KR101353104B1 (ko) * 2009-04-14 2014-01-17 닛본 덴끼 가부시끼가이샤 이미지 시그니처 추출 장치
JP5862413B2 (ja) 2012-03-28 2016-02-16 富士通株式会社 情報変換規則生成プログラム、情報変換規則生成装置および情報変換規則生成方法
WO2013169303A1 (en) 2012-05-09 2013-11-14 Yknots Industries Llc Adaptive haptic feedback for electronic devices
US20150302270A1 (en) * 2012-08-07 2015-10-22 Metaio Gmbh A method of providing a feature descriptor for describing at least one feature of an object representation
US9361538B2 (en) * 2012-12-26 2016-06-07 Microsoft Technology Licensing, Llc Real time photometric edge description
WO2014138745A2 (en) * 2013-03-08 2014-09-12 Curalate, Inc. Method and system for clustering and classifying online visual information
US9939901B2 (en) 2014-09-30 2018-04-10 Apple Inc. Haptic feedback assembly
US10031925B2 (en) * 2014-10-15 2018-07-24 Thinkcx Technologies, Inc. Method and system of using image recognition and geolocation signal analysis in the construction of a social media user identity graph
US9390315B1 (en) * 2015-06-25 2016-07-12 A9.Com, Inc. Image match for featureless objects
CN108596883B (zh) * 2018-04-12 2021-07-13 福州大学 一种基于深度学习和距离约束的航拍图像防震锤滑移故障诊断方法
KR101963446B1 (ko) 2018-10-18 2019-03-28 남양진흥기업(주) 터널 시공방법 및 그 구조
CN113095156B (zh) * 2021-03-23 2022-08-16 西安深信科创信息技术有限公司 一种基于逆灰度方式的双流网络签名鉴定方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09330414A (ja) * 1996-06-12 1997-12-22 Yoshihiko Nomura 画像マッチング方法およびその装置
US7382897B2 (en) * 2004-04-27 2008-06-03 Microsoft Corporation Multi-image feature matching using multi-scale oriented patches
CN1267849C (zh) * 2004-07-02 2006-08-02 清华大学 基于断纹检测的指纹识别方法
US7375745B2 (en) * 2004-09-03 2008-05-20 Seiko Epson Corporation Method for digital image stitching and apparatus for performing the same
US7570794B2 (en) * 2005-09-02 2009-08-04 Gm Global Technology Operations, Inc. System and method for evaluating a machined surface of a cast metal component
EP1912160B1 (de) 2006-10-11 2012-05-16 Mitsubishi Electric R&D Centre Europe B.V. Bilddeskriptor zur Bilderkennung
GB0700468D0 (en) 2007-01-10 2007-02-21 Mitsubishi Electric Inf Tech Improved image identification
GB0712388D0 (en) 2007-06-26 2007-08-01 Mitsubishi Electric Inf Tech High Performance Image Identification
GB0719833D0 (en) 2007-10-10 2007-11-21 Mitsubishi Electric Inf Tech Enhanced image identification

Also Published As

Publication number Publication date
KR101506060B1 (ko) 2015-03-25
GB0800364D0 (en) 2008-02-20
CN101911100B (zh) 2015-08-26
EP2240887A1 (de) 2010-10-20
JP2011511340A (ja) 2011-04-07
WO2009087340A1 (en) 2009-07-16
JP5431362B2 (ja) 2014-03-05
CN101911100A (zh) 2010-12-08
US8660385B2 (en) 2014-02-25
KR20100112155A (ko) 2010-10-18
US20110038541A1 (en) 2011-02-17

Similar Documents

Publication Publication Date Title
US8660385B2 (en) Feature-based signatures for image identification
US8831355B2 (en) Scale robust feature-based identifiers for image identification
Wu et al. Busternet: Detecting copy-move image forgery with source/target localization
CN101523415B (zh) 导出图像的表示的方法、对图像进行识别和比较的方法及其装置
Anand et al. A copy move forgery detection to overcome sustained attacks using dyadic wavelet transform and SIFT methods
Muzaffer et al. A fast and effective digital image copy move forgery detection with binarized SIFT
Nawaz et al. Image authenticity detection using DWT and circular block-based LTrP features
Kaur et al. Efficient hybrid passive method for the detection and localization of copy-moveand spliced images
Al-Shamasneh et al. Image splicing forgery detection using feature-based of sonine functions and deep features
Tang et al. Structural feature-based image hashing and similarity metric for tampering detection
Kumar et al. Non-overlapping block-level difference-based image forgery detection and localization (NB-localization)
Shaobao et al. Copy-move forgery detection via dimensionality reduction and double quantization feature
Saber et al. Digital image forgery detection by utilize combined feature extraction techniques
Li et al. An image copy move forgery detection method using QDCT
Dixit et al. Copy–move image forgery detection using gray-tones with texture description
Shallal et al. Enhanced Detection of Copy-Move Forgery by Fusing Scores From Handcrafted and Deep Learning-Based Detection Systems
Krishnalal et al. Advances in Copy Move Forgery Detection in Digital Images: A Comparative Examination of Conventional Approaches and Deep Learning Based Models
Sewan et al. Forged copy-move recognition using convolutional neural network
Funda A hybrid copy-move forgery detection method with FAST and SURF
Srivastava et al. A Review of Copy-Move Image Forgery Detection
Oyiza et al. An improved discrete cosine transformation block based scheme for copy-move image forgery detection
Ankilla et al. Forgery and Tampering Detection for Images Using Deep Learning
Hamidon et al. An Analysis of Copy-Move Forgery Detection based on Discrete Cosine Transform against JPEG Compression and Brightness Changes Attack
Mahdi et al. A Review Study on Forgery and Tamper Detection Techniques in Digital Images
Fattahi et al. Detection of copy-move forgery in digital images using scale invariant feature transform algorithm and the spearman relationship

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100806

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20120605

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20130710

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: MITSUBISHI ELECTRIC CORPORATION

Owner name: MITSUBISHI ELECTRIC R & D CENTRE EUROPE B.V.

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008029651

Country of ref document: DE

Effective date: 20140213

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 647869

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140215

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20140101

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 647869

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140101

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140501

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140502

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008029651

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

26N No opposition filed

Effective date: 20141002

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008029651

Country of ref document: DE

Effective date: 20141002

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140630

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140630

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140630

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140401

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140402

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140101

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20080630

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20190711 AND 20190717

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602008029651

Country of ref document: DE

Owner name: RAKUTEN GROUP, INC., JP

Free format text: FORMER OWNER: MITSUBISHI ELECTRIC CO., TOKYO, JP

Ref country code: DE

Ref legal event code: R081

Ref document number: 602008029651

Country of ref document: DE

Owner name: RAKUTEN GROUP, INC., JP

Free format text: FORMER OWNER: MITSUBISHI ELECTRIC CORP., TOKYO, JP

Ref country code: DE

Ref legal event code: R082

Ref document number: 602008029651

Country of ref document: DE

Representative=s name: MAUCHER JENKINS PATENTANWAELTE & RECHTSANWAELT, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602008029651

Country of ref document: DE

Owner name: RAKUTEN, INC., JP

Free format text: FORMER OWNER: MITSUBISHI ELECTRIC CORP., TOKYO, JP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602008029651

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G06K0009460000

Ipc: G06V0030180000

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602008029651

Country of ref document: DE

Representative=s name: DENNEMEYER & ASSOCIATES RECHTSANWALTSGESELLSCH, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602008029651

Country of ref document: DE

Owner name: RAKUTEN GROUP, INC., JP

Free format text: FORMER OWNER: RAKUTEN, INC., TOKYO, JP

Ref country code: DE

Ref legal event code: R082

Ref document number: 602008029651

Country of ref document: DE

Representative=s name: DENNEMEYER & ASSOCIATES S.A., DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602008029651

Country of ref document: DE

Representative=s name: DENNEMEYER & ASSOCIATES RECHTSANWALTSGESELLSCH, DE

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250520

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20250520

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20250520

Year of fee payment: 18