EP4612642A1 - Procede et systeme de correction de bruit fixe d'une image - Google Patents
Procede et systeme de correction de bruit fixe d'une imageInfo
- Publication number
- EP4612642A1 EP4612642A1 EP23817304.1A EP23817304A EP4612642A1 EP 4612642 A1 EP4612642 A1 EP 4612642A1 EP 23817304 A EP23817304 A EP 23817304A EP 4612642 A1 EP4612642 A1 EP 4612642A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- images
- fixed noise
- correction parameter
- image
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
- H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
- H04N25/673—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction by using reference sources
- H04N25/674—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction by using reference sources based on the scene itself, e.g. defocusing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Definitions
- the present invention relates to a method for correcting fixed noise in at least one image. It also relates to a system for correcting fixed noise in at least one image.
- Document US9900526 proposes using a visual shutter to calibrate a scene. This visual shutter corresponds to an object that appears almost uniform and covers a large part of the camera.
- Document US9208542 proposes a method for reducing noise at the pixel level in thermal images. This method allows you to count the number of times a pixel is smaller or larger than its neighboring pixels and, accordingly, to correct the pixel in question.
- the aim of the present invention is to resolve at least one of the aforementioned drawbacks.
- This objective is achieved with a fixed noise correction method in at least one image.
- the process includes the following steps:
- the present invention is based on several images acquired from different scenes taken with the same image sensor in a short time. This is achieved with a rotating image sensor which can reconstruct a panorama using images taken while rotating the image sensor: each image corresponds to a different piece or sector of panoramic scene.
- An acquisition frequency E of the image sensor is greater than a fixed noise change frequency between the acquisition of two images. The acquisition frequency E is sufficient so that any changes in the fixed noise parameters are very low.
- the invention then makes it possible to avoid the defects of traditional methods based on regularity, because each fixed noise correction is estimated not from one, but from several acquired images.
- the acquired images are substantially regular over a majority of the image, and that in each pixel one can find one or more very regular images locally around this pixel among the acquired images.
- regular the fact that the intensity difference in the image considered does not undergo a sudden change over a majority of the image.
- An image is also said to be "locally regular” if the pixel values in a small neighborhood can be described with a simple mathematical formula, for example the equation of a plane, with few errors.
- Natural images are generally considered to be, with high probability, regular in a majority of pixels. This corresponds, for example, to flattened areas of the image, or with slight gradients. High contrast areas, such as the edges of objects, or heavily textured areas, such as a pile of branches, are not considered smooth.
- non-uniformity correction parameters can be re-estimated frequently enough to compensate for variations in non-uniformity of the sensor.
- non-uniformity variations we mean variations in fixed noise. It is considered here, given the high image acquisition frequency, that the value of the fixed noise in each image is substantially identical. By “substantially identical”, we mean that the fixed noise has a negligible evolution between the first and the last of the N images acquired.
- the at least one fixed noise correction parameter determined by the processing module according to the invention minimizes a functional of the form
- G, O and f. are image-sized matrices, and * is the Hadamard product.
- G and O correspond respectively to a gain and offset matrix.
- the values of the matrices O and G are the parameters which are estimated.
- a variable number of parameters can be determined from the N images acquired. In the case where only O is estimated, the invention gives more robust and rapid results. In the case where several parameters are estimated, the signal is better represented and the correction is more efficient in practice.
- the values of the matrices O and G are the parameters which are estimated in the present case.
- the step of determining at least one fixed noise correction parameter by the processing module may comprise the following step:
- the step of determining at least one fixed noise correction parameter by the processing module can be carried out via a direct implementation.
- Direct implementation may include the following steps:
- the first filtering of the N images can include the use of a high pass filter.
- a judicious choice of G and h allows a direct implementation, where a succession of filtering makes it possible to directly obtain the minimum of the functional. At least one correction parameter can be determined in this way.
- the judicious choice of G and h corresponds to
- ⁇ is the matrix of the size of the image containing 1 in each box
- F is a high pass filter on the acquired image, such as subtracting the median of neighboring pixels from the value of each pixel
- a direct implementation of the minimization of the functional then consists first of filtering the N images according to the high pass filter F.
- the high pass filter corresponds to a spatial filter. Then the results of these filterings are filtered by the median operator along the N images.
- the step of determining at least one fixed noise correction parameter by a processing module may also include the following step:
- This temporal smoothing step makes it possible to refine the results of the determined correction parameter(s).
- a system configured for the correction of fixed noise of at least one image, the system comprising:
- At least one rotating image sensor arranged to acquire at least N substantially distinct and regular images, so that the combination of distinct images forms a panorama of a surrounding scene, the fixed noise included in each of the N images acquired being substantially identical
- a processing module arranged and/or programmed to determine at least one fixed noise correction parameter from the N images acquired by the image sensor, F at least one correction parameter minimizing a functional, and for correct the fixed noise in each of the N images acquired from said estimate of F at least one determined correction parameter.
- a computer program product comprising instructions which, when the program is executed by a computer, lead it to implement the steps of the method according to 'invention.
- FIG.l is a schematic profile sectional view of a system according to the invention according to one embodiment.
- FIG.2 is a flowchart of the method of the invention according to one embodiment.
- variants of the invention comprising only a selection of characteristics described or illustrated subsequently isolated from the other characteristics described or illustrated (even if this selection is isolated within a sentence including these other characteristics), if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.
- This selection includes at least one preferably functional characteristic without structural details, and/or with only part of the structural details if this part only is sufficient to confer a technical advantage or to differentiate the invention from the state of the art. anterior.
- the system comprises a rotating image sensor 1 arranged to acquire N substantially distinct and regular images, so that the combination of the distinct images forms a panorama of a surrounding scene, the fixed noise included in each of the N acquired images being substantially identical .
- the system may include several image sensors, whether rotatable or not.
- the image sensor 1 is rotated in order to acquire several images of the environment by following a panorama.
- the acquired images are distinct from each other, that is to say that each acquired image images a sector of the panorama.
- An image is defined as a set of pixel values measured during capture by an image sensor 1.
- a scene corresponds to a physical reality in front of the image sensor 1 and which is captured by said sensor of images 1 which produces an image.
- the rotary image sensor 1 corresponds, in a preferred embodiment, to an infrared rotary image sensor 1.
- the image sensor is made up of a set of sensors each corresponding to a pixel. Indeed, depending on the model, the infrared rotary image sensor 1 acquires multiple images to reconstruct a panorama.
- a panorama therefore corresponds to a set of scenes, or images, placed end to end to produce a 360 degree observation, preferably at least 90 degrees.
- the rotating image sensor 1 rotates at a rotation frequency typically from 2Hz to 0.5Hz.
- the image sensor 1 can rotate clockwise and vice versa.
- the rotary image sensor 1 acquires images according to a minimum acquisition frequency F.
- F can take values such as 7.5Hz, 45Hz or 64Hz.
- the invention can be applied to other systems moving or rotating the system at a sufficiently high frequency that multiple images are acquired according to the presented embodiment.
- the system also includes a processing module (not shown in the figure) arranged and/or programmed to determine at least one fixed noise correction parameter from the N images acquired by the image sensor, F at least a correction parameter minimizing a functional, and to correct the fixed noise in each of the N images acquired from said estimate of F at least one determined correction parameter.
- a processing module (not shown in the figure) arranged and/or programmed to determine at least one fixed noise correction parameter from the N images acquired by the image sensor, F at least a correction parameter minimizing a functional, and to correct the fixed noise in each of the N images acquired from said estimate of F at least one determined correction parameter.
- the method for correcting fixed noise in at least one image comprises three steps (El to E3).
- Step El corresponds to an acquisition by the image sensor 1 of N substantially distinct and regular images, so that the combination of the distinct images forms a panorama of a surrounding scene, the fixed noise included in each of the N acquired images being substantially identical.
- the acquisition of images is treated in the same way as that presented [Fig.l].
- Step E2 corresponds to the determination of at least one fixed noise correction parameter by a processing module 2 from the N acquired images, F at least one correction parameter minimizing a functional.
- the processing module 2 is configured to directly determine at least one fixed noise correction parameter by minimizing a functional.
- the processing module 2 is then configured to estimate a set of parameters for all of the N acquired images, the set of parameters comprising a gain value and an offset value.
- the functional returns a number, often positive definite, which is linked to the quantity of fixed noise present in the group of N acquired images considered.
- the objective is to reduce this value to eliminate the fixed noise.
- the functionality varies depending on the correction parameters defined (offset, gain, etc.).
- the value of the functional corresponds to the value of a function linked to the quantity of noise present in the image in which these corrections are applied.
- the set of correction parameters determined by the processing module 2 minimizes this functional.
- the functional is often composed of a data attachment term and a regularization term.
- the choice of the two terms reflects regularity assumptions made about the scenes that we expect to observe in practice.
- a “total variation” term is used as a regularization term, and a quadratic penalization, on the norm of the corrections gains (minus one) and estimated offsets, as an attachment to the data.
- the data attachment term reflects that the intensities of the images f.
- correction parameters are applied to the image, here an offset parameter ( of order 0) and a gain parameter (of order 1). Parameters that minimize said quantity are sought.
- the image index i ranging from 1 to N
- the fixed noise is assumed to be fixed on this set of acquired images.
- the total variations are summed for the different images because all of the images must have their discontinuities reduced by the correct fixed noise correction.
- Other aggregation methods could also be used such as the minimum or the median.
- the fixed noise varies slightly with time, the corrective parameters are therefore updated for a new group of images.
- the correction parameters are estimated from several images.
- new group we mean a batch of N images acquired during a rotation by the image sensor. Once the correction has been estimated for N images, it is assumed that, for the following N images, the fixed noise will have changed very little. We can therefore distribute determined correction parameters and adapt them.
- the processing module 2 is configured to determine at least one fixed noise correction parameter from the N images acquired via a direct implementation.
- This embodiment consists of taking each of the acquired images individually and applying processing equivalent to a high-pass filter on each of the N acquired images. More precisely, this high pass filter consists of taking, for each pixel of the acquired image, the median value of its eight direct neighboring pixels and subtracting it from the value of the pixel in question.
- the result of the high pass filter on each image contains several components, such as the natural difference in the content of the scene in each pixel compared to the neighbors and the noise composed of a temporally dependent component (fixed noise) and a temporally independent component (photon noise and electronic).
- a number of other alternative filters are possible. For example, for the first determination step, we can use the subtraction of the neighbor average or the subtraction of any other low-pass filter (which is equivalent to applying a high-pass filter).
- Machine learning can also be used to independently predict the correction at each pixel on each image.
- step E2 also includes a step of filtering the N images processed to extract at least one correction parameter common to each of the N images acquired. Filtering makes it possible to greatly reduce the contribution of the observed scenes, which implies that the scenes must correspond to varied scenes. This is the case here since, according to the invention, a rotating image sensor is used, each image acquired covering a different angle of view and with little overlap with the previous image. The result of the processing step with a high-pass filter on each acquired image is therefore filtered to produce a single image, i.e. a single estimate at each pixel. More precisely, a median at each pixel of all the images is produced, recovered and serves as a correction parameter.
- This second filtering has the effect of greatly reducing in each pixel the contribution of the temporally independent noise and the observed scenes, while the fixed noise is preserved.
- the common correction parameter therefore corresponds to an estimate of the fixed noise present in the acquired images.
- other filters than that presented in this embodiment are possible such as averaging after removing the elements beyond the extreme quantiles or filtering the value appearing the most.
- This second filtering step is equivalent to seeking a consensus between the independent estimates of the correction parameters of each image.
- temporal smoothing of F at least one common correction parameter is used in order to refine the results of F at minus a common correction parameter.
- Step E3 of the method corresponds to the correction of the fixed noise included in each of the N images acquired by the processing module from F at least one determined fixed noise correction parameter.
- the correction is applied in the same way. This involves applying the correction parameter(s) determined to the N images, i.e. multiplying each image by the gain and adding the determined offset. There correction can be applied following a calibration of the image sensor predetermined at the factory for example.
- each of the means of the device according to the invention previously described is a technical means.
- each of the means of the device according to the invention previously described may comprise at least one computer, a central or calculation unit, an analog electronic circuit (preferably dedicated), a digital electronic circuit (preferably dedicated), and/or a microprocessor (preferably dedicated), and/or software means.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| FR2213248A FR3143163B1 (fr) | 2022-12-13 | 2022-12-13 | Procede et systeme de correction de bruit fixe d’une image |
| PCT/EP2023/082076 WO2024125939A1 (fr) | 2022-12-13 | 2023-11-16 | Procede et systeme de correction de bruit fixe d'une image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4612642A1 true EP4612642A1 (fr) | 2025-09-10 |
Family
ID=85937110
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP23817304.1A Pending EP4612642A1 (fr) | 2022-12-13 | 2023-11-16 | Procede et systeme de correction de bruit fixe d'une image |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US20260010983A1 (fr) |
| EP (1) | EP4612642A1 (fr) |
| FR (1) | FR3143163B1 (fr) |
| WO (1) | WO2024125939A1 (fr) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20250200963A1 (en) * | 2023-12-18 | 2025-06-19 | Nvidia Corporation | Automatic determination of noise profiles for image sensors |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7880777B2 (en) * | 2005-05-26 | 2011-02-01 | Fluke Corporation | Method for fixed pattern noise reduction in infrared imaging cameras |
| US7995859B2 (en) | 2008-04-15 | 2011-08-09 | Flir Systems, Inc. | Scene based non-uniformity correction systems and methods |
| US9208542B2 (en) | 2009-03-02 | 2015-12-08 | Flir Systems, Inc. | Pixel-wise noise reduction in thermal images |
| US9900526B2 (en) | 2011-06-10 | 2018-02-20 | Flir Systems, Inc. | Techniques to compensate for calibration drifts in infrared imaging devices |
| EP2719165B1 (fr) * | 2011-06-10 | 2018-05-02 | Flir Systems, Inc. | Techniques de correction de non-uniformité pour dispositifs d'imagerie infrarouge |
| US9973692B2 (en) * | 2013-10-03 | 2018-05-15 | Flir Systems, Inc. | Situational awareness by compressed display of panoramic views |
| US10417745B2 (en) * | 2016-06-28 | 2019-09-17 | Raytheon Company | Continuous motion scene based non-uniformity correction |
| CN112424821B (zh) * | 2018-05-15 | 2024-04-23 | 泰立戴恩菲力尔商业系统公司 | 基于由旋转成像器捕获的图像的全景图像构建 |
-
2022
- 2022-12-13 FR FR2213248A patent/FR3143163B1/fr active Active
-
2023
- 2023-11-16 WO PCT/EP2023/082076 patent/WO2024125939A1/fr not_active Ceased
- 2023-11-16 EP EP23817304.1A patent/EP4612642A1/fr active Pending
- 2023-11-16 US US19/137,114 patent/US20260010983A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| FR3143163B1 (fr) | 2024-12-20 |
| FR3143163A1 (fr) | 2024-06-14 |
| WO2024125939A1 (fr) | 2024-06-20 |
| US20260010983A1 (en) | 2026-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2457379B1 (fr) | Procédé d'estimation d'un défaut d'un système de capture d'images et systèmes associés | |
| CN108541374B (zh) | 一种图像融合方法、装置及终端设备 | |
| US8755628B2 (en) | Image de-hazing by solving transmission value | |
| EP1694058A1 (fr) | Procédé et dispositif de capture d'images comprenant une mesure de mouvements locaux | |
| EP1749278A2 (fr) | Procede de traitement de donnees d'image, par réduction de bruit d'image, et camera integrant des moyens de mise en oeuvre du procede | |
| Cao et al. | Spatially adaptive column fixed-pattern noise correction in infrared imaging system using 1D horizontal differential statistics | |
| Burge et al. | Optimal defocus estimates from individual images for autofocusing a digital camera | |
| EP3200153A1 (fr) | Procédé de détection de cibles au sol et en mouvement dans un flux vidéo acquis par une caméra aéroportée | |
| FR3019352A1 (fr) | Systeme d'identification d'un modele d'appareil photographique associe a une image compressee au format jpeg, procede, utilisations et applications associes | |
| EP4612642A1 (fr) | Procede et systeme de correction de bruit fixe d'une image | |
| Liu et al. | Efficient polarization demosaicking via low-cost edge-aware and inter-channel correlation | |
| EP3301644B1 (fr) | Procédé de construction d'une carte de profondeur d'une scène et/ou d'une image entièrement focalisée | |
| Chittilappilly et al. | SVM based defect detection for industrial applications | |
| FR3078427A1 (fr) | Detection dynamique de lumiere parasite dans une image numerique | |
| WO2024226097A1 (fr) | Système et procédé d'apprentissage auto-supervisé de réseaux de restauration d'image | |
| EP3963539B1 (fr) | Procede et dispositif pour retirer la remanence dans une image infrarouge d'une scene changeante | |
| FR3140724A1 (fr) | Detection et correction de pixels defectueux | |
| Ratliff et al. | Scene-based correction of fixed pattern noise in hyperspectral image data using temporal reordering | |
| Zheng et al. | An improved adaptive preprocessing method for TDI CCD images | |
| FR3025683A1 (fr) | Procede et dispositif d'estimation de parametres d'egalisation d'image numerique | |
| FR3095064A1 (fr) | Procédés et dispositifs de levée de doute biométrique | |
| EP4066204B1 (fr) | Procédé et dispositif de traitement d'images | |
| EP3963540B1 (fr) | Procede et dispositif pour retirer la remanence dans une image infrarouge d'une scene statique | |
| Song et al. | Extreme Low-Light Image Denoising | |
| EP4684352A1 (fr) | Procede de traitement d'une image brute, produit programme d'ordinateur et dispositif de traitement associes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20250602 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20260219 |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) |