WO2021218765A1 - 图像去噪方法及装置、电子设备以及存储介质 - Google Patents

图像去噪方法及装置、电子设备以及存储介质 Download PDF

Info

Publication number
WO2021218765A1
WO2021218765A1 PCT/CN2021/088927 CN2021088927W WO2021218765A1 WO 2021218765 A1 WO2021218765 A1 WO 2021218765A1 CN 2021088927 W CN2021088927 W CN 2021088927W WO 2021218765 A1 WO2021218765 A1 WO 2021218765A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
training
network
convolutional layer
initial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/088927
Other languages
English (en)
French (fr)
Inventor
艾吉松
徐科
孔德辉
王宁
刘欣
游晶
朱方
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sanechips Technology Co Ltd
Original Assignee
Sanechips Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sanechips Technology Co Ltd filed Critical Sanechips Technology Co Ltd
Priority to EP21795374.4A priority Critical patent/EP4145384B1/en
Priority to US17/921,271 priority patent/US12394023B2/en
Publication of WO2021218765A1 publication Critical patent/WO2021218765A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • This application relates to the field of image processing technology.
  • Image denoising has always been a very important part of the field of image processing, especially in recent years, the meteorological light-level shooting in the surveillance field.
  • the lighting conditions in the dark night environment are very bad, and the sensor is not sensitive enough, resulting in a lot of noise in the captured pictures. Therefore, the captured images or videos have lower resolution compared with those captured under good lighting conditions; This not only affects the visual effect, but also affects the accuracy of the recognition result for the image or video from which the moving target needs to be recognized. Therefore, there is an urgent need for a denoising method to improve image quality.
  • One aspect of the embodiments of the present application provides an image denoising method, including: obtaining an image to be processed; and inputting the image to be processed into an image denoising model to obtain a denoised image; wherein the image denoising model is A model that combines U-shaped network, residual network and dense network.
  • An aspect of the embodiments of the present application provides an image denoising device, including: a first acquisition module configured to acquire an image to be processed; and a second acquisition module configured to input the image to be processed into an image denoising model
  • the image after denoising is obtained; among them, the image denoising model is a model formed by combining U-shaped network, residual network and dense network.
  • An aspect of the embodiments of the present application provides an electronic device, including: one or more processors; a memory configured to store one or more programs; when the one or more programs are executed by the one or more processors , So that the one or more processors implement the image denoising method provided in the embodiment of the present application.
  • One aspect of the embodiments of the present application provides a storage medium that stores a computer program, and the computer program is executed by a processor to implement the image denoising method provided in the embodiments of the present application.
  • FIG. 1 is a flowchart of the image denoising method provided by this application.
  • FIG. 2 is a schematic diagram of a structure of the image denoising model provided by this application.
  • FIG. 3 is a schematic diagram of a structure of the first dense residual sub-module in the image denoising model provided by this application.
  • FIG. 4 is another flowchart of the image denoising method provided by this application.
  • FIG. 5A is a flow chart of obtaining the first training noise image and the corresponding first training true value image provided by this application.
  • Fig. 5B is a flow chart of generating the second training truth image provided by this application.
  • FIG. 6 is a schematic diagram of a structure of the generation network in the confrontation network provided by this application.
  • FIG. 7 is a schematic structural diagram of the discrimination network in the confrontation network provided by this application.
  • FIG. 8 is a schematic diagram of a structure of the image denoising device provided by this application.
  • FIG. 9 is a schematic diagram of another structure of the image denoising device provided by this application.
  • FIG. 10 is a schematic diagram of another structure of the image denoising device provided by this application.
  • FIG. 11 is a schematic diagram of a structure of an electronic device provided by this application.
  • Image denoising is essential to the improvement of image quality.
  • a better adaptive denoising algorithm based on neural network is a supervised learning type neural network algorithm.
  • the training sample includes a pair of input samples and output samples, and the parameters in the neural network are updated through the gradient descent algorithm, so that the output of the training sample input through the neural network is close to the true value sample.
  • FIG. 1 is a flowchart of the image denoising method provided by this application.
  • the image denoising method can be applied to scenes in which images are denoised, and can be executed by an image denoising device.
  • the image denoising device can be implemented by software and/or hardware, and the image denoising device can be integrated into an electronic device.
  • the image denoising method provided by the present application may include step 101 and step 102.
  • step 101 an image to be processed is acquired.
  • the image to be processed is input into the image denoising model, and the denoised image is obtained.
  • the image denoising model can be a model formed by combining a U-shaped network, a residual network, and a dense network.
  • the image to be processed may be an image captured by a front-end monitoring device, or an image frame in a video captured by a front-end monitoring device.
  • the image to be processed in this application can also be an image in other fields, for example, a medical image.
  • the image denoising model in this application may be a pre-trained model.
  • This model is a model formed by combining U-shaped network, residual network and dense network.
  • the U-shaped network in this application that is, Unet, refers to a U-shaped network including feature extraction, that is, an encoding part, and upsampling, that is, a decoding part.
  • the residual network in this application refers to a network that includes a direct mapping part and a residual part, and the direct mapping part and the residual part perform an addition operation.
  • the dense network in this application refers to a network that includes a direct mapping part and a residual part, and the direct mapping part and the residual part perform channel connection operations.
  • the image denoising model in this application can combine the characteristics of the residual network and the dense network to make better use of the deep and shallow features of the image to be processed; it can remove the noise while retaining the details of the image to be processed as much as possible Therefore, on the basis of achieving better denoising performance, the image quality after denoising is taken into account.
  • FIG. 2 is a schematic diagram of a structure of the image denoising model provided by this application.
  • the image denoising model provided by this application may include: an input layer, a first convolutional layer, at least one dense residual module, a dense residual block, at least one upsampling module, and a second volume connected in sequence.
  • Build-up layer, third convolutional layer, and output layer may include:
  • the output terminal of the input layer also performs a subtraction operation with the output terminal of the third convolutional layer, and the result of the subtraction operation is input to the input terminal of the output layer.
  • the output terminal of the first convolutional layer also performs an addition operation with the output terminal of the second convolutional layer, and the result of the addition operation is input to the input terminal of the third convolutional layer.
  • the dense residual module includes a first dense residual sub-module and a convolution sub-module that are sequentially connected
  • the up-sampling module includes an up-sampling sub-module and a second dense residual sub-module that are sequentially connected.
  • the output terminal of the first dense residual sub-module also performs an addition operation with the input terminal of the up-sampling sub-module.
  • the number of dense residual modules may be 4, and the number of up-sampling modules may be 4.
  • mapping relationship between the dense residual modules and the up-sampling modules can be established. According to the mapping relationship, the output terminal of the first dense residual sub-module in the dense residual module and the input terminal of the up-sampling sub-module in the corresponding up-sampling module in the mapping relationship are added.
  • the resolution of the image to be processed input by the input layer is N*N, and the number of channels is 3.
  • the convolution kernel of the first convolution layer is 3*3, and the number of channels becomes 128.
  • the convolution kernel in the convolution sub-module is also 3*3, with a span of 2.
  • the convolution kernels of the second convolution layer and the third convolution layer are both 3*3.
  • the number of channels in the second convolutional layer is 128.
  • the number of channels in the third convolutional layer is 3.
  • FIG. 3 is a schematic structural diagram of the first dense residual sub-module in the image denoising model provided by this application.
  • the first dense residual sub-module may include: a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, and a seventh convolutional layer that are sequentially connected.
  • the input end of the fourth convolutional layer is also added with the input end of the seventh convolutional layer.
  • the input end of the fifth convolutional layer and the input end of the fourth convolutional layer perform a fusion operation.
  • the input end of the sixth convolutional layer and the output end of the fourth convolutional layer and the input end of the fourth convolutional layer perform a fusion operation.
  • the input end of the seventh convolutional layer, the input end of the fourth convolutional layer, the output end of the fifth convolutional layer, and the output end of the fourth convolutional layer perform a fusion operation.
  • the fusion operation in this application refers to the channel joining operation in a dense network.
  • the structure of the second dense residual sub-module and the dense residual block is the same as the structure of the first dense residual sub-module, and will not be repeated here.
  • the convolution kernels of the fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer are all 3*3.
  • the convolution kernel of the seventh convolution layer is 1*1.
  • M denote the number of channels in the fourth convolutional layer, fifth convolutional layer, sixth convolutional layer, and seventh convolutional layer.
  • the image denoising method provided in this application may include: obtaining an image to be processed, inputting the image to be processed into an image denoising model, and obtaining a denoised image, where the image denoising model is a combination of U-shaped network and residual network And the model of dense network formation.
  • the image denoising model in this application can combine the characteristics of the residual network and the dense network to make better use of the deep and shallow features of the image to be processed, so as to remove the noise while retaining the details of the image to be processed as much as possible Therefore, on the basis of achieving better denoising performance, the image quality after denoising is taken into account.
  • FIG. 4 is another flowchart of the image denoising method provided by this application.
  • the embodiment of the present application describes in detail the steps of how to train the image denoising model on the basis of the implementable manner and various optional solutions shown in FIG. 1.
  • the embodiment of the present application only shows the steps of training an image denoising model.
  • the image denoising method provided by the embodiment of the present application may include the following steps 401 to 404.
  • step 401 a first training noise image and a corresponding first training ground truth image are acquired.
  • the first training noise image and the corresponding first training truth image can be acquired through actual shooting.
  • the first training noise image and the corresponding first training ground truth image can be obtained through actual shooting and confrontation of the network.
  • FIG. 5A is a flow chart of obtaining the first training noise image and the corresponding first training true value image provided by this application. As shown in FIG. 5A, the acquisition process may include the following steps 501-505.
  • a second training true value image is generated according to multiple target images taken under the first preset light source brightness.
  • the first preset light source brightness may be 200 lux.
  • the camera module is enabled, and the noise removal algorithm and the bad pixel removal algorithm in the image signal processing module are all turned off.
  • the number of target images can be 200.
  • the average image of the multiple target images may be determined based on the multiple target images, and the average of the multiple target images may be determined.
  • the image is used as the second training ground truth image.
  • the steps of removing bad pixels, aligning the intensity, and obtaining the average value of the multiple target images may be performed to generate the final second training true value image. This embodiment will be described in detail later.
  • step 502 multiple images taken by randomly adjusting the brightness of the light source within the range of the brightness of the second preset light source and the brightness of the third preset light source are used as the second training noise image.
  • the second preset light source brightness is less than the third preset light source brightness
  • the third preset light source brightness is less than the first preset light source brightness
  • multiple images taken by randomly adjusting the brightness of the light source within the range of the brightness of the second preset light source and the brightness of the third preset light source are used as the second training noise image.
  • the second preset light source brightness may be 0.1 lux, and the third preset light source brightness may be 100 lux.
  • the number of second training noise images may be 200.
  • the camera parameters when shooting each second training noise image for example, analog gain and digital gain, can be recorded.
  • the scene can be changed, and multiple target images and the second training noise image in different scenes can be shot.
  • a second training ground truth image is generated.
  • one second training ground truth image corresponds to multiple second training noise images.
  • step 503 according to the second training ground truth image and the second training noise image, the initial confrontation network is trained, and the confrontation network formed by the final training is obtained.
  • the confrontation network in the embodiment of the present application may include a generation network and a decision network. Both the generation network and the discrimination network in the confrontation network can be U-shaped networks.
  • the initial generation network in the initial confrontation network is used to generate a noise image
  • the initial decision network in the initial confrontation network is used to determine the similarity between the noise image output by the initial generation network and the actual second training noise image.
  • the initial confrontation network is independently and iteratively trained until convergence.
  • the specific process can include the following steps 1-step 3.
  • step 1 input the camera parameters corresponding to the normal distribution noise, the second training ground truth image, and any second training noise image to the initial generation network in the initial confrontation network to obtain the output noise image.
  • step 2 input the noise image, the second training noise image, the camera parameters corresponding to the second training noise image, and the second training truth value image into the initial discrimination network in the initial confrontation network to obtain the initial discrimination The output probability of the network.
  • step 3 according to the output probability, the camera parameters corresponding to the second training noise image, the second training ground truth image, the second training noise image and the noise image, determine the loss function of the initial discriminant network and the loss of the initial generation network Function; when it is determined that the initial confrontation network does not converge according to the loss function of the initial discriminant network and the loss function of the initial generation network, alternate execution and return to execute the normal distribution noise, the second training true value image, and the second training noise image corresponding
  • the camera parameters are input to the initial generation network in the initial confrontation network, the steps of obtaining the output noise image, and returning to the noise image, the second training noise image, the camera parameters corresponding to the second training noise image, and the second training truth
  • the value image is input into the initial discriminant network in the initial confrontation network to obtain the output probability of the initial discriminant network, until the convergence of the confrontation network is determined according to the loss function of the initial discriminant network and the loss function of the initial generation network, and the converged confrontation The network is determined as the confrontation network formed by the final training
  • the output probability of the initial discrimination network is a value between 0 and 1, which represents the probability of true and false noise images.
  • 0 represents the noise image generated by the generating network
  • 1 represents the second training noise image actually collected.
  • N c is the conditional signal for generating the noise image, including the analog gain, digital gain and the second training true value image of the camera;
  • N f is the noise image generated by the generating network,
  • N r is the second training noise image that is actually collected,
  • D( *) represents the output of the discriminant network,
  • E(*) represents the average value.
  • step 504 the third training ground truth image obtained in advance is input into the confrontation network, and the third training noise image output by the generation network of the confrontation network is obtained.
  • the third training ground truth image obtained in advance may be input into the confrontation network to obtain the third training noise image output by the generation network of the confrontation network.
  • both the second training noise image and the third training noise image are used as the first training image, and the second training truth value image and the third training truth value image are both used as the corresponding first training truth value image.
  • the second training noise image actually collected and the third training noise image generated by the adversarial network can be used as the first training noise image of the training image denoising model, and the second training true value image And the third training truth value image is used as the corresponding first training truth value image.
  • the adversarial network to generate training sample pairs greatly enriches the number of training samples. Based on these rich training samples, the image denoising model can be fully trained, thereby improving the denoising performance of the image denoising model.
  • step 402 the first training noise image is input to the initial image denoising model, and the output result is obtained.
  • step 403 a loss function is determined according to the output result and the corresponding first training truth image.
  • step 404 when the loss function is greater than the preset threshold, adjust the network structure and network parameters of the initial image denoising model according to the loss function, determine the updated image denoising model, and use the updated image denoising model as the new
  • the initial image denoising model of returns to the step of inputting the first trained noise image into the initial image denoising model and obtaining the output result until the loss function is less than or equal to the preset threshold, and when the loss function is less than or equal to the preset threshold
  • the image denoising model of is determined as the image denoising model.
  • Steps 402 to 404 are the process of iteratively training the image denoising model.
  • the image denoising model based on these first training noise images can significantly improve the denoising performance and generalization ability of the image denoising model.
  • Fig. 5B is a flow chart of generating the second training truth image provided by this application. As shown in FIG. 5B, the process of generating the second training ground truth image may include the following steps 601 to 604.
  • step 601 an average image of the multiple images is generated based on multiple images taken in a dark environment.
  • the pixel value of each pixel in the average image is the average of the pixel values of the pixels at the corresponding positions of all the images taken in the dark environment.
  • step 602 for each target image, the first pixel value of each pixel of the target image is compared with the second pixel value of the pixel at the corresponding position in the average image.
  • step 603 if the absolute value of the difference between the first pixel value and the second pixel value is greater than the preset first pixel difference threshold, bilinear interpolation is used to determine the updated value of the first pixel value to form The updated target image.
  • the process from step 601 to step 603 is a process of removing dead pixels.
  • Bad pixels will affect the accuracy of true image estimation because they do not follow a random process that generates noise at normal pixel locations.
  • the absolute value of the difference between the first pixel value and the second pixel value is greater than the preset first pixel difference threshold, indicating that the pixel point corresponding to the first pixel value is a defective pixel point.
  • the bilinear interpolation method is used to determine the updated value of the first pixel value. After all the dead pixels are corrected, an updated target image is formed.
  • the absolute value of the difference between the first pixel value of all pixels and the second pixel value of the pixel at the corresponding position of the average image is less than or equal to the preset first pixel difference threshold, then The target image is used as the updated target image.
  • step 604 a second training ground truth image is generated according to the updated target image.
  • an average image of a plurality of updated target images may be used as the second training truth image.
  • step 604 may specifically be: for each updated target image, determine the average pixel value of all pixels in the updated target image; according to the average pixel value of each updated target image Value, to determine the average value of the average pixel value of the multiple updated target images; the absolute value of the difference between the corresponding average pixel value and the average pixel value is greater than the updated second pixel difference threshold
  • the target image is deleted to form a filtered update target image; according to the filtered update target image, a second training truth image is generated.
  • the average value and variance of the average pixel values of the multiple updated target images are determined. Obtain a confidence interval based on the mean and variance, and then delete the images outside the confidence interval.
  • generating the second training truth value image according to the filtered update target image includes: for each filtered update target image, if the first position in the filtered update target image is If the pixel value is less than or equal to the preset first pixel threshold or greater than or equal to the preset second pixel threshold, the pixel value at the first position is updated to all the pixel values at the first position of the updated target image after filtering, The pixel value with the most occurrences generates a filtered secondary update target image, where the first pixel threshold is less than the second pixel threshold; the average image of all filtered secondary update target images is determined as the second training true value image.
  • the preset first pixel threshold may be 0, and the preset second pixel threshold may be 255.
  • FIG. 6 is a schematic diagram of a structure of the generation network in the confrontation network provided by this application.
  • the generation network can include: sequentially connected input layer, 3 layers of first convolution module, first pooling layer, 3 layers of second convolution module, second pooling layer, 8 layers of third volume Convolution module, first upsampling layer, 3 layers of fourth convolution module, second upsampling layer, 3 layers of fifth convolution module, and convolution layer.
  • Each convolution module includes a convolution layer, a normalization layer, and an activation layer.
  • the convolution kernels involved in the generation network are all 3*3. The resolution and the number of channels of the image in each layer or each module are shown in the figure.
  • the resolution of the image in the input layer is 256*256, and the number of channels is 8; the resolution of the image in the convolutional layer of the first convolution module is 256*256, and the number of channels is 64; the convolution of the second convolution module The resolution of the image in the buildup layer is 128*128, and the number of channels is 128; the resolution of the image in the convolutional layer of the third convolution module is 64*64, and the number of channels is 256; the convolution of the fourth convolution module The resolution of the image in the build-up layer is 128*128, and the number of channels is 128; and, the resolution of the image in the convolutional layer of the fifth convolution module is 256*256, and the number of channels is 64.
  • FIG. 7 is a schematic structural diagram of the discrimination network in the confrontation network provided by this application.
  • the decision network can include: sequentially connected input layer, 3 layers of first convolution module, first pooling layer, 3 layers of second convolution module, second pooling layer, 3 layers of third volume Integration module, third pooling layer, 2-layer fourth convolution module, fourth pooling layer, fifth convolution module, fully connected layer, and activation layer.
  • the activation function of the activation layer can be a Sigmoid function.
  • the convolution module includes a convolution layer, a normalization layer, and an activation layer.
  • the convolution kernels involved in the judgment network are all 3*3. The resolution and the number of channels of the image in each layer or each module are shown in the figure.
  • the resolution of the image in the input layer is 256*256, and the number of channels is 12; the resolution of the image in the convolutional layer of the first convolution module is 256*256, and the number of channels is 64; the convolution of the second convolution module The resolution of the image in the buildup layer is 128*128, and the number of channels is 128; the resolution of the image in the convolutional layer of the third convolution module is 64*64, and the number of channels is 256; the convolution of the fourth convolution module The resolution of the image in the build-up layer is 32*32, and the number of channels is 512; and, the resolution of the image in the convolutional layer of the fifth convolution module is 16*16, and the number of channels is 64.
  • the second training noise image actually collected and the third training noise image generated by the adversarial network can be used as the first training noise image of the training image denoising model.
  • the second training truth value image and the third training truth value image are both used as the corresponding first training truth value image.
  • Using the confrontation network to generate training sample pairs greatly enriches the number of training samples. Based on these rich training samples, the image denoising model can be fully trained, thereby improving the denoising performance of the image denoising model.
  • FIG. 8 is a schematic diagram of a structure of the image denoising device provided by this application.
  • the image denoising device provided by the present application may include: a first acquisition module 81 and a second acquisition module 82.
  • the first acquiring module 81 may be configured to acquire the image to be processed.
  • the second acquiring module 82 may be configured to input the image to be processed into the image denoising model to acquire the denoised image.
  • the image denoising model can be a model formed by combining a U-shaped network, a residual network, and a dense network.
  • the image denoising model may include: an input layer, a first convolutional layer, at least one dense residual module, a dense residual block, at least one up-sampling module, and a second convolutional layer connected in sequence , The third convolutional layer and the output layer.
  • the output terminal of the input layer also performs a subtraction operation with the output terminal of the third convolutional layer, and the result of the subtraction operation is input to the input terminal of the output layer.
  • the output terminal of the first convolutional layer also performs an addition operation with the output terminal of the second convolutional layer, and the result of the addition operation is input to the input terminal of the third convolutional layer.
  • the dense residual module may include a first dense residual sub-module and a convolution sub-module connected in sequence
  • the up-sampling module may include an up-sampling sub-module and a second dense residual sub-module connected in sequence.
  • the output terminal of the first dense residual sub-module also performs an addition operation with the input terminal of the up-sampling sub-module.
  • the first dense residual sub-module may include: a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, and a seventh convolutional layer that are sequentially connected.
  • the input end of the fourth convolutional layer is also added with the input end of the seventh convolutional layer.
  • the input end of the fifth convolutional layer and the input end of the fourth convolutional layer perform a fusion operation.
  • the input end of the sixth convolutional layer and the output end of the fourth convolutional layer and the input end of the fourth convolutional layer perform a fusion operation.
  • the input end of the seventh convolutional layer, the input end of the fourth convolutional layer, the output end of the fifth convolutional layer, and the output end of the fourth convolutional layer perform a fusion operation.
  • the image denoising device provided in the embodiment of the present application can be configured to execute the image denoising method in any of the foregoing implementable manners.
  • the implementation principle and technical effect of the image denoising device provided in the embodiment of the present application are similar, and will not be repeated here.
  • FIG. 9 is a schematic diagram of another structure of the image denoising device provided by this application.
  • the image denoising device provided by the present application may further include the following modules: a third acquisition module 91, a fourth acquisition module 92, a first determination module 93, and a second determination module 94.
  • the third acquiring module 91 may be configured to acquire the first training noise image and the corresponding first training ground truth image.
  • the fourth obtaining module 92 may be configured to input the first training noise image into the initial image denoising model to obtain an output result.
  • the first determining module 93 may be configured to determine the loss function according to the output result and the corresponding first training truth value image.
  • the second determining module 94 may be configured to adjust the network structure and network parameters of the initial image denoising model according to the loss function when the loss function is greater than the preset threshold, determine the updated image denoising model, and change the updated image
  • the denoising model is used as the new initial image denoising model. Return to the step of inputting the first trained noise image into the initial image denoising model and obtaining the output result until the loss function is less than or equal to the preset threshold, and the loss function is less than or The image denoising model when it is equal to the preset threshold is determined as the image denoising model.
  • the image denoising device provided in the embodiment of the present application can be configured to execute the image denoising method in any of the foregoing implementable manners.
  • the implementation principle and technical effect of the image denoising device provided in the embodiment of the present application are similar, and will not be repeated here.
  • FIG. 10 is a schematic diagram of another structure of the image denoising device provided by this application.
  • the specific structure of the third acquisition module 91 will be described in detail below on the basis of the implementation manner shown in FIG. 9.
  • the third acquisition module 91 may include: a generation sub-module 911, a first determination sub-module 912, a first acquisition sub-module 913, a second acquisition sub-module 914, and a second determination sub-module 915.
  • the generating sub-module 911 may be configured to generate the second training truth image according to multiple target images taken under the first preset light source brightness.
  • the generating sub-module 911 may be specifically configured to: generate an average image of multiple images according to multiple images taken in a dark environment; compare each target image with respect to each target image. The first pixel value of the pixel and the second pixel value of the pixel at the corresponding position in the average image; if the absolute value of the difference between the first pixel value and the second pixel value is greater than the preset first pixel difference threshold, then The bilinear interpolation method is used to determine the updated value of the first pixel value to form an updated target image; and, according to the updated target image, a second training true value image is generated.
  • the generating submodule 911 may be specifically configured to: determine the updated target for each updated target image The average pixel value of all pixels in the image; according to the average pixel value of each updated target image, determine the average value of the average pixel value of multiple updated target images; compare the corresponding average pixel value with the average pixel value The updated target image whose absolute value of the difference of the average value is greater than the preset second pixel difference threshold is deleted to form a filtered update target image; and, according to the filtered update target image, a second training true value image is generated .
  • the generating submodule 911 may be specifically configured to: for each filtered update target image, if the filtered update target image The pixel value at the first position in the update target image is less than or equal to the preset first pixel threshold or greater than or equal to the preset second pixel threshold, then the pixel value at the first position is updated to all filtered update targets Among the pixel values at the first position of the image, the pixel value with the most occurrences is generated to generate a filtered secondary update target image, where the first pixel threshold is less than the second pixel threshold; and, all filtered secondary updates The average image of the target image is determined to be the second training true value image.
  • the first determining sub-module 912 may be configured to use multiple images taken by randomly adjusting the brightness of the light source within the range of the brightness of the second preset light source and the brightness of the third preset light source as the second training noise image.
  • the second preset light source brightness is less than the third preset light source brightness
  • the third preset light source brightness is less than the first preset light source brightness
  • the first acquisition sub-module 913 may be configured to train the initial confrontation network according to the second training ground truth image and the second training noise image, and acquire the confrontation network formed by the final training.
  • the first acquisition submodule 913 may be specifically configured to: input the camera parameters corresponding to the normal distribution noise, the second training ground truth image, and any second training noise image to the initial countermeasure network The initial generation network in, to obtain the output noise image; input the noise image, the second training noise image, the camera parameters corresponding to the second training noise image, and the second training truth image to the initial judgment in the initial countermeasure network In the network, obtain the output probability of the initial discrimination network; and, according to the output probability, the camera parameters corresponding to the second training noise image, the second training truth value image, the second training noise image, and the noise image, determine the initial discrimination network The loss function and the loss function of the initial generation network.
  • the initial confrontation network When it is determined that the initial confrontation network does not converge according to the loss function of the initial discriminant network and the loss function of the initial generation network, alternate execution and return to execute the normal distribution noise, the second training truth image,
  • the camera parameters corresponding to the second training noise image are input to the initial generation network in the initial confrontation network to obtain the output noise image, and return the noise image, the second training noise image, and the second training noise image
  • the corresponding camera parameters and the second training ground truth image are input to the initial discriminant network in the initial confrontation network to obtain the output probability of the initial discriminant network until it is determined according to the loss function of the initial discriminant network and the loss function of the initial generation network
  • the confrontation network converges, and the convergent confrontation network is determined as the confrontation network formed by the final training.
  • the generation network and the discrimination network in the confrontation network may both be U-shaped networks.
  • the second acquisition sub-module 914 may be configured to input the third training ground truth image acquired in advance into the confrontation network, and acquire the third training noise image output by the generation network of the confrontation network.
  • the second determination sub-module 915 may be configured to use both the second training noise image and the third training noise image as the first training image, and both the second training truth image and the third training truth image as the corresponding first training image. Train the ground truth image.
  • the image denoising device provided by the present application can be configured to implement the image denoising method in any of the above-mentioned implementable modes.
  • the implementation principle and technical effect of the image denoising device provided by the present application are similar and will not be repeated here.
  • FIG. 11 is a schematic diagram of a structure of an electronic device provided by this application.
  • the electronic device may include a processor 111 and a memory 112; the number of processors 111 in the electronic device may be one or more.
  • one processor 111 is taken as an example; the processor in the electronic device 111 and the memory 112; can be connected by a bus or in other ways.
  • the connection by a bus is taken as an example.
  • the memory 112 can be configured to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the image denoising method in the embodiment of the present application (for example, an image denoising device).
  • the processor 111 runs the software programs, instructions, and modules stored in the memory 112 to thereby implement various functional applications and data processing of the electronic device, that is, to realize the aforementioned image denoising method.
  • the memory 112 may mainly include a program storage area and a data storage area.
  • the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the electronic device and the like.
  • the memory 112 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • This application also provides a storage medium containing computer-executable instructions.
  • the computer-executable instructions implement an image denoising method when executed by a computer processor.
  • the method may include: acquiring an image to be processed; and inputting the image to be processed into In the image denoising model, the denoised image is obtained; among them, the image denoising model is a model formed by combining a U-shaped network, a residual network, and a dense network.
  • the computer-executable instructions are not limited to implementing the method operations described above, and may also implement related operations in the image denoising method in any implementable manner of the present application.
  • the various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic or any combination thereof.
  • some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, although the present application is not limited thereto.
  • Computer program instructions can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code written in any combination of one or more programming languages or Object code.
  • ISA instruction set architecture
  • the block diagram of any logic flow in the drawings of the present application may represent program steps, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions.
  • the computer program can be stored on the memory.
  • the memory can be of any type suitable for the local technical environment and can be implemented using any suitable data storage technology, such as but not limited to read only memory (ROM), random access memory (RAM), optical storage devices and systems (digital multi-function optical discs) DVD or CD) etc.
  • Computer-readable media may include non-transitory storage media.
  • the data processor can be any type suitable for the local technical environment, such as but not limited to general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSP), application-specific integrated circuits (ASIC), programmable logic devices (FGPA) And processors based on multi-core processor architecture.
  • DSP digital signal processors
  • ASIC application-specific integrated circuits
  • FGPA programmable logic devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Processing (AREA)

Abstract

本申请涉及图像处理技术领域，并提供一种图像去噪方法及装置、电子设备和存储介质。图像去噪方法包括：获取待处理图像，将待处理图像输入至图像去噪模型中，获取去噪后的图像，其中，图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。

Description

图像去噪方法及装置、电子设备以及存储介质 技术领域
本申请涉及图像处理技术领域。
背景技术
图像去噪一直是图像处理领域非常重要的一部分,特别近年来，监控领域流行星光级拍摄。黑夜环境的光照条件非常恶劣，传感器感光不足，导致采集出来的图片会遗留很多的噪声，因此，获取到的图像或视频与光照条件好的情况下采集出来的图像或视频相比分辨力降低；这不仅影响视觉效果，对于需要从中识别运动目标的图像或视频，更是影响了识别结果的准确性。因此，亟需一种去噪方法来实现图像质量的提升。
发明内容
本申请实施例的一个方面提供一种图像去噪方法，包括：获取待处理图像；以及，将待处理图像输入至图像去噪模型中，获取去噪后的图像；其中，图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。
本申请实施例的一个方面提供一种图像去噪装置，包括：第一获取模块，被配置为获取待处理图像；以及，第二获取模块，被配置为将待处理图像输入至图像去噪模型中，获取去噪后的图像；其中，图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。
本申请实施例的一个方面提供一种电子设备，包括：一个或多个处理器；存储器，被配置为存储一个或多个程序；当该一个或多个程序被该一个或多个处理器执行，使得该一个或多个处理器实现本申请实施例提供的图像去噪方法。
本申请实施例的一个方面提供一种存储介质，存储介质存储有计算机程序，计算机程序被处理器执行时实现本申请实施例提供的图像去噪方法。
附图说明
图1为本申请提供的图像去噪方法的一种流程图。
图2为本申请提供的图像去噪模型的一种结构示意图。
图3为本申请提供的图像去噪模型中的第一稠密残差子模块的一种一种结构示意图。
图4为本申请提供的图像去噪方法的另一种流程图。
图5A为本申请提供的获取第一训练噪声图像以及对应的第一训练真值图像的一种流程图。
图5B为本申请提供的生成第二训练真值图像的一种流程图。
图6为本申请提供的对抗网络中的生成网络的一种结构示意图。
图7为本申请提供的对抗网络中的判别网络的一种结构示意图。
图8为本申请提供的图像去噪装置的一种结构示意图。
图9为本申请提供的图像去噪装置的另一种结构示意图。
图10为本申请提供的图像去噪装置的又一种结构示意图。
图11为本申请提供的电子设备的一种结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚明白，下文中将结合附图对本申请的实施例进行详细说明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互任意组合。
图像去噪对于图像质量的提高至关重要。目前，一种效果较好的基于神经网络的自适应去噪算法为监督式学习类型神经网络算法。在该算法中，训练样本包括输入样本与输出样本对，通过梯度下降算法更新神经网络中的参数，使得训练样本的输入经过神经网络后的输出接近真值样本。
但是，目前的自适应去噪算法无法在实现较好的去噪性能的同时，兼顾去噪后的图像质量。
图1为本申请提供的图像去噪方法的一种流程图。该图像去噪方法可适用于对图像进行去噪的场景，且可以由图像去噪装置来执行。 该图像去噪装置可以由软件和/或硬件的方式实现，该图像去噪装置可以集成于电子设备中。如图1所示，本申请提供的图像去噪方法可包括步骤101和步骤102。
在步骤101中，获取待处理图像。
在步骤102中，将待处理图像输入至图像去噪模型中，获取去噪后的图像。该图像去噪模型可以为结合U型网络、残差网络以及稠密网络形成的模型。
在一种可实施方式中，待处理图像可以为前端监控设备拍摄的图像，或者，前端监控设备拍摄的视频中的图像帧。当然，本申请中的待处理图像还可以为其他领域中的图像，例如，医学图像等。
本申请中的图像去噪模型可为预先训练好的模型。该模型为结合U型网络、残差网络以及稠密网络形成的模型。本申请中的U型网络，即，Unet，指的是包括特征提取，即编码部分，以及，上采样，即解码部分的呈U型的网络。本申请中的残差网络指的是包括直接映射部分和残差部分，且直接映射部分和残差部分进行相加操作的网络。本申请中的稠密网络指的是包括直接映射部分和残差部分，且直接映射部分和残差部分进行通道连结操作的网络。
本申请中的图像去噪模型，可以结合残差网络和稠密网络的特点，更好地利用待处理图像的深层和浅层特征；能实现在去掉噪声的同时，尽可能保留待处理图像的细节，从而，在实现较好的去噪性能的基础上，兼顾去噪后的图像质量。
图2为本申请提供的图像去噪模型的一种结构示意图。如图2所示，本申请提供的图像去噪模型可以包括：依次连接的输入层、第一卷积层、至少一个稠密残差模块、稠密残差块、至少一个上采样模块、第二卷积层、第三卷积层以及输出层。
输入层的输出端还与第三卷积层的输出端进行相减操作，相减操作后的结果输入至输出层的输入端。
第一卷积层的输出端还与第二卷积层的输出端进行相加操作，相加操作后的结果输入至第三卷积层的输入端。
稠密残差模块包括依次连接的第一稠密残差子模块以及卷积子 模块，上采样模块包括依次连接的上采样子模块以及第二稠密残差子模块。第一稠密残差子模块的输出端还与上采样子模块的输入端进行相加操作。
在一种可实施方式中，稠密残差模块的数量可以为4个，上采样模块的数量可以为4个。
在稠密残差模块的数量以及上采样模块的数量均为多个时，可以建立稠密残差模块与上采样模块的映射关系。根据该映射关系，实现稠密残差模块中的第一稠密残差子模块的输出端与该映射关系中对应的上采样模块中的上采样子模块的输入端进行相加操作。
在图像去噪模型中，输入层输入的待处理图像的分辨率为N*N，通道数为3。第一卷积层的卷积核为3*3，通道数变为128。卷积子模块中的卷积核也为3*3，跨度为2。第二卷积层以及第三卷积层的卷积核均为3*3。第二卷积层中通道数为128。第三卷积层的通道数为3。
图3为本申请提供的图像去噪模型中的第一稠密残差子模块的一种结构示意图。如图3所示，更具体地，第一稠密残差子模块可包括：依次连接的第四卷积层、第五卷积层、第六卷积层以及第七卷积层。
其中，第四卷积层的输入端还与第七卷积层的输入端进行相加操作。第五卷积层的输入端与第四卷积层的输入端进行融合操作。第六卷积层的输入端与第四卷积层的输出端以及第四卷积层的输入端进行融合操作。第七卷积层的输入端与第四卷积层的输入端、第五卷积层的输出端以及第四卷积层的输出端进行融合操作。
在一种可实施方式中，本申请中的融合操作指的是稠密网络中的通道连结操作。
需要说明的是，第二稠密残差子模块以及稠密残差块的结构与第一稠密残差子模块的结构相同，此处不再赘述。
第四卷积层、第五卷积层以及第六卷积层的卷积核均为3*3。第七卷积层的卷积核为1*1。以M表示第四卷积层、第五卷积层、第六卷积层以及第七卷积层中的通道数。
本申请提供的图像去噪方法可包括：获取待处理图像，将待处理图像输入至图像去噪模型中，获取去噪后的图像，其中，图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。本申请中的图像去噪模型，可以结合残差网络和稠密网络的特点，更好地利用待处理图像的深层和浅层特征，能实现在去掉噪声的同时，尽可能保留待处理图像的细节，从而，在实现较好的去噪性能的基础上，兼顾去噪后的图像质量。
图4为本申请提供的图像去噪方法的另一种流程图。本申请实施例在图1所示的可实施方式及各种可选的方案的基础上，对如何训练图像去噪模型的步骤作详细说明。本申请实施例仅示出训练图像去噪模型的步骤。如图4所示，本申请实施例提供的图像去噪方法可包括如下步骤401-步骤404。
在步骤401中，获取第一训练噪声图像以及对应的第一训练真值图像。
在一种可实施方式中，可以通过实际拍摄获取第一训练噪声图像以及对应的第一训练真值图像。
在另一种可实施方式中，可以通过实际拍摄与对抗网络，获取第一训练噪声图像以及对应的第一训练真值图像。
图5A为本申请提供的获取第一训练噪声图像以及对应的第一训练真值图像的一种流程图。如图5A所示，获取过程可包括如下步骤501-步骤505。
在步骤501中，根据在第一预设光源亮度下拍摄的多张目标图像，生成第二训练真值图像。
在一种可实施方式中，第一预设光源亮度可以为200勒克斯。在第一预设光源亮度下拍摄目标图像时，使能摄像头模组，并将图像信号处理模块中的去噪算法、去坏点算法全部关闭。在拍摄时，保持摄像头以及场景绝对静止。目标图像的数量可以为200张。
在一种可实施方式中，在根据多张目标图像，生成第二训练真值图像时，可以是根据多张目标图像，确定该多张目标图像的平均图像，将该多张目标图像的平均图像作为第二训练真值图像。
在另一种可实施方式中，可以对该多张目标图像进行去坏点、强度对齐以及均值获取等步骤，生成最终的第二训练真值图像。该实施方式将在后文进行详细描述。
在步骤502中，将在第二预设光源亮度与第三预设光源亮度范围内，随机调节光源亮度拍摄的多张图像，作为第二训练噪声图像。
其中，第二预设光源亮度小于第三预设光源亮度，第三预设光源亮度小于第一预设光源亮度。
在本申请实施例中，将在第二预设光源亮度与第三预设光源亮度范围内，随机调节光源亮度拍摄的多张图像，作为第二训练噪声图像。
第二预设光源亮度可以为0.1勒克斯，第三预设光源亮度可以为100勒克斯。第二训练噪声图像的数量可以为200张。在拍摄第二训练噪声图像时，可以记录下拍摄每张第二训练噪声图像时的摄像头参数，例如，模拟增益与数字增益。
需要说明的是，在本申请实施例中可以变换场景，拍摄不同场景中的多张目标图像以及第二训练噪声图像。同一场景中的多张目标图像，生成第二训练真值图像。针对一个场景来说，一个第二训练真值图像对应多张第二训练噪声图像。
在步骤503中，根据第二训练真值图像以及第二训练噪声图像，训练初始对抗网络，获取最终训练形成的对抗网络。
在一种可实施方式中，本申请实施例中的对抗网络可包括生成网络和判决网络。对抗网络中的生成网络以及判别网络均可以为U型网络。
初始对抗网络中的初始生成网络用于生成噪声图像，初始对抗网络中的初始判决网络用于判决初始生成网络输出的噪声图像与实际的第二训练噪声图像的相似性。根据初始生成网络的损失函数以及初始判决网络的损失函数，单独交替迭代训练初始对抗网络，直至收敛。具体过程可包括如下步骤1-步骤3。
在步骤1中，将正态分布噪声、第二训练真值图像、任一第二训练噪声图像对应的摄像头参数，输入至初始对抗网络中的初始生成 网络，获取输出的噪声图像。
在步骤2中，将噪声图像、该第二训练噪声图像、该第二训练噪声图像对应的摄像头参数，以及第二训练真值图像，输入至初始对抗网络中的初始判别网络中，获取初始判别网络的输出概率。
在步骤3中，根据输出概率、该第二训练噪声图像对应的摄像头参数、第二训练真值图像、该第二训练噪声图像以及噪声图像，确定初始判别网络的损失函数以及初始生成网络的损失函数；当根据初始判别网络的损失函数以及初始生成网络的损失函数确定初始对抗网络不收敛时，交替执行返回执行将正态分布噪声、第二训练真值图像、该第二训练噪声图像对应的摄像头参数，输入至初始对抗网络中的初始生成网络，获取输出的噪声图像的步骤，以及返回将噪声图像、该第二训练噪声图像、该第二训练噪声图像对应的摄像头参数以及第二训练真值图像，输入至初始对抗网络中的初始判别网络中，获取初始判别网络的输出概率的步骤，直至根据初始判别网络的损失函数以及初始生成网络的损失函数确定对抗网络收敛，并将收敛的对抗网络确定为最终训练形成的对抗网络。
初始判别网络的输出概率为0～1之间的值，来表示真假噪声图像的概率。0表示为生成网络生成的噪声图像，1表示为实际采集到的第二训练噪声图像。
其中，对抗网络中的生成网络的损失函数为：L G＝E[(D(N c,N r,N f)) 2]+E[(D(N c,N f,N r)-1) 2]，判决网络的损失函数为：L D＝E[(D(N c,N r,N f)-1) 2]+E[(D(N c,N f,N r)) 2]。
N c为生成噪声图片的条件信号，包括摄像头的模拟增益、数字增益以及第二训练真值图像；N f为生成网络生成的噪声图片，N r为真实采集的第二训练噪声图像，D(*)表示判别网络的输出，E(*)表示求均值。
在步骤504中，将预先获取的第三训练真值图像输入至对抗网络中，获取对抗网络的生成网络输出的第三训练噪声图像。
在训练得到对抗网络之后，可以将预先获取的第三训练真值图像输入至对抗网络中，获取对抗网络的生成网络输出的第三训练噪声 图像。
在步骤505中，将第二训练噪声图像以及第三训练噪声图像均作为第一训练图像，将第二训练真值图像以及第三训练真值图像均作为对应的第一训练真值图像。
在本申请实施例中，可以将实际采集到的第二训练噪声图像，以及对抗网络生成的第三训练噪声图像均作为训练图像去噪模型的第一训练噪声图像，将第二训练真值图像以及第三训练真值图像均作为对应的第一训练真值图像。利用对抗网络生成训练样本对，大大丰富了训练样本的数量，基于这些丰富的训练样本，可以对图像去噪模型进行充分训练，从而，提高了图像去噪模型的去噪性能。
在步骤402中，将第一训练噪声图像输入至初始图像去噪模型中，获取输出结果。
在步骤403中，根据输出结果以及对应的第一训练真值图像，确定损失函数。
在步骤404中，当损失函数大于预设阈值时，根据损失函数，调整初始图像去噪模型的网络结构以及网络参数，确定更新后的图像去噪模型，将更新后的图像去噪模型作为新的初始图像去噪模型，返回执行将第一训练噪声图像输入至初始图像去噪模型中，获取输出结果的步骤，直至损失函数小于或者等于预设阈值，将损失函数小于或者等于预设阈值时的图像去噪模型确定为图像去噪模型。
步骤402-步骤404为迭代训练图像去噪模型的过程。
由于步骤401中获取到的第一训练噪声图像比较丰富，因此，基于这些第一训练噪声图像训练图像去噪模型，可以显著提高图像去噪模型的去噪性能和泛化能力。
图5B为本申请提供的生成第二训练真值图像的一种流程图。如图5B所示，生成第二训练真值图像的过程可以包括如下步骤601-步骤604。
在步骤601中，根据在无光线环境中拍摄的多张图像，生成多张图像的平均图像。
该平均图像中的每个像素点的像素值为无光线环境中拍摄的所 有图像对应位置像素点的像素值的平均值。
在步骤602中，针对每张目标图像，比较目标图像的每个像素点的第一像素值与平均图像中对应位置处的像素点的第二像素值。
在步骤603中，若第一像素值与第二像素值的差值的绝对值大于预设第一像素差值阈值，则采用双线性插值法，确定第一像素值更新后的值，形成更新后的目标图像。
步骤601-步骤603的过程为去坏点过程。坏点像素会影响真值图像估计的准确性，因为它们不遵循在正常像素位置产生噪声的随机过程。第一像素值与第二像素值的差值的绝对值大于预设第一像素差值阈值，说明第一像素值对应的像素点为缺陷像素点。采用双线性插值法来确定第一像素值更新后的值。对所有坏点修正完成后，形成了更新后的目标图像。若某张目标图像中，所有像素点的第一像素值与平均图像的对应位置的像素点的第二像素值的差值的绝对值均小于或者等于预设第一像素差值阈值，则将该目标图像作为更新后的目标图像。
在步骤604中，根据更新后的目标图像，生成第二训练真值图像。
在一种可实施方式中，可以将多个更新后的目标图像的平均图像，作为第二训练真值图像。
在另一种可实施方式中，步骤604具体可以为：针对每个更新后的目标图像，确定更新后的目标图像中所有像素点的平均像素值；根据每个更新后的目标图像的平均像素值，确定多个更新后的目标图像的平均像素值的平均值；将对应的平均像素值与平均像素值的平均值的差值的绝对值大于预设第二像素差值阈值的更新后的目标图像删除，形成过滤后的更新目标图像；根据过滤后的更新目标图像，生成第二训练真值图像。
该可实施方式实现了图片强度对齐。由于曝光算法的不精确，仍然需要考虑同一场景，同一亮度下采集到图片的微小变化。将对应的平均像素值与平均像素值的平均值的差值的绝对值大于预设第二像素差值阈值的更新后的目标图像删除。
根据每个更新后的目标图像的平均像素值，确定多个更新后的目标图像的平均像素值的平均值以及方差。基于均值和方差得到一个置信区间，然后把在置信区间以外的图像删除。
在一种可实施方式中，根据过滤后的更新目标图像，生成第二训练真值图像，包括：针对每个过滤后的更新目标图像，若过滤后的更新目标图像中的第一位置处的像素值小于或者等于预设第一像素阈值或者大于或者等于预设第二像素阈值，则将第一位置处的像素值更新为所有过滤后的更新目标图像的第一位置处的像素值中，出现次数最多的像素值，生成过滤后的二次更新目标图像，其中，第一像素阈值小于第二像素阈值；将所有过滤后的二次更新目标图像的平均图像，确定为第二训练真值图像。示例性地，预设第一像素阈值可以为0，预设第二像素阈值可以为255。
图6为本申请提供的对抗网络中的生成网络的一种结构示意图。如图6所示，生成网络可以包括：依次连接的输入层、3层第一卷积模块、第一池化层、3层第二卷积模块、第二池化层、8层第三卷积模块、第一上采样层、3层第四卷积模块、第二上采样层、3层第五卷积模块以及卷积层。每个卷积模块包括卷积层、归一化层以及激活层。生成网络中涉及到的卷积核均为3*3。每层或者每个模块中的图像的分辨率以及通道数如图中所示。输入层中的图像的分辨率为256*256，通道数为8；第一卷积模块的卷积层中的图像的分辨率为256*256，通道数为64；第二卷积模块的卷积层中的图像的分辨率为128*128，通道数为128；第三卷积模块的卷积层中的图像的分辨率为64*64，通道数为256；第四卷积模块的卷积层中的图像的分辨率为128*128，通道数为128；以及，第五卷积模块的卷积层中的图像的分辨率为256*256，通道数为64。
图7为本申请提供的对抗网络中的判别网络的一种结构示意图。如图7所示，判决网络可包括：依次连接的输入层、3层第一卷积模块、第一池化层、3层第二卷积模块、第二池化层、3层第三卷积模块、第三池化层、2层第四卷积模块、第四池化层、第五卷积模块、全连接层以及激活层。其中，激活层的激活函数可以为Sigmoid函数。 卷积模块均包括卷积层、归一化层以及激活层。判决网络中涉及到的卷积核均为3*3。每层或者每个模块中的图像的分辨率以及通道数如图中所示。输入层中的图像的分辨率为256*256，通道数为12；第一卷积模块的卷积层中的图像的分辨率为256*256，通道数为64；第二卷积模块的卷积层中的图像的分辨率为128*128，通道数为128；第三卷积模块的卷积层中的图像的分辨率为64*64，通道数为256；第四卷积模块的卷积层中的图像的分辨率为32*32，通道数为512；以及，第五卷积模块的卷积层中的图像的分辨率为16*16，通道数为64。
根据本申请实施例提供的图像去噪方法，可以将实际采集到的第二训练噪声图像，以及对抗网络生成的第三训练噪声图像均作为训练图像去噪模型的第一训练噪声图像，将第二训练真值图像以及第三训练真值图像均作为对应的第一训练真值图像。利用对抗网络生成训练样本对，大大丰富了训练样本的数量，基于这些丰富的训练样本，可以对图像去噪模型进行充分训练，从而，提高了图像去噪模型的去噪性能。
图8为本申请提供的图像去噪装置的一种结构示意图。如图8所示，本申请提供的图像去噪装置可包括：第一获取模块81以及第二获取模块82。
第一获取模块81，可被配置为获取待处理图像。
第二获取模块82，可被配置为将待处理图像输入至图像去噪模型中，获取去噪后的图像。
其中，图像去噪模型可为结合U型网络、残差网络以及稠密网络形成的模型。
在一种可实施方式中，图像去噪模型可包括：依次连接的输入层、第一卷积层、至少一个稠密残差模块、稠密残差块、至少一个上采样模块、第二卷积层、第三卷积层以及输出层。
其中，输入层的输出端还与第三卷积层的输出端进行相减操作，相减操作后的结果输入至输出层的输入端。
第一卷积层的输出端还与第二卷积层的输出端进行相加操作，相加操作后的结果输入至第三卷积层的输入端。
稠密残差模块可包括依次连接的第一稠密残差子模块以及卷积子模块，上采样模块可包括依次连接的上采样子模块以及第二稠密残差子模块。第一稠密残差子模块的输出端还与上采样子模块的输入端进行相加操作。
在一种可实施方式中，第一稠密残差子模块可包括：依次连接的第四卷积层、第五卷积层、第六卷积层以及第七卷积层。
其中，第四卷积层的输入端还与第七卷积层的输入端进行相加操作。
第五卷积层的输入端与第四卷积层的输入端进行融合操作。
第六卷积层的输入端与第四卷积层的输出端以及第四卷积层的输入端进行融合操作。
第七卷积层的输入端与第四卷积层的输入端、第五卷积层的输出端以及第四卷积层的输出端进行融合操作。
本申请实施例提供的图像去噪装置可被配置为执行上述任意可实施方式中的图像去噪方法，本申请实施例提供的图像去噪装置实现原理和技术效果类似，此处不再赘述。
图9为本申请提供的图像去噪装置的另一种结构示意图。下面在图8所示可实施方式的基础上，对图像去噪装置包括的其他模块作一详细说明。如图9所示，本申请提供的图像去噪装置还可包括如下模块：第三获取模块91、第四获取模块92、第一确定模块93以及第二确定模块94。
第三获取模块91，可被配置为获取第一训练噪声图像以及对应的第一训练真值图像。
第四获取模块92，可被配置为将第一训练噪声图像输入至初始图像去噪模型中，获取输出结果。
第一确定模块93，可被配置为根据输出结果以及对应的第一训练真值图像，确定损失函数。
第二确定模块94，可被配置为当损失函数大于预设阈值时，根据损失函数，调整初始图像去噪模型的网络结构以及网络参数，确定更新后的图像去噪模型，将更新后的图像去噪模型作为新的初始图像 去噪模型，返回执行将第一训练噪声图像输入至初始图像去噪模型中，获取输出结果的步骤，直至损失函数小于或者等于预设阈值，将损失函数小于或者等于预设阈值时的图像去噪模型确定为图像去噪模型。
本申请实施例提供的图像去噪装置可被配置为执行上述任意可实施方式中的图像去噪方法，本申请实施例提供的图像去噪装置实现原理和技术效果类似，此处不再赘述。
图10为本申请提供的图像去噪装置的又一种结构示意图。下面在图9所示的可实施方式的基础上，对第三获取模块91的具体结构作一详细说明。如图10所示，第三获取模块91可包括：生成子模块911、第一确定子模块912、第一获取子模块913、第二获取子模块914以及第二确定子模块915。
生成子模块911，可被配置为根据在第一预设光源亮度下拍摄的多张目标图像，生成第二训练真值图像。
在一种可实施方式中，生成子模块911可具体被配置为：根据在无光线环境中拍摄的多张图像，生成多张图像的平均图像；针对每张目标图像，比较目标图像的每个像素点的第一像素值与平均图像中对应位置处的像素点的第二像素值；若第一像素值与第二像素值的差值的绝对值大于预设第一像素差值阈值，则采用双线性插值法，确定第一像素值更新后的值，形成更新后的目标图像；以及，根据更新后的目标图像，生成第二训练真值图像。
在一种可实施方式中，在根据更新后的目标图像，生成第二训练真值图像的方面，生成子模块911可具体被配置为：针对每个更新后的目标图像，确定更新后的目标图像中所有像素点的平均像素值；根据每个更新后的目标图像的平均像素值，确定多个更新后的目标图像的平均像素值的平均值；将对应的平均像素值与平均像素值的平均值的差值的绝对值大于预设第二像素差值阈值的更新后的目标图像删除，形成过滤后的更新目标图像；以及，根据过滤后的更新目标图像，生成第二训练真值图像。
在一种可实施方式中，在根据过滤后的更新目标图像，生成第二训练真值图像的方面，生成子模块911可具体被配置为：针对每个 过滤后的更新目标图像，若过滤后的更新目标图像中的第一位置处的像素值小于或者等于预设第一像素阈值或者大于或者等于预设第二像素阈值，则将第一位置处的像素值更新为所有过滤后的更新目标图像的第一位置处的像素值中，出现次数最多的像素值，生成过滤后的二次更新目标图像，其中，第一像素阈值小于第二像素阈值；以及，将所有过滤后的二次更新目标图像的平均图像，确定为第二训练真值图像。
第一确定子模块912，可被配置为将在第二预设光源亮度与第三预设光源亮度范围内，随机调节光源亮度拍摄的多张图像，作为第二训练噪声图像。
其中，第二预设光源亮度小于第三预设光源亮度，第三预设光源亮度小于第一预设光源亮度。
第一获取子模块913，可被配置为根据第二训练真值图像以及第二训练噪声图像，训练初始对抗网络，获取最终训练形成的对抗网络。
在一种可实施方式中，第一获取子模块913可具体被配置为：将正态分布噪声、第二训练真值图像、任一第二训练噪声图像对应的摄像头参数，输入至初始对抗网络中的初始生成网络，获取输出的噪声图像；将噪声图像、该第二训练噪声图像、该第二训练噪声图像对应的摄像头参数以及第二训练真值图像，输入至初始对抗网络中的初始判别网络中，获取初始判别网络的输出概率；以及，根据输出概率、该第二训练噪声图像对应的摄像头参数、第二训练真值图像、该第二训练噪声图像以及噪声图像，确定初始判别网络的损失函数以及初始生成网络的损失函数，当根据初始判别网络的损失函数以及初始生成网络的损失函数确定初始对抗网络不收敛时，交替执行返回执行将正态分布噪声、第二训练真值图像、该第二训练噪声图像对应的摄像头参数，输入至初始对抗网络中的初始生成网络，获取输出的噪声图像的步骤，以及，返回将噪声图像、该第二训练噪声图像、该第二训练噪声图像对应的摄像头参数以及第二训练真值图像，输入至初始对抗网络中的初始判别网络中，获取初始判别网络的输出概率的步骤，直至根据初始判别网络的损失函数以及初始生成网络的损失函数确定 对抗网络收敛，将收敛的对抗网络确定为最终训练形成的对抗网络。
在一种可实施方式中，对抗网络中的生成网络以及判别网络可均为U型网络。
第二获取子模块914，可被配置为将预先获取的第三训练真值图像输入至对抗网络中，获取对抗网络的生成网络输出的第三训练噪声图像。
第二确定子模块915，可被配置为将第二训练噪声图像以及第三训练噪声图像均作为第一训练图像，将第二训练真值图像以及第三训练真值图像均作为对应的第一训练真值图像。
本申请提供的图像去噪装置可被配置为实现上述任意可实施方式中的图像去噪方法，本申请提供的图像去噪装置实现原理和技术效果类似，此处不再赘述。
图11为本申请提供的电子设备的一种结构示意图。如图11所示，该电子设备可包括处理器111和存储器112；电子设备中处理器111的数量可以是一个或多个，图11中以一个处理器111为例；电子设备中的处理器111和存储器112；可以通过总线或其他方式连接，图11中以通过总线连接为例。
存储器112作为一种计算机可读存储介质，可被配置为存储软件程序、计算机可执行程序以及模块，如本申请实施例中的图像去噪方法对应的程序指令/模块(例如，图像去噪装置中的第一获取模块81以及第二获取模块82)。处理器111通过运行存储在存储器112中的软件程序、指令以及模块，从而电子设备的各种功能应用以及数据处理，即实现上述的图像去噪方法。
存储器112可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据电子设备的使用所创建的数据等。此外，存储器112可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。
本申请还提供一种包含计算机可执行指令的存储介质，计算机可执行指令在由计算机处理器执行时实现图像去噪方法，该方法可包 括：获取待处理图像；以及，将待处理图像输入至图像去噪模型中，获取去噪后的图像；其中，图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。
当然，根据本申请提供的包含计算机可执行指令的存储介质，计算机可执行指令不限于实现如上所述的方法操作，还可以实现本申请任意可实施方式中的图像去噪方法中的相关操作。
以上所述，仅为本申请的示例性实施例而已，并非用于限定本申请的保护范围。
一般来说，本申请的多种可实施方式可以在硬件或专用电路、软件、逻辑或其任何组合中实现。例如，一些方面可以被实现在硬件中，而其它方面可以被实现在可以被控制器、微处理器或其它计算装置执行的固件或软件中，尽管本申请不限于此。
本申请实施例可以通过移动装置的数据处理器执行计算机程序指令来实现，例如在处理器实体中，或者通过硬件，或者通过软件和硬件的组合。计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码。
本申请附图中的任何逻辑流程的框图可以表示程序步骤，或者可以表示相互连接的逻辑电路、模块和功能，或者可以表示程序步骤与逻辑电路、模块和功能的组合。计算机程序可以存储在存储器上。存储器可以具有任何适合于本地技术环境的类型并且可以使用任何适合的数据存储技术实现，例如但不限于只读存储器(ROM)、随机访问存储器(RAM)、光存储器装置和系统(数码多功能光碟DVD或CD光盘)等。计算机可读介质可以包括非瞬时性存储介质。数据处理器可以是任何适合于本地技术环境的类型，例如但不限于通用计算机、专用计算机、微处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、可编程逻辑器件(FGPA)以及基于多核处理器架构的处理器。
通过示范性和非限制性的示例，上文已提供了对本申请的示范实施例的详细描述。但结合附图和权利要求来考虑，对以上实施例的 多种修改和调整对本领域技术人员来说是显而易见的，但不偏离本发明的范围。因此，本发明的恰当范围将根据权利要求确定。

Claims (13)

  1. 一种图像去噪方法，包括：
    获取待处理图像；以及
    将所述待处理图像输入至图像去噪模型中，获取去噪后的图像；其中，所述图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。
  2. 根据权利要求1所述的方法，其中，所述图像去噪模型包括：依次连接的输入层、第一卷积层、至少一个稠密残差模块、稠密残差块、至少一个上采样模块、第二卷积层、第三卷积层以及输出层；
    其中，所述输入层的输出端与所述第三卷积层的输出端进行相减操作，相减操作后的结果输入至所述输出层的输入端；
    所述第一卷积层的输出端与所述第二卷积层的输出端进行相加操作，相加操作后的结果输入至所述第三卷积层的输入端；
    所述至少一个稠密残差模块包括依次连接的第一稠密残差子模块以及卷积子模块；
    所述至少一个上采样模块包括依次连接的上采样子模块以及第二稠密残差子模块；以及
    所述第一稠密残差子模块的输出端还与所述上采样子模块的输入端进行相加操作。
  3. 根据权利要求2所述的方法，其中，所述第一稠密残差子模块包括：依次连接的第四卷积层、第五卷积层、第六卷积层以及第七卷积层；
    其中，所述第四卷积层的输入端与所述第七卷积层的输入端进行相加操作；
    所述第五卷积层的输入端与所述第四卷积层的输入端进行融合操作；
    所述第六卷积层的输入端与所述第四卷积层的输出端以及所述第四卷积层的输入端进行融合操作；以及
    所述第七卷积层的输入端与所述第四卷积层的输入端、所述第五卷积层的输出端以及所述第四卷积层的输出端进行融合操作。
  4. 根据权利要求1-3任一项所述的方法，在将所述待处理图像输入至图像去噪模型中，获取去噪后的图像之前，还包括：
    获取第一训练噪声图像以及对应的第一训练真值图像；
    将所述第一训练噪声图像输入至初始图像去噪模型中，获取输出结果；
    根据所述输出结果以及所述对应的第一训练真值图像，确定损失函数；以及
    响应于确定所述损失函数大于预设阈值，根据所述损失函数，调整所述初始图像去噪模型的网络结构以及网络参数，确定更新后的图像去噪模型，将所述更新后的图像去噪模型作为新的初始图像去噪模型，返回执行将所述第一训练噪声图像输入至初始图像去噪模型中，获取输出结果的步骤，直至所述损失函数小于或者等于所述预设阈值，将所述损失函数小于或者等于所述预设阈值时的图像去噪模型确定为所述图像去噪模型。
  5. 根据权利要求4所述的方法，其中，所述获取第一训练噪声图像以及对应的第一训练真值图像，包括：
    根据在第一预设光源亮度下拍摄的多张目标图像，生成第二训练真值图像；
    将在第二预设光源亮度与第三预设光源亮度范围内，随机调节光源亮度拍摄的多张图像，作为第二训练噪声图像；其中，所述第二预设光源亮度小于所述第三预设光源亮度，所述第三预设光源亮度小于所述第一预设光源亮度；
    根据所述第二训练真值图像以及所述第二训练噪声图像，训练初始对抗网络，获取最终训练形成的对抗网络；
    将预先获取的第三训练真值图像输入至所述对抗网络中，获取所述对抗网络的生成网络输出的第三训练噪声图像；以及
    将所述第二训练噪声图像以及所述第三训练噪声图像均作为所述第一训练图像，将所述第二训练真值图像以及所述第三训练真值图像均作为所述对应的第一训练真值图像。
  6. 根据权利要求5所述的方法，其中，所述根据在第一预设光源亮度下拍摄的多张目标图像，生成第二训练真值图像，包括：
    根据在无光线环境中拍摄的多张图像，生成所述多张图像的平均图像；
    针对每张目标图像，比较所述目标图像的每个像素点的第一像素值与所述平均图像中对应位置处的像素点的第二像素值；
    响应于确定所述第一像素值与所述第二像素值的差值的绝对值大于预设第一像素差值阈值，采用双线性插值法，确定所述第一像素值更新后的值，形成更新后的目标图像；以及
    根据所述更新后的目标图像，生成所述第二训练真值图像。
  7. 根据权利要求6所述的方法，其中，所述根据所述更新后的目标图像，生成第二训练真值图像，包括：
    针对每个更新后的目标图像，确定所述更新后的目标图像中所有像素点的平均像素值；
    根据每个更新后的目标图像的平均像素值，确定多个更新后的目标图像的平均像素值的平均值；
    将对应的平均像素值与所述平均像素值的平均值的差值的绝对值大于预设第二像素差值阈值的更新后的目标图像删除，形成过滤后的更新目标图像；以及
    根据所述过滤后的更新目标图像，生成所述第二训练真值图像。
  8. 根据权利要求7所述的方法，其中，所述根据所述过滤后的更新目标图像，生成所述第二训练真值图像，包括：
    针对每个过滤后的更新目标图像，响应于确定所述过滤后的更新目标图像中的第一位置处的像素值小于或者等于预设第一像素阈 值或者大于或者等于预设第二像素阈值，将所述第一位置处的像素值更新为所有过滤后的更新目标图像的第一位置处的像素值中，出现次数最多的像素值，生成过滤后的二次更新目标图像；其中，所述第一像素阈值小于所述第二像素阈值；以及
    将所有过滤后的二次更新目标图像的平均图像，确定为所述第二训练真值图像。
  9. 根据权利要求5所述的方法，其中，根据所述第二训练真值图像以及所述第二训练噪声图像，训练初始对抗网络，获取最终训练形成的对抗网络，包括：
    将正态分布噪声、所述第二训练真值图像、任一第二训练噪声图像对应的摄像头参数，输入至初始对抗网络中的初始生成网络，获取输出的噪声图像；
    将所述噪声图像、所述第二训练噪声图像、所述第二训练噪声图像对应的摄像头参数以及所述第二训练真值图像，输入至所述初始对抗网络中的初始判别网络中，获取所述初始判别网络的输出概率；以及
    根据所述输出概率、所述第二训练噪声图像对应的摄像头参数、所述第二训练真值图像、所述第二训练噪声图像以及所述噪声图像，确定所述初始判别网络的损失函数以及所述初始生成网络的损失函数，响应于确定根据所述初始判别网络的损失函数以及所述初始生成网络的损失函数确定所述初始对抗网络不收敛，交替执行返回执行将正态分布噪声、所述第二训练真值图像、所述第二训练噪声图像对应的摄像头参数，输入至初始对抗网络中的初始生成网络，获取输出的噪声图像的步骤，以及，返回将所述噪声图像、所述第二训练噪声图像、所述第二训练噪声图像对应的摄像头参数以及所述第二训练真值图像，输入至所述初始对抗网络中的初始判别网络中，获取所述初始判别网络的输出概率的步骤，直至根据所述初始判别网络的损失函数以及所述初始生成网络的损失函数确定对抗网络收敛，将收敛的对抗网络确定为所述最终训练形成的对抗网络。
  10. 根据权利要求5-9任一项所述的方法，其中，所述对抗网络中的生成网络以及判别网络均为U型网络。
  11. 一种图像去噪装置，包括：
    第一获取模块，被配置为获取待处理图像；以及
    第二获取模块，被配置为将所述待处理图像输入至图像去噪模型中，获取去噪后的图像；其中，所述图像去噪模型为结合U型网络、残差网络以及稠密网络形成的模型。
  12. 一种电子设备，包括：
    一个或多个处理器；以及
    存储器，被配置为存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现根据权利要求1-10中任一项所述的图像去噪方法。
  13. 一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现根据权利要求1-10中任一项所述的图像去噪方法。
PCT/CN2021/088927 2020-04-27 2021-04-22 图像去噪方法及装置、电子设备以及存储介质 Ceased WO2021218765A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21795374.4A EP4145384B1 (en) 2020-04-27 2021-04-22 Image denoising method and apparatus, electronic device and storage medium
US17/921,271 US12394023B2 (en) 2020-04-27 2021-04-22 Image denoising method and apparatus, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010346311.9A CN113643189B (zh) 2020-04-27 2020-04-27 图像去噪方法、装置和存储介质
CN202010346311.9 2020-04-27

Publications (1)

Publication Number Publication Date
WO2021218765A1 true WO2021218765A1 (zh) 2021-11-04

Family

ID=78332093

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/088927 Ceased WO2021218765A1 (zh) 2020-04-27 2021-04-22 图像去噪方法及装置、电子设备以及存储介质

Country Status (4)

Country Link
US (1) US12394023B2 (zh)
EP (1) EP4145384B1 (zh)
CN (1) CN113643189B (zh)
WO (1) WO2021218765A1 (zh)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114298202A (zh) * 2021-12-23 2022-04-08 上海高德威智能交通系统有限公司 一种图像加密方法、装置、电子设备及存储介质
CN114494047A (zh) * 2022-01-11 2022-05-13 辽宁师范大学 一种基于双增强残差网络的生物图像去噪方法
CN114724144A (zh) * 2022-05-16 2022-07-08 北京百度网讯科技有限公司 文本识别方法、模型的训练方法、装置、设备及介质
CN114757841A (zh) * 2022-03-30 2022-07-15 西北核技术研究所 一种基于噪声特性的激光诱导荧光图像去噪方法
CN114898113A (zh) * 2022-04-29 2022-08-12 际络科技(上海)有限公司 一种对象检测方法、装置、电子设备及存储介质
CN116029946A (zh) * 2023-03-29 2023-04-28 中南大学 基于异构残差注意神经网络模型的图像去噪方法及系统
CN116723412A (zh) * 2023-08-10 2023-09-08 四川玉米星球科技有限公司 一种照片中背景光影均化方法及文本拍摄扫描系统
CN116739907A (zh) * 2022-03-01 2023-09-12 Oppo广东移动通信有限公司 图像去噪方法、装置、设备和计算机可读存储介质
CN116915549A (zh) * 2023-05-18 2023-10-20 重庆邮电大学 一种基于轻量级高效LEU-Net的1-比特大规模MIMO信道估计方法
CN117197014A (zh) * 2023-09-12 2023-12-08 南京诺源医疗器械有限公司 一种可降噪的肺部医学图像融合方法、系统和电子设备
CN117291824A (zh) * 2022-06-20 2023-12-26 北京小米移动软件有限公司 图像处理方法、装置、电子设备及存储介质
CN117315090A (zh) * 2023-09-27 2023-12-29 中国科学院自动化研究所 基于跨模态风格学习的图像生成方法及装置
CN117830079A (zh) * 2023-12-27 2024-04-05 北京智象未来科技有限公司 真实图片预测方法、装置、设备、存储介质
CN117876943A (zh) * 2024-03-12 2024-04-12 荣耀终端有限公司 一种图像处理方法、图像增强模型的训练方法及电子设备
CN118279181A (zh) * 2024-05-31 2024-07-02 杭州海康威视数字技术股份有限公司 可调参图像复原模型的训练方法和可调参的图像复原方法
CN121724861A (zh) * 2026-02-12 2026-03-24 清华大学 一种基于多专家判别的显微去噪方法

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023041375A (ja) * 2021-09-13 2023-03-24 キヤノン株式会社 情報処理装置、情報処理方法及びプログラム
CN114418873B (zh) * 2021-12-29 2022-12-20 英特灵达信息技术(深圳)有限公司 一种暗光图像降噪方法及装置
CN115115531B (zh) * 2022-01-14 2025-10-28 长城汽车股份有限公司 图像去噪方法、装置、车辆及存储介质
CN114897725B (zh) * 2022-05-09 2025-01-07 马上消费金融股份有限公司 图像降噪方法、装置、设备及存储介质
CN115222667A (zh) * 2022-06-21 2022-10-21 Oppo广东移动通信有限公司 图像处理方法及装置、计算机可读存储介质和电子设备
CN115526857A (zh) * 2022-09-26 2022-12-27 深圳先进技术研究院 一种pet图像去噪的方法、终端设备及可读存储介质
CN116051408B (zh) * 2023-01-06 2023-10-27 郑州轻工业大学 一种基于残差自编码的图像深度去噪方法
CN116630837B (zh) * 2023-03-13 2025-02-11 北京邮电大学 双阶段视频去噪模型训练方法、视频去噪方法和系统
CN116721023A (zh) * 2023-05-22 2023-09-08 北京奕斯伟计算技术股份有限公司 图像去噪模型训练方法、图像去噪方法、装置和电子设备
CN116912111A (zh) * 2023-06-21 2023-10-20 中国联合网络通信集团有限公司 一种图像去噪方法、图像处理模型的训练方法及装置
CN116977650B (zh) * 2023-07-31 2024-11-12 西北工业大学深圳研究院 图像去噪方法、图像去噪装置、电子设备及存储介质
CN116862903B (zh) * 2023-07-31 2024-06-25 梅卡曼德（北京）机器人科技有限公司 缺陷检测模型训练方法及装置、缺陷检测方法和电子设备
CN117291846B (zh) * 2023-11-27 2024-02-27 北京大学第三医院（北京大学第三临床医学院） 一种应用于喉显微外科手术的oct系统及图像去噪方法
CN117351216B (zh) * 2023-12-05 2024-02-02 成都宜图智享信息科技有限公司 一种基于监督深度学习的图像自适应去噪方法
CN117392615B (zh) * 2023-12-12 2024-03-15 南昌理工学院 一种基于监控视频的异常识别方法及系统
CN117474797B (zh) * 2023-12-28 2024-03-19 南京信息工程大学 一种多尺度互补学习的图像去噪方法及装置
CN117495687B (zh) * 2023-12-29 2024-04-02 清华大学深圳国际研究生院 一种水下图像增强方法
CN117541501B (zh) * 2024-01-09 2024-05-31 清华大学 扫描光场自监督网络去噪方法、装置、电子设备及介质
CN117934944A (zh) * 2024-01-24 2024-04-26 广东工业大学 一种基于裁剪量化的图像分类方法、装置及设备
CN117726542B (zh) * 2024-02-18 2024-06-21 北京理工大学 基于扩散模型的可控噪声去除方法及系统
CN118229818B (zh) * 2024-04-10 2025-05-13 国家卫星气象中心(国家空间天气监测预警中心) 一种基于气象卫星数据的合成云图生成方法及系统
CN118365954B (zh) * 2024-05-14 2025-04-11 广东电网有限责任公司 钢筋绑扎质量的确定方法、装置、处理器和电子设备
CN118608417B (zh) * 2024-05-31 2025-01-24 重庆交通大学 基于深度学习的遥感图像复合噪声去除方法
CN118898552B (zh) * 2024-10-08 2025-02-28 浙江大学滨江研究院 基于改进聚类的多阶段oct图像去噪方法、设备和存储介质
CN119992307B (zh) * 2025-04-17 2026-02-13 上海达华测绘科技有限公司 基于声呐图像的水下目标检测方法、装置、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190128989A1 (en) * 2017-11-01 2019-05-02 Siemens Healthcare Gmbh Motion artifact reduction of magnetic resonance images with an adversarial trained network
CN110232361A (zh) * 2019-06-18 2019-09-13 中国科学院合肥物质科学研究院 基于三维残差稠密网络的人体行为意图识别方法与系统
CN110443867A (zh) * 2019-08-01 2019-11-12 太原科技大学 基于生成对抗网络的ct图像超分辨率重构方法
CN110610464A (zh) * 2019-08-15 2019-12-24 天津中科智能识别产业技术研究院有限公司 一种基于稠密残差神经网络的人脸图像超分辨率方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110753935A (zh) * 2017-04-25 2020-02-04 小利兰·斯坦福大学托管委员会 用于医学成像的使用深度卷积神经网络的剂量减少
US11756160B2 (en) * 2018-07-27 2023-09-12 Washington University ML-based methods for pseudo-CT and HR MR image estimation
US11164067B2 (en) * 2018-08-29 2021-11-02 Arizona Board Of Regents On Behalf Of Arizona State University Systems, methods, and apparatuses for implementing a multi-resolution neural network for use with imaging intensive applications including medical imaging
CN110473154B (zh) * 2019-07-31 2021-11-16 西安理工大学 一种基于生成对抗网络的图像去噪方法
CN110599409B (zh) * 2019-08-01 2022-02-22 西安理工大学 基于多尺度卷积组与并行的卷积神经网络图像去噪方法
CN110782395B (zh) * 2019-10-28 2024-02-09 西安电子科技大学 图像处理方法及装置、电子设备和计算机可读存储介质
CN110910329B (zh) * 2019-11-27 2022-09-02 中国科学技术大学 一种以需求为导向的图像去噪方法
US11508037B2 (en) * 2020-03-10 2022-11-22 Samsung Electronics Co., Ltd. Systems and methods for image denoising using deep convolutional networks
US11948274B1 (en) * 2021-01-05 2024-04-02 Pixar Deep learned super resolution for feature film production
US20230419447A1 (en) * 2022-06-23 2023-12-28 Bitmovin Gmbh Lightweight Dense Residual Network for Video Super-Resolution on Mobile Devices
US20240062332A1 (en) * 2022-08-12 2024-02-22 Beth Israel Deaconess Medical Center, Inc. System and method for improving sharpness of magnetic resonance images using a deep learning neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190128989A1 (en) * 2017-11-01 2019-05-02 Siemens Healthcare Gmbh Motion artifact reduction of magnetic resonance images with an adversarial trained network
CN110232361A (zh) * 2019-06-18 2019-09-13 中国科学院合肥物质科学研究院 基于三维残差稠密网络的人体行为意图识别方法与系统
CN110443867A (zh) * 2019-08-01 2019-11-12 太原科技大学 基于生成对抗网络的ct图像超分辨率重构方法
CN110610464A (zh) * 2019-08-15 2019-12-24 天津中科智能识别产业技术研究院有限公司 一种基于稠密残差神经网络的人脸图像超分辨率方法

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114298202A (zh) * 2021-12-23 2022-04-08 上海高德威智能交通系统有限公司 一种图像加密方法、装置、电子设备及存储介质
CN114494047A (zh) * 2022-01-11 2022-05-13 辽宁师范大学 一种基于双增强残差网络的生物图像去噪方法
CN114494047B (zh) * 2022-01-11 2024-04-02 辽宁师范大学 一种基于双增强残差网络的生物图像去噪方法
CN116739907A (zh) * 2022-03-01 2023-09-12 Oppo广东移动通信有限公司 图像去噪方法、装置、设备和计算机可读存储介质
CN114757841A (zh) * 2022-03-30 2022-07-15 西北核技术研究所 一种基于噪声特性的激光诱导荧光图像去噪方法
CN114898113A (zh) * 2022-04-29 2022-08-12 际络科技(上海)有限公司 一种对象检测方法、装置、电子设备及存储介质
CN114724144A (zh) * 2022-05-16 2022-07-08 北京百度网讯科技有限公司 文本识别方法、模型的训练方法、装置、设备及介质
CN114724144B (zh) * 2022-05-16 2024-02-09 北京百度网讯科技有限公司 文本识别方法、模型的训练方法、装置、设备及介质
CN117291824A (zh) * 2022-06-20 2023-12-26 北京小米移动软件有限公司 图像处理方法、装置、电子设备及存储介质
CN116029946A (zh) * 2023-03-29 2023-04-28 中南大学 基于异构残差注意神经网络模型的图像去噪方法及系统
CN116915549A (zh) * 2023-05-18 2023-10-20 重庆邮电大学 一种基于轻量级高效LEU-Net的1-比特大规模MIMO信道估计方法
CN116723412B (zh) * 2023-08-10 2023-11-10 四川玉米星球科技有限公司 一种照片中背景光影均化方法及文本拍摄扫描系统
CN116723412A (zh) * 2023-08-10 2023-09-08 四川玉米星球科技有限公司 一种照片中背景光影均化方法及文本拍摄扫描系统
CN117197014A (zh) * 2023-09-12 2023-12-08 南京诺源医疗器械有限公司 一种可降噪的肺部医学图像融合方法、系统和电子设备
CN117197014B (zh) * 2023-09-12 2024-02-20 南京诺源医疗器械有限公司 一种可降噪的肺部医学图像融合方法、系统和电子设备
CN117315090A (zh) * 2023-09-27 2023-12-29 中国科学院自动化研究所 基于跨模态风格学习的图像生成方法及装置
CN117830079A (zh) * 2023-12-27 2024-04-05 北京智象未来科技有限公司 真实图片预测方法、装置、设备、存储介质
CN117876943A (zh) * 2024-03-12 2024-04-12 荣耀终端有限公司 一种图像处理方法、图像增强模型的训练方法及电子设备
CN118279181A (zh) * 2024-05-31 2024-07-02 杭州海康威视数字技术股份有限公司 可调参图像复原模型的训练方法和可调参的图像复原方法
CN121724861A (zh) * 2026-02-12 2026-03-24 清华大学 一种基于多专家判别的显微去噪方法

Also Published As

Publication number Publication date
EP4145384A1 (en) 2023-03-08
US20230230206A1 (en) 2023-07-20
US12394023B2 (en) 2025-08-19
CN113643189A (zh) 2021-11-12
EP4145384A4 (en) 2024-05-15
CN113643189B (zh) 2025-01-10
EP4145384B1 (en) 2026-02-11

Similar Documents

Publication Publication Date Title
WO2021218765A1 (zh) 图像去噪方法及装置、电子设备以及存储介质
TWI897954B (zh) 保持用於幀中的目標物件的固定尺寸
US11704775B2 (en) Bright spot removal using a neural network
US11538175B2 (en) Method and apparatus for detecting subject, electronic device, and computer readable storage medium
CN110428366B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
KR102097905B1 (ko) 이미지 객체 인식 장치 및 방법
WO2021057474A1 (zh) 主体对焦方法、装置、电子设备和存储介质
CN104796583B (zh) 相机噪声模型产生及使用方法以及使用该方法的装置
Song et al. Matching in the dark: A dataset for matching image pairs of low-light scenes
CN107465855A (zh) 图像的拍摄方法及装置、无人机
KR20220089602A (ko) 무보정 광각 이미지 기반 가변 합성곱 신경망 학습 방법 및 장치
CN107742115A (zh) 一种基于视频监控的运动目标物检测追踪的方法及系统
CN111008555B (zh) 一种无人机图像弱小目标增强提取方法
CN116797471B (zh) 一种基于深度学习的水下目标图像检测方法和系统
US20230132230A1 (en) Efficient Video Execution Method and System
US20230125040A1 (en) Temporally Consistent Neural Network Processing System
CN113269695A (zh) 一种图像去模糊方法、系统、装置及存储介质
CN118644562B (zh) 一种模型训练方法、三维点云获取方法及电子设备
CN111144312B (zh) 图像处理方法、装置、设备和介质
Wischow et al. Camera Condition Monitoring and Readjustment by means of Noise and Blur
Qian et al. Adaptive domain learning for cross-domain image denoising
CN118279829A (zh) 一种低照度宽视场视频图像变化检测方法及装置
CN119810001A (zh) 图像处理方法、装置、计算机设备和存储介质
CN117541485A (zh) 图像显示方法、装置、计算机设备和存储介质
WO2022153474A1 (ja) 画像処理装置、画像処理方法、およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21795374

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021795374

Country of ref document: EP

Effective date: 20221128

WWG Wipo information: grant in national office

Ref document number: 17921271

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2021795374

Country of ref document: EP