WO2021135702A1 - 一种视频去噪方法和电子设备 - Google Patents

一种视频去噪方法和电子设备 Download PDF

Info

Publication number
WO2021135702A1
WO2021135702A1 PCT/CN2020/129864 CN2020129864W WO2021135702A1 WO 2021135702 A1 WO2021135702 A1 WO 2021135702A1 CN 2020129864 W CN2020129864 W CN 2020129864W WO 2021135702 A1 WO2021135702 A1 WO 2021135702A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
gpu
denoising
video frame
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/129864
Other languages
English (en)
French (fr)
Inventor
黄懿
杜凌霄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Pte Ltd
Original Assignee
Bigo Technology Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bigo Technology Pte Ltd filed Critical Bigo Technology Pte Ltd
Priority to EP20908710.5A priority Critical patent/EP4086840B1/en
Priority to US17/789,974 priority patent/US12354242B2/en
Publication of WO2021135702A1 publication Critical patent/WO2021135702A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation

Definitions

  • This application relates to the field of image processing technology, and in particular to a video denoising method and an electronic device.
  • the existing image denoising algorithms include pixel-based denoising algorithms, block matching-based denoising algorithms, transform domain-based denoising algorithms, and machine learning denoising algorithms. In actual use, these algorithms either have poor denoising effect, or the algorithm takes a long time to execute.
  • Non-local Bayes (NLB) denoising algorithm is a compromise choice in denoising effect and execution time obtained by experiments in recent years.
  • the NLB denoising method is a two-step denoising.
  • a negative definite matrix may appear due to the subtraction operation of the covariance matrix, which makes the image denoising effect not completely stable.
  • the embodiments of the present application provide a video denoising method and a corresponding electronic device to improve the stability of the denoising effect.
  • an embodiment of the present application discloses a video denoising method, which is applied to an electronic device having a central processing unit (CPU) and an image processing unit GPU.
  • the method includes:
  • the CPU obtains video data, and decodes the video data to obtain a video frame image
  • the GPU loads the video frame image from the CPU
  • the GPU denoises the video frame image according to a preset non-local average NLM denoising algorithm to obtain a first image
  • the GPU performs denoising on the first image according to a preset non-local Bayesian NLB denoising algorithm to obtain a second image;
  • the CPU obtains the second image from the GPU, and encodes the second image to obtain denoised video data.
  • the embodiment of the application also discloses a video denoising method, which is applied to an electronic device with a CPU and a GPU, and the method includes:
  • the CPU obtains video data, and decodes the video data to obtain a video frame image
  • the GPU loads the video frame image from the CPU
  • the GPU uses a preset first image denoising algorithm to denoise the video frame image process to obtain the first image, or uses a preset first image denoising algorithm to denoise the video frame Image process denoising to obtain a first image, and using a preset second image denoising algorithm to denoise the first image to obtain a second image;
  • the CPU obtains a first image from the GPU and encodes the first image to obtain denoised video data; or obtains a second image from the GPU and encodes the second image to obtain Video data after denoising.
  • the embodiment of the application also discloses an electronic device, including a central processing unit CPU and an image processor GPU;
  • the CPU includes:
  • the video data decoding module is used to obtain video data, and decode the video data to obtain a video frame image
  • a video data encoding module configured to obtain a second image from the GPU, and encode the second image to obtain denoised video data
  • the GPU includes:
  • a video frame image loading module configured to load the video frame image from the CPU
  • the first image denoising module is configured to denoise the video frame image according to a preset non-local average NLM denoising algorithm to obtain a first image;
  • the second image denoising module is used to denoise the first image according to a preset non-local Bayesian NLB denoising algorithm to obtain a second image.
  • the embodiment of the application also discloses an electronic device, including a CPU and a GPU;
  • the CPU includes:
  • the video data decoding module is used to obtain video data, and decode the video data to obtain a video frame image
  • the video data encoding module is configured to obtain a first image from the GPU, and encode the first image to obtain denoised video data; or, obtain a second image from the GPU, and perform analysis on the second image.
  • the image is encoded to obtain the denoised video data;
  • the GPU includes:
  • a video frame image loading module configured to load the video frame image from the CPU
  • a noise level determining module configured to determine the noise level of the video frame image
  • the image denoising module is configured to use a preset first image denoising algorithm to process the video frame image to obtain the first image according to the noise level, or use a preset first image denoising algorithm to The video frame image process denoises to obtain a first image, and a preset second image denoising algorithm is used to denoise the first image to obtain a second image.
  • the embodiment of the present application also discloses a device, including: a processor, a memory, and a computer program stored on the memory and capable of running on the processor.
  • the computer program is executed by the processor, the implementation is as follows: Steps of any of the video denoising methods described above.
  • the embodiment of the present application also discloses a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video denoising method as described in any of the above are implemented .
  • the NLM denoising algorithm is used to replace the NLB denoising algorithm to perform the first denoising, avoiding the use of the NLB denoising algorithm in the first step. Denoising, leading to the problem of unstable denoising effect.
  • FIG. 1 is a flowchart of steps of a video denoising method provided by an embodiment of the present application
  • FIG. 2 is a flowchart of the steps of another video denoising method provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a first search box corresponding to a first sub-image in an embodiment of the present application
  • FIG. 4 is a flowchart of the steps of another video denoising method provided by an embodiment of the present application.
  • FIG. 5 is a structural block diagram of an electronic device provided by an embodiment of the present application.
  • Fig. 6 is a structural block diagram of another electronic device provided by an embodiment of the present application.
  • the NLB denoising method uses two-step denoising.
  • the reason for the two-step denoising is that the denoising effect of the NLB algorithm depends on the accuracy of the grouping of similar blocks.
  • the first step due to the presence of noise, the grouping of blocks will be disturbed.
  • the image with a lot of noise interference removed will be more accurate for block matching.
  • the original noisy image and the denoised image are divided into blocks according to the denoised image, and the noisy image is used in the NLB algorithm for denoising again.
  • Another reason is that the mean and variance of the image after denoising is closer to the image without noise.
  • the NLB denoising algorithm is:
  • P is the estimation of the image block after denoising
  • Is the expectation of a noisy image block
  • ⁇ is the noise variance
  • the traditional NLB denoising method adopts the NLB denoising algorithm in two steps of denoising.
  • denoising in one step is the NLB denoising algorithm in two steps of denoising.
  • One of the core concepts of the embodiments of the present application is that the two-step denoising mechanism of the NLB denoising method is used, and the two steps of denoising can be optimized separately.
  • non-local mean NLM (non-local means) denoising can be used in the first step of denoising.
  • the NLM denoising algorithm is used to denoise, the NLB denoising is used for the second denoising.
  • the NLM denoising algorithm is also optimized and accelerated in the embodiment of the present application.
  • the embodiment of the present application also optimizes the NLB denoising algorithm and accelerates the operation.
  • the above-mentioned improved algorithm is not only suitable for video denoising, but also for denoising images in other fields, such as drone images, remote sensing images, scanned images, fax images, medical images, military reconnaissance images, and so on.
  • Step 101 The CPU obtains video data, and decodes the video data to obtain a video frame image
  • the user can input the captured video to an electronic device with a CPU and GPU for denoising processing.
  • the CPU obtains the video data taken by the user, decodes the video data into multiple video frame images, and performs image denoising processing by the GPU.
  • both CPU and GPU can run image denoising algorithms. Based on the structural characteristics of GPU, GPU is more suitable for image processing.
  • the execution process of the image denoising algorithm can be improved according to the multi-threaded parallel structure of the GPU, and the image denoising process can be accelerated by the GPU to shorten the processing time.
  • Step 102 The GPU loads the video frame image from the CPU;
  • the GPU can allocate the video memory size according to the size of the video frame image, and load the video frame image into the video memory.
  • Step 103 The GPU denoises the video frame image according to a preset non-local average NLM denoising algorithm to obtain a first image;
  • the NLB denoising method uses two-step denoising.
  • the two-step denoising is implemented by using the NLB denoising algorithm.
  • the NLM denoising algorithm is used to replace the NLB denoising algorithm to perform the first step of denoising.
  • Step 104 The GPU denoises the first image according to a preset non-local Bayesian NLB denoising algorithm to obtain a second image;
  • the GPU uses the NLB denoising algorithm to denoise each pixel of the first image to obtain the second image.
  • Step 105 The CPU obtains the second image from the GPU, and encodes the second image to obtain denoised video data.
  • the CPU encodes all the second images to obtain the denoised video data.
  • the NLM denoising algorithm is used to replace the NLB denoising algorithm to perform the first denoising, avoiding the use of the NLB denoising algorithm in the first step. Denoising, leading to the problem of unstable denoising effect.
  • FIG. 2 there is shown a step flow chart of another video denoising method provided by an embodiment of the present application.
  • the modified method is applied to an electronic device with a central processing unit (CPU) and an image processing unit (GPU).
  • the specific method may include the following steps :
  • Step 201 The CPU obtains video data, and decodes the video data to obtain a video frame image
  • Step 202 The CPU fills the video frame image
  • the image block is usually determined with the pixel as the center. For example, a 5x5 image block is determined by one pixel. For the pixels on the border of the video frame, the image block cannot be determined with it as the center, so the image block matching operation cannot be performed. Therefore, it is necessary to fill in the original video frame image, so that the pixels of the original boundary can also perform the operation of the image block without separate judgment.
  • Step 203 The GPU loads the video frame image from the CPU;
  • the GPU is loaded from the CPU, and the video frame image is filled by the CPU.
  • Step 204 The GPU denoises the video frame image according to a preset non-local average NLM denoising algorithm to obtain a first image;
  • the NLM denoising algorithm is optimized to improve the effect of image denoising.
  • a reference block for each pixel in the image, construct a patch with it as the center, called a reference block, for example, an image block with a size of 5x5; then construct a patch with the reference block as the center Search box, such as a 16x16 search box.
  • search for similar image blocks of the reference image block in the search box Specifically, for each pixel in the search box, use this point as the center to construct an adjacent image block, and then determine whether the adjacent image block is Similar image blocks.
  • the similarity of two image blocks can be measured by pixel distance. Compare the pixel distance between all pixels in the two image blocks:
  • i is the reference image block
  • j is the adjacent image block.
  • the 25 pixels of one image block are sequentially set to the pixel distance of the pixel at the corresponding position of the other image block.
  • the pixel distance is measured in absolute value.
  • the pixel distance refers to the difference in pixel values. For example, if the value of one pixel is 255 and the value of another pixel is 240, the pixel distance is 15.
  • the adjacent image block is considered to be a similar image block.
  • Z(i) is a normalized parameter, that is, the sum of all weights.
  • h 2 is the size of the image block.
  • the existing weight calculation method only considers the pixel distance, and the traditional NLM algorithm will blur the edges of the image while denoising.
  • the calculation method of the weight is improved, and the weight is calculated by considering the coordinate distance, which improves the edge preservation of the algorithm and better preserves the details of the image edge after denoising.
  • the coordinate distance refers to the distance between two pixels in the image, that is, the closer the point to the reference point, the greater the weight.
  • the weight can be specifically calculated according to the following formula:
  • C(i) is a normalized parameter, Is the original pixel distance, which represents the pixel distance of the entire image block; d(i,j) represents the pixel distance between pixel i and pixel j.
  • D(i,j) represents the coordinate distance between the pixel point i and the pixel point j
  • ⁇ s is the variance of the pixel distance
  • the curve of the weight function is made steeper through the function f(x), which can reduce the weight of image blocks with lower similarity.
  • Similar image blocks of the reference image block will include the reference image block itself, so the reference image block itself occupies a large weight. If the center pixel of the reference block itself is a noise point, it will not be conducive to noise removal.
  • This embodiment of this application has been improved by multiplying the weight of the reference image block itself by a coefficient less than 1 (for example, 0.5), by reducing the weight of the reference image block itself, thereby reducing the pixel itself is a single point of noise. The error.
  • the step 204 may include the following sub-steps:
  • Sub-step S11 the GPU allocates threads to the pixels of the video frame image
  • the GPU executes the improved NLM algorithm for the pixel allocation thread of the video frame image.
  • the GPU can allocate a thread for multiple pixels, that is, a thread sequentially denoises multiple pixels according to the improved NLM algorithm; the GPU can also One thread is allocated to one pixel, that is, one thread only denoises one pixel according to the improved NLM algorithm.
  • GPU acceleration algorithm execution with a unified computing device architecture may also be used.
  • GPU threads are organized in a grid, and each grid contains several thread blocks. Many threads in the same thread block have the same instruction address, which can not only execute in parallel, but also share memory (Shared memory) realizes intra-block communication.
  • each thread block is allocated to the hardware streaming processor SM (Streaming Multiprocessor) to perform operations; and the threads in the thread block will use warp as a unit to perform grouping calculations on the threads.
  • SM streaming Multiprocessor
  • the warp size of CUDA is all 32, which means that 32 threads will be combined into a warp to execute together.
  • the instructions executed by the threads in the same warp are the same, but the processed data is different.
  • a stream processor can only execute one warp in a thread block at a time.
  • the thread block is usually set to a multiple of 32. For example, the thread block can be set to 8x8, 16x16, 32x32 and other sizes.
  • the feature settings based on CUDA are optimized as follows:
  • the GPU divides the video frame image into a plurality of first sub-images; the GPU allocates threads corresponding to pixels belonging to the same first sub-image to the same thread block; The GPU allocates the data required for the execution of the NLM denoising algorithm to the thread block; when the threads in the same thread block are executed according to the warp, the threads in the warp coordinate the required data
  • the shared memory of the GPU is read.
  • the video frame image is divided into multiple 8x8 sub-images, and the threads responsible for processing the pixels of this sub-image are allocated to the same thread block. If each pixel in the sub-image is in charge of one thread, that is, 64 threads are responsible for an 8x8 sub-image, the size of the thread block can be set to 64, and the thread block with the size of 64 is executed in two warps.
  • the data required by the thread block to execute the NLM denoising algorithm refers to the data of the search box.
  • FIG. 3 is a schematic diagram of the first search box corresponding to the first sub-image in an embodiment of this application.
  • search for a first similar image block corresponding to the first reference image block centered on this pixel in the same first search box For a pixel in the same first sub-image, search for a first similar image block corresponding to the first reference image block centered on this pixel in the same first search box. Since the threads of the same first sub-image are in the same thread block, the data of the first search box can be read to the shared memory based on the cooperative operation of the threads in the same thread block.
  • each thread needs to read the 16x16 search box.
  • the thread block of size 64 each thread only needs to read 4 pixels to the shared memory, which greatly reduces the amount of data read by the thread and improves the processing speed of the algorithm.
  • the current NLM denoising algorithm provided by Opencv reads data on the global memory of the GPU, which is a very slow data reading method.
  • the embodiment of the present application can greatly increase the data reading speed by reading all the data required by the NLM denoising algorithm into the shared memory.
  • Sub-step S12 the thread determines the first reference image block of the pixel and the corresponding first similar image block
  • the thread searches for the first similar image block corresponding to the first reference image block in the first search box. Specifically, the thread determines an adjacent image block with each pixel in the first search box as the center, and then determines whether all adjacent image blocks are similar image blocks. In some cases, a large number of first similar image blocks may be found (for example, hundreds of first similar image blocks). If the subsequent algorithm operation process is executed with a large number of similar blocks, it will take up too much memory. Because the GPU algorithm is not suitable for occupying too much memory, it is not suitable for sorting, and it is not suitable for further filtering among the first similar images found.
  • the search area may be reduced, so that all adjacent image blocks in the first search box are not judged whether they are the first similar image blocks; and According to the preset first pixel interval, adjacent image blocks are selected to determine whether they are the first similar image blocks.
  • the sub-step S12 may further include:
  • Sub-step S121 the thread determines the first reference image block with the pixel as the center
  • Sub-step S122 the thread obtains the corresponding first search box from the shared memory of the GPU;
  • the thread can obtain the corresponding first search box from the shared memory .
  • sub-step S123 the thread determines the first similar image block corresponding to the first reference image block in the first search box at a preset first pixel interval.
  • Sub-step S13 the thread determines the pixel distance and the coordinate distance of the first similar image relative to the first reference image block
  • the thread calculates the weight of the first similar image block by using the pixel distance and the coordinate distance;
  • sub-step S15 the thread uses the weight of the first similar image block to determine the pixel value of the pixel after denoising
  • the weighted average value of the weights of the similar image blocks and the pixel values of the central pixels of the similar image blocks is used as the denoised pixel value of the central pixel of the reference image block.
  • the GPU uses all the images determined by the denoised pixels as the first image.
  • Step 205 The GPU denoises the first image according to a preset non-local Bayesian NLB denoising algorithm to obtain a second image;
  • the GPU uses the NLB denoising algorithm to denoise each pixel of the first image to obtain the second image.
  • Step 206 The CPU obtains the second image from the GPU, and encodes the second image to obtain denoised video data.
  • the CPU encodes all the second images to obtain the denoised video data.
  • the embodiment of the application also optimizes the NLB denoising algorithm.
  • P is the estimated value of the image block after denoising
  • the mean value of the noisy image block The mean value of the noisy image block
  • the covariance matrix of the noisy image block ⁇ is the noise variance.
  • the formula for the first step of denoising is:
  • Is the estimated value of the image block after the first step of denoising Is the estimated value of the image block after denoising in the second step, Is the mean value of the image block in the image after the first step of denoising, It is the covariance matrix of the image block in the image after the first step of denoising.
  • the second step denoising formula is improved as:
  • the mean value of the image blocks in the original image is taken into the calculation of the mean, instead of the mean value of the image blocks in the image after the first step of denoising. Therefore, the second step of denoising can recover some The detailed information of the original image.
  • the step 206 may specifically include the following sub-steps:
  • Sub-step S21 the GPU allocates threads to the pixels of the first image
  • the feature settings based on CUDA are optimized as follows:
  • the GPU After the GPU allocates threads, the GPU divides the first image into a plurality of second sub-images; the GPU allocates threads corresponding to pixels belonging to the same second sub-image to the same thread block; The GPU allocates the data required for the execution of the NLB denoising algorithm to the thread block; when the threads in the same thread block are executed according to the warp, the threads in the warp coordinate the required data
  • the shared memory of the GPU is read.
  • the data required by the thread block to execute the NLB denoising algorithm refers to the data of the search box.
  • the same second search box is set for the pixels in the same second sub-image. For example, for a pixel in the same second sub-image, search for a second similar image block corresponding to a second reference image block centered on this pixel in the same second search box. Since the threads of the same second sub-image are in the same thread block, the data of the second search box can be read to the shared memory based on the cooperative operation of the threads in the same thread block. After reading the data required to execute the NLB denoising algorithm into the shared memory, all threads in the same thread block can read the data from the shared memory.
  • the number of registers in the GPU is limited, that is, the variables that can be opened on the GPU are limited, and the number of registers used (the number of variables) will affect concurrency.
  • the NLB denoising algorithm is more complicated and needs to use more local variables. Therefore, in the actual code implementation, on the one hand, it is necessary to allow the subsequent steps of the algorithm to re-use the previously opened array and so on. On the other hand, let the most frequently used variables in the algorithm be stored in the register.
  • the NLB denoising algorithm requires a large amount of local memory to store information such as similar blocks due to the need to calculate the covariance matrix. Therefore, a large amount of memory multiplexing can be used in the embodiments of the present application to reduce memory overhead.
  • Sub-step S22 the thread determines the second reference image block of the pixel and the corresponding second similar image block
  • the thread searches for the second similar image block corresponding to the second reference image block in the second search box. Specifically, the thread determines a neighboring image block with each pixel in the second search box as the center, and then performs all the neighboring images The block judges whether it is a similar image block.
  • the sub-step S22 may further include:
  • sub-step S221 the thread determines the second reference image block with the pixel as the center
  • Sub-step S222 the thread obtains the corresponding second search box from the shared memory of the GPU;
  • the thread can obtain the corresponding second search box from the shared memory .
  • the thread determines the second similar image block corresponding to the second reference image block in the second search box at a preset second pixel interval.
  • Sub-step S23 the thread uses the second similar image block and the second reference image block to calculate the covariance matrix of the second reference image block;
  • the covariance matrix of the second reference image block is the second step denoising formula after the above improvement
  • the thread uses the first reference image block and the first similar image block to calculate the average value of the first reference image block;
  • the mean value of the first reference image block is in the above-mentioned improved second step denoising formula It needs to be calculated based on the first reference image block of the determined pixel and the corresponding first similar image block in the original video frame image.
  • the thread uses the covariance matrix of the second reference image block and the mean value of the first reference image block to calculate the denoised pixel value of the second reference image block;
  • the estimated value after denoising of the second reference image block can be obtained according to the improved second step denoising formula described above.
  • sub-step S26 the thread uses all denoised second reference image blocks containing the same pixel to determine the denoised pixel value of the pixel.
  • all pixels in the second reference image block are denoised. That is, a pixel is denoised in the second reference image block centered on it, but also denoised in the second reference image block centered on other pixels. Therefore, a pixel will have more than one pixel.
  • the final estimated value of a pixel after denoising is: the average value of the estimated value of the pixel of all the second reference image blocks after the denoising of the pixel.
  • the GPU uses all the images determined by the denoised pixels as the second image.
  • the GPU may also determine the noise level of the video frame image; if the noise level is greater than or equal to the preset noise level threshold, the GPU pair The first image is denoised according to a preset non-local Bayesian NLB denoising algorithm to obtain a second image. If the noise level is less than the preset noise level threshold, the GPU does not perform denoising on the first image according to the preset non-local Bayesian NLB denoising algorithm to obtain the second image; the noise level is determined by The noise variance sigma is measured.
  • the noise level threshold is 10
  • the noise level of the video frame image is less than 10
  • only the NLM denoising algorithm is needed for denoising; if the noise level is greater than or equal to 10, the NLM denoising is used After the algorithm performs denoising, the NLB denoising algorithm is used to denoise.
  • the step of determining the noise level of the video frame image by the GPU may include:
  • the GPU determines the number of loaded video frame images; if the number of loaded video frame images is less than a preset frame number threshold, the GPU determines the noise level of the video frame images at a preset frame interval; the GPU The average value of the multiple noise levels determined is used as the noise level of the video frame image. If the number of the loaded video frame images is greater than or equal to the preset frame number threshold, the GPU selects a preset number of video frame images from the loaded video frame images; the GPU sets the preset number According to the preset frame interval, determine the noise level of the video frame image; the GPU will determine the average value of the multiple noise levels obtained as the noise level of the video frame image.
  • the preset frame number threshold is 500
  • an image is taken every other frame to evaluate its noise level
  • the noise level of all images in the entire video is the average value of the calculated noise level.
  • the noise level of all images in the entire video is the average value of the calculated noise level.
  • the NLM denoising algorithm is used to replace the NLB denoising algorithm to perform the first denoising, avoiding the use of the NLB denoising algorithm in the first step. Denoising, leading to the problem of unstable denoising effect.
  • FIG. 4 there is shown a step flow chart of another video denoising method provided by an embodiment of the present application.
  • the modified method is applied to an electronic device with a central processing unit CPU and an image processor GPU.
  • the specific steps may include the following steps :
  • Step 401 The CPU obtains video data, and decodes the video data to obtain a video frame image;
  • the CPU may also fill in the video frame image.
  • Step 402 the GPU loads the video frame image from the CPU
  • Step 403 The GPU determines the noise level of the video frame image
  • the step 403 may include: the GPU determines the number of loaded video frame images; if the number of the loaded video frame images is less than the preset number of frames threshold, the GPU presses the preset The frame interval determines the noise level of the video frame image; the GPU determines the average value of the multiple noise levels obtained as the noise level of the video frame image.
  • the GPU selects a preset number of video frame images from the loaded video frame images; the GPU sets the preset number According to the preset frame interval, determine the noise level of the video frame image; the GPU will determine the average value of the multiple noise levels obtained as the noise level of the video frame image.
  • the GPU uses a preset first image denoising algorithm to process the video frame image to obtain the first image according to the noise level, or uses a preset first image denoising algorithm to obtain the first image. Denoising the video frame image process to obtain a first image, and using a preset second image denoising algorithm to denoise the first image to obtain a second image;
  • the step 404 may include sub-steps:
  • sub-step S31 when the noise level is less than a preset noise level threshold, the GPU uses a preset first image denoising algorithm to denoise the video frame image to obtain a first image;
  • the sub-step S31 may include: the GPU allocates a thread to the pixels of the video frame image; the thread determines the first reference image block of the pixel and the corresponding first similar image block; the thread determines the The pixel distance and the coordinate distance of the first similar image relative to the first reference image block; the thread uses the pixel distance and the coordinate distance to calculate the weight of the first similar image block; the thread uses the The weight of the first similar image block determines the pixel value after the denoising of the pixel; the GPU uses all the images determined by the denoised pixel as the first image.
  • the GPU uses a preset first image denoising algorithm to process the video frame image to obtain a first image, and uses The preset second image denoising algorithm denoises the first image to obtain the second image.
  • the sub-step S32 may include:
  • the GPU allocates threads to the pixels of the video frame image; the thread determines the first reference image block of the pixel and the corresponding first similar image block; the thread determines that the first similar image is relative to the first similar image The pixel distance and coordinate distance of a reference image block; the thread uses the pixel distance and the coordinate distance to calculate the weight of the first similar image block; the thread uses the weight of the first similar image block to determine The pixel value after the denoising of the pixel; the GPU uses all the images determined by the denoised pixels as the first image.
  • the GPU allocates threads to the pixels of the first image; the thread determines the second reference image block of the pixel and the corresponding second similar image block; the thread uses the second similar image block and the A second reference image block, calculating the covariance matrix of the second reference image block; the thread uses the first reference image block and the first similar image block to calculate the mean value of the first reference image block; The thread uses the covariance matrix of the second reference image block and the mean value of the first reference image block to calculate the pixel value of the second reference image block after denoising; For all denoised second reference image blocks, determine the denoised pixel value of the pixel; the GPU uses all the denoised pixels to determine the image as the second image.
  • step 405 the CPU obtains a first image from the GPU, and encodes the first image to obtain denoised video data; or, obtains a second image from the GPU, and compares the second image Perform encoding to obtain denoised video data.
  • the GPU evaluates the noise level of the video frame image, and the GPU selects different denoising strategies according to different noise levels. Then the denoising work is performed frame by frame, and the video frame image after the noise is removed will be encoded.
  • the encoded video occupies less memory; for videos with obvious noise, the user's subjective experience can also be significantly improved after denoising.
  • FIG. 5 there is shown a structural block diagram of an electronic device provided by an embodiment of the present application, which may specifically include a central processing unit CPU51 and an image processor GPU52;
  • the CPU 51 may include:
  • the video data decoding module 511 is configured to obtain video data, and decode the video data to obtain a video frame image
  • the video data encoding module 512 is configured to obtain a second image from the GPU, and encode the second image to obtain denoised video data;
  • the GPU52 may include:
  • the video frame image loading module 521 is configured to load the video frame image from the CPU
  • the first image denoising module 522 is configured to denoise the video frame image according to a preset non-local average NLM denoising algorithm to obtain a first image;
  • the second image denoising module 523 is configured to denoise the first image according to a preset non-local Bayesian NLB denoising algorithm to obtain a second image.
  • the first image denoising module 522 may include:
  • the first thread allocation sub-module is used to allocate threads to the pixels of the video frame image
  • the first image determination sub-module is configured to use all the images determined by the denoised pixels as the first image
  • the thread may include:
  • the first block matching module is used to determine the first reference image block of the pixel and the corresponding first similar image block;
  • a distance determining module configured to determine the pixel distance and coordinate distance of the first similar image relative to the first reference image block
  • a weight calculation module for the thread to calculate the weight of the first similar image block by using the pixel distance and the coordinate distance;
  • the first pixel estimation module is configured to use the weight of the first similar image block to determine the pixel value of the pixel after denoising.
  • the second image denoising module 523 may include:
  • the second thread allocation sub-module is used to allocate threads to the pixels of the first image
  • the second image determination sub-module is configured to use all the images determined by the denoised pixels as the first image
  • the thread may include:
  • the second block matching module is used to determine the second reference image block of the pixel and the corresponding second similar image block
  • a covariance matrix calculation module configured to calculate the covariance matrix of the second reference image block by using the second similar image block and the second reference image block;
  • An image block average value calculation module configured to use the first reference image block and the first similar image block to calculate the average value of the first reference image block
  • An image block pixel estimation module configured to use the covariance matrix of the second reference image block and the mean value of the first reference image block to calculate the denoised pixel value of the second reference image block;
  • the second pixel estimation module is configured to use all denoised second reference image blocks containing the same pixel to determine the denoised pixel value of the pixel.
  • the GPU 52 may further include:
  • the noise level determining module is configured to determine the noise level of the video frame image before the second image denoising module performs denoising on the first image according to a preset NLB denoising algorithm to obtain the second image ;
  • the second image denoising module includes:
  • the second image denoising sub-module is configured to denoise the first image according to the preset non-local Bayesian NLB denoising algorithm if the noise level is greater than or equal to the preset noise level threshold, Get the second image.
  • the GPU 52 may further include:
  • the first sub-image segmentation module is configured to segment the video frame image into a plurality of first sub-images after the first thread allocation sub-module allocates threads to the pixels of the video frame image;
  • the first thread block configuration module is used to configure threads corresponding to pixels belonging to the same first sub-image to the same thread block;
  • the first data allocation module is configured to allocate the data required when the NLM denoising algorithm is executed to the thread block;
  • the thread may also include:
  • the first cooperative reading module is configured to read the required data to the shared memory of the GPU when the threads in the same thread block execute according to the warp.
  • the GPU 52 may further include:
  • the second sub-image segmentation module is configured to segment the first image into a plurality of second sub-images after the second thread allocation sub-module allocates threads to the pixels of the video frame image;
  • the second thread block configuration module is used to configure threads corresponding to pixels belonging to the same second sub-image to the same thread block;
  • the second data allocation module is configured to allocate the data required when the NLB denoising algorithm is executed to the thread block;
  • the thread may also include:
  • the second cooperative reading module is used for cooperatively reading the required data to the shared memory of the GPU when the threads in the same thread block are executed according to the warp.
  • the first block matching module may include:
  • the first reference image block determination sub-module is configured to determine the first reference image block with the pixel as the center;
  • the first search box obtaining sub-module is used to obtain the corresponding first search box from the shared memory of the GPU;
  • the first similar image block determination submodule is configured to determine the first similar image block corresponding to the first reference image block in the first search box according to a preset first pixel interval.
  • the second block matching module may include:
  • the second reference image block determining sub-module is configured to determine the second reference image block with the pixel as the center;
  • the second search box obtaining sub-module is used to obtain the corresponding second search box from the shared memory of the GPU;
  • the second similar image block determination submodule is configured to determine a second similar image block corresponding to the second reference image block in the second search box according to a preset second pixel interval.
  • the noise level determining module may include:
  • the image number determination sub-module is used to determine the number of loaded video frame images
  • the first noise level determining sub-module is configured to determine the noise level of the video frame image according to the preset frame interval if the number of the loaded video frame images is less than the preset frame number threshold;
  • the second noise level determining sub-module is configured to use the determined average value of multiple noise levels as the noise level of the video frame image.
  • the noise level determining module may further include:
  • the video frame image selection sub-module is configured to select a preset number of video frame images from the loaded video frame images if the number of the loaded video frame images is greater than or equal to the preset frame number threshold;
  • the third noise level determining sub-module is configured to determine the noise level of the video frame image according to the preset frame interval for the preset number of video frame images;
  • the fourth noise level determination sub-module is used to use the determined average value of multiple noise levels as the noise level of the video frame image.
  • the CPU 51 may further include:
  • the image filling module is used to fill the video frame image after the video data decoding module decodes the video data to obtain the video frame image.
  • the NLM denoising algorithm is used to replace the NLB denoising algorithm to perform the first denoising, avoiding the use of the NLB denoising algorithm in the first step. Denoising, leading to the problem of unstable denoising effect.
  • FIG. 6 there is shown a structural block diagram of another electronic device provided by an embodiment of the present application, which may specifically include a central processing unit CPU61 and an image processor GPU62;
  • the CPU 61 may include:
  • the video data decoding module 611 is configured to obtain video data, and decode the video data to obtain a video frame image
  • the video data encoding module 612 is configured to obtain a first image from the GPU, and encode the first image to obtain denoised video data; or, obtain a second image from the GPU, and perform processing on the first image. Two images are encoded to obtain denoised video data;
  • the GPU 62 may include:
  • the video frame image loading module 621 is configured to load the video frame image from the CPU
  • the noise level determining module 622 is configured to determine the noise level of the video frame image
  • the image denoising module 623 is configured to use a preset first image denoising algorithm to process the video frame image to obtain a first image according to the noise level, or use a preset first image denoising algorithm Denoising the video frame image to obtain a first image, and using a preset second image denoising algorithm to denoise the first image to obtain a second image.
  • the image denoising module 623 may include:
  • the first image denoising sub-module is configured to use a preset first image denoising algorithm to denoise the video frame image to obtain a first image when the noise level is less than a preset noise level threshold;
  • the second image denoising sub-module is configured to use the preset first image denoising algorithm to denoise the video frame image to obtain the first image when the noise level is greater than or equal to the preset noise level threshold. , And use a preset second image denoising algorithm to denoise the first image to obtain a second image.
  • the first image denoising submodule may include:
  • the first thread allocation unit is used to allocate threads to the pixels of the video frame image
  • the first image determining unit is configured to use all the images determined by the denoised pixels as the first image
  • the thread may include:
  • the first block matching module is used to determine the first reference image block of the pixel and the corresponding first similar image block;
  • a distance determining module configured to determine the pixel distance and coordinate distance of the first similar image relative to the first reference image block
  • a weight calculation module configured to calculate the weight of the first similar image block by using the pixel distance and the coordinate distance
  • the first pixel estimation module is configured to use the weight of the first similar image block to determine the pixel value of the pixel after denoising.
  • the second image denoising submodule may include:
  • the second thread allocation unit is used to allocate threads to the pixels of the first image
  • the second image determining unit is configured to use all the images determined by denoising pixels as the first image
  • the thread may include:
  • the second block matching module is used to determine the second reference image block of the pixel and the corresponding second similar image block
  • a covariance matrix calculation module configured to calculate the covariance matrix of the second reference image block by using the second similar image block and the second reference image block;
  • An image block average value calculation module configured to use the first reference image block and the first similar image block to calculate the average value of the first reference image block
  • An image block pixel estimation module configured to use the covariance matrix of the second reference image block and the mean value of the first reference image block to calculate the denoised pixel value of the second reference image block;
  • the second pixel estimation module is configured to use all denoised second reference image blocks containing the same pixel to determine the denoised pixel value of the pixel.
  • the GPU evaluates the noise level of the video frame image, and the GPU selects different denoising strategies according to different noise levels. Then the denoising work is performed frame by frame, and the video frame image after the noise is removed will be encoded.
  • the encoded video occupies less memory; for videos with obvious noise, the user's subjective experience can also be significantly improved after denoising.
  • the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.
  • the embodiment of the present application also provides a device, including:
  • It includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor.
  • the computer program is executed by the processor, each process of the foregoing video denoising method embodiment is realized, and the same The technical effect, in order to avoid repetition, will not be repeated here.
  • the embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored.
  • a computer program is stored on which a computer program is stored.
  • the computer program is executed by a processor, each process of the above-mentioned video denoising method embodiment is realized, and the same technology can be achieved. The effect, in order to avoid repetition, will not be repeated here.
  • any reference signs placed between parentheses should not be constructed as a limitation to the claims.
  • the word “comprising” does not exclude the presence of elements or steps not listed in the claims.
  • the word “a” or “an” preceding an element does not exclude the presence of multiple such elements.
  • the application can be realized by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices may be embodied in the same hardware item.
  • the use of the words first, second, and third, etc. do not indicate any order. These words can be interpreted as names.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例提供了一种视频去噪方法和电子设备，所述方法包括：所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；所述GPU从所述CPU加载所述视频帧图像；所述GPU对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像；所述CPU从所述GPU获取所述第二图像，并对所述第二图像进行编码得到去噪后的视频数据。在本申请实施例中，在实现NLB去噪方式的两步去噪方式时，采用NLM去噪算法替换NLB去噪算法进行第一步去噪，避免了使用NLB去噪算法在进行第一步去噪，导致去噪效果不稳定的问题。

Description

一种视频去噪方法和电子设备
本申请要求在2019年12月31日提交中国专利局、申请号201911418935.0、发明名称为“一种视频去噪方法和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，特别是涉及一种视频去噪方法和一种电子设备。
背景技术
随着网络技术的发展，越来越多人喜欢将拍摄的视频分享到社交平台，或者在社交平台观看其他人分享的视频。
很多情况下，人们都是随手打开手机的摄像头拍摄视频，摄像头在光照条件的不佳的时候采集出来的视频会有明显的噪声，这些噪声会对用户的主观体验造成影响，也会影响视频的编解码和压缩。
目前，现有的图像去噪算法包括基于像素点的去噪算法、基于块匹配的去噪算法、基于变换域的去噪算法和机器学习去噪算法。在实际使用中这些算法要不就是去噪效果不好、要不就是算法执行时间长。
非局部贝叶斯NLB(non-local bayes)去噪算法是近年实验得到的一个在去噪效果和执行时间上较为折中的选择。
NLB去噪方式为两步去噪，其中第一步去噪由于对协方差矩阵的减法操作可能出现负定矩阵，使得图像去噪效果不完全稳定。
发明内容
鉴于上述问题，提出了本申请实施例提供一种视频去噪方法和相应的一种电子设备，以提高去噪效果的稳定性。
为了解决上述问题，本申请实施例公开了一种视频去噪方法，应用于具有中央处理器CPU和图像处理器GPU的电子设备，所述方法包括：
所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
所述GPU从所述CPU加载所述视频帧图像；
所述GPU对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像；
所述CPU从所述GPU获取所述第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
本申请实施例还公开了一种视频去噪方法，应用于具有CPU和GPU的电子设备，所述方法包括：
所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
所述GPU从所述CPU加载所述视频帧图像；
所述GPU确定所述视频帧图像的噪声等级；
所述GPU根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像；
所述CPU从所述GPU获取第一图像，并对所述第一图像进行编码得到去噪后的视频数据；或者，从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
本申请实施例还公开了一种电子设备，包括中央处理器CPU和图像处理器GPU；
所述CPU包括：
视频数据解码模块，用于获取视频数据，并对所述视频数据进行解码得到视频帧图像；
视频数据编码模块，用于从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据；
所述GPU包括：
视频帧图像加载模块，用于从所述CPU加载所述视频帧图像；
第一图像去噪模块，用于对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
第二图像去噪模块，用于对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像。
本申请实施例还公开了一种电子设备，包括CPU和GPU；
所述CPU包括：
视频数据解码模块，用于获取视频数据，并对所述视频数据进行解码得到视频帧图像；
视频数据编码模块，用于从所述GPU获取第一图像，并对所述第一图像进行编码得到去噪后的视频数据；或者，从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据；
所述GPU包括：
视频帧图像加载模块，用于从所述CPU加载所述视频帧图像；
噪声等级确定模块，用于确定所述视频帧图像的噪声等级；
图像去噪模块，用于根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像。
本申请实施例还公开了一种装置，包括：处理器、存储器及存储在所述存储器上并能够在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如上任一项所述的视频去噪方法的步骤。
本申请实施例还公开了一种计算机可读存储介质，所述计算机可读存储介质上存储计算机程序，所述计算机程序被处理器执行时实现如上任一项所述的视频去噪方法的步骤。
本申请实施例包括以下优点：
在本申请实施例中，在实现NLB去噪方式的两步去噪方式时，采用NLM去噪算法替换NLB去噪算法进行第一步去噪，避免了使用NLB去噪算法在进行第一步去噪，导致去噪效果不稳定的问题。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种视频去噪方法的步骤流程图；
图2是本申请实施例提供的另一种视频去噪方法的步骤流程图；
图3是本申请实施例中第一子图像对应的第一搜索框的示意图；
图4是本申请实施例提供的又一种视频去噪方法的步骤流程图；
图5是本申请实施例提供的一种电子设备的结构框图；
图6是本申请实施例提供的另一种电子设备的结构框图。
具体实施例
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。
NLB去噪方式采用两步去噪，采用两步去噪的原因是，NLB算法去噪的效果依赖相似块分组的准确性，第一步由于存在噪声，块的分组会受到干扰，在完成了第一步去噪后，去除了很多噪声干扰的图像用来做块匹配会更加精确。此时将原噪声图像和去噪后的图像都按照去噪后图像进行分块，并在NLB算法中用到含噪声图像进行再去噪。另外一个原因是去噪过一次之后的图像的均值和方差更加接近没有噪声的图像。
NLB去噪算法为：
Figure PCTCN2020129864-appb-000001
其中，P为去噪后的图像块的估计；
Figure PCTCN2020129864-appb-000002
为噪声图像块的期望，
Figure PCTCN2020129864-appb-000003
为噪声图像块的协方差矩阵，I为单位矩阵，σ为噪声方差。
传统的NLB去噪方式两步去噪都采用NLB去噪算法，在一步去噪时，
由于相似块的协方差矩阵
Figure PCTCN2020129864-appb-000004
可能减去σ 2I而变为负定，从而使得概率计算没有意义。
本申请实施例的核心构思之一在于，沿用NLB去噪方式的两步去噪机制，可以对两步去噪分别进行优化。
由于第一步去噪最大的贡献在于提供了一个更加准确的块匹配，因此为了加速，本申请实施例中，在第一步去噪可以采用非局部平均值NLM(non-local means)去噪算法去噪，而弃用NLB去噪算法的计算。在采用NLM去噪算法去噪后，采用NLB去噪进行第二去噪，实验证明采取这样的策略对最终去噪效果几乎没有影响。
在第一步去噪中，除了使用NLM去噪算法保证计算的数学意义，本申请实施例中也对NLM去噪算法进行了算法优化和运算加速。
在第二步去噪中，本申请实施例也对NLB去噪算法进行了算法优化和运算加速。
通过上述的优化，可以使得在采用本申请实施例的视频去噪方法时，能达到较好的去噪效果，并且缩小算法运算时长。
上述改进后的算法除了适用于视频去噪，也适用于其他领域的图像的去噪，例如无人机图像、遥感图像、扫描图像、传真图像、医学图像、军事侦察图像等等。
以下，通过具体实施例对本申请的视频去噪方法进行说明。
参照图1，示出了本申请实施例提供的一种视频去噪方法的步骤流程图，改方法应用于具有中央处理器CPU和图像处理器GPU的电子设备，所述具体可以包括如下步骤：
步骤101，所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
在实际中，用户可以将拍摄的视频输入到具有CPU和GPU的电子设备进行去噪处理。CPU获取用户拍摄的视频数据，将视频数据解码为多个视频帧图像，由GPU进行图像去噪处理。在实际中，CPU和GPU都可以运行图像去噪算法，基于GPU的结构特点，GPU更适合图像处理。
本申请实施例中，可以根据GPU的多线程并行的结构特点，对图像去噪算法的执行过程进行改进，通过GPU加速图像去噪过程，缩短处理时间。
步骤102，所述GPU从所述CPU加载所述视频帧图像；
具体的，GPU可以根据视频帧图像的大小分配显存大小，将视频帧图像加载到显存中。
步骤103，所述GPU对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
NLB去噪方式采用的是两步去噪，传统方式中，两步去噪都是采用NLB去噪算法实现。本申请实施例中，采用NLM去噪算法替换NLB去噪算法进行第一步去噪。
步骤104，所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像；
GPU用NLB去噪算法对第一图像的各个像素点进行去噪，得到第二图像。
步骤105，所述CPU从所述GPU获取所述第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
在GPU对加载的所有视频帧图像都去噪得到第二图像后，CPU对所有的第二图像进行编码，得到去噪后的视频数据。
在本申请实施例中，在实现NLB去噪方式的两步去噪方式时，采用NLM去噪算法替换NLB去噪算法进行第一步去噪，避免了使用NLB去噪算法在进行第一步去噪，导致去噪效果不稳定的问题。
参照图2，示出了本申请实施例提供的另一种视频去噪方法的步骤流程图，改方法应用于具有中央处理器CPU和图像处理器GPU的电子设备，所述具体可以包括如下步骤：
步骤201，所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
步骤202，所述CPU对所述视频帧图像进行填充；
由于NLM去噪算法和NLB去噪算法都是基于图像块匹配的算法，图像块通常以像素点为中心确定。例如，以一像素点确定5x5的图像块。对于视频帧图像边界的像素点，由于无法以其为中心确定图像块，导致不能进行图像块匹配的运算。因此需要对原始的视频帧图像进行填充，使得原本边界的像素点也能进行图像块的运算，而不用单独去判断。
步骤203，所述GPU从所述CPU加载所述视频帧图像；
GPU从CPU中加载，由CPU填充之后的视频帧图像。
步骤204，所述GPU对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
在本申请实施例对NLM去噪算法进行了优化，以提高图像去噪的效果。传统的NLM算法中，对图像中的每个像素点，构造一个以其为中心的图像块(patch)，称为参考块，例如，尺寸为5x5的图像块；然后构造一个以参考快为中心的搜索框，例如尺寸16x16的搜索框。
然后在搜索框中搜索参考图像块的相似图像块，具体的，对于搜索框中的每一个像素点，以这个点为中心再构造出一个相邻图像块，然后判断这个相邻图像块是否为相似图像块。两个图像块的相似度可以采用像素距离来衡量，比较两个图像块内所有像素点之间的像素距离：
Figure PCTCN2020129864-appb-000005
其中，i为参考图像块，j为相邻图像块。
例如，若像素块为5x5，则依次将一个图像块的25个像素点，与另一个图像块对应位置的像素点的像素距离。其中，像素距离以绝对值衡量。像素距离是指像素值的差距，例如一个像素点的值为255，另一个像素点的值为240，则像素距离为15。
当
Figure PCTCN2020129864-appb-000006
小于预设的像素距离阈值时，则认为该相邻图像块为相似图像块。
对于两个图像块来说，两个图像块的差异性越大，那么相邻图像块的中心像素点所占的权重就应该越小，直观而言，差异性和权重具有反比关系。具体的求权重公式为：
Figure PCTCN2020129864-appb-000007
其中Z(i)为一个归一化参数，即所有权重的和。
Figure PCTCN2020129864-appb-000008
其中，h 2为图像块的大小。
现有的权重计算方式只考虑像素距离，传统的NLM算法在去噪的同时会使得图像的边变得模糊。对此，在本申请实施例中，对权重的计算方式进行改进，增加考虑坐标距离来计算权重，提高算法的保边性，更好的保留去噪后图像边缘的细节。坐标距离是指两个像素点在图像中的距离，即离参考点越近的点，占的权重越大。
在本申请实施例中，具体可以按照如下公式计算权重：
Figure PCTCN2020129864-appb-000009
其中，C(i)为一个归一化参数，
Figure PCTCN2020129864-appb-000010
为原始的像素距离，表示整个图像块的像素距离；d(i,j)表示像素点i和像素点j的像素距离。
Figure PCTCN2020129864-appb-000011
用于将权重函数变得更为陡峭。
Figure PCTCN2020129864-appb-000012
为坐标距离，D(i,j)表示像素点i和像素点j的坐标距离，σ s为像素距离的方差。
Figure PCTCN2020129864-appb-000013
为像素距离，|z(j)-z(i)|表示块中心像素点i和j的像素距离，σ r为预估的噪声方差。
进一步的，通过函数f(x)使得权重函数的曲线更为陡峭，可以减少相似度较低的图像块的权重。
另一方面，参考图像块的相似图像块会包括参考图像块本身，因此参考图像块本身所占的权重很大，如果参考块的中心像素点本身即为噪点，会不利于噪声剔除。对此本申请实施例进行了改进，将参考图像块本身的权重乘以一个小于1的系数(例如0.5)，通过减少参考图像块本身的权重，从而降低本像素点自身是单点噪声而产生的误差。
在本申请实施例中，所述步骤204可以包括如下子步骤：
子步骤S11，所述GPU对所述视频帧图像的像素点分配线程；
GPU为视频帧图像的像素点分配线程执行改进的NLM算法，GPU可以为多 个像素点分配一个线程，即由一个线程依次按照改进的NLM算法对多个像素点进行去噪；GPU也可以为一个像素点分配一个线程，即一个线程只按照改进的NLM算法对一个像素点进行去噪。
在本申请实施例中，除了对NLM算法进行改进之外，还可以采用具有统一计算设备架构CUDA(Compute Unified Device Architecture)的GPU加速算法的执行。GPU线程以网格(grid)的方式组织，每个网格中又包含若干个线程块(block)，同一线程块中的众多线程拥有相同的指令地址，不仅能够并行执行，而且能够通过共享存储器(Shared memory)实现块内通信。
具有CUDA的CPU在实际执行过程中，会以线程块为单位。把一个个线程块分配给硬件的流处理器SM(Streaming Multiprocessor)进行运算；而线程块中的线程又会以线程束(warp)为单位，对线程进行分组计算。目前CUDA的线程束大小都是32，也就是说32个线程会被组成一个线程束来一起执行。同一个线程束中的线程执行的指令是相同的，只是处理的数据不同。一个流处理器一次只会执行一个线程块里的一个线程束，为了不浪费线程束的计算资源，通常线程块设置为32的倍数。例如，设置线程块可以设置为8x8，16x16，32x32等尺寸。
在本申请实施例中，基于CUDA的特性设置如下优化：
在GPU分配线程后，所述GPU将所述视频帧图像切分为多个第一子图像；所述GPU将属于同一第一子图像的像素点对应的线程，配置到同一线程块；所述GPU对所述线程块，分配在执行所述NLM去噪算法时所需要的数据；在同一线程块中的线程按照线程束执行时，所述线程束中的线程将所述所需的数据协同读取到所述GPU的共享内存。
例如，将视频帧图像切分为多个8x8的子图像，将负责对这个子图像的像素点进行处理的线程，配置到同一个线程块。若子图像中的每个像素点由一个线程负责，即由64个线程负责一个8x8的子图像，线程块的大小可以设置为64，大小为64的线程块以2个线程束来执行。线程块在执行NLM去噪算法时所需要的数据是指搜索框的数据。
在本申请实施例改进的NLM去噪算法中，对同一第一子图像中的像素点设置了同一第一搜索框。如图3所示为本申请实施例中第一子图像对应的第一搜索框的示意图。对同一第一子图像中像素点，在同一第一搜索框内搜索以这个像素 点为中心的第一参考图像块对应的第一相似图像块。由于同一个第一子图像的线程在同一个线程块，可以基于同一线程块内线程的协同操作，将第一搜索框的数据读取到共享内存。
以16x16的搜索框为例，若不通过协同操作，每个线程都需要读取16x16的搜索框。在使用线程块的协同操作后，大小为64的线程块中，每个线程只需要读取4个像素到共享内存，大大减少了线程读取的数据量，提高算法处理速度。
在将执行NLM去噪算法所需的数据读取到共享内存后，同一线程块的线程都可以从共享内存读取数据。
另外，目前Opencv提供的NLM去噪算法读取数据都是在GPU的全局内存(global memory)上进行，这是一种非常缓慢数据读取方式。对此，本申请实施例通过将NLM去噪算法所需的数据都读取到共享内存中，可以大大提高数据读取速度。
子步骤S12，所述线程确定像素点的第一参考图像块和对应的第一相似图像块；
线程在第一搜索框内搜索第一参考图像块对应的第一相似图像块。具体的，线程以第一搜索框内每一个像素点为中心确定一相邻图像块，然后对所有相邻图像块判断是否为相似图像块。在一些情况下，可能会查找到大量的第一相似图像块(例如，上百个第一相似图像块)，如果以大量的相似块执行后续的算法运行过程，会占用过多的内存。由于GPU算法不适合占用过多内存，不适合做排序，也不适合在查找到的第一相似图像中进一步筛选。
为了减少搜索到的第一相似图像块的数量，在本申请实施例中，可以通过减小搜索区域，不对第一搜索框内所有相邻图像块都进行判断是否为第一相似图像块；而是按预设第一像素间隔，选取相邻图像块来判断是否为第一相似图像块。
在本申请实施例中，所述子步骤S12可以进一步包括：
子步骤S121，所述线程以像素点为中心确定第一参考图像块；
子步骤S122，所述线程从所述GPU的共享内存，获取对应的第一搜索框；
由于线程在执行NLM去噪算法所需的数据(包括第一搜索框的数据)，已经通过线程块的协同操作读取到共享内存了，因此线程可以从共享内存，获取对应的第一搜索框。
子步骤S123，所述线程按预设第一像素间隔，在所述第一搜索框内的确定第一参考图像块对应的第一相似图像块。
子步骤S13，所述线程确定所述第一相似图像相对所述第一参考图像块的像素距离和坐标距离；
子步骤S14，所述线程采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；
子步骤S15，所述线程采用所述第一相似图像块的权重确定所述像素点去噪后的像素值；
在计算得到所有相似图像块的权重后，采用相似图像块的权重和相似图像块中心像素点的像素值加权平均的值，作为参考图像块的中心像素点的去噪后的像素值。
子步骤S16，所述GPU将所有去噪后的像素点确定的图像作为第一图像。
步骤205，所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像；
GPU用NLB去噪算法对第一图像的各个像素点进行去噪，得到第二图像。
步骤206，所述CPU从所述GPU获取所述第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
在GPU对加载的所有视频帧图像都进行去噪得到第二图像后，CPU对所有的第二图像进行编码，得到去噪后的视频数据。
本申请实施例还对NLB去噪算法进行了优化。
传统的NLB去噪算法为：
Figure PCTCN2020129864-appb-000014
其中，P为去噪声后的图像块的估计值，
Figure PCTCN2020129864-appb-000015
带噪声的图像块的均值，
Figure PCTCN2020129864-appb-000016
带噪声图像块的协方差矩阵，σ为噪声方差。
按照现有的NLB两步去噪方式：
第一步去噪的公式为：
Figure PCTCN2020129864-appb-000017
第二步去噪的公式为：
Figure PCTCN2020129864-appb-000018
其中，
Figure PCTCN2020129864-appb-000019
为第一步去噪后的图像块的估计值，
Figure PCTCN2020129864-appb-000020
为第二步去噪后的图像块的估计值，
Figure PCTCN2020129864-appb-000021
为第一步去噪后的图像中，图像块的均值，
Figure PCTCN2020129864-appb-000022
为第一步去噪后的图像中，图像块的协方差矩阵。
在实际中，采用第二步去噪后的公式进行去噪时，会导致损失一些图像细节信息。对此，本申请实施例中，将第二步去噪公式改进为：
Figure PCTCN2020129864-appb-000023
改进后的第二步去噪公式中，在计算均值时带入原始图像中图像块的均值，而不是第一步去噪后图像中图像块的均值，因此第二步去噪可以恢复出一些原始图像的细节信息。
在本申请实施例中，所述步骤206具体可以包括如下子步骤：
子步骤S21，所述GPU对所述第一图像的像素点分配线程；
在本申请实施例中，基于CUDA的特性设置如下优化：
在GPU分配线程后，所述GPU将所述第一图像切分为多个第二子图像；所述GPU将属于同一第二子图像的像素点对应的线程，配置到同一线程块；所述GPU对所述线程块，分配在执行所述NLB去噪算法时所需要的数据；在同一线程块中的线程按照线程束执行时，所述线程束中的线程将所述所需的数据协同读取到所述GPU的共享内存。线程块在执行NLB去噪算法时所需要的数据是指搜索框的数据。
在本申请实施例改进的NLB去噪算法中，对同一第二子图像中的像素点设置了同一第二搜索框。如对同一第二子图像中像素点，在同一第二搜索框内搜索以这个像素点为中心的第二参考图像块对应的第二相似图像块。由于同一个第二子图像的线程在同一个线程块，可以基于同一线程块内线程的协同操作，将第二搜索框的数据读取到共享内存。在将执行NLB去噪算法所需的数据读取到共享内存后，同一线程块的线程都可以从共享内存读取数据。
GPU中寄存器的数目有限，也就是能够在GPU上开辟的变量有限，寄存器使用的数量(变量的个数)会影响并发性。
NLB去噪算法比较复杂，需要使用较多的局部变量。因此在实际的代码实现中，一方面，需要让算法后续的步骤可以重新利用前面开辟的数组等。另一方面， 让需要算法中使用最频繁的变量存在寄存器中。
NLB去噪算法由于需要计算协方差矩阵，存储相似块等信息需要用到大量本地内存。因此本申请实施例中可以使用大量的内存复用以降低内存开销。
通过上述优化，可以减少寄存器的使用可以使得同时运行更多的线程，从而提高算法的并发性，大大缩短算法计算时间，在实际中能缩短30％的计算时间。
子步骤S22，所述线程确定像素点的第二参考图像块和对应的第二相似图像块；
线程在第二搜索框内搜索第二参考图像块对应的第二相似图像块，具体的，线程以第二搜索框内每一个像素点为中心确定一相邻图像块，然后对所有相邻图像块判断是否为相似图像块。
为了减少搜索到的第二相似图像块的数量，在本申请实施例中，通过减小搜索区域，不对第二搜索框内所有相邻图像块都进行判断是否为第二相似图像块；而是按预设第二像素间隔，选取相邻图像块来判断是否为第二相似图像块。
所述子步骤S22可以进一步包括：
子步骤S221，所述线程以像素点为中心确定第二参考图像块；
子步骤S222，所述线程从所述GPU的共享内存，获取对应的第二搜索框；
由于线程在执行NLB去噪算法所需的数据(包括第二搜索框的数据)，已经通过线程块的协同操作读取到共享内存了，因此线程可以从共享内存，获取对应的第二搜索框。
子步骤S223，所述线程按预设第二像素间隔，在所述第二搜索框内的确定第二参考图像块对应的第二相似图像块。
子步骤S23，所述线程采用所述第二相似图像块与所述第二参考图像块，计算所述第二参考图像块的协方差矩阵；
第二参考图像块的协方差矩阵即上述改进后的第二步去噪公式中的
Figure PCTCN2020129864-appb-000024
子步骤S24，所述线程采用所述第一参考图像块和所述第一相似图像块，计算所述第一参考图像块的均值；
第一参考图像块的均值即上述改进后的第二步去噪公式中的
Figure PCTCN2020129864-appb-000025
需要根据在原始的视频帧图像中，确定像素点的第一参考图像块和对应的第一相似图像块计算得到。
子步骤S25，所述线程采用所述第二参考图像块的协方差矩阵和所述第一参考图像块的均值，计算所述第二参考图像块去噪后的像素值；
具体可以按照上述改进后的第二步去噪公式，得到第二参考图像块去噪后的估计值。
子步骤S26，所述线程采用包含同一像素点的所有去噪后的第二参考图像块，确定该像素点去噪后的像素值。
经过改进后的第二步去噪公式后，第二参考图像块内所有的像素点都进行了去噪。即一个像素点，除了在以其为中心的第二参考图像块内被去噪，还会在以其他像素点为中心的第二参考图像块内被去噪，因此一个像素点会有具有多个去噪后的估计值。一个像素点最终的去噪后估计值为：包含该像素点的所有去噪后的第二参考图像块的该像素点估计值的平均值。
子步骤S27，所述GPU将所有去噪后的像素点确定的图像作为第二图像。
在本申请实施例中，在执行步骤206之前，还可以由所述GPU确定所述视频帧图像的噪声等级；若所述噪声等级大于或等于所述预设噪声等级阈值，则所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像。若所述噪声等级小于预设噪声等级阈值，则所述GPU不执行对所述第一图像，按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像；噪声等级由噪声方差sigma来衡量，例如，假设噪声等级阈值为10，若视频帧图像的噪声等级小于10，则只需采用NLM去噪算法进行去噪；若噪声等级大于或等于10，在采用NLM去噪算法进行去噪后，再采用NLB去噪算法去噪。
在本申请实施例中，所述GPU确定所述视频帧图像的噪声等级的步骤可以包括：
所述GPU确定加载的视频帧图像的数目；若所述加载的视频帧图像的数目小于预设帧数目阈值，则所述GPU按预设帧间隔，确定视频帧图像的噪声等级；所述GPU将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。若所述加载的视频帧图像的数目大于或等于所述预设帧数目阈值，则所述GPU从加载的视频帧图像中选取预设数目的视频帧图像；所述GPU对所述预设数目的视频帧图像，按预设帧间隔，确定视频帧图像的噪声等级；所述GPU将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
例如，假设预设帧数目阈值为500，对于小于500帧的视频，每隔一帧取出一帧图像评估其噪声等级，整个视频所有图像的噪声等级为计算到的噪声等级的平均值。对于大于500帧的视频，取前500帧，每隔一帧取出一帧图像评估其噪声等级，整个视频所有图像的噪声等级为计算到的噪声等级的平均值。
在本申请实施例中，在实现NLB去噪方式的两步去噪方式时，采用NLM去噪算法替换NLB去噪算法进行第一步去噪，避免了使用NLB去噪算法在进行第一步去噪，导致去噪效果不稳定的问题。
参照图4，示出了本申请实施例提供的又一种视频去噪方法的步骤流程图，改方法应用于具有中央处理器CPU和图像处理器GPU的电子设备，所述具体可以包括如下步骤：
步骤401，所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
在本申请实施例中，所述CPU还可以对所述视频帧图像进行填充。
步骤402，所述GPU从所述CPU加载所述视频帧图像；
步骤403，所述GPU确定所述视频帧图像的噪声等级；
在本申请实施例中，所述步骤403可以包括：所述GPU确定加载的视频帧图像的数目；若所述加载的视频帧图像的数目小于预设帧数目阈值，则所述GPU按预设帧间隔，确定视频帧图像的噪声等级；所述GPU将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
若所述加载的视频帧图像的数目大于或等于所述预设帧数目阈值，则所述GPU从加载的视频帧图像中选取预设数目的视频帧图像；所述GPU对所述预设数目的视频帧图像，按预设帧间隔，确定视频帧图像的噪声等级；所述GPU将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
步骤404，所述GPU根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像；
在本申请实施例中，所述步骤404可以包括子步骤：
子步骤S31，当所述噪声等级小于预设噪声等级阈值时，所述GPU采用预设 的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像；
所述子步骤S31可以包括：所述GPU对所述视频帧图像的像素点分配线程；所述线程确定像素点的第一参考图像块和对应的第一相似图像块；所述线程确定所述第一相似图像相对所述第一参考图像块的像素距离和坐标距离；所述线程采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；所述线程采用所述第一相似图像块的权重确定所述像素点去噪后的像素值；所述GPU将所有去噪后的像素点确定的图像作为第一图像。
子步骤S32，当所述噪声等级大于或等于所述预设噪声等级阈值时，所述GPU采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像。
所述子步骤S32可以包括：
所述GPU对所述视频帧图像的像素点分配线程；所述线程确定像素点的第一参考图像块和对应的第一相似图像块；所述线程确定所述第一相似图像相对所述第一参考图像块的像素距离和坐标距离；所述线程采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；所述线程采用所述第一相似图像块的权重确定所述像素点去噪后的像素值；所述GPU将所有去噪后的像素点确定的图像作为第一图像。
所述GPU对所述第一图像的像素点分配线程；所述线程确定像素点的第二参考图像块和对应的第二相似图像块；所述线程采用所述第二相似图像块与所述第二参考图像块，计算所述第二参考图像块的协方差矩阵；所述线程采用所述第一参考图像块和所述第一相似图像块，计算所述第一参考图像块的均值；所述线程采用所述第二参考图像块的协方差矩阵和所述第一参考图像块的均值，计算所述第二参考图像块去噪后的像素值；所述线程采用包含同一像素点的所有去噪后的第二参考图像块，确定该像素点去噪后的像素值；所述GPU将所有去噪后的像素点确定的图像作为第二图像。
步骤405，所述CPU从所述GPU获取第一图像，并对所述第一图像进行编码得到去噪后的视频数据；或者，从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
在本申请实施例中，用户拍摄的视频被解码成视频帧图像后，GPU评估视频 帧图像的噪声等级，GPU根据不同的噪声等级选择不同的去噪策略。然后逐帧进行去噪工作，去除噪声后的视频帧图像会被进行编码。进行编码后的视频占用内存更少；对于含噪声明显的视频，去噪后用户的主观感受也能得到显著改善。
需要说明的是，对于方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请实施例并不受所描述的动作顺序的限制，因为依据本申请实施例，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作并不一定是本申请实施例所必须的。
参照图5，示出了本申请实施例提供的一种电子设备的结构框图，具体可以包括中央处理器CPU51和图像处理器GPU52；
所述CPU51可以包括：
视频数据解码模块511，用于获取视频数据，并对所述视频数据进行解码得到视频帧图像；
视频数据编码模块512，用于从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据；
所述GPU52可以包括：
视频帧图像加载模块521，用于从所述CPU加载所述视频帧图像；
第一图像去噪模块522，用于对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
第二图像去噪模块523，用于对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像。
在本申请实施例中，所述第一图像去噪模块522可以包括：
第一线程分配子模块，用于对所述视频帧图像的像素点分配线程；
第一图像确定子模块，用于将所有去噪后的像素点确定的图像作为第一图像；
所述线程可以包括：
第一块匹配模块，用于确定像素点的第一参考图像块和对应的第一相似图像块；
距离确定模块，用于确定所述第一相似图像相对所述第一参考图像块的像素 距离和坐标距离；
权重计算模块，用于所述线程采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；
第一像素估计模块，用于采用所述第一相似图像块的权重确定所述像素点去噪后的像素值。
在本申请实施例中，所述第二图像去噪模块523可以包括：
第二线程分配子模块，用于对所述第一图像的像素点分配线程；
第二图像确定子模块，用于将所有去噪后的像素点确定的图像作为第一图像；
所述线程可以包括：
第二块匹配模块，用于确定像素点的第二参考图像块和对应的第二相似图像块；
协方差矩阵计算模块，用于采用所述第二相似图像块与所述第二参考图像块，计算所述第二参考图像块的协方差矩阵；
图像块均值计算模块，用于采用所述第一参考图像块和所述第一相似图像块，计算所述第一参考图像块的均值；
图像块像素估计模块，用于采用所述第二参考图像块的协方差矩阵和所述第一参考图像块的均值，计算所述第二参考图像块去噪后的像素值；
第二像素估计模块，用于采用包含同一像素点的所有去噪后的第二参考图像块，确定该像素点去噪后的像素值。
在本申请实施例中，所述GPU52还可以包括：
噪声等级确定模块，用于在所述第二图像去噪模块对所述第一图像，按照预设的NLB去噪算法进行去噪，得到第二图像之前，确定所述视频帧图像的噪声等级；
所述第二图像去噪模块包括：
第二图像去噪子模块，用于若所述噪声等级大于或等于所述预设噪声等级阈值，则对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像。
在本申请实施例中，所述GPU52还可以包括：
第一子图像切分模块，用于在所述第一线程分配子模块对所述视频帧图像的像素点分配线程之后，将所述视频帧图像切分为多个第一子图像；
第一线程块配置模块，用于将属于同一第一子图像的像素点对应的线程，配置到同一线程块；
第一数据分配模块，用于对所述线程块，分配在执行所述NLM去噪算法时所需要的数据；
所述线程还可以包括：
第一协同读取模块，用于在同一线程块中的线程按照线程束执行时，将所述所需的数据协同读取到所述GPU的共享内存。
在本申请实施例中，所述GPU52还可以包括：
第二子图像切分模块，用于在所述第二线程分配子模块对所述视频帧图像的像素点分配线程之后，将所述第一图像切分为多个第二子图像；
第二线程块配置模块，用于将属于同一第二子图像的像素点对应的线程，配置到同一线程块；
第二数据分配模块，用于对所述线程块，分配在执行所述NLB去噪算法时所需要的数据；
所述线程还可以包括：
第二协同读取模块，用于在同一线程块中的线程按照线程束执行时，将所述所需的数据协同读取到所述GPU的共享内存。
在本申请实施例中，所述第一块匹配模块可以包括：
第一参考图像块确定子模块，用于以像素点为中心确定第一参考图像块；
第一搜索框获取子模块，用于从所述GPU的共享内存，获取对应的第一搜索框；
第一相似图像块确定子模块，用于按预设第一像素间隔，在所述第一搜索框内的确定第一参考图像块对应的第一相似图像块。
在本申请实施例中，所述第二块匹配模块可以包括：
第二参考图像块确定子模块，用于以像素点为中心确定第二参考图像块；
第二搜索框获取子模块，用于从所述GPU的共享内存，获取对应的第二搜索框；
第二相似图像块确定子模块，用于按预设第二像素间隔，在所述第二搜索框内的确定第二参考图像块对应的第二相似图像块。
在本申请实施例中，所述噪声等级确定模块可以包括：
图像数目确定子模块，用于确定加载的视频帧图像的数目；
第一噪声等级确定子模块，用于若所述加载的视频帧图像的数目小于预设帧数目阈值，则按预设帧间隔，确定视频帧图像的噪声等级；
第二噪声等级确定子模块，用于将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
在本申请实施例中，所述噪声等级确定模块还可以包括：
视频帧图像选取子模块，用于若所述加载的视频帧图像的数目大于或等于所述预设帧数目阈值，则从加载的视频帧图像中选取预设数目的视频帧图像；
第三噪声等级确定子模块，用于对所述预设数目的视频帧图像，按预设帧间隔，确定视频帧图像的噪声等级；
第四噪声等级确定子模块，用于将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
在本申请实施例中，所述CPU51还可以包括：
图像填充模块，用于在所述视频数据解码模块对所述视频数据进行解码得到视频帧图像之后，对所述视频帧图像进行填充。
在本申请实施例中，在实现NLB去噪方式的两步去噪方式时，采用NLM去噪算法替换NLB去噪算法进行第一步去噪，避免了使用NLB去噪算法在进行第一步去噪，导致去噪效果不稳定的问题。
参照图6，示出了本申请实施例提供的另一种电子设备的结构框图，具体可以包括中央处理器CPU61和图像处理器GPU62；
所述CPU61可以包括：
视频数据解码模块611，用于获取视频数据，并对所述视频数据进行解码得到视频帧图像；
视频数据编码模块612，用于从所述GPU获取第一图像，并对所述第一图像进行编码得到去噪后的视频数据；或者，从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据；
所述GPU62可以包括：
视频帧图像加载模块621，用于从所述CPU加载所述视频帧图像；
噪声等级确定模块622，用于确定所述视频帧图像的噪声等级；
图像去噪模块623，用于根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像。
在本申请实施例中，所述图像去噪模块623可以包括：
第一图像去噪子模块，用于当所述噪声等级小于预设噪声等级阈值时，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像；
第二图像去噪子模块，用于当所述噪声等级大于或等于所述预设噪声等级阈值时，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像。
在本申请实施例中，所述第一图像去噪子模块可以包括：
第一线程分配单元，用于对所述视频帧图像的像素点分配线程；
第一图像确定单元，用于将所有去噪后的像素点确定的图像作为第一图像；
所述线程可以包括：
第一块匹配模块，用于确定像素点的第一参考图像块和对应的第一相似图像块；
距离确定模块，用于确定所述第一相似图像相对所述第一参考图像块的像素距离和坐标距离；
权重计算模块，用于采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；
第一像素估计模块，用于采用所述第一相似图像块的权重确定所述像素点去噪后的像素值。
在本申请实施例中，所述第二图像去噪子模块可以包括：
第二线程分配单元，用于对所述第一图像的像素点分配线程；
第二图像确定单元，用于将所有去噪后的像素点确定的图像作为第一图像；
所述线程可以包括：
第二块匹配模块，用于确定像素点的第二参考图像块和对应的第二相似图像块；
协方差矩阵计算模块，用于采用所述第二相似图像块与所述第二参考图像块，计算所述第二参考图像块的协方差矩阵；
图像块均值计算模块，用于采用所述第一参考图像块和所述第一相似图像块，计算所述第一参考图像块的均值；
图像块像素估计模块，用于采用所述第二参考图像块的协方差矩阵和所述第一参考图像块的均值，计算所述第二参考图像块去噪后的像素值；
第二像素估计模块，用于采用包含同一像素点的所有去噪后的第二参考图像块，确定该像素点去噪后的像素值。
在本申请实施例中，用户拍摄的视频被解码成视频帧图像后，GPU评估视频帧图像的噪声等级，GPU根据不同的噪声等级选择不同的去噪策略。然后逐帧进行去噪工作，去除噪声后的视频帧图像会被进行编码。进行编码后的视频占用内存更少；对于含噪声明显的视频，去噪后用户的主观感受也能得到显著改善。
对于装置实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
本申请实施例还提供了一种装置，包括：
包括处理器、存储器及存储在所述存储器上并能够在所述处理器上运行的计算机程序，该计算机程序被处理器执行时实现上述视频去噪方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例还提供了一种计算机可读存储介质，计算机可读存储介质上存储计算机程序，计算机程序被处理器执行时实现上述视频去噪方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。本文中所称的“一个实施例”、“实施例”或者“一个或者多个实施例”意味着，结合实施例描述的特定特征、结构或者特性包括在本申请的至少一个实施例中。此外，请注意，这里“在一个实施例中”的词语例子不一定全指同一个实施例。在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本申请的实 施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本申请可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (19)

  1. 一种视频去噪方法，其特征在于，应用于具有中央处理器CPU和图像处理器GPU的电子设备，所述方法包括：
    所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
    所述GPU从所述CPU加载所述视频帧图像；
    所述GPU对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
    所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像；
    所述CPU从所述GPU获取所述第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
  2. 根据权利要求1所述的方法，其特征在于，所述GPU对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像，包括：
    所述GPU对所述视频帧图像的像素点分配线程；
    所述线程确定像素点的第一参考图像块和对应的第一相似图像块；
    所述线程确定所述第一相似图像相对所述第一参考图像块的像素距离和坐标距离；
    所述线程采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；
    所述线程采用所述第一相似图像块的权重确定所述像素点去噪后的像素值；
    所述GPU将所有去噪后的像素点确定的图像作为第一图像。
  3. 根据权利要求2所述的方法，其特征在于，所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像，包括：
    所述GPU对所述第一图像的像素点分配线程；
    所述线程确定像素点的第二参考图像块和对应的第二相似图像块；
    所述线程采用所述第二相似图像块与所述第二参考图像块，计算所述第二参考图像块的协方差矩阵；
    所述线程采用所述第一参考图像块和所述第一相似图像块，计算所述第一参考图像块的均值；
    所述线程采用所述第二参考图像块的协方差矩阵和所述第一参考图像块的均值，计算所述第二参考图像块去噪后的像素值；
    所述线程采用包含同一像素点的所有去噪后的第二参考图像块，确定该像素点去噪后的像素值；
    所述GPU将所有去噪后的像素点确定的图像作为第二图像。
  4. 根据权利要求1所述的方法，其特征在于，在所述GPU对所述第一图像，按照预设的NLB去噪算法进行去噪，得到第二图像，之前还包括：
    所述GPU确定所述视频帧图像的噪声等级；
    所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像，包括：
    若所述噪声等级大于或等于所述预设噪声等级阈值，则所述GPU对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像。
  5. 根据权利要求2所述的方法，其特征在于，在所述GPU对所述视频帧图像的像素点分配线程之后，还包括：
    所述GPU将所述视频帧图像切分为多个第一子图像；
    所述GPU将属于同一第一子图像的像素点对应的线程，配置到同一线程块；
    所述GPU对所述线程块，分配在执行所述NLM去噪算法时所需要的数据；
    在同一线程块中的线程按照线程束执行时，所述线程束中的线程将所述所需的数据协同读取到所述GPU的共享内存。
  6. 根据权利要求3所述的方法，其特征在于，所述GPU对所述第一图像的像素点分配线程之后，还包括：
    所述GPU将所述第一图像切分为多个第二子图像；
    所述GPU将属于同一第二子图像的像素点对应的线程，配置到同一线程块；
    所述GPU对所述线程块，分配在执行所述NLB去噪算法时所需要的数据；
    在同一线程块中的线程按照线程束执行时，所述线程束中的线程将所述所需的数据协同读取到所述GPU的共享内存。
  7. 根据权利要求2所述的方法，其特征在于，所述线程确定像素点的第一参考图像块和对应的第一相似图像块，包括：
    所述线程以像素点为中心确定第一参考图像块；
    所述线程从所述GPU的共享内存，获取对应的第一搜索框；
    所述线程按预设第一像素间隔，在所述第一搜索框内的确定第一参考图像块对应的第一相似图像块。
  8. 根据权利要求3所述的方法，其特征在于，所述线程确定像素点的第二参考图像块和对应的第二相似图像块，包括：
    所述线程以像素点为中心确定第二参考图像块；
    所述线程从所述GPU的共享内存，获取对应的第二搜索框；
    所述线程按预设第二像素间隔，在所述第二搜索框内的确定第二参考图像块对应的第二相似图像块。
  9. 根据权利要求4所述的方法，其特征在于，所述GPU确定所述视频帧图像的噪声等级，包括：
    所述GPU确定加载的视频帧图像的数目；
    若所述加载的视频帧图像的数目小于预设帧数目阈值，则所述GPU按预设帧间隔，确定视频帧图像的噪声等级；
    所述GPU将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
  10. 根据权利要求9所述的方法，其特征在于，所述GPU确定所述视频帧图像的噪声等级，还包括：
    若所述加载的视频帧图像的数目大于或等于所述预设帧数目阈值，则所述GPU从加载的视频帧图像中选取预设数目的视频帧图像；
    所述GPU对所述预设数目的视频帧图像，按预设帧间隔，确定视频帧图像的噪声等级；
    所述GPU将确定得到的多个噪声等级的平均值，作为视频帧图像的噪声等级。
  11. 根据权利要求1所述的方法，其特征在于，在所述CPU对所述视频数据进行解码得到视频帧图像之后，还包括：
    所述CPU对所述视频帧图像进行填充。
  12. 一种视频去噪方法，其特征在于，应用于具有CPU和GPU的电子设备，所述方法包括：
    所述CPU获取视频数据，并对所述视频数据进行解码得到视频帧图像；
    所述GPU从所述CPU加载所述视频帧图像；
    所述GPU确定所述视频帧图像的噪声等级；
    所述GPU根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像；
    所述CPU从所述GPU获取第一图像，并对所述第一图像进行编码得到去噪后的视频数据；或者，从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据。
  13. 根据权利要求12所述的方法，其特征在于，所述GPU根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像，包括：
    当所述噪声等级小于预设噪声等级阈值时，所述GPU采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像；
    当所述噪声等级大于或等于所述预设噪声等级阈值时，所述GPU采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像。
  14. 根据权利要求13所述的方法，其特征在于，所述GPU采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，包括：
    所述GPU对所述视频帧图像的像素点分配线程；
    所述线程确定像素点的第一参考图像块和对应的第一相似图像块；
    所述线程确定所述第一相似图像相对所述第一参考图像块的像素距离和坐标距离；
    所述线程采用所述像素距离和所述坐标距离，计算所述第一相似图像块的权重；
    所述线程采用所述第一相似图像块的权重确定所述像素点去噪后的像素值；
    所述GPU将所有去噪后的像素点确定的图像作为第一图像。
  15. 根据权利要求14所述的方法，其特征在于，所述GPU采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像，包括：
    所述GPU对所述第一图像的像素点分配线程；
    所述线程确定像素点的第二参考图像块和对应的第二相似图像块；
    所述线程采用所述第二相似图像块与所述第二参考图像块，计算所述第二参考图像块的协方差矩阵；
    所述线程采用所述第一参考图像块和所述第一相似图像块，计算所述第一参考图像块的均值；
    所述线程采用所述第二参考图像块的协方差矩阵和所述第一参考图像块的均值，计算所述第二参考图像块去噪后的像素值；
    所述线程采用包含同一像素点的所有去噪后的第二参考图像块，确定该像素点去噪后的像素值；
    所述GPU将所有去噪后的像素点确定的图像作为第二图像。
  16. 一种电子设备，其特征在于，包括：中央处理器CPU和图像处理器GPU；
    所述CPU包括：
    视频数据解码模块，用于获取视频数据，并对所述视频数据进行解码得到视频帧图像；
    视频数据编码模块，用于从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据；
    所述GPU包括：
    视频帧图像加载模块，用于从所述CPU加载所述视频帧图像；
    第一图像去噪模块，用于对所述视频帧图像按照预设的非局部平均值NLM去噪算法进行去噪，得到第一图像；
    第二图像去噪模块，用于对所述第一图像按照预设的非局部贝叶斯NLB去噪算法进行去噪，得到第二图像。
  17. 一种电子设备，其特征在于，包括CPU和GPU；
    所述CPU包括：
    视频数据解码模块，用于获取视频数据，并对所述视频数据进行解码得到视频帧图像；
    视频数据编码模块，用于从所述GPU获取第一图像，并对所述第一图像进行编码得到去噪后的视频数据；或者，从所述GPU获取第二图像，并对所述第二图像进行编码得到去噪后的视频数据；
    所述GPU包括：
    视频帧图像加载模块，用于从所述CPU加载所述视频帧图像；
    噪声等级确定模块，用于确定所述视频帧图像的噪声等级；
    图像去噪模块，用于根据所述噪声等级，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，或者，采用预设的第一图像去噪算法对所述视频帧图像进程去噪得到第一图像，并采用预设的第二图像去噪算法对所述第一图像进行去噪得到第二图像。
  18. 一种装置，其特征在于，包括：处理器、存储器及存储在所述存储器上并能够在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如权利要求1-11或12-15中任一项所述的视频去噪方法的步骤。
  19. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储计算机程序，所述计算机程序被处理器执行时实现如权利要求1-11或12-15中任一项所述的视频去噪方法的步骤。
PCT/CN2020/129864 2019-12-31 2020-11-18 一种视频去噪方法和电子设备 Ceased WO2021135702A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20908710.5A EP4086840B1 (en) 2019-12-31 2020-11-18 Video denoising method and electronic device
US17/789,974 US12354242B2 (en) 2019-12-31 2020-11-18 Method for denoising videos and electronic device therefor

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911418935.0A CN111179201B (zh) 2019-12-31 2019-12-31 一种视频去噪方法和电子设备
CN201911418935.0 2019-12-31

Publications (1)

Publication Number Publication Date
WO2021135702A1 true WO2021135702A1 (zh) 2021-07-08

Family

ID=70650721

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/129864 Ceased WO2021135702A1 (zh) 2019-12-31 2020-11-18 一种视频去噪方法和电子设备

Country Status (4)

Country Link
US (1) US12354242B2 (zh)
EP (1) EP4086840B1 (zh)
CN (1) CN111179201B (zh)
WO (1) WO2021135702A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111179201B (zh) * 2019-12-31 2023-04-11 广州市百果园信息技术有限公司 一种视频去噪方法和电子设备
CN112702605B (zh) * 2020-12-24 2024-10-29 百果园技术(新加坡)有限公司 视频转码系统、视频转码方法、电子设备和存储介质
US12597102B2 (en) * 2021-04-28 2026-04-07 Asml Netherlands B.V. Image enhancement in charged particle inspection
US20230102843A1 (en) * 2021-09-27 2023-03-30 Nvidia Corporation User-configurable memory allocation
CN114648469B (zh) * 2022-05-24 2022-09-27 上海齐感电子信息科技有限公司 视频图像去噪方法及其系统、设备和存储介质
CN116596789B (zh) * 2023-05-18 2025-10-21 青岛海洋科技中心 非局部均值滤波去噪的加速方法
CN116681616B (zh) * 2023-06-09 2025-10-21 中国人民解放军国防科技大学 一种基于自监督学习的卷积神经网络图像去噪方法
CN117094906A (zh) * 2023-08-23 2023-11-21 和也健康科技有限公司 基于n2v和emi移除算法的电磁成像检测双重去噪方法及装置
CN117196979B (zh) * 2023-09-06 2026-04-17 普联技术有限公司 图像噪声处理方法、装置、电子设备及存储介质
CN117218032A (zh) * 2023-09-26 2023-12-12 北京瞰瞰智能科技有限公司 图像快速降噪的方法、装置、计算机设备及存储介质
CN117011193B (zh) * 2023-09-28 2023-12-05 生态环境部长江流域生态环境监督管理局生态环境监测与科学研究中心 一种轻量化凝视卫星视频去噪方法及去噪系统
CN118338074B (zh) * 2024-06-12 2024-08-20 广州市艾索技术有限公司 一种多媒体音视频数据管理系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102254323A (zh) * 2011-06-10 2011-11-23 西安电子科技大学 基于treelet融合和水平集分割的遥感图像变化检测
CN102393955A (zh) * 2011-07-18 2012-03-28 西安电子科技大学 用于图像复原的全信息非局部约束全变分方法
CN107085833A (zh) * 2017-04-13 2017-08-22 长安大学 基于梯度倒数自适应开关均中值融合的遥感图像滤波方法
CN107292851A (zh) * 2017-07-14 2017-10-24 重庆理工大学 一种基于伪3d变换的bm3d图像降噪方法
US20180089864A1 (en) * 2016-09-23 2018-03-29 Carestream Health, Inc. Cbct image processing method
CN109360156A (zh) * 2018-08-17 2019-02-19 上海交通大学 基于生成对抗网络的图像分块的单张图像去雨方法
CN111179201A (zh) * 2019-12-31 2020-05-19 广州市百果园信息技术有限公司 一种视频去噪方法和电子设备

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4993403B2 (ja) * 2006-06-02 2012-08-08 富士ゼロックス株式会社 画像処理装置、画像符号化装置、画像復号化装置、画像処理システム及びプログラム
CN103745447B (zh) * 2014-02-17 2016-05-25 东南大学 一种非局部均值滤波的快速并行实现方法
US10607321B2 (en) * 2016-06-22 2020-03-31 Intel Corporation Adaptive sharpness enhancement control
US10511846B1 (en) * 2016-09-01 2019-12-17 Google Llc Real-time adaptive video denoiser with moving object detection
US10469876B2 (en) * 2016-12-22 2019-11-05 Mediatek Inc. Non-local adaptive loop filter combining multiple denoising technologies and grouping image patches in parallel
CN107730464B (zh) * 2017-10-09 2021-03-30 四川大学 基于块匹配的图像降噪并行算法
US10643308B2 (en) * 2017-10-11 2020-05-05 Gopro, Inc. Double non-local means denoising
US11216912B2 (en) * 2017-10-18 2022-01-04 Gopro, Inc. Chrominance denoising
KR102324609B1 (ko) * 2017-12-22 2021-11-10 한국전자통신연구원 다지점 영상회의 장치 및 그 제어 방법
CN109615591A (zh) * 2018-11-27 2019-04-12 东莞信大融合创新研究院 一种基于gpu并行加速的三维块匹配降噪方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102254323A (zh) * 2011-06-10 2011-11-23 西安电子科技大学 基于treelet融合和水平集分割的遥感图像变化检测
CN102393955A (zh) * 2011-07-18 2012-03-28 西安电子科技大学 用于图像复原的全信息非局部约束全变分方法
US20180089864A1 (en) * 2016-09-23 2018-03-29 Carestream Health, Inc. Cbct image processing method
CN107085833A (zh) * 2017-04-13 2017-08-22 长安大学 基于梯度倒数自适应开关均中值融合的遥感图像滤波方法
CN107292851A (zh) * 2017-07-14 2017-10-24 重庆理工大学 一种基于伪3d变换的bm3d图像降噪方法
CN109360156A (zh) * 2018-08-17 2019-02-19 上海交通大学 基于生成对抗网络的图像分块的单张图像去雨方法
CN111179201A (zh) * 2019-12-31 2020-05-19 广州市百果园信息技术有限公司 一种视频去噪方法和电子设备

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
LIANG GUANGSHUN, WANG RI-WEI;WEN XIAN-BIN: "Image Denoising based on Bilateral Filtering and Non-Local Means", GUANGDIANZI-JIGUANG - JOURNAL OF OPTRONICS-LASER, TIANJIN DAXUE JIDIAN FENXIAO, TIANJIN, CN, vol. 26, no. 11, 1 November 2015 (2015-11-01), CN, pages 2231 - 2235, XP055828216, ISSN: 1005-0086, DOI: 10.16136/j.joel.2015.11.0554 *
See also references of EP4086840A4 *
SUN YUZHE, LU LEI;LUO XIAOSHU;GUO LEI;HAO ZHANLONG;TANG TANG: "Image Denoising Based on Bilateral Filtering Combined with Non-local Means Filtering", JOURNAL OF GUANGXI NORMAL UNIVERSITY (NATURAL SCIENCE EDITION), vol. 35, no. 2, 1 April 2017 (2017-04-01), pages 32 - 38, XP055828217, ISSN: 1001-6600, DOI: 10.16088/j.issn.1001-6600.2017.02.005 *
ZHANG FANG: "Research on Image Poisson Denoising", INFORMATION SCIENCE AND TECHNOLOGY, CHINESE MASTER’S THESES FULL-TEXT DATABASE, 15 February 2018 (2018-02-15), XP055828218 *

Also Published As

Publication number Publication date
CN111179201A (zh) 2020-05-19
EP4086840A4 (en) 2024-06-12
CN111179201B (zh) 2023-04-11
US12354242B2 (en) 2025-07-08
EP4086840B1 (en) 2026-04-01
EP4086840A1 (en) 2022-11-09
US20220392019A1 (en) 2022-12-08

Similar Documents

Publication Publication Date Title
CN111179201B (zh) 一种视频去噪方法和电子设备
US20230080693A1 (en) Image processing method, electronic device and readable storage medium
US20210287047A1 (en) Training method for image processing model, image processing method, network device, and storage medium
CN117408890A (zh) 一种视频图像传输质量增强方法及系统
CN113344804B (zh) 一种弱光图像增强模型的训练方法和弱光图像增强方法
CN111047543B (zh) 图像增强方法、装置和存储介质
JP5440241B2 (ja) 画像強調装置、画像強調方法及び画像強調プログラム
CN115439384B (zh) 一种无鬼影多曝光图像融合方法、装置
Hou et al. Underwater image dehazing and denoising via curvature variation regularization
CN112085705A (zh) 一种基于改进樽海鞘群算法的图像分割方法及装置
WO2021232963A1 (zh) 视频去噪方法、装置、移动终端和存储介质
Conde et al. Raw image reconstruction from RGB on smartphones. NTIRE 2025 challenge report
US20250363791A1 (en) System and Method for Low-Light Image Enhancement Using Hierarchical Adaptive Wavelet Decomposition with Cross-Scale Feature Fusion
CN118488219B (zh) 一种用于无人机航拍的图像压缩方法及装置
CN107330863B (zh) 一种基于噪声估计的图像去噪方法
Mori et al. Fast template matching based on normalized cross correlation using adaptive block partitioning and initial threshold estimation
CN121527577A (zh) 一种基于多尺度特征解耦与动态融合的图像质量优化方法
CN119027341B (zh) 视频处理方法、系统、装置、设备及介质
CN113627342B (zh) 视频深度特征提取优化的方法、系统、设备及存储介质
CN114550173B (zh) 图像预处理方法、装置、电子设备以及可读存储介质
CN118747833B (zh) 光源估计模型训练、光源估计方法和装置
CN117692652B (zh) 一种基于深度学习的可见光与红外视频融合编码方法
CN116681618B (zh) 图像去噪方法、电子设备及存储介质
CN113438386A (zh) 一种应用于视频处理的动静判定方法及装置
CN109584275B (zh) 一种目标跟踪方法、装置、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20908710

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020908710

Country of ref document: EP

Effective date: 20220801

WWG Wipo information: grant in national office

Ref document number: 17789974

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2020908710

Country of ref document: EP