WO2021114592A1 - 视频降噪方法、装置、终端及存储介质 - Google Patents

视频降噪方法、装置、终端及存储介质 Download PDF

Info

Publication number
WO2021114592A1
WO2021114592A1 PCT/CN2020/095359 CN2020095359W WO2021114592A1 WO 2021114592 A1 WO2021114592 A1 WO 2021114592A1 CN 2020095359 W CN2020095359 W CN 2020095359W WO 2021114592 A1 WO2021114592 A1 WO 2021114592A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pixel
noise reduction
pixels
target image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/095359
Other languages
English (en)
French (fr)
Inventor
李本超
李峰
刘程浩
刘毅
艾通
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP20898101.9A priority Critical patent/EP3993396B1/en
Publication of WO2021114592A1 publication Critical patent/WO2021114592A1/zh
Priority to US17/572,604 priority patent/US12094085B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/268—Signal distribution or switching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/70—Circuits for processing colour signals for colour killing
    • H04N9/71—Circuits for processing colour signals for colour killing combined with colour gain control
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G06T2207/20028—Bilateral filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20182—Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Definitions

  • This application relates to the field of multimedia technology, and in particular to a video noise reduction method, device, terminal and storage medium.
  • the pixels in the neighborhood often include the pixels that have completed the filtering process before, so that the subsequent pixels
  • the filtering process is a serial process, which results in a slower algorithm operation speed.
  • a video noise reduction method device, terminal, and storage medium are provided.
  • a video noise reduction method executed by a terminal, and the method includes:
  • the spatial filtering is used to eliminate the dependency between the pixels of the target image
  • time-domain filtering is performed on the pixels of the target image in parallel to obtain a second image
  • the first denoised image is the image of the target image
  • the first image and the second image are fused to obtain a second noise reduction image corresponding to the target image that has undergone noise reduction processing.
  • a video noise reduction device including:
  • the spatial filtering module is used to perform spatial filtering on the pixels of the target image in the video to be processed to obtain the first image; the spatial filtering is used to eliminate the dependency between the pixels of the target image;
  • the time domain filtering module is configured to perform time domain filtering on the pixels of the target image in parallel according to the frame difference between the first image and the first noise reduction image to obtain a second image.
  • the first noise reduction image The image is an image that has undergone noise reduction processing corresponding to the previous frame image of the target image;
  • the fusion module is configured to predict the corresponding first gain of the pixel of the second image in the second noise-reduced image according to the second gain coefficient corresponding to the pixel of the target image in the first noise-reduced image Coefficient; and according to the first gain coefficient, the first image and the second image are fused to obtain a second noise reduction image corresponding to the target image that has undergone noise reduction processing.
  • a non-volatile storage medium storing computer-readable instructions.
  • the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video noise reduction method.
  • a terminal includes a memory and a processor, and computer-readable instructions are stored in the memory.
  • the processor executes the steps of the video noise reduction method.
  • FIG. 1 is a video image collected by a low-performance camera configured by a laptop computer according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of a video conference process provided by an embodiment of the present application.
  • FIG. 3 is a structural block diagram of a video noise reduction system provided by an embodiment of the present application.
  • FIG. 4 is a flowchart of a video noise reduction method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of image filtering before pixel dependence is released according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of image filtering provided by an embodiment of the present application after pixel dependence is released;
  • FIG. 7 is a schematic diagram of a spatial filtering effect comparison provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a comparison before and after noise reduction processing provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a key process of a video noise reduction method provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an algorithm flow chart of a video noise reduction method provided by an embodiment of the present application.
  • FIG. 11 is a block diagram of a video noise reduction device provided by an embodiment of the present application.
  • FIG. 12 is a structural block diagram of a terminal provided by an embodiment of the present application.
  • the embodiment of the present application mainly relates to a scene of performing noise reduction processing on a video, and takes the noise reduction processing on a remote conference video as an example for description.
  • Remote video conferencing is an important part of the various functions of office collaboration products. It has very strict requirements on the captured video, and usually requires the use of high-definition cameras for video capture. When using a weaker camera for video capture, the captured video generally has noise. If these noises are not processed, the experience of the video conference will be poor.
  • Figure 1 is a video image captured by a low-performance camera configured on a laptop computer. As can be seen from Figure 1, the collected video images contain a lot of noise.
  • the embodiments of the present application can also be applied to perform noise reduction processing on the video collected by the mobile phone camera during a video call, or perform noise reduction processing on the video collected by the monitoring device, etc. The embodiment of the application does not limit this.
  • the video noise reduction method provided by the embodiments of the present application solves the dependency between pixels in the image to meet the needs of parallel computing. Because the parallel computing capability of GPU (Graphics Processing Unit, graphics processing unit) is stronger than that of CPU Therefore, the video noise reduction method provided by the embodiments of the present application replaces the CPU by calling the Metal (an image processing interface provided by Apple) or DirectX (an image processing interface provided by Microsoft) provided by the GPU. Realize the parallel processing of each pixel. Thereby, the processing speed of the noise reduction processing on the video is improved and the CPU usage is reduced. That is, the video noise reduction method provided by the embodiments of this application can achieve fast video noise reduction with a very low CPU occupancy rate.
  • FIG. 2 is a schematic diagram of a video conference process provided by an embodiment of the present application.
  • the video image collected by the camera is displayed locally after noise reduction processing and other operations, for example, displayed on the screen of a laptop computer.
  • the encoder encodes the video image after noise reduction processing, and transmits it to the remote end through the network.
  • the remote decoder decodes the video image and displays the decoded video image at the remote end.
  • the remote end can also be a notebook computer.
  • the video noise reduction system 300 may be used to implement video noise reduction, and includes: a terminal 310 and a video service platform 320.
  • the terminal 310 may be connected to the video service platform 320 through a wireless network or a wired network.
  • the terminal 310 may be at least one of a smart phone, a video camera, a desktop computer, a tablet computer, an MP4 player, and a laptop portable computer.
  • the terminal 310 installs and runs an application program that supports remote video conferences.
  • the terminal 310 may be a terminal used by a user, and an account of the user is logged in an application program running on the terminal.
  • the video service platform 320 includes at least one of a server, multiple servers, and a cloud computing platform.
  • the video service platform 320 is used to provide background services for remote video conferences, such as user management, video stream forwarding, and so on.
  • the video service platform 320 includes: an access server, a data management server, a user management server, and a database.
  • the access server is used to provide access services for the terminal 310.
  • the data management server is used to forward the video stream uploaded by the terminal, etc.
  • the same service is provided in a load balancing manner or the same service is provided in the manner of a main server and a mirror server, which is not limited in the embodiment of the present application.
  • the database is used to store user account information.
  • the account information is the data information that the user has authorized to collect.
  • the terminal 310 may generally refer to one of multiple terminals, and this embodiment only uses the local terminal 310 and two remote terminals 310 for illustration. Those skilled in the art may know that the number of the aforementioned terminals may be more or less. For example, the above-mentioned remote terminal may be only one, or the above-mentioned remote terminal may be dozens or hundreds, or more. The embodiment of the present application does not limit the number and types of terminals 310.
  • FIG. 4 is a flowchart of a video noise reduction method provided by an embodiment of the present application, as shown in FIG. 4. The method includes the following steps:
  • the terminal spatially filters the pixels of the target image in the video to be processed to obtain the first image; the spatial filtering is used to eliminate the dependency between pixels in the target image.
  • the terminal may implement spatial filtering of the pixels of the target image based on the first filter, that is, input the target image into the first filter, and the output of the first filter is the first filter after the spatial filtering.
  • the first filter may be an improved bilateral filter, and the first filter may process pixels of the target image in parallel.
  • the first filter is described below:
  • the bilateral filtering algorithm is a non-linear edge-preserving filtering algorithm, which is a compromise processing method that combines the spatial proximity of the image and the similarity of pixel values.
  • the bilateral filtering algorithm considers both spatial information and gray-scale similarity to achieve the purpose of edge preservation and denoising. It has simple, non-iterative, and local characteristics. Among them, edge preservation and denoising refers to replacing the original pixel value of the pixel by the average value of the neighboring pixels of the currently processed pixel.
  • the pixels of the image to be processed are usually first from left to right, then from top to bottom (or from top to bottom, then from top to bottom).
  • spatial filtering it is often achieved by performing linear or nonlinear processing on neighboring pixels of the currently processed pixel. Because in the process of filtering the image to be processed, when the processing order is performed on the subsequent pixels, the pixels in the neighborhood of the pixel often include the pixels that have completed the spatial filtering process before, resulting in the subsequent order.
  • the pixel points of has a dependency on the pixels that have been filtered, and this dependency causes the spatial filtering of the entire image to become a serial processing process.
  • the elimination of pixel dependence refers to the elimination of the dependence relationship between pixels.
  • I(p) represents the pixel value of the currently processed pixel in the image
  • I(q) represents the neighboring pixel of the currently processed pixel in the image
  • the pixel value of a point p represents the coordinates of the currently processed pixel in the image
  • q represents the coordinate of the neighboring pixel of the currently processed pixel in the image
  • ⁇ (p,q) represents the weight related to the position of the pixel
  • g ( ⁇ ) represents the Gaussian function
  • the ⁇ s and ⁇ r sub-tables represent the variance of the Gaussian function.
  • the I(q) corresponding to the neighborhood pixel before the currently processed pixel is the pixel value after spatial filtering
  • the I(q) corresponding to the neighborhood pixel after the currently processed pixel is (q) is the original pixel value of the neighborhood pixel.
  • the neighborhood pixel of the currently processed pixel refers to the pixel within the neighborhood of the currently processed pixel.
  • the size of the neighborhood of the pixel is different, and the number of the neighborhood of the pixel is different.
  • the neighborhood of a pixel can be four neighborhoods, that is, the upper neighborhood, the lower neighborhood, the left neighborhood, and the right neighborhood; the neighborhood pixels of the pixel are the four pixels adjacent to the top, bottom, left, and right of the pixel.
  • the neighborhood of a pixel can be eight neighborhoods, namely, the upper neighborhood, the upper left neighborhood, the upper right neighborhood, the lower neighborhood, the lower left neighborhood, the lower right neighborhood, the left neighborhood, and the right neighborhood; the neighborhood of the pixel
  • the pixel points are eight pixels surrounding the pixel point.
  • the neighborhood of the pixel can also be selected in other ways.
  • FIG. 5 is a schematic diagram of an image filter provided by an embodiment of the present application before pixel dependence is released.
  • the currently processed pixel is the central pixel, and the central pixel corresponds to 12 neighboring pixels.
  • the neighboring pixels located on the left and above the center pixel are the pixels that have been processed.
  • the neighboring pixels located to the right and below the center pixel are unprocessed pixels.
  • the first improvement is made to the above-mentioned process, that is, the above-mentioned bilateral filter is improved.
  • the pixel dependence between pixels is calculated, and the above-mentioned first filter is obtained.
  • the first filter is also based on the bilateral filtering algorithm. The difference is that when the pixel of the target image is filtered by the above formulas (1) and (2), the pixel value of the neighboring pixel of the pixel is The value of I(q) uses the original pixel value of the image, that is, the pixel value after filtering is not used. In this way, each pixel no longer depends on the pixel that is arranged before the current pixel in the processing order, and the influence of the pixel that is arranged before the current pixel in the processing order on the current pixel after filtering is eliminated.
  • FIG. 6 is a schematic diagram of an image filter provided by an embodiment of the present application after pixel dependence is released.
  • the currently processed pixel is the central pixel, and the central pixel corresponds to 12 neighboring pixels.
  • These 12 neighboring pixels are all unprocessed pixels, that is, the pixels of the neighboring pixels.
  • the values are all initial pixel values.
  • FIG. 7 is a schematic diagram of a spatial filtering effect comparison provided by an embodiment of the present application.
  • FIG. 7 exemplarily shows the target image, the image filtered by the bilateral filter, and the image filtered by the first filter.
  • the terminal can call the image processing interface provided by the GPU, such as Metal or DirectX. And so on, transfer the steps of performing spatial filtering on the pixels of the target image to the GPU for implementation.
  • the terminal can also call the image processing interface of the graphics processor, through the image processing interface to obtain the pixels of the target image in the video to be processed in parallel, and perform spatial filtering on the pixels obtained in parallel, thereby realizing parallel processing
  • the pixel points of the target image in the video are processed for spatial filtering, which accelerates the entire spatial filtering process, saves CPU resources and reduces the CPU occupancy rate.
  • the terminal acquires a first noise reduction image, where the first noise reduction image is an image that has undergone noise reduction processing corresponding to a previous frame of the target image.
  • the terminal after the terminal performs spatial filtering on the pixels of the target image, it may also perform temporal filtering on the pixels of the target image. Before performing temporal filtering on the pixels of the target image, the terminal may obtain the first denoising image corresponding to the previous frame of the target image. The subsequent step of temporally filtering the target image is performed based on the first noise-reduced image and the above-mentioned first image.
  • the terminal determines the frame difference between the first image and the first denoising image.
  • the terminal may store the noise-reduction pixel value of each pixel of the first noise-reduction image that has undergone noise reduction processing in the form of a two-dimensional array.
  • the terminal may also store the filtered pixel value of each pixel of the first image in the form of a two-dimensional array, and the pixel of the first image corresponds to each pixel of the first denoising image one-to-one.
  • the size of the array is the product of the height of the target image and the width of the image.
  • the terminal can calculate the difference between the noise-reduced pixel value of the pixel in the first noise-reduced image and the corresponding filtered pixel value in the first image, and use the difference as the pixel corresponding to the pixel.
  • Pixel frame difference the frame difference between the first image and the first denoising image is obtained, and the frame difference may be in the form of a two-dimensional array.
  • the terminal performs temporal filtering on the pixels of the target image in parallel according to the frame difference between the first image and the first denoising image to obtain a second image.
  • the terminal may input the frame difference between the first image and the first noise-reduced image and the target image into the second filter ,
  • the time domain filtering is performed based on the second filter, and the output of the second filter is the second image.
  • the second filter may be an improved Kalman filter based on the Kalman filter algorithm, that is, the third improvement in the embodiment of the present application is to improve the Kalman filter based on the Kalman filter algorithm to obtain The second filter described above.
  • the second filter is described below:
  • the process of time-domain filtering based on the Kalman filtering algorithm mainly includes two steps, one is prediction and the other is correction.
  • the prediction step the terminal predicts the corresponding pixel value and variance of any pixel in the target image based on the noise-reduced pixel value and variance corresponding to any pixel in the first noise-reduced image.
  • the correction step the terminal determines the gain coefficient corresponding to each pixel, and determines the gain coefficient, the corresponding pixel value of the pixel in the target image, and the corresponding noise reduction pixel value of the pixel in the first noise reduction image.
  • the gain coefficient is a parameter of the relationship between the pixel values of the corresponding pixels between two frames of images.
  • P k-1 represents the corresponding variance of the pixel in the first denoised image.
  • Q represents the variance offset coefficient, which is an empirical parameter in the Kalman filter algorithm. In this embodiment, Q is a constant
  • K k represents the corresponding gain coefficient of the pixel in the predicted noise reduction image of the target image.
  • R represents the gain offset coefficient, which is also an empirical parameter in the Kalman filter algorithm.
  • R is a parameter that changes following an iterative operation. It can be understood that both Q and R are empirical parameter factors, and Kalman filters with different performances can be obtained by adjusting them.
  • P k represents the variance that the pixel needs to use in the next frame of image.
  • the video noise reduction method provided by the embodiment of the present application optimizes the formula (4), introduces the frame difference when calculating the variance, and obtains the formula (8).
  • ⁇ represents the frame difference between the first image and the first noise-reduction image.
  • the video noise reduction method provided by the embodiment of the present application adds formula (9) and formula (10), and optimizes formula (5) to obtain formula (11).
  • R k represents the corresponding gain bias coefficient of the pixel in the target image
  • R k-1 represents the corresponding gain bias coefficient of the pixel in the first noise reduction image
  • K k-1 represents the pixel in the first noise reduction image.
  • U k represents the motion compensation coefficient.
  • this step can be implemented through the following sub-step 4041 to sub-step 4043. Since the terminal can perform temporal filtering on the pixels of the target image in parallel, in sub-step 4041 to sub-step 4044, any pixel in the target image is taken as an example for description. The pixels are the same. When all the pixels of the target image are processed by the terminal, the second image is obtained.
  • the terminal predicts the corresponding first gain coefficient of the pixel of the second image in the second noise-reduced image according to the second gain coefficient of the pixel of the target image in the first noise-reduced image. For details, refer to sub-step 4041 to sub-step 4044.
  • the terminal determines the second variance of the pixel according to the corresponding first variance of the pixel in the first noise-reduction image, the frame difference between the first image and the first noise-reduction image, and the variance offset coefficient.
  • the corresponding first variance of the pixel in the first noise-reduction image is P k-1
  • the frame difference between the first image and the first noise-reduction image is ⁇
  • the variance offset coefficient is Q.
  • the terminal obtains the second gain coefficient and the second gain offset coefficient corresponding to the pixel in the first noise reduction image, and determines the first gain coefficient corresponding to the pixel according to the second gain coefficient and the second gain offset coefficient.
  • Gain offset coefficient the first gain coefficient corresponding to the pixel according to the second gain coefficient and the second gain offset coefficient.
  • the second gain coefficient corresponding to the pixel in the first noise reduction image is K k-1
  • the second gain offset coefficient corresponding to the pixel in the first noise reduction image is R k-1
  • the first gain offset coefficient R k corresponding to the pixel can be calculated.
  • the terminal determines the motion compensation coefficient corresponding to the pixel point according to the frame difference.
  • the motion compensation coefficient U k corresponding to the pixel can be calculated according to formula (10).
  • the terminal determines the first gain coefficient corresponding to the pixel point according to the second variance, the first gain offset coefficient corresponding to the pixel point, and the motion compensation coefficient.
  • the second variance obtained from the above sub-step 4041 to sub-step 4043 The first gain offset coefficient R k and the motion compensation coefficient U k are calculated to obtain the first gain coefficient K k corresponding to the pixel point.
  • the terminal can also use formula (7) and the second variance To determine the third-party difference P k that the pixel needs to use in the next frame of image.
  • the terminal fuses the first image and the second image according to the first gain coefficient to obtain a second noise reduction image corresponding to the target image that has undergone noise reduction processing.
  • the terminal also obtains the first gain coefficient corresponding to the pixels of the second image in the process of performing temporal filtering on the pixels of the target image to obtain the second image.
  • the terminal may use the product of the difference between the first gain coefficient K k corresponding to the pixel and the preset value and the first pixel value x k of the pixel as the first fusion value, and the pixel The product of the corresponding first gain coefficient and the second pixel value Z k of the pixel is used as the second fusion value.
  • the first pixel value is the pixel value of the pixel after time-domain filtering
  • the second pixel value is the pixel value of the pixel after being spatially filtered.
  • the terminal sums the first fusion value and the second fusion value to obtain the noise reduction pixel value corresponding to the pixel.
  • the above summation process can be realized according to formula (12).
  • the terminal may use the first gain coefficient as a weighting coefficient for fusing the first image and the second image.
  • the difference between the first gain coefficient corresponding to the pixel of the second image in the second noise reduction image and the preset value 1 is used as the fusion weight of the pixel of the second image;
  • the first gain coefficient corresponding to the pixels in the second noise-reduced image is used as the fusion weight of the pixels of the first image, and the pixel values of the first image and the second image are weighted and fused to obtain the second noise-reduced image.
  • FIG. 8 is a schematic diagram of a comparison before and after noise reduction processing provided by an embodiment of the present application.
  • Figure 8 includes the target image before noise reduction and the target image after noise reduction. It can be seen from the figure that the noise in the target image after noise reduction is significantly reduced compared to the target image before noise reduction, that is, the embodiment of the present application provides The video noise reduction method effectively realizes the noise reduction of the target image.
  • a third filter can be set.
  • the third filter has the same structure as the first filter.
  • the third filter, the first filter, and the second filter can be processed by calling the GPU.
  • the interface processes each pixel in the target image in parallel to achieve noise reduction processing on the target image.
  • FIG. 9 is a schematic diagram of a key process of a video noise reduction method provided by an embodiment of the present application.
  • the figure includes input, noise reduction processing and output three parts, the input is the target image f C and the first noise reduction image
  • the first filter and the third filter are represented by image noise reduction filters F1 and F2, respectively.
  • the second filter is represented by the Kalman filter Fk. Parallel acceleration through GPU image processing interface.
  • the terminal When the terminal performs noise reduction on the target image, it processes the target image f C through the image noise reduction filter F1 to obtain the first image Calculate the first denoised image according to the processing result And the first image Frame difference between f D and the frame difference f D f C and a target input image Fk Kalman filter, the Kalman filter output image and the second image Fk noise reduction filter output F2 is fused, Obtain the second denoising image corresponding to the target image that has been denoised
  • the second denoising image may also be stored in the Kalman filter to participate in subsequent image operations.
  • FIG. 10 is a schematic diagram of the algorithm flow of a video noise reduction method provided in an embodiment of the present application.
  • Spatial filtering of the target image includes: Among them, the arrow indicates assignment.
  • Time-domain filtering of the target image includes: Time-domain filtering of any pixel of the target image includes: Calculate the frame difference; R k ⁇ 1+R k-1 (1+K k-1 ) -1 , calculate the gain offset coefficient;
  • the first noise reduction map Calculate the first gain coefficient; Calculate the pixel value after time-domain filtering of the pixel; Calculate the noise reduction pixel value; Calculate the variance to be used in the next frame of image and return
  • the video noise reduction method provided by the embodiments of this application has a fourth improvement, that is, the format of the input image is set to adopt the YCbCr (YUV) format, and the image is processed for noise reduction.
  • the first filter and the second filter respectively perform spatial filtering and temporal filtering on the brightness component of the target image, that is, only perform noise reduction processing on the Y channel that characterizes the brightness detail information.
  • the pixels of the target image are filtered in a spatial domain that eliminates pixel dependence, so that there is no longer a dependency relationship between pixels in the target image, and the first image and the first image obtained by the spatial filtering are filtered according to the spatial domain.
  • the frame difference between the noise reduction images is used to perform temporal filtering on the pixels of the target image in parallel, so that the video noise reduction process is converted from serial processing to parallel processing, and the noise reduction processing process is accelerated.
  • FIG. 11 is a block diagram of a video noise reduction device provided by an embodiment of the present application.
  • the device is used to perform the steps of the foregoing video noise reduction method.
  • the device includes: a spatial filtering module 1101, a temporal filtering module 1102, and a fusion module 1103.
  • the various modules included in the video noise reduction device may be implemented in whole or in part by software, hardware or a combination thereof.
  • the spatial filtering module 1101 is used to perform spatial filtering on the pixels of the target image in the video to be processed to obtain the first image; the spatial filtering is used to eliminate the dependency between the pixels of the target image.
  • the temporal filtering module 1102 is used to perform temporal filtering on the pixels of the target image in parallel according to the frame difference between the first image and the first noise-reduced image to obtain a second image, and the first noise-reduced image is the image of the target image.
  • the previous image corresponds to the image that has been processed for noise reduction.
  • the fusion module 1103 is used for predicting the corresponding first gain coefficient of the pixel of the second image in the second noise-reduced image according to the second gain coefficient of the pixel of the target image in the first noise-reduced image; and A gain coefficient is used to fuse the first image and the second image to obtain a second noise reduction image corresponding to the target image that has undergone noise reduction processing.
  • the spatial filtering module 1101 is further configured to obtain the initial pixel value of the neighboring pixels of each pixel for all pixels of the target image in the to-be-processed video; and according to the neighboring pixels The initial pixel value of, the pixel is spatially filtered.
  • the video noise reduction device further includes: an interface calling module for calling the image processing interface of the graphics processor; and a parallel acquisition module for acquiring pixels of the target image in the video to be processed in parallel through the image processing interface Points; and filtering the pixels obtained in parallel through the image processing interface.
  • the temporal filtering module 1102 is also used to obtain each pixel of the target image in parallel; for any pixel of the target image, according to the first variance of the pixel in the first denoising image , The frame difference between the first image and the first denoising image and the variance offset coefficient to determine the second variance of the pixel; according to the second variance, the first gain offset coefficient corresponding to the pixel, and the motion compensation coefficient, determine The first gain coefficient corresponding to the pixel; according to the first gain coefficient, the initial pixel value of the pixel and the corresponding noise reduction pixel value of the pixel in the first noise reduction image, the first pixel value after the time domain filtering of the pixel is determined And obtain a second image according to the first pixel value after time domain filtering of each pixel of the target image.
  • the video noise reduction device further includes: a first determining module, configured to determine the motion compensation coefficient according to the frame difference.
  • the video noise reduction device further includes: an acquisition module for acquiring a second gain coefficient and a second gain offset coefficient corresponding to a pixel in the first noise reduction image; and a second determination module for According to the second gain coefficient and the second gain offset coefficient, the first gain offset coefficient corresponding to the pixel point is determined.
  • the time-domain filtering module 1102 is also used for any pixel of the second image to calculate the difference between the first gain coefficient corresponding to the pixel and the preset value and the first pixel value of the pixel Take the product of the first gain coefficient corresponding to the pixel and the second pixel value of the pixel as the second fusion value, and the second pixel value is the pixel value of the pixel after spatial filtering; and The first fusion value and the second fusion value are summed to obtain the noise reduction pixel value corresponding to the pixel point.
  • the spatial filtering and the temporal filtering respectively process the brightness components of the pixels.
  • the pixels of the target image are filtered in a spatial domain that eliminates pixel dependence, so that there is no longer a dependency relationship between pixels in the target image, and the first image and the first image obtained by the spatial filtering are filtered according to the spatial domain.
  • the frame difference between the noise reduction images is used to perform temporal filtering on the pixels of the target image in parallel, so that the video noise reduction process is converted from serial processing to parallel processing, and the noise reduction processing process is accelerated.
  • the device provided in the above embodiment runs an application program, only the division of the above-mentioned functional modules is used as an example.
  • the above-mentioned function allocation can be completed by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above.
  • the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments.
  • FIG. 12 is a structural block diagram of a terminal 1200 provided by an embodiment of the present application.
  • the terminal 1200 can be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, moving picture expert compression standard audio layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, moving picture expert compressing standard audio Level 4) Player, laptop or desktop computer.
  • the terminal 1200 may also be called user equipment, portable terminal, laptop terminal, desktop terminal and other names.
  • the terminal 1200 includes a processor 1201 and a memory 1202.
  • the processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on.
  • the processor 1201 may adopt at least one hardware form among DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array, Programmable Logic Array). achieve.
  • the processor 1201 may also include a main processor and a coprocessor.
  • the main processor is a processor used to process data in the awake state, also called a CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor used to process data in the standby state.
  • the processor 1201 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used to render and draw content that needs to be displayed on the display screen.
  • the processor 1201 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computing operations related to machine learning.
  • AI Artificial Intelligence
  • the memory 1202 may include one or more computer-readable storage media, which may be non-transitory.
  • the memory 1202 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.
  • the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1201 to implement the video reduction provided by the method embodiment of the present application. Noise method.
  • the terminal 1200 may optionally further include: a peripheral device interface 1203 and at least one peripheral device.
  • the processor 1201, the memory 1202, and the peripheral device interface 1203 may be connected by a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface 1203 through a bus, a signal line, or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 1204, a display screen 1205, a camera component 1206, an audio circuit 1207, a positioning component 1208, and a power supply 1209.
  • the peripheral device interface 1203 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 1201 and the memory 1202.
  • the processor 1201, the memory 1202, and the peripheral device interface 1203 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1201, the memory 1202, and the peripheral device interface 1203 or The two can be implemented on a separate chip or circuit board, which is not limited in this embodiment.
  • the radio frequency circuit 1204 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals.
  • the radio frequency circuit 1204 communicates with a communication network and other communication devices through electromagnetic signals.
  • the radio frequency circuit 1204 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the radio frequency circuit 1204 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on.
  • the radio frequency circuit 1204 can communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocol includes, but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity, wireless fidelity) networks.
  • the radio frequency circuit 1204 may also include a circuit related to NFC (Near Field Communication), which is not limited in this application.
  • the display screen 1205 is used to display a UI (User Interface, user interface).
  • the UI can include graphics, text, icons, videos, and any combination thereof.
  • the display screen 1205 also has the ability to collect touch signals on or above the surface of the display screen 1205.
  • the touch signal may be input to the processor 1201 as a control signal for processing.
  • the display screen 1205 may also be used to provide virtual buttons and/or virtual keyboards, also called soft buttons and/or soft keyboards.
  • the display screen 1205 may be one display screen 1205, which is provided with the front panel of the terminal 1200; in other embodiments, there may be at least two display screens 1205, which are respectively arranged on different surfaces of the terminal 1200 or in a folded design; In still other embodiments, the display screen 1205 may be a flexible display screen, which is arranged on the curved surface or the folding surface of the terminal 1200. Furthermore, the display screen 1205 can also be set as a non-rectangular irregular pattern, that is, a special-shaped screen.
  • the display screen 1205 may be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).
  • the camera assembly 1206 is used to capture images or videos.
  • the camera assembly 1206 includes a front camera and a rear camera.
  • the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal.
  • the camera assembly 1206 may also include a flash.
  • the flash can be a single-color flash or a dual-color flash. Dual color temperature flash refers to a combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.
  • the audio circuit 1207 may include a microphone and a speaker.
  • the microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 1201 for processing, or input to the radio frequency circuit 1204 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively set in different parts of the terminal 1200.
  • the microphone can also be an array microphone or an omnidirectional collection microphone.
  • the speaker is used to convert the electrical signal from the processor 1201 or the radio frequency circuit 1204 into sound waves.
  • the speaker can be a traditional thin-film speaker or a piezoelectric ceramic speaker.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into human audible sound waves, but also convert the electrical signal into human inaudible sound waves for distance measurement and other purposes.
  • the audio circuit 1207 may also include a headphone jack.
  • the positioning component 1208 is used to locate the current geographic location of the terminal 1200 to implement navigation or LBS (Location Based Service, location-based service).
  • the positioning component 1208 may be a positioning component based on the GPS (Global Positioning System, Global Positioning System) of the United States, the Beidou system of China, the Granus system of Russia, or the Galileo system of the European Union.
  • the power supply 1209 is used to supply power to various components in the terminal 1200.
  • the power source 1209 may be alternating current, direct current, disposable batteries, or rechargeable batteries.
  • the rechargeable battery may support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal 1200 further includes one or more sensors 1210.
  • the one or more sensors 1210 include, but are not limited to: an acceleration sensor 1211, a gyroscope sensor 1212, a pressure sensor 1213, a fingerprint sensor 1214, an optical sensor 1215, and a proximity sensor 1216.
  • the acceleration sensor 1211 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1200.
  • the acceleration sensor 1211 can be used to detect the components of gravitational acceleration on three coordinate axes.
  • the processor 1201 may control the display screen 1205 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1211.
  • the acceleration sensor 1211 may also be used for the collection of game or user motion data.
  • the gyroscope sensor 1212 can detect the body direction and rotation angle of the terminal 1200, and the gyroscope sensor 1212 can cooperate with the acceleration sensor 1211 to collect the user's 3D actions on the terminal 1200. Based on the data collected by the gyroscope sensor 1212, the processor 1201 can implement the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 1213 may be disposed on the side frame of the terminal 1200 and/or the lower layer of the display screen 1205.
  • the processor 1201 performs left and right hand recognition or quick operation according to the holding signal collected by the pressure sensor 1213.
  • the processor 1201 can control the operability controls on the UI interface according to the pressure operation of the user on the display screen 1205.
  • the operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the fingerprint sensor 1214 is used to collect the user's fingerprint.
  • the processor 1201 identifies the user's identity according to the fingerprint collected by the fingerprint sensor 1214, or the fingerprint sensor 1214 identifies the user's identity according to the collected fingerprint. When it is recognized that the user's identity is a trusted identity, the processor 1201 authorizes the user to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings.
  • the fingerprint sensor 1214 may be provided on the front, back or side of the terminal 1200. When a physical button or a manufacturer logo is provided on the terminal 1200, the fingerprint sensor 1214 can be integrated with the physical button or the manufacturer logo.
  • the optical sensor 1215 is used to collect the ambient light intensity.
  • the processor 1201 may control the display brightness of the display screen 1205 according to the ambient light intensity collected by the optical sensor 1215. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1205 is increased; when the ambient light intensity is low, the display brightness of the display screen 1205 is decreased.
  • the processor 1201 may also dynamically adjust the shooting parameters of the camera assembly 1206 according to the ambient light intensity collected by the optical sensor 1215.
  • the proximity sensor 1216 also called a distance sensor, is usually arranged on the front panel of the terminal 1200.
  • the proximity sensor 1216 is used to collect the distance between the user and the front of the terminal 1200.
  • the processor 1201 controls the display screen 1205 to switch from the on-screen state to the off-screen state; when the proximity sensor 1216 detects When the distance between the user and the front surface of the terminal 1200 gradually increases, the processor 1201 controls the display screen 1205 to switch from the rest screen state to the bright screen state.
  • FIG. 12 does not constitute a limitation on the terminal 1200, and may include more or less components than those shown in the figure, or combine certain components, or adopt different component arrangements.
  • the embodiment of the present application also provides a computer-readable storage medium that stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the processor executes the steps of the video noise reduction method.
  • the steps of the video noise reduction method may be the steps in the video noise reduction method of each of the foregoing embodiments.
  • Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Picture Signal Circuits (AREA)

Abstract

一种视频降噪方法，视频降噪方法包括：对待处理视频中的目标图像的像素点进行空域滤波，得到第一图像；空域滤波用于消除目标图像的像素点之间的依赖关系；根据第一图像和第一降噪图像之间的帧差，并行对目标图像的像素点进行时域滤波，得到第二图像；根据目标图像的像素点在第一降噪图像中对应的第二增益系数，预测第二图像的像素点在第二降噪图像中对应的第一增益系数；及根据第一增益系数，对第一图像和第二图像进行融合，得到第二降噪图像。

Description

视频降噪方法、装置、终端及存储介质
本申请要求于2019年12月12日提交中国专利局，申请号为201911288617.7，申请名称为“视频降噪方法、装置、终端及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及多媒体技术领域，特别涉及一种视频降噪方法、装置、终端及存储介质。
背景技术
随着多媒体技术的发展，办公协同产品成为越来越多中大型企业必不可少会议工具，其中，远程视频会议是办公协同产品中较为重要的组成部分，为企业提供了极大的便利。在远程视频会议的通信过程中，摄像头采集到的视频往往包含大量的噪声，如果不降低这些的噪声，会导致视频会议的效果很差。
相关技术中，在对视频图像进行滤波处理过程中，在对顺序在后面的像素点进行滤波处理时，其邻域内的像素点往往包含了之前完成了滤波处理的像素点，使得后面的像素点对已经处理过的像素点存在依赖关系，及滤波处理过程为串行处理过程，导致算法运算速度较慢。
发明内容
根据本申请提供的各种实施例，提供了一种视频降噪方法、装置、终端及存储介质。
一种视频降噪方法，由终端执行，所述方法包括：
对待处理视频中的目标图像的像素点进行空域滤波，得到第一图像；所述空域滤波用于消除所述目标图像的像素点之间的依赖关系；
根据所述第一图像和第一降噪图像之间的帧差，并行对所述目标图像的像素点进行时域滤波，得到第二图像，所述第一降噪图像为所述目标图像的上一帧图像对应的已经过降噪处理的图像；
根据所述目标图像的像素点在所述第一降噪图像中对应的第二增益系数，预测所述第二图像的像素点在第二降噪图像中对应的第一增益系数；及
根据所述第一增益系数，对所述第一图像和所述第二图像进行融合，得到所述目标图像对应的已经过降噪处理的第二降噪图像。
一种视频降噪装置，包括：
空域滤波模块，用于对待处理视频中的目标图像的像素点进行空域滤波，得到第一图像；所述空域滤波用于消除所述目标图像的像素点之间的依赖关系；
时域滤波模块，用于根据所述第一图像和第一降噪图像之间的帧差，并行对所述目标图像的像素点进行时域滤波，得到第二图像，所述第一降噪图像为所述目标图像的上一帧图像对应的已经过降噪处理的图像；及
融合模块，用于根据所述目标图像的像素点在所述第一降噪图像中对应的第二增益系数，预测所述第二图像的像素点在第二降噪图像中对应的第一增益系数；及根据所述第一增益系数，对所述第一图像和所述第二图像进行融合，得到所述目标图像对应的已经过降噪处理的第二降噪图像。
一种存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行视频降噪方法的步骤。
一种终端，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行视频降噪方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种通过笔记本电脑配置的低性能摄像头采集到的视频图像；
图2是本申请实施例提供的一种视频会议的流程示意图；
图3是本申请实施例提供的一种视频降噪系统的结构框图；
图4是本申请实施例提供的一种视频降噪方法的流程图；
图5是本申请实施例提供的一种图像滤波解除像素依赖前的示意图；
图6是本申请实施例提供的一种图像滤波解除像素依赖后的示意图；
图7是本申请实施例提供的一种空域滤波效果对比的示意图；
图8是本申请实施例提供的一种降噪处理前后对比示意图；
图9是本申请实施例提供的一种视频降噪方法的关键流程示意图；
图10是本申请实施例提供的一种视频降噪方法的算法流程示意图；
图11是本申请实施例提供的一种视频降噪装置的框图；及
图12是本申请实施例提供的一种终端的结构框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
本申请实施例主要涉及对视频进行降噪处理的场景，并以对远程会议视频进行降噪处理为例进行说明。远程视频会议是办公协同产品各项功能中较为重要的一部分，其对采集到的视频有着非常严格的要求，通常需要使用高清摄像头来进行视频采集。在使用性能较弱的摄像头进行视频采集时，采集到的视频普遍存在噪声，如果不对这些噪声进行处理，会导致视频会议的体验很差。例如，参见图1所示，图1是通过笔记本电脑配置的低性能摄像头采集到的视频图像。从图1中可以看到，采集到的视频图像中包含有大量的噪声。另外，本申请实施例还可以应用于视频通话过程中对手机摄像头采集的视频进行降噪处理，或者对监控设备采集的视频进行降噪处理等，本申请实施例对此不进行限制。
下面简单介绍一下本申请实施例提供的视频降噪方法。为了使摄像头采集的视频能够满足远程视频会议的需求，通常需要对采集的视频进行降噪处理。目前已有多种对视频进行降噪处理的方法，而这些方法通常都是通过终端的CPU运行视频降噪相关的算法来实现视频的降噪处理。由于办公协同产品中不仅仅包括远程会议视频这一项功能，还包括流程审批、项目管理等多种功能，因此如果远程视频会议功能占用了大部分的CPU资源，则会导致办公协同产品的其他功能无法正常使用，或者导致办公协同产品对CPU的处理能力要求较高导致无法在大多数场景下使用。而本申请实施例提供的视频降噪方法，通过解除图像中像素点之间的依赖关系，使其满足并行计算的需求，由于GPU(Graphics Processing Unit，图形处理器)的并行计算能力强于CPU，因此本申请实施例提供的视频降噪方法通过调用GPU提供的Metal(一种苹果公司提供的图像处理接口)或者DirectX(一种微软公司提供的图像处理接口)等图像处理接口，来取代CPU实现对各像素点的并行处理。从而提高了对视频进行降噪处理时的处理速度并且减少了对CPU的占用。即本申请实施例提供的视频降噪方法，可以以很低的CPU占用率实现快速的视频降噪，降噪后的视频流再传输到远端显示，保证良好的视频会议体验的同时为办公协同产品的其他功能提供了大量的CPU资源。上述流程可参见图2所示，图2 是本申请实施例提供的一种视频会议的流程示意图。如图2所示，摄像头采集的视频图像经过降噪处理等操作后在本地显示，如在笔记本电脑的屏幕上进行显示。编码器对经过降噪处理后的视频图像进行编码，通过网络传输到远端，远端的解码器对视频图像进行解码，将解码后的视频图像在远端显示，该远端也可以为笔记本电脑。
图3是本申请实施例提供的一种视频降噪系统300的结构框图，该视频降噪系统300可以用于实现视频降噪，包括：终端310和视频服务平台320。
终端310可以通过无线网络或有线网络与视频服务平台320相连。终端310可以是智能手机、摄像机、台式计算机、平板电脑、MP4播放器和膝上型便携计算机中的至少一种。终端310安装和运行有支持远程视频会议的应用程序。示意性的，终端310可以是用户使用的终端，终端运行的应用程序内登录有该用户的账号。
视频服务平台320包括一台服务器、多台服务器和云计算平台中的至少一种。视频服务平台320用于提供远程视频会议的后台服务，如用户管理、视频流转发等。可选的，视频服务平台320包括：接入服务器、数据管理服务器、用户管理服务器和数据库。接入服务器用于提供终端310的接入服务。数据管理服务器用于对终端上传的视频流进行转发等。数据管理服务器可以是一台或多台，当数据管理服务器是多台时，存在至少两台数据管理服务器用于提供不同的服务，和/或，存在至少两台数据管理服务器用于提供相同的服务。比如以负载均衡方式提供同一种服务或者以主服务器和镜像服务器的方式提供同一种服务，本申请实施例对此不加以限定。数据库用于存储用户的账号信息。该账号信息为用户已授权采集的数据信息。
终端310可以泛指多个终端中的一个，本实施例仅以本地终端310和两个远程终端310来举例说明。本领域技术人员可以知晓，上述终端的数量可以更多或更少。比如上述远程终端可以仅为一个，或者上述远程终端为几十个或几百个，或者更多数量。本申请实施例对终端310的数量和类型不加以限定。
图4是本申请实施例提供的一种视频降噪方法的流程图，如图4所示。该方法包括以下步骤：
401、终端对待处理视频中的目标图像的像素点空域滤波，得到第一图像；空域滤波用于消除目标图像中各像素点之间的依赖关系。
在本申请实施例中，终端可以基于第一滤波器实现对目标图像的像素点的空域滤波，即将目标图像输入第一滤波器中，则第一滤波器的输出即为经过空域滤波后的第一图像。其中，第一滤波器可以为改进后的双边滤波器，该第一滤波器可以并行处理目标图像的像素点。
下面对第一滤波器进行说明：
在图像去噪领域，双边滤波算法是一种非线性的保边滤波算法，是结合了图像的空间邻近度和像素值相似度的一种折中处理方法。该双边滤波算法同时考虑了空域信息和灰度相似性，以达到保边去噪的目的，具有简单、非迭代、局部的特点。其中保边去噪指的是通过当前处理的像素点的邻域像素点的平均值来替代该像素点的原始像素值。在通过基于双边滤波算法的双边滤波器对待处理图像进行滤波处理过程中，对于待处理图像的像素点，通常是按照先从左到右，再从上到下(或先从上到下，再从左到右，或者其他完成整张图像扫描的顺序方式)的方式使用滤波模板对整张图像进行扫描。而在对像素点进行空域滤波时，往往是通过对当前处理的像素点的邻域像素点进行线性或者非线性的处理来实现。由于在对待处理图像进行滤波处理过程中，对处理顺序在后面的像素点进行滤波处理的时候，该像素点邻域内的像素点往往包含之前完成了空域滤波处理的像素点，则导致顺序在后面的像素点对已经滤波处理过的像素点存在依赖关系，而这样的依赖关系，导致了整个图像的空域滤波处理变成了一种串行处理过程。其中，解除像素依赖是指消除像素点之间的依赖关系。
第一滤波器的原理可以参见公式(1)和(2)所示。
Figure PCTCN2020095359-appb-000001
Figure PCTCN2020095359-appb-000002
其中，
Figure PCTCN2020095359-appb-000003
表示图像中当前处理的像素点经过空域滤波后的像素值，I(p)表示图像中当前处理的像素点的像素值，I(q)表示的是图像中当前处理的像素点的邻域像素点的像素值，p表示图像中当前处理的像素点的坐标，q表示图像中当前处理的像素点的邻域像素点的坐标，ω(p，q)表示与像素点位置相关的权重，g(·)表示高斯函数，σ s和σ r分表表示高斯函数的方差。
需要说明的是，顺序在当前处理的像素点之前的邻域像素点对应的I(q)为经过空域滤波后的像素值，而顺序在当前处理的像素点之后的邻域像素点对应的I(q)为该邻域像素点的原始的像素值。
可以理解，当前处理的像素点的邻域像素点，是指当前处理的像素点的邻域内的像素点。像素点的邻域大小不同，像素点的邻域像素点的数量不同。像素点的邻域可以是四邻域，即上邻域、下邻域、左邻域和右邻域；像素点的邻域像素点为该像素点上下左右相邻的四个像素点。像素点的邻域可以是八邻域，即上邻域、左上邻域、右上邻域、下邻域、左下邻域、右下邻域、左邻域和右邻域；像素点的邻域像素点为包围该像素点的八个像素点。当然，在其他的实施例中，像素点的邻域还可以有其他的选取方式。
例如，参见图5所示，图5是本申请实施例提供的一种图像滤波解除像素依赖前的示意图。在图5中，当前处理的像素点为中心像素点，该中心像素点对应12个邻域像素点，位置在中心像素点的左侧和上方的邻域像素点为已经过处理的像素点，位置在中心像素点的右侧和下方的邻域像素点为未经过处理的像素点。
由于上述空域滤波处理过程是串行处理过程，相对于并行处理过程耗时长，因此在本申请实施例中，对上述处理过程进行了第一个改进，即对上述双边滤波器进行了改进，解除了像素点之间的像素依赖，得到上述第一滤波 器。该第一滤波器同样基于双边滤波算法，不同点在于，在通过上述公式(1)和(2)对目标图像的像素点进行滤波处理时，该像素点的邻域像素点的像素值，即I(q)的值，全部采用图像最原始的像素值，即不使用经过滤波处理过后的像素值。这样各像素点不再依赖处理顺序排在当前像素点之前的像素点，解除了处理顺序排在当前像素点之前的像素点在滤波之后对当前像素点带来的影响。
例如，参见图6所示，图6是本申请实施例提供的一种图像滤波解除像素依赖后的示意图。在图6中，当前处理的像素点为中心像素点，该中心像素点对应12个邻域像素点，这12个邻域像素点均为未经过处理的像素点，即邻域像素点的像素值均为初始像素值。
由于解除了像素点之间的像素依赖，因此，终端基于第一滤波器对每个像素点进行空域滤波的处理过程相同。本步骤可以为：对于目标图像的全部像素点，终端可以获取每个像素点的邻域像素点的初始像素值。然后终端可以根据该邻域像素点的初始像素值，通过第一滤波器对该像素点进行空域滤波，得到该像素点经过空域滤波后的像素值。当终端基于第一滤波器将目标图像的全部像素点均处理完毕时，即得到第一图像，该第一图像中的各像素点的像素值为经过空域滤波后的像素值。参见图7所示，图7是本申请实施例提供的一种空域滤波效果对比的示意图。图7中实例性的示出了目标图像、通过双边滤波器滤波后的图像，通过第一滤波器滤波后的图像。
需要说明的是，由于GPU的并行计算能力强于CPU，因此在本申请实施例中，对上述处理过程进行了第二个改进，即终端可以通过调用GPU提供的图像处理接口，如Metal或DirectX等，将上述对目标图像的像素点进行空域滤波的步骤转移到GPU中实现。
相应的，终端还可以调用图形处理器的图像处理接口，通过该图像处理接口并行地获取待处理视频中的目标图像的像素点，并对并行获取的像素点进行空域滤波，从而实现并行地对待处理视频中的目标图像的像素点进行空域滤波，对整个空域滤波过程进行了加速，并且节省了CPU的资源，降低了 CPU的占用率。
402、终端获取第一降噪图像，该第一降噪图像为目标图像的上一帧图像对应的已经过降噪处理的图像。
在本申请实施例中，终端在对目标图像的像素点进行空域滤波后，还可以对目标图像的像素点进行时域滤波。在对目标图像的像素点进行时域滤波之前，终端可以获取目标图像的上一帧图像对应的第一降噪图像。基于该第一降噪图像和上述第一图像来执行后续对目标图像进行时域滤波的步骤。
403、终端确定第一图像和第一降噪图像之间的帧差。
在本申请实施例中，终端在获取第一降噪图像后，可以以二维数组的形式存储该第一降噪图像的每个经过降噪处理后的像素点的降噪像素值。相应的，终端也可以以二维数组的形式存储该第一图像的每个像素点的滤波像素值，且第一图像的像素点与第一降噪图像的各像素点一一对应，二维数组的大小为目标图像的高度与图像的宽度的乘积。终端可以对于任一像素点，计算该像素点在第一降噪图像中对应的降噪像素值和在第一图像中对应的滤波像素值的差值，将该差值作为该像素点对应的像素帧差。从而得到第一图像和第一降噪图像之间的帧差，该帧差可以为二维数组的形式。
404、终端根据第一图像和第一降噪图像之间的帧差，并行对目标图像的像素点进行时域滤波，得到第二图像。
在本申请实施例中，终端在得到第一图像和第一降噪图像之间的帧差后，可以将第一图像和第一降噪图像之间的帧差和目标图像输入第二滤波器中，基于第二滤波器进行时域滤波，该第二滤波器的输出即为该第二图像。其中，第二滤波器可以为改进后的基于卡尔曼滤波算法的卡尔曼滤波器，即在本申请施例中的第三个改进是对基于卡尔曼滤波算法的卡尔曼滤波器进行改进，得到上述第二滤波器。
下面对第二滤波器进行说明：
基于卡尔曼滤波算法进行时域滤波的过程主要包括两个步骤，一个是预测，一个是矫正。在进行预测步骤时，终端基于任一像素点在第一降噪图像 中对应的降噪像素值和方差，来预测该像素点在目标图像中对应的像素值和方差。在进行矫正步骤时，终端确定各像素点对应的增益系数，根据该增益系数、像素点在目标图像中对应的像素值以及像素点在第一降噪图像中对应的降噪像素值，确定该像素点时域滤波后的第一像素值。增益系数是两帧图像之间响应的像素点的像素值之间关系的参数。
上述步骤的实现原理可以参见下述公式(3)-(7)。
Figure PCTCN2020095359-appb-000004
其中，
Figure PCTCN2020095359-appb-000005
表示预测的像素点在目标图像的像素值，
Figure PCTCN2020095359-appb-000006
表示像素点在第一降噪图像中对应的降噪像素值。
Figure PCTCN2020095359-appb-000007
其中，
Figure PCTCN2020095359-appb-000008
表示预测的像素点在目标图像的方差。P k-1表示像素点在第一降噪图像中对应的方差。Q表示方差偏置系数，是卡尔曼滤波算法中的经验参数，在本实施例中，Q为常数
Figure PCTCN2020095359-appb-000009
其中，K k表示像素点在预测的目标图像的降噪图像中对应的增益系数。R表示增益偏置系数，也是卡尔曼滤波算法中的经验参数，在本实施例中，R为跟随迭代操作变化的参数。可以理解，Q和R都是经验参数因子，通过调整它们可以得到不同性能的卡尔曼滤波器。
Figure PCTCN2020095359-appb-000010
其中，P k表示像素点在下一帧图像需要用到的方差。
为了使算法运算速度更快，本申请实施例提供的视频降噪方法对公式(4)进行了优化，在计算方差时引入了帧差，得到公式(8)。
Figure PCTCN2020095359-appb-000011
其中，Δ表示第一图像和第一降噪图像之间的帧差。
而为了解决降噪滤波过程中的运动抖动问题，本申请实施例提供的视频降噪方法增加了公式(9)和公式(10)，并对公式(5)进行了优化得到公式(11)。
R k＝1+R k-1(1+K k-1) -1     (9)
其中，R k表示该像素点在目标图像对应增益偏置系数，R k-1表示像素点在第一降噪图像中对应的增益偏置系数，K k-1表示像素点在第一降噪图像中对应的增益系数。
Figure PCTCN2020095359-appb-000012
其中，U k表示运动补偿系数。
Figure PCTCN2020095359-appb-000013
相应的，本步骤可以通过以下子步骤4041至子步骤4043来实现。由于终端可以并行对目标图像的像素点进行时域滤波，在子步骤4041至子步骤4044中，示例性的以目标图像中的任一像素点为例进行说明，其他像素点的处理方式与该像素点相同。当终端将目标图像的全部像素点均处理完毕时，即得到第二图像。
405、终端根据目标图像的像素点在第一降噪图像中对应的第二增益系数，预测第二图像的像素点在第二降噪图像中对应的第一增益系数。具体参考子步骤4041至子步骤4044。
4041、终端根据像素点在第一降噪图像中对应的第一方差、第一图像和第一降噪图像之间的帧差以及方差偏置系数，确定该像素点的第二方差。
例如，该像素点在第一降噪图像中对应的第一方差为P k-1，第一图像和第一降噪图像之间的帧差为Δ，方差偏置系数为Q，根据上述公式(8)即可计算得到该像素点的第二方差
Figure PCTCN2020095359-appb-000014
4042、终端获取像素点在第一降噪图像中对应的第二增益系数以及第二增益偏置系数，根据该第二增益系数和该第二增益偏置系数，确定该像素点对应的第一增益偏置系数。
例如，该像素点在第一降噪图像中对应的第二增益系数为K k-1，该像素点在第一降噪图像中对应的第二增益偏置系数为R k-1，根据公式(9)即可计算得到该像素点对应的第一增益偏置系数R k。
4043、终端根据帧差确定该像素点对应的运动补偿系数。
例如，帧差为Δ，根据公式(10)即可计算得到该像素点对应的运动补偿系数U k。
4044、终端根据第二方差、像素点对应的第一增益偏置系数以及运动补偿系数，确定该像素点对应的第一增益系数。
例如，根据上述公式(11)，对上述子步骤4041至子步骤4043得到的第二方差
Figure PCTCN2020095359-appb-000015
第以增益偏置系数R k以及运动补偿系数U k进行计算，得到该像素点对应的第一增益系数K k。
需要说明的是，终端在得到该像素点对应的第一增益系数K k后，还可以根据公式(7)以及第二方差
Figure PCTCN2020095359-appb-000016
来确定该像素点在下一帧图像需要使用的第三方差P k。
406、终端根据第一增益系数，对第一图像和第二图像进行融合，得到目标图像对应的已经过降噪处理的第二降噪图像。
在本申请实施例中，终端在对目标图像的像素点进行时域滤波得到第二图像的过程中，还得到了第二图像的像素点对应的第一增益系数。对于任一像素点，终端可以将该像素点对应的第一增益系数K k与预设数值的差值和该像素点的第一像素值x k的乘积作为第一融合值，将该像素点对应的第一增益系数和该像素点的第二像素值Z k的乘积作为第二融合值。其中，第一像素值为该像素点在经过时域滤波后的像素值，第二像素值为该像素点在经过空域滤波后的像素值。终端对第一融合值和第二融合值求和，得到该像素点对应的降噪像素值。相应的，可以根据公式(12)来实现上述求和过程。
Figure PCTCN2020095359-appb-000017
其中，
Figure PCTCN2020095359-appb-000018
表示像素点对应的降噪像素值。
可以理解，终端在获取到第二图像的像素点在第二降噪图像中对应的第一增益系数后，可将第一增益系数用作融合第一图像和第二图像的权重系数。具体地，将第二图像的像素点在第二降噪图像中对应的第一增益系数与预设数值1之间的差值，用作第二图像的像素点的融合权重；将第二图像的像素点在第二降噪图像中对应的第一增益系数，用作第一图像的像素点的融合权重， 对第一图像和第二图像的像素值加权融合，得到第二降噪图像。
当全部像素点均融合完毕时，即得到降噪后的目标图像。例如，参见图8所示，图8是本申请实施例提供的一种降噪处理前后对比示意图。图8中包括降噪前的目标图像和降噪后的目标图像，由图中可知，降噪后的目标图像中的噪点相较于降噪前的目标图像明显减少，即本申请实施例提供的视频降噪方法有效的实现了对目标图像的降噪处理。
需要说明的是，上述步骤401至步骤405是本申请实施例提供的视频降噪方法的可选的实现方式，相应的该视频降噪方法还可以不按照上述步骤401至步骤405的顺序执行，或者，可选的还可以设置第三滤波器，该第三滤波器与第一滤波器的结构相同，该第三滤波器和上述第一滤波器以及上述第二滤波器通过调用GPU的图像处理接口并行对目标图像中的每个像素点进行处理，以实现对目标图像的降噪处理。
例如，参见图9所示，图9是本申请实施例提供的一种视频降噪方法的关键流程示意图。如图9所示，图中包括输入、降噪处理以及输出三个部分，输入的是目标图像f C以及第一降噪图像
Figure PCTCN2020095359-appb-000019
在降噪部分，第一滤波器和第三滤波器分别用图像降噪滤波器F1和F2来表示。第二滤波器用卡尔曼滤波器Fk来表示。通过GPU的图像处理接口进行并行加速。终端在对目标图像进行降噪时，通过图像降噪滤波器F1对目标图像f C进行处理，得到第一图像
Figure PCTCN2020095359-appb-000020
根据处理结果计算第一降噪图像
Figure PCTCN2020095359-appb-000021
和第一图像
Figure PCTCN2020095359-appb-000022
之间的帧差f D，将帧差f D和目标图像f C输入卡尔曼滤波器Fk，将卡尔曼滤波器Fk的输出结果第二图像和图像降噪滤波器F2的输出结果进行融合，得到目标图像对应的已经过降噪处理的第二降噪图像
Figure PCTCN2020095359-appb-000023
在另外的实施例中，该第二降噪图像还可以保存在卡尔曼滤波器中，参与后续图像的运算。
与图9所示的流程相对应的算法流程可以参见图10所示，图10是本申请实施例提供的一种视频降噪方法的算法流程示意图。其中，初始化参数包括：P＝0，Q＝0.05，R＝0，K＝0，上一帧图像经F1得到的像素值初始化为零。对目标图像进行空域滤波包括：
Figure PCTCN2020095359-appb-000024
其中，箭头表示 赋值。对目标图像进行时域滤波包括：
Figure PCTCN2020095359-appb-000025
对目标图像的任一像素点进行时域滤波包括：
Figure PCTCN2020095359-appb-000026
计算帧差；R k←1+R k-1(1+K k-1) -1，计算增益偏置系数；
Figure PCTCN2020095359-appb-000027
将第一降噪图
Figure PCTCN2020095359-appb-000028
Figure PCTCN2020095359-appb-000029
计算第一增益系数；
Figure PCTCN2020095359-appb-000030
计算像素点时域滤波后的像素值；
Figure PCTCN2020095359-appb-000031
计算降噪像素值；
Figure PCTCN2020095359-appb-000032
计算下一帧图像要用到的方差，返回
Figure PCTCN2020095359-appb-000033
还需要说明的是，由于本申请实施例提供的视频降噪方法中，对图像进行空域滤波时，解除了各像素点的依赖关系，使GPU可以对各像素点进行并行计算，并且，在对图像进行时域滤波时，同样不存在像素依赖的问题，也可以使GPU对各像素进行并行计算，从而整个视频降噪过程均可以并行处理。当复杂的降噪处理过程都迁移到GPU上完成时，电脑端的CPU占用率将会变的非常低。并且，为了进一步的加速降噪的处理过程，本申请实施例提供的视频降噪方法进行了第四改进，即将输入的图像的格式设置为采用YCbCr(YUV)格式，在对图像进行降噪处理时，第一滤波器和第二滤波器分别对目标图像的亮度分量进行空域滤波和时域滤波，即只对表征亮度细节信息的Y通道进行降噪处理。
为了更清楚的展示本申请实施例提供的视频降噪方法的在节约CPU占用率方面的效果，本申请还进行了对比实验，在对比实验中，采用两个不同型号的笔记本电脑进行对比。对比结果可以参见表1所示。
表1：
Figure PCTCN2020095359-appb-000034
根据表1可知，相对于不使用解耦以及GPU并行计算，在使用解耦以及 GPU并行计算时，CPU的占用率有明显的下降。
在本申请实施例中，通过对目标图像的像素点进行解除像素依赖的空域滤波，使得目标图像中的各像素点之间不再存在依赖关系，并且根据空域滤波得到的第一图像和第一降噪图像之间的帧差，来并行对目标图像的像素点进行时域滤波，使得视频降噪过程由串行处理转换为并行处理，加速了降噪处理过程。
图11是本申请实施例提供的一种视频降噪装置的框图。该装置用于执行上述视频降噪方法执行时的步骤，参见图11，装置包括：空域滤波模块1101、时域滤波模块1102以及融合模块1103。视频降噪装置中包括的各个模块可全部或部分通过软件、硬件或其组合来实现。
空域滤波模块1101，用于对待处理视频中的目标图像的像素点进行空域滤波，得到第一图像；空域滤波用于消除目标图像的像素点之间的依赖关系。
时域滤波模块1102，用于根据第一图像和第一降噪图像之间的帧差，并行对目标图像的像素点进行时域滤波，得到第二图像，第一降噪图像为目标图像的上一帧图像对应的已经过降噪处理的图像。
融合模块1103，用于根据目标图像的像素点在第一降噪图像中对应的第二增益系数，预测第二图像的像素点在第二降噪图像中对应的第一增益系数；及根据第一增益系数，对第一图像和第二图像进行融合，得到目标图像对应的已经过降噪处理的第二降噪图像。
在一个实施例中，空域滤波模块1101，还用于对于所述待处理视频中的目标图像的全部像素点，获取每个像素点的邻域像素点的初始像素值；及根据邻域像素点的初始像素值，对像素点进行空域滤波。
在一个实施例中，视频降噪装置还包括：接口调用模块，用于调用图形处理器的图像处理接口；及并行获取模块，用于通过图像处理接口并行获取待处理视频中的目标图像的像素点；及通过图像处理接口对并行获取的像素点进行滤波。
在一个实施例中，时域滤波模块1102，还用于并行获取目标图像的每个 像素点；对于目标图像的任一像素点，根据像素点在第一降噪图像中对应的第一方差、第一图像和第一降噪图像之间的帧差以及方差偏置系数，确定像素点的第二方差；根据第二方差、像素点对应的第一增益偏置系数以及运动补偿系数，确定像素点对应的第一增益系数；根据第一增益系数、像素点的初始像素值以及像素点在第一降噪图像中对应的降噪像素值，确定像素点时域滤波后的第一像素值；及根据所述目标图像的每个像素点时域滤波后的第一像素值，得到第二图像。
在一个实施例中，视频降噪装置还包括：第一确定模块，用于根据帧差，确定运动补偿系数。
在一个实施例中，视频降噪装置还包括：获取模块，用于获取像素点在第一降噪图像中对应的第二增益系数以及第二增益偏置系数；及第二确定模块，用于根据第二增益系数和第二增益偏置系数，确定像素点对应的第一增益偏置系数。
在一个实施例中，时域滤波模块1102，还用于第二图像的任一像素点，将像素点对应的第一增益系数与预设数值之间的差值和像素点的第一像素值的乘积作为第一融合值；将像素点对应的第一增益系数和像素点的第二像素值的乘积作为第二融合值，第二像素值为像素点在经过空域滤波后的像素值；及对第一融合值和第二融合值求和，得到像素点对应的降噪像素值。
在一个实施例中，其特征在于，空域滤波和时域滤波分别对像素点的亮度分量进行处理。
在本申请实施例中，通过对目标图像的像素点进行解除像素依赖的空域滤波，使得目标图像中的各像素点之间不再存在依赖关系，并且根据空域滤波得到的第一图像和第一降噪图像之间的帧差，来并行对目标图像的像素点进行时域滤波，使得视频降噪过程由串行处理转换为并行处理，加速了降噪处理过程。
需要说明的是：上述实施例提供的装置在运行应用程序时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分 配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的装置与方法实施例属于同一构思，其具体实现过程详见方法实施例。
图12是本申请实施例提供的一种终端1200的结构框图。该终端1200可以是：智能手机、平板电脑、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、笔记本电脑或台式电脑。终端1200还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1200包括有：处理器1201和存储器1202。
处理器1201可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器1201可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器1201也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器1201可以在集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器1201还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1202可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器1202还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1202中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1201所执行以实现本申请中方法实施例提供的视频降噪方法。
在一些实施例中，终端1200还可选包括有：外围设备接口1203和至少一个外围设备。处理器1201、存储器1202和外围设备接口1203之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口1203相连。具体地，外围设备包括：射频电路1204、显示屏1205、摄像头组件1206、音频电路1207、定位组件1208和电源1209中的至少一种。
外围设备接口1203可被用于将I/O(Input/Output，输入/输出)相关的至少一个外围设备连接到处理器1201和存储器1202。在一些实施例中，处理器1201、存储器1202和外围设备接口1203被集成在同一芯片或电路板上；在一些其他实施例中，处理器1201、存储器1202和外围设备接口1203中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路1204用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路1204通过电磁信号与通信网络以及其他通信设备进行通信。射频电路1204将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路1204包括：天线系统、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路1204可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路1204还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏1205用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏1205是触摸显示屏时，显示屏1205还具有采集在显示屏1205的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器1201进行处理。此时，显示屏1205还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏1205可以为一个，设置终端1200的前面板；在另一些实施例中，显示屏1205可以为至少两个，分别设置在终端1200的不同表面或呈折叠 设计；在再一些实施例中，显示屏1205可以是柔性显示屏，设置在终端1200的弯曲表面上或折叠面上。甚至，显示屏1205还可以设置成非矩形的不规则图形，也即异形屏。显示屏1205可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode,有机发光二极管)等材质制备。
摄像头组件1206用于采集图像或视频。可选地，摄像头组件1206包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端的前面板，后置摄像头设置在终端的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件1206还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路1207可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器1201进行处理，或者输入至射频电路1204以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在终端1200的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器1201或射频电路1204的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路1207还可以包括耳机插孔。
定位组件1208用于定位终端1200的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件1208可以是基于美国的GPS(Global Positioning System，全球定位系统)、中国的北斗系统、俄罗斯的格雷纳斯系统或欧盟的伽利略系统的定位组件。
电源1209用于为终端1200中的各个组件进行供电。电源1209可以是交流 电、直流电、一次性电池或可充电电池。当电源1209包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端1200还包括有一个或多个传感器1210。该一个或多个传感器1210包括但不限于：加速度传感器1211、陀螺仪传感器1212、压力传感器1213、指纹传感器1214、光学传感器1215以及接近传感器1216。
加速度传感器1211可以检测以终端1200建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器1211可以用于检测重力加速度在三个坐标轴上的分量。处理器1201可以根据加速度传感器1211采集的重力加速度信号，控制显示屏1205以横向视图或纵向视图进行用户界面的显示。加速度传感器1211还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器1212可以检测终端1200的机体方向及转动角度，陀螺仪传感器1212可以与加速度传感器1211协同采集用户对终端1200的3D动作。处理器1201根据陀螺仪传感器1212采集的数据，可以实现如下功能：动作感应(比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器1213可以设置在终端1200的侧边框和/或显示屏1205的下层。当压力传感器1213设置在终端1200的侧边框时，可以检测用户对终端1200的握持信号，由处理器1201根据压力传感器1213采集的握持信号进行左右手识别或快捷操作。当压力传感器1213设置在显示屏1205的下层时，由处理器1201根据用户对显示屏1205的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器1214用于采集用户的指纹，由处理器1201根据指纹传感器1214采集到的指纹识别用户的身份，或者，由指纹传感器1214根据采集到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器1201授权该用户执行相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、 下载软件、支付及更改设置等。指纹传感器1214可以被设置终端1200的正面、背面或侧面。当终端1200上设置有物理按键或厂商Logo时，指纹传感器1214可以与物理按键或厂商Logo集成在一起。
光学传感器1215用于采集环境光强度。在一个实施例中，处理器1201可以根据光学传感器1215采集的环境光强度，控制显示屏1205的显示亮度。具体地，当环境光强度较高时，调高显示屏1205的显示亮度；当环境光强度较低时，调低显示屏1205的显示亮度。在另一个实施例中，处理器1201还可以根据光学传感器1215采集的环境光强度，动态调整摄像头组件1206的拍摄参数。
接近传感器1216，也称距离传感器，通常设置在终端1200的前面板。接近传感器1216用于采集用户与终端1200的正面之间的距离。在一个实施例中，当接近传感器1216检测到用户与终端1200的正面之间的距离逐渐变小时，由处理器1201控制显示屏1205从亮屏状态切换为息屏状态；当接近传感器1216检测到用户与终端1200的正面之间的距离逐渐变大时，由处理器1201控制显示屏1205从息屏状态切换为亮屏状态。
本邻域技术人员可以理解，图12中示出的结构并不构成对终端1200的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
本申请实施例还提供了一种计算机可读存储介质，存储有计算机可读指令，计算机可读指令被处理器执行时，使得处理器执行上述视频降噪方法的步骤。此处视频降噪方法的步骤可以是上述各个实施例的视频降噪方法中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存 储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (15)

  1. 一种视频降噪方法，其特征在于，由终端执行，所述方法包括：
    对待处理视频中的目标图像的像素点进行空域滤波，得到第一图像；所述空域滤波用于消除所述目标图像的像素点之间的依赖关系；
    根据所述第一图像和第一降噪图像之间的帧差，并行对所述目标图像的像素点进行时域滤波，得到第二图像，所述第一降噪图像为所述目标图像的上一帧图像对应的已经过降噪处理的图像；
    根据所述目标图像的像素点在所述第一降噪图像中对应的第二增益系数，预测所述第二图像的像素点在第二降噪图像中对应的第一增益系数；及
    根据所述第一增益系数，对所述第一图像和所述第二图像进行融合，得到所述目标图像对应的已经过降噪处理的所述第二降噪图像。
  2. 根据权利要求1所述的方法，其特征在于，所述对待处理视频中的目标图像的像素点进行空域滤波，包括：
    针对所述待处理视频中的目标图像的全部像素点，获取每个所述像素点的邻域像素点的初始像素值；及
    根据所述邻域像素点的初始像素值，对所述像素点进行空域滤波。
  3. 根据权利要求1所述的方法，其特征在于，所述对待处理视频中的目标图像的像素点进行空域滤波，包括：
    调用图形处理器的图像处理接口；
    通过所述图像处理接口并行获取待处理视频中的目标图像的像素点；及
    通过所述图像处理接口对并行获取的像素点进行滤波。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述第一图像和第一降噪图像之间的帧差，并行对所述目标图像的像素点进行时域滤波，得到第二图像，包括：
    并行获取所述目标图像的每个像素点；
    对于所述目标图像的任一像素点，根据所述像素点在所述第一降噪图像中对应的第一方差、所述第一图像和第一降噪图像之间的帧差以及方差偏置 系数，确定所述像素点的第二方差；
    根据所述第二方差、所述像素点对应的第一增益偏置系数以及运动补偿系数，确定所述像素点对应的第一增益系数；
    根据所述第一增益系数、所述像素点的初始像素值以及所述像素点在所述第一降噪图像中对应的降噪像素值，确定所述像素点时域滤波后的第一像素值；及
    根据所述目标图像的每个像素点时域滤波后的第一像素值，得到第二图像。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述第二方差、所述像素点对应的第一增益偏置系数以及运动补偿系数，确定所述像素点对应的第一增益系数之前，所述方法还包括：
    根据所述帧差，确定所述运动补偿系数。
  6. 根据权利要求4所述的方法，其特征在于，所述根据所述第二方差、所述像素点对应的第一增益偏置系数以及运动补偿系数，确定所述像素点对应的第一增益系数之前，所述方法还包括：
    获取所述像素点在所述第一降噪图像中对应的第二增益系数以及第二增益偏置系数；及
    根据所述第二增益系数和所述第二增益偏置系数，确定所述像素点对应的第一增益偏置系数。
  7. 根据权利要求4所述的方法，其特征在于，所述根据所述第一增益系数，对所述第一图像和所述第二图像进行融合，得到所述目标图像对应的已经过降噪处理的第二降噪图像，包括：
    对于所述第二图像的任一像素点，将所述像素点对应的第一增益系数与预设数值之间的差值和所述像素点的第一像素值的乘积作为第一融合值；
    将所述像素点对应的第一增益系数和所述像素点的第二像素值的乘积作为第二融合值，所述第二像素值为所述像素点在经过所述空域滤波后的像素值；及
    对所述第一融合值和所述第二融合值求和，得到所述像素点对应的降噪像素值。
  8. 根据权利要求1所述的方法，其特征在于，所述空域滤波和时域滤波分别对像素点的亮度分量进行处理。
  9. 一种视频降噪装置，其特征在于，所述装置包括：
    空域滤波模块，用于对待处理视频中的目标图像的像素点进行空域滤波，得到第一图像；所述空域滤波用于消除所述目标图像的像素点之间的依赖关系；
    时域滤波模块，用于根据所述第一图像和第一降噪图像之间的帧差，并行对所述目标图像的像素点进行时域滤波，得到第二图像，所述第一降噪图像为所述目标图像的上一帧图像对应的已经过降噪处理的图像；及
    融合模块，用于根据所述目标图像的像素点在所述第一降噪图像中对应的第二增益系数，预测所述第二图像的像素点在第二降噪图像中对应的第一增益系数；及根据所述第一增益系数，对所述第一图像和所述第二图像进行融合，得到所述目标图像对应的已经过降噪处理的所述第二降噪图像。
  10. 根据权利要求9所述的装置，其特征在于，所述空域滤波模块，还用于对于所述待处理视频中的目标图像的全部像素点，获取每个所述像素点的邻域像素点的初始像素值；及根据所述邻域像素点的初始像素值，对所述像素点进行空域滤波。
  11. 根据权利要求10所述的装置，其特征在于，所述装置还包括：
    接口调用模块，用于调用图形处理器的图像处理接口；及
    并行获取模块，用于通过所述图像处理接口并行获取待处理视频中的目标图像的像素点；及通过所述图像处理接口对并行获取的像素点进行滤波。
  12. 根据权利要求9所述的装置，其特征在于，所述时域滤波模块，还用于并行获取所述目标图像的每个像素点；对于所述目标图像的任一像素点，根据所述像素点在所述第一降噪图像中对应的第一方差、所述第一图像和第一降噪图像之间的帧差以及方差偏置系数，确定所述像素点的第二方差；根 据所述第二方差、所述像素点对应的第一增益偏置系数以及运动补偿系数，确定所述像素点对应的第一增益系数；根据所述第一增益系数、所述像素点的初始像素值以及所述像素点在所述第一降噪图像中对应的降噪像素值，确定所述像素点时域滤波后的第一像素值；及根据所述目标图像的每个像素点时域滤波后的第一像素值，得到第二图像。
  13. 根据权利要求9所述的装置，其特征在于，所述时域滤波模块，还用于对于所述第二图像的任一像素点，将所述像素点对应的第一增益系数与预设数值之间的差值和所述像素点的第一像素值的乘积作为第一融合值；将所述像素点对应的第一增益系数和所述像素点的第二像素值的乘积作为第二融合值，所述第二像素值为所述像素点在经过所述空域滤波后的像素值；及对所述第一融合值和所述第二融合值求和，得到所述像素点对应的降噪像素值。
  14. 一种终端，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如权利要求1至8中任一项所述的方法的步骤。
  15. 一种存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如权利要求1至8中任一项所述的方法的步骤。
PCT/CN2020/095359 2019-12-12 2020-06-10 视频降噪方法、装置、终端及存储介质 Ceased WO2021114592A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20898101.9A EP3993396B1 (en) 2019-12-12 2020-06-10 Video denoising method, device, terminal, and storage medium
US17/572,604 US12094085B2 (en) 2019-12-12 2022-01-10 Video denoising method and apparatus, terminal, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911288617.7A CN110933334B (zh) 2019-12-12 2019-12-12 视频降噪方法、装置、终端及存储介质
CN201911288617.7 2019-12-12

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/572,604 Continuation US12094085B2 (en) 2019-12-12 2022-01-10 Video denoising method and apparatus, terminal, and storage medium

Publications (1)

Publication Number Publication Date
WO2021114592A1 true WO2021114592A1 (zh) 2021-06-17

Family

ID=69863644

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/095359 Ceased WO2021114592A1 (zh) 2019-12-12 2020-06-10 视频降噪方法、装置、终端及存储介质

Country Status (4)

Country Link
US (1) US12094085B2 (zh)
EP (1) EP3993396B1 (zh)
CN (1) CN110933334B (zh)
WO (1) WO2021114592A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115330628A (zh) * 2022-08-18 2022-11-11 盐城众拓视觉创意有限公司 基于图像处理的视频逐帧去噪方法
CN119831905A (zh) * 2024-12-19 2025-04-15 广东汇天航空航天科技有限公司 图像校正方法及装置

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110933334B (zh) * 2019-12-12 2021-08-03 腾讯科技（深圳）有限公司 视频降噪方法、装置、终端及存储介质
CN112740265A (zh) * 2020-04-28 2021-04-30 深圳市大疆创新科技有限公司 红外图像降噪方法、装置及设备
CN113362260B (zh) * 2021-07-21 2025-01-07 Oppo广东移动通信有限公司 图像优化方法及装置、存储介质及电子设备
AU2023327801A1 (en) * 2022-08-26 2025-03-20 Cuvos Pty Ltd A signal processing system
CN117876243A (zh) * 2022-09-30 2024-04-12 深圳市中兴微电子技术有限公司 视频降噪方法、电子设备及计算机可读存储介质
CN116228589B (zh) * 2023-03-22 2023-08-29 新创碳谷集团有限公司 一种视觉检测相机噪声点消除方法、设备及存储介质
CN116777775B (zh) * 2023-06-14 2025-10-10 杭州微影软件有限公司 一种针对红外图像的图像处理方法、装置及电子设备
CN118365554B (zh) * 2024-06-19 2024-08-23 深圳市超像素智能科技有限公司 视频降噪方法、装置、电子设备及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102769722A (zh) * 2012-07-20 2012-11-07 上海富瀚微电子有限公司 时域与空域结合的视频降噪装置及方法
CN104735300A (zh) * 2015-03-31 2015-06-24 中国科学院自动化研究所 基于权重滤波的视频去噪装置及方法
US20180220129A1 (en) * 2017-01-30 2018-08-02 Intel Corporation Motion, coding, and application aware temporal and spatial filtering for video pre-processing
CN109410124A (zh) * 2016-12-27 2019-03-01 深圳开阳电子股份有限公司 一种视频图像的降噪方法及装置
CN110933334A (zh) * 2019-12-12 2020-03-27 腾讯科技（深圳）有限公司 视频降噪方法、装置、终端及存储介质

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7110455B2 (en) * 2001-08-14 2006-09-19 General Instrument Corporation Noise reduction pre-processor for digital video using previously generated motion vectors and adaptive spatial filtering
KR20100036601A (ko) * 2008-09-30 2010-04-08 엘지전자 주식회사 영상 잡음 제거 장치 및 방법
US9451183B2 (en) * 2009-03-02 2016-09-20 Flir Systems, Inc. Time spaced infrared image enhancement
US8638395B2 (en) * 2009-06-05 2014-01-28 Cisco Technology, Inc. Consolidating prior temporally-matched frames in 3D-based video denoising
CN101964863B (zh) * 2010-05-07 2012-10-24 镇江唐桥微电子有限公司 一种自适应的时空域视频图像降噪方法
CN102497497B (zh) * 2011-12-05 2013-07-31 四川九洲电器集团有限责任公司 一种图像去噪算法中阈值动态调整的方法
US9497429B2 (en) * 2013-03-15 2016-11-15 Pelican Imaging Corporation Extended color processing on pelican array cameras
CN103369209B (zh) * 2013-07-31 2016-08-17 上海通途半导体科技有限公司 视频降噪装置及方法
CN103533214B (zh) * 2013-10-01 2017-03-22 中国人民解放军国防科学技术大学 一种基于卡尔曼滤波和双边滤波的视频实时去噪方法
WO2016109585A1 (en) * 2014-12-31 2016-07-07 Flir Systems, Inc. Image enhancement with fusion
CN107979712B (zh) 2016-10-20 2020-02-21 上海富瀚微电子股份有限公司 一种视频降噪方法及装置
CN108174056A (zh) 2016-12-07 2018-06-15 南京理工大学 一种时空域联合的微光视频降噪方法
CN109743473A (zh) 2019-01-11 2019-05-10 珠海全志科技股份有限公司 视频图像3d降噪方法、计算机装置及计算机可读存储介质
CN110493494B (zh) * 2019-05-31 2021-02-26 杭州海康威视数字技术股份有限公司 图像融合装置及图像融合方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102769722A (zh) * 2012-07-20 2012-11-07 上海富瀚微电子有限公司 时域与空域结合的视频降噪装置及方法
CN104735300A (zh) * 2015-03-31 2015-06-24 中国科学院自动化研究所 基于权重滤波的视频去噪装置及方法
CN109410124A (zh) * 2016-12-27 2019-03-01 深圳开阳电子股份有限公司 一种视频图像的降噪方法及装置
US20180220129A1 (en) * 2017-01-30 2018-08-02 Intel Corporation Motion, coding, and application aware temporal and spatial filtering for video pre-processing
CN110933334A (zh) * 2019-12-12 2020-03-27 腾讯科技（深圳）有限公司 视频降噪方法、装置、终端及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3993396A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115330628A (zh) * 2022-08-18 2022-11-11 盐城众拓视觉创意有限公司 基于图像处理的视频逐帧去噪方法
CN115330628B (zh) * 2022-08-18 2023-09-12 盐城众拓视觉创意有限公司 基于图像处理的视频逐帧去噪方法
CN119831905A (zh) * 2024-12-19 2025-04-15 广东汇天航空航天科技有限公司 图像校正方法及装置

Also Published As

Publication number Publication date
US20220130023A1 (en) 2022-04-28
EP3993396C0 (en) 2025-10-15
CN110933334A (zh) 2020-03-27
EP3993396B1 (en) 2025-10-15
US12094085B2 (en) 2024-09-17
EP3993396A4 (en) 2022-08-31
EP3993396A1 (en) 2022-05-04
CN110933334B (zh) 2021-08-03

Similar Documents

Publication Publication Date Title
US12094085B2 (en) Video denoising method and apparatus, terminal, and storage medium
CN110502954B (zh) 视频分析的方法和装置
KR102635373B1 (ko) 이미지 처리 방법 및 장치, 단말 및 컴퓨터 판독 가능 저장 매체
WO2021008456A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2020221012A1 (zh) 图像特征点的运动信息确定方法、任务执行方法和设备
CN111754386B (zh) 图像区域屏蔽方法、装置、设备及存储介质
CN110839174A (zh) 图像处理的方法、装置、计算机设备以及存储介质
CN110189348B (zh) 头像处理方法、装置、计算机设备及存储介质
JP2022550126A (ja) レンズの動きを制御する方法及び装置並びコンピュータ装置及びプログラム
CN108320263A (zh) 一种图像处理的方法、装置及移动终端
WO2022033272A1 (zh) 图像处理方法以及电子设备
CN111565309A (zh) 显示设备及其畸变参数确定方法、装置、系统及存储介质
CN110248197A (zh) 语音增强方法及装置
CN111050211B (zh) 视频处理方法、装置及存储介质
CN109754439B (zh) 标定方法、装置、电子设备及介质
CN110797042B (zh) 音频处理方法、装置及存储介质
CN111860064B (zh) 基于视频的目标检测方法、装置、设备及存储介质
CN113407774B (zh) 封面确定方法、装置、计算机设备及存储介质
CN107730443B (zh) 图像处理方法、装置及用户设备
CN114897915A (zh) 图像分割方法、装置、电子设备和存储介质
CN111757146B (zh) 视频拼接的方法、系统及存储介质
CN113420172A (zh) 图片分享方法、装置、计算机设备及介质
CN113129221A (zh) 图像处理方法、装置、设备及存储介质
CN114219746B (zh) 超广角图像获取方法、装置、存储介质及电子设备
CN110660031B (zh) 图像锐化方法及装置、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20898101

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 20898101.9

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2020898101

Country of ref document: EP

Effective date: 20220127

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2020898101

Country of ref document: EP