WO2016019484A1 - Appareil et procédé pour fournir une super-résolution d'une image à basse résolution - Google Patents

Appareil et procédé pour fournir une super-résolution d'une image à basse résolution Download PDF

Info

Publication number
WO2016019484A1
WO2016019484A1 PCT/CN2014/000755 CN2014000755W WO2016019484A1 WO 2016019484 A1 WO2016019484 A1 WO 2016019484A1 CN 2014000755 W CN2014000755 W CN 2014000755W WO 2016019484 A1 WO2016019484 A1 WO 2016019484A1
Authority
WO
WIPO (PCT)
Prior art keywords
resolution
image
low
sub
ground truth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2014/000755
Other languages
English (en)
Inventor
Xiaoou Tang
Chao Dong
Chen Change Loy
Kaiming He
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN201480082564.2A priority Critical patent/CN106796716B/zh
Priority to PCT/CN2014/000755 priority patent/WO2016019484A1/fr
Publication of WO2016019484A1 publication Critical patent/WO2016019484A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Definitions

  • the present application generally relates to a field of image processing, more particularly, to an apparatus and a method for providing super-resolution of a low-resolution image.
  • SR Super-resolution
  • SR is a class of techniques that enhance the resolution of an imaging system.
  • Recent state-of-the-art methods for super-resolution of a single image are mostly example-based. These methods either exploit internal similarities of the same image, or learn mapping functions from external low-resolution and high-resolution exemplar pairs.
  • the external example-based methods are often provided with abundant samples, but are challenged by the difficulties of modeling the data effectively and compactly.
  • CNN Convolutional neural networks
  • an apparatus for providing super-resolution of a low-resolution image may comprise: a patch extracting and representing device comprising a first set of filters configured to extract patches from the low-resolution image and represent each of the extracted patches as a high dimensional vector, a mapping device comprising a second set of filters configured to map nonlinearly each of the high dimensional vectors onto a high-resolution patch-wise representation and an aggregating device configured to aggregate the high-resolution patch-wise representations to generate a high-resolution image for the low-resolution image.
  • a method for providing super-resolution of a low-resolution image may comprise: extracting patches from the low-resolution image and representing each of the extracted patches as a high dimensional vector by a patch extracting and representing device comprising a first set of filters; mapping nonlinearly each of the high dimensional vectors onto a high-resolution patch-wise representation by a mapping device comprising a second set of filters; and aggregating the high-resolution patch- wise representations to generate a high-resolution image.
  • a method for training a convolutional neural network system for providing super-resolution of a low-resolution image may comprise: 1) sampling a low-resolution sub-image and its corresponding ground truth high-resolution sub-image from a predetermined training set; 2) reconstructing the low-resolution sub-image to a high-resolution sub-image by the convolutional neural network system; 3) generating a reconstruction error by comparing dissimilarity between the reconstructed high-resolution sub-image and the ground truth high-resolution sub-image; 4) back-propagating the reconstruction error through the convolutional neural network system so as to adjust weights on connections between neurons of the convolutional neural network system; and repeating steps l)-4) until an average value of the reconstruction error is lower than a preset threshold.
  • an apparatus for providing super-resolution of a low-resolution image may comprise a reconstructing unit configured to reconstruct the low-resolution image to a high-resolution image based on predetermined parameters and a training unit configured to train the convolutional neural network system with a predetermined training set so as to determine the parameters used by the reconstructing unit.
  • the reconstructing unit may comprise: a patch extracting and representing device configured to extract patches from the low-resolution image and represent each of the extracted patches as a high dimensional vector based on the predetermined parameters; a mapping device configured to map nonlinearly each of the high dimensional vectors onto a high-resolution patch-wise representation; and a aggregating device configured to aggregate the high-resolution patch-wise representations to generate a high-resolution image.
  • the patch extracting and representing device, the mapping device and the aggregating device comprise a plurality of convolutional layers, and the plurality of convolutional layers are sequentially connected to each other to form a convolutional neural network system.
  • the present application does not explicitly learn the dictionaries or manifolds for modeling the patch space. These are implicitly achieved via the convolutional layers. Furthermore, the patch extraction and aggregation are also formulated as convolutional layers, so are involved in the optimization. In the method and apparatus of the present application, the entire convolutional neural network is fully obtained through training, with little pre/post-processing. With a lightweight structure, the apparatus and method of the present application have achieved superior performance than the state-of-the-art methods.
  • Fig. 1 is a schematic diagram illustrating an apparatus for providing super-resolution of a low-resolution image consistent with an embodiment of the present application.
  • Fig. 2 is a schematic diagram illustrating an apparatus for providing super-resolution of a low-resolution image consistent with another embodiment of the present application.
  • FIG. 3 is a schematic diagram illustrating a convolutional neural network system, consistent with some disclosed embodiments.
  • Fig. 4. is a schematic diagram illustrating a training unit of the apparatus, consistent with some disclosed embodiments.
  • Fig. 5. is a schematic diagram illustrating a training set preparation device of the training unit, consistent with some disclosed embodiments.
  • Fig. 6 is a schematic diagram illustrating an apparatus for providing super-resolution of a low-resolution image when it is implemented in software, consistent with some disclosed embodiments.
  • Fig. 7 is a schematic flowchart illustrating a method for providing super-resolution of a low-resolution image, consistent with some disclosed embodiments.
  • FIG. 8 is a schematic flowchart illustrating a method for training a convolutional neural network system for providing super-resolution of a low-resolution image, consistent with some disclosed embodiments.
  • FIG. 1 is a schematic diagram illustrating an exemplary apparatus 1000 for providing super-resolution of a low-resolution image consistent with some disclosed embodiments.
  • the apparatus 1000 may comprise a patch extracting and representing device 100, a mapping device 200 and an aggregating device 300.
  • the patch extracting and representing device 100 may comprise a first set of filters configured to extract patches from the low-resolution image and represent each of the extracted patches as a high dimensional vector.
  • the mapping device 200 may comprise a second set of filters configured to map nonlinearly each of the high dimensional vectors onto a high-resolution patch-wise.
  • the aggregating device 300 may be configured to aggregate the high-resolution patch-wise representations to generate a high-resolution image for the low-resolution image.
  • the first set of filters is configured to extract patches from the low-resolution image and represent each of the extracted patches as a high dimensional vector by rule of a first nonlinear function of first parameters, in which the first parameters are determined from predetermined parameters associated with the low-resolution image.
  • the second set of filters is configured to map nonlinearly each of the high dimensional vectors onto a high-resolution patch-wise representation by rule of a second nonlinear function of second parameters, in which the second parameters are determined from predetermined parameters associated with the high dimensional vector.
  • the first set of filters, the second set of filters and the aggregating device as mentioned in the above will be further discussed in detail.
  • the low-resolution image is denoted by F
  • the high-resolution image is denoted by F(Y) which is as similar as possible to a ground truth high-resolution image X.
  • the first set of filters is configured to extract patches from the low-resolution image Y and represent each of the extracted patches as a high dimensional vector.
  • these vectors comprise a set of feature maps, of which the number equals to the dimensionality of the vectors.
  • a popular strategy in image restoration is to densely extract patches and then represent them by a set of pre-trained bases such as PCA (Principal Component Analysis), DCT (Discrete Cosine Transformation), Haar, etc.
  • the first set of filters may be simulated as an operation F .
  • Fj(Y) F' (W ! * Y + Bj) , (1) where W ⁇ and B ⁇ represent the filters and biases respectively.
  • F'(x) is a nonlinear function e.g., max(0,x), tanh(x) or l/(l+exp(-x)).
  • W ⁇ is of a size cx lx 1 xtii, where c is the number of channels in the input image, for example, if the input image is a color image, then c is 3, and i is the spatial size of a filter, and ti ⁇ is the number of filters.
  • W ⁇ applies ti ⁇ convolutions on the image, and each convolution has a kernel size cx/ix i.
  • the output is composed of ti ⁇ feature maps.
  • B ⁇ is an ⁇ -dimensional vector, whose each element is associated with a filter.
  • the second set of filters is configured to map nonlinearly each of the high dimensional vectors onto another high-dimensional vector.
  • the first set of filters extracts an ni-dimensional feature for each patch.
  • the second set of filters maps each of these ni-dimensional vectors into an n 2 -dimensional vector.
  • Each mapped vector is conceptually the representation of a high-resolution patch.
  • the second set of filters may be simulated as an operation F 2 :
  • F 2 (Y) F"(W2*F 1 (Y) + B 2 ), (2) where W 2 is of a size tiixl xl xn 2 , and B 2 is a n 2 -dimensional vector.
  • F"(x) is a nonlinear function e.g., max(0,x), tanh(x) or l/(l+exp(-x)).
  • each of the output n 2 -dimensional vectors is conceptually a representation of a high-resolution patch that will be used for reconstruction.
  • the aggregating device 300 aggregates the high-resolution patch-wise representations to generate a high-resolution image.
  • the aggregating device 300 may be simulated as an operation F3:
  • F(Y) W 3 * F 2 (Y) + B 3 , (3) where W3 is of a size n2Xf. 3 X.f3Xc, and B3 is a c-dimensional vector.
  • the filters may act like an averaging filter. If the representations of the high-resolution patches are in some other domains (e.g., coefficients in terms of some bases), W3 may behave as first projecting the coefficients onto the image domain and then averaging. In either way, W3 is a set of linear filters.
  • the apparatus 1000 may further comprise a comparing device (not shown) which is configured to sample a ground truth high-resolution image corresponding to the low-resolution image from a predetermined training set and compare dissimilarity between the aggregated high-resolution image and the sampled ground truth high-resolution image to generate a reconstruction error.
  • the reconstruction error comprises a mean squared error.
  • the reconstruction error is back-propagated in order to determine the parameters, i.e., Wi t W2 , W3, B ⁇ t B2 , and B3.
  • the apparatus 1000 may further comprise an upscaling unit (not shown), and the upscaling unit is configured to upscale the low-resolution image to a predetermined size.
  • the low-resolution image may be upscaled by using bicubic interpolation.
  • the upscaling is the only pre-processing in the embodiment.
  • Fig. 2 is a schematic diagram illustrating an apparatus 1000' for providing super-resolution of a low-resolution image consistent with another embodiment of the present application.
  • the apparatus 1000' may comprise a reconstructing unit 100' and a training unit 200' .
  • the reconstructing unit 100' is configured to reconstruct the low-resolution image to a high-resolution image based on predetermined parameters.
  • the reconstructing unit 100' may further comprise a patch extracting and representing device 110', a mapping device 120' and a aggregating device 130' .
  • the patch extracting and representing device 110', the mapping device 120' and the aggregating device 130' may connect together to form a convolutional neural network system.
  • Fig. 3 illustrates the layer configuration of the convolutional neural network system in mathematic simulation model.
  • each of the patch extracting and representing device 110', the mapping device 120' and the aggregating device 130' may be simulated as at least one convolutional layer, respectively. Different operations are performed at different convolutional layers, respectively.
  • the patch extracting and representing unit 110' is configured to extract patches from the low-resolution image and represent each of the extracted patches as a high dimensional vector based on the predetermined parameters. This is equivalent to convolving the image by a set of filters as mentioned above.
  • the mapping device 120' is configured to nonlinearly map each of the high dimensional vectors onto a high-resolution patch-wise representation. This is equivalent to applying a second set of filters as mentioned above which have a trivial spatial support lxl. Alternatively, it is possible to add more convolutional layers (whose spatial supports are lxl) to increase the non-linearity. But this can significantly increase the complexity of the convolutional neural network system, and thus demands more training data and time.
  • the aggregating device 130' is configured to aggregate the high-resolution patch-wise representations to generate a high-resolution image.
  • the training unit 200' is configured to train the convolutional neural network system with a predetermined training set so as to determine the parameters, for example W ⁇ , W 2 , W 3 , B B 2 , B 3 , used by the reconstructing unit.
  • the training unit 200' may comprise a sampling device 210', a comparing device 220', and a back-propagating device 230' .
  • the sampling device 210' may be configured to sample a low-resolution sub-image and its corresponding ground truth high-resolution sub-image from a predetermined training set and input the low-resolution sub-image to the convolutional neural network system.
  • “sub-images” means these samples are treated as small “images” rather than “patches”, in the sense that "patches” are overlapping and require some averaging as post-processing but "sub-images" need not.
  • the comparing device 220' may be configured to compare dissimilarity between the reconstructed high-resolution sub-image based on the input low-resolution sub-image from the convolutional neural network system and the corresponding ground truth high-resolution sub-image to generate a reconstruction error.
  • the reconstruction error may comprise a mean squared error, and the error is minimized by using stochastic gradient descent with the standard back propagation.
  • the back-propagating device 230' is configured to back-propagate the reconstruction error through the convolutional neural network system so as to adjust weights on connections between neurons of the convolutional neural network system.
  • the convolutional neural network system do not preclude the usage of other kinds of reconstruction error, if only the reconstruction error are derivable. If a better perceptually motivated metric is given during the training, it is flexible for the convolutional neural network system to adapt to that metric.
  • the training unit 200' may further comprise a training set preparation device configured to prepare the predetermined training set for training the convolutional neural network system.
  • Fig. 5 is a schematic diagram illustrating the training set preparation device of the training unit 200' .
  • the training set preparation device may comprise a cropper 241 ', a low-resolution sub-image generator 242', a pairing device 243' and a collector 244' .
  • the cropper 24 may be configured to crop randomly a plurality of sub-images from a randomly selected training image to generate a set of ground truth high-resolution sub-images. For example, the cropper 24 may crop n sub-images of mxm pixels each.
  • the low-resolution sub-image generator 242' may be configured to generate a set of low-resolution sub-images based on the set of ground truth high-resolution sub-images.
  • the pairing device 243' may be configured to pair each of the ground truth high-resolution sub-images with a corresponding low-resolution sub-image.
  • the collector 244' may be configured to collect all the pairs to form the predetermined training set.
  • the low-resolution sub-image generator 242' may comprise a blurring device, a sampling device and an upscaling device.
  • the blurring device may be configured to blur each of the ground truth high-resolution sub-images by a Gaussian kernel.
  • the sampling device may be configured to sample the blurred ground truth high-resolution sub-image by a predetermined scaling factor.
  • the upscaling device may be configured to upscale the sampled ground truth high-resolution sub-image by a predetermined scaling factor to generate the set of low-resolution sub-images.
  • the apparatus 1000 and 1000' may be implemented using certain hardware, software, or a combination thereof.
  • the embodiments of the present invention may be adapted to a computer program product embodied on one or more computer readable storage media (comprising but not limited to disk storage, CD-ROM, optical memory and the like) containing computer program codes.
  • Fig. 6 is a schematic diagram illustrating an apparatus 1000 and 1000' for providing super-resolution of a low-resolution image when it is implemented in software, consistent with some disclosed embodiments.
  • apparatus 1000 and 1000' may include a general purpose computer, a computer cluster, a mainstream computer, a computing device dedicated for providing online contents, or a computer network comprising a group of computers operating in a centralized or distributed fashion.
  • apparatus 1000 and 1000' may include one or more processors (processors 102, 104, 106 etc.), a memory 112, a storage device 116, and a bus to facilitate information exchange among various devices of apparatus 1000.
  • Processors 102-106 may include a central processing unit (“CPU"), a graphic processing unit (“GPU”), or other suitable information processing devices.
  • processors 102-106 can include one or more printed circuit boards, and/or one or more microprocessor chips. Processors 102-106 can execute sequences of computer program instructions to perform various methods that will be explained in greater detail below.
  • Memory 112 can include, among other things, a random access memory (“RAM”) and a read-only memory (“ROM”). Computer program instructions can be stored, accessed, and read from memory 112 for execution by one or more of processors 102-106. For example, memory 112 may store one or more software applications. Further, memory 112 may store an entire software application or only a part of a software application that is executable by one or more of processors 102-106. It is noted that although only one block is shown in Fig. 1, memory 112 may include multiple physical devices installed on a central computing device or on different computing devices.
  • Fig. 7 is a schematic flowchart illustrating a method 2000 for providing super-resolution of a low-resolution image, consistent with some disclosed embodiments.
  • the method 2000 may be described in detail with respect to Fig. 7.
  • patches are extracted from the low-resolution image and each of the extracted patches is represented as a high dimensional vector by the patch extracting and representing device comprising the first set of filters.
  • these vectors comprise a set of feature maps, of which the number equals to the dimensionality of the vectors.
  • a popular strategy in image restoration is to densely extract patches and then represent them by a set of pre-trained bases such as PCA, DCT, Haar, etc.
  • each of the high dimensional vectors is mapped nonlinearly onto a high-resolution patch-wise representation by a mapping device comprising a second set of filters.
  • the first set of filters extracts an ni-dimensional feature for each patch.
  • the second set of filters maps each of these ni-dimensional vectors into an n 2 -dimensional vector.
  • each mapped vector is conceptually the representation of a high-resolution patch. These vectors comprise another set of feature maps.
  • step S230 the high-resolution patch-wise representations are aggregated to generate a high-resolution image.
  • these steps S210-S230 may be simulated by the above-mentioned formulae (l)-(3).
  • the patches may be extracted from the low-resolution image and representing each of the extracted patches as a high dimensional vector by rule of a first nonlinear function of first parameters, in which the first parameters are determined from predetermined parameters associated with the low-resolution image.
  • F'(first parameters) may be max(0, first parameters), tan zf first parameters) or l/( l+exp( -first parameters )).
  • each of the high dimensional vectors onto a high-resolution patch-wise representation may be mapped nonlinearly by rule of a function of a second nonlinear function of second parameters, in which the second parameters are determined from predetermined parameters associated with the high dimensional vector.
  • F"(second parameters) may be max(0, second parameters), tanh( second parameters) or l/( 1 + exp( -second parameters ) ) .
  • the method 2000 may further comprise a step of sampling a ground truth high-resolution image corresponding to the low-resolution image from a predetermined training set and a step of comparing dissimilarity between the aggregated high-resolution image and the corresponding ground truth high-resolution image to generate a reconstruction error.
  • the reconstruction error is back-propagated in order to determine the parameters, i.e., Wi t W 2, W3, B ⁇ t B 2, and B 3 .
  • the method 2000 further comprises a step of preparing the predetermined training set.
  • a plurality of sub-images is first cropped from a randomly selected training image to generate a set of ground truth high-resolution sub-images. For example, n sub-images of fflXffl pixels each may be cropped.
  • a set of low-resolution sub-images are generated based on the set of ground truth high-resolution sub-images.
  • each of the ground truth high-resolution sub-images is paired with a corresponding low-resolution sub-image. Then, all the pairs are collected to form the predetermined training set.
  • each of the ground truth high-resolution sub-images is blurred by a Gaussian kernel, and the blurred ground truth high-resolution sub-image is downsampled by a predetermined scaling factor. Then, the downsampled sub-images are upscaled by the same predetermined scaling factor to generate the set of low-resolution sub-images.
  • the method 2000 may further comprise a step of upscaling the low-resolution image to a predetermined size (not shown) before the patches are extracted from the low-resolution image.
  • a method 3000 for training a convolutional neural network system for providing super-resolution of a low-resolution image is illustrated.
  • the method 3000 may be described in detail with respect to Fig. 8.
  • a low-resolution sub-image and its corresponding ground truth high-resolution sub-image are sampled from a predetermined training set at step S310.
  • a high-resolution sub-image is reconstructed from the low-resolution sub-image by the convolutional neural network system.
  • a reconstruction error is generated by comparing dissimilarity between the reconstructed high-resolution sub-image and the ground truth high-resolution sub-image.
  • the reconstruction error is back-propagated through the convolutional neural network system so as to adjust weights on connections between neurons of the convolutional neural network system. Repeating steps S310-S340 until an average value of the reconstruction error is lower than a preset threshold, for example, half of the mean square error between the low-resolution sub-images and high-resolution sub-image in the predetermined training set.
  • the present application does not explicitly learn the dictionaries or manifolds for modeling the patch space. These are implicitly achieved via the convolutional layers. Furthermore, the patch extraction and aggregation are also formulated as convolutional layers, so are involved in the optimization. In the method and apparatus of the present application, the entire convolutional neural network is fully obtained through training, with little pre/post-processing. With a lightweight structure, the apparatus and method of the present application have achieved superior performance than the state-of-the-art methods.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un appareil pour fournir une super-résolution d'une image à basse résolution. L'appareil peut comprendre : un dispositif d'extraction et de représentation de pièce comprenant un premier ensemble de filtres, configuré pour extraire des pièces à partir de l'image à basse résolution et représenter chacune des pièces extraites sous la forme d'un vecteur de grande dimension, un dispositif de mappage comprenant un second ensemble de filtres, configuré pour mapper de manière non linéaire chacun des vecteurs de grande dimension sur une représentation par pièces à haute résolution, et un dispositif d'agrégation configuré pour agréger les représentations par pièces à haute résolution pour générer une image à haute résolution pour l'image à basse résolution. L'invention concerne également un procédé pour fournir une super-résolution d'une image à basse résolution.
PCT/CN2014/000755 2014-08-08 2014-08-08 Appareil et procédé pour fournir une super-résolution d'une image à basse résolution Ceased WO2016019484A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201480082564.2A CN106796716B (zh) 2014-08-08 2014-08-08 用于为低分辨率图像提供超分辨率的设备和方法
PCT/CN2014/000755 WO2016019484A1 (fr) 2014-08-08 2014-08-08 Appareil et procédé pour fournir une super-résolution d'une image à basse résolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2014/000755 WO2016019484A1 (fr) 2014-08-08 2014-08-08 Appareil et procédé pour fournir une super-résolution d'une image à basse résolution

Publications (1)

Publication Number Publication Date
WO2016019484A1 true WO2016019484A1 (fr) 2016-02-11

Family

ID=55262969

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/000755 Ceased WO2016019484A1 (fr) 2014-08-08 2014-08-08 Appareil et procédé pour fournir une super-résolution d'une image à basse résolution

Country Status (2)

Country Link
CN (1) CN106796716B (fr)
WO (1) WO2016019484A1 (fr)

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106251289A (zh) * 2016-07-21 2016-12-21 北京邮电大学 一种基于深度学习和自相似性的视频超分辨率重建方法
CN106447609A (zh) * 2016-08-30 2017-02-22 上海交通大学 一种基于深度卷积神经网络的图像超分辨方法
CN106991646A (zh) * 2017-03-28 2017-07-28 福建帝视信息科技有限公司 一种基于密集连接网络的图像超分辨率方法
CN108537733A (zh) * 2018-04-11 2018-09-14 南京邮电大学 基于多路径深度卷积神经网络的超分辨率重建方法
CN108615222A (zh) * 2018-04-17 2018-10-02 中国矿业大学 一种基于多对多连接的深度卷积网络图像超分辨率系统
CN108694700A (zh) * 2017-04-10 2018-10-23 三星电子株式会社 用于深度学习图像超分辨率的系统和方法
CN109325915A (zh) * 2018-09-11 2019-02-12 合肥工业大学 一种用于低分辨率监控视频的超分辨率重建方法
CN109360148A (zh) * 2018-09-05 2019-02-19 北京悦图遥感科技发展有限公司 基于混合随机降采样的遥感影像超分辨率重构方法和装置
CN109801214A (zh) * 2018-05-29 2019-05-24 京东方科技集团股份有限公司 图像重构装置及方法、设备、计算机可读存储介质
US10410322B2 (en) 2017-04-05 2019-09-10 Here Global B.V. Deep convolutional image up-sampling
CN110310227A (zh) * 2019-06-27 2019-10-08 电子科技大学 一种基于高低频信息分解的图像超分辨率重建方法
CN110322528A (zh) * 2019-06-26 2019-10-11 浙江大学 基于3t、7t的核磁共振脑部图像血管重建方法
CN110390638A (zh) * 2019-07-22 2019-10-29 北京工商大学 一种高分辨率三维体素模型重建方法
WO2019231235A1 (fr) * 2018-05-29 2019-12-05 엘지전자 주식회사 Dispositif de traitement de signal et dispositif d'affichage d'image comprenant celui-ci
CN110675324A (zh) * 2018-07-02 2020-01-10 上海寰声智能科技有限公司 4k超高清图像清晰化处理方法
CN110738601A (zh) * 2019-10-23 2020-01-31 智慧视通(杭州)科技发展有限公司 一种基于三维人脸模型低分辨率人脸图像超分辨率重建方法
CN110738099A (zh) * 2019-08-30 2020-01-31 中山大学 一种基于自适应双分支网络的低分辨率行人再识别方法
CN111461990A (zh) * 2020-04-03 2020-07-28 华中科技大学 一种基于深度学习分步实现超分辨成像的方法
CN111563843A (zh) * 2020-04-30 2020-08-21 苏州大学 一种图像超分辨重构方法、系统及相关装置
CN111696034A (zh) * 2020-05-11 2020-09-22 青岛小鸟看看科技有限公司 图像处理方法、装置及电子设备
CN112184555A (zh) * 2020-10-21 2021-01-05 天津大学 一种基于深度交互学习的立体图像超分辨率重建方法
CN112598575A (zh) * 2020-12-22 2021-04-02 电子科技大学 一种基于特征处理的图像信息融合及超分辨率重建方法
CN112825271A (zh) * 2019-11-21 2021-05-21 通用电气精准医疗有限责任公司 使用深度神经网络和测量数据的递归抽取来重建医疗图像的系统和方法
CN112907441A (zh) * 2020-12-29 2021-06-04 中央财经大学 一种基于地下水卫星图像超分辨率的空间降尺度方法
EP3714404A4 (fr) * 2017-11-26 2021-08-18 Yeda Research and Development Co. Ltd Rehaussement et manipulation de signal à l'aide d'un réseau profond spécifique d'un signal
CN115601242A (zh) * 2022-12-13 2023-01-13 电子科技大学（Cn） 一种适用于硬件部署的轻量级图像超分辨率重建方法
WO2025041222A1 (fr) * 2023-08-21 2025-02-27 Nec Corporation Appareil de mise en correspondance d'images, procédé de mise en correspondance d'images, appareil d'entraînement, procédé d'entraînement et support non transitoire lisible par ordinateur

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110785709B (zh) * 2017-06-30 2022-07-15 科磊股份有限公司 从低分辨率图像产生高分辨率图像以用于半导体应用
CN108416733A (zh) * 2018-02-07 2018-08-17 杭州电子科技大学 一种不降低质量且快速的图像超分辨率方法
CN108765343B (zh) * 2018-05-29 2021-07-20 Oppo（重庆）智能科技有限公司 图像处理的方法、装置、终端及计算机可读存储介质
CN112598579B (zh) * 2020-12-28 2024-08-27 苏州科达特种视讯有限公司 面向监控场景的图像超分辨率方法、装置及存储介质
TWI879417B (zh) * 2023-04-20 2025-04-01 創惟科技股份有限公司 影像處理方法及影像處理裝置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020172434A1 (en) * 2001-04-20 2002-11-21 Mitsubishi Electric Research Laboratories, Inc. One-pass super-resolution images
CN101299235A (zh) * 2008-06-18 2008-11-05 中山大学 一种基于核主成分分析的人脸超分辨率重构方法
CN101639937A (zh) * 2009-09-03 2010-02-03 复旦大学 一种基于人工神经网络的超分辨率方法
CN101710386A (zh) * 2009-12-25 2010-05-19 西安交通大学 一种基于相关特征和非线性映射的超分辨率人脸识别方法
CN102306306A (zh) * 2011-08-30 2012-01-04 西安交通大学 使用局部二值模式和径向基函数的对特征进行超分辨率重建的字符识别方法
CN102306374A (zh) * 2011-08-30 2012-01-04 西安交通大学 一种位置块非线性映射的人脸图像超分辨率重建方法
CN103500165A (zh) * 2013-08-21 2014-01-08 新华通讯社 一种结合聚类和双key值的高维向量检索方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102880878B (zh) * 2011-07-15 2015-05-06 富士通株式会社 一种基于单幅图像进行超分辨率分析的方法及系统
CN102354394B (zh) * 2011-09-22 2015-03-11 中国科学院深圳先进技术研究院 图像超分辨方法及系统
CN103839242A (zh) * 2014-01-15 2014-06-04 中国科学院电子学研究所 一种基于高维索引的图像快速超分辨率增强方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020172434A1 (en) * 2001-04-20 2002-11-21 Mitsubishi Electric Research Laboratories, Inc. One-pass super-resolution images
CN101299235A (zh) * 2008-06-18 2008-11-05 中山大学 一种基于核主成分分析的人脸超分辨率重构方法
CN101639937A (zh) * 2009-09-03 2010-02-03 复旦大学 一种基于人工神经网络的超分辨率方法
CN101710386A (zh) * 2009-12-25 2010-05-19 西安交通大学 一种基于相关特征和非线性映射的超分辨率人脸识别方法
CN102306306A (zh) * 2011-08-30 2012-01-04 西安交通大学 使用局部二值模式和径向基函数的对特征进行超分辨率重建的字符识别方法
CN102306374A (zh) * 2011-08-30 2012-01-04 西安交通大学 一种位置块非线性映射的人脸图像超分辨率重建方法
CN103500165A (zh) * 2013-08-21 2014-01-08 新华通讯社 一种结合聚类和双key值的高维向量检索方法

Cited By (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106251289A (zh) * 2016-07-21 2016-12-21 北京邮电大学 一种基于深度学习和自相似性的视频超分辨率重建方法
CN106447609A (zh) * 2016-08-30 2017-02-22 上海交通大学 一种基于深度卷积神经网络的图像超分辨方法
CN106991646A (zh) * 2017-03-28 2017-07-28 福建帝视信息科技有限公司 一种基于密集连接网络的图像超分辨率方法
CN106991646B (zh) * 2017-03-28 2020-05-26 福建帝视信息科技有限公司 一种基于密集连接网络的图像超分辨率方法
US10410322B2 (en) 2017-04-05 2019-09-10 Here Global B.V. Deep convolutional image up-sampling
US10621696B2 (en) 2017-04-05 2020-04-14 Here Global B.V. Deep convolutional image up-sampling
CN108694700A (zh) * 2017-04-10 2018-10-23 三星电子株式会社 用于深度学习图像超分辨率的系统和方法
CN108694700B (zh) * 2017-04-10 2023-12-01 三星电子株式会社 用于深度学习图像超分辨率的系统和方法
US11907835B2 (en) 2017-11-26 2024-02-20 Yeda Research And Development Co. Ltd. Signal enhancement and manipulation using a signal-specific deep network
EP3714404A4 (fr) * 2017-11-26 2021-08-18 Yeda Research and Development Co. Ltd Rehaussement et manipulation de signal à l'aide d'un réseau profond spécifique d'un signal
CN108537733A (zh) * 2018-04-11 2018-09-14 南京邮电大学 基于多路径深度卷积神经网络的超分辨率重建方法
CN108537733B (zh) * 2018-04-11 2022-03-11 南京邮电大学 基于多路径深度卷积神经网络的超分辨率重建方法
CN108615222A (zh) * 2018-04-17 2018-10-02 中国矿业大学 一种基于多对多连接的深度卷积网络图像超分辨率系统
CN109801214A (zh) * 2018-05-29 2019-05-24 京东方科技集团股份有限公司 图像重构装置及方法、设备、计算机可读存储介质
WO2019231235A1 (fr) * 2018-05-29 2019-12-05 엘지전자 주식회사 Dispositif de traitement de signal et dispositif d'affichage d'image comprenant celui-ci
CN110675324A (zh) * 2018-07-02 2020-01-10 上海寰声智能科技有限公司 4k超高清图像清晰化处理方法
CN110675324B (zh) * 2018-07-02 2023-10-10 上海寰声智能科技有限公司 4k超高清图像清晰化处理方法
CN109360148A (zh) * 2018-09-05 2019-02-19 北京悦图遥感科技发展有限公司 基于混合随机降采样的遥感影像超分辨率重构方法和装置
CN109360148B (zh) * 2018-09-05 2023-11-07 北京悦图遥感科技发展有限公司 基于混合随机降采样的遥感影像超分辨率重构方法和装置
CN109325915A (zh) * 2018-09-11 2019-02-12 合肥工业大学 一种用于低分辨率监控视频的超分辨率重建方法
CN109325915B (zh) * 2018-09-11 2022-11-08 合肥工业大学 一种用于低分辨率监控视频的超分辨率重建方法
CN110322528A (zh) * 2019-06-26 2019-10-11 浙江大学 基于3t、7t的核磁共振脑部图像血管重建方法
CN110310227B (zh) * 2019-06-27 2020-09-08 电子科技大学 一种基于高低频信息分解的图像超分辨率重建方法
CN110310227A (zh) * 2019-06-27 2019-10-08 电子科技大学 一种基于高低频信息分解的图像超分辨率重建方法
CN110390638B (zh) * 2019-07-22 2023-07-28 北京工商大学 一种高分辨率三维体素模型重建方法
CN110390638A (zh) * 2019-07-22 2019-10-29 北京工商大学 一种高分辨率三维体素模型重建方法
CN110738099A (zh) * 2019-08-30 2020-01-31 中山大学 一种基于自适应双分支网络的低分辨率行人再识别方法
CN110738099B (zh) * 2019-08-30 2022-06-07 中山大学 一种基于自适应双分支网络的低分辨率行人再识别方法
CN110738601A (zh) * 2019-10-23 2020-01-31 智慧视通(杭州)科技发展有限公司 一种基于三维人脸模型低分辨率人脸图像超分辨率重建方法
CN112825271A (zh) * 2019-11-21 2021-05-21 通用电气精准医疗有限责任公司 使用深度神经网络和测量数据的递归抽取来重建医疗图像的系统和方法
CN111461990A (zh) * 2020-04-03 2020-07-28 华中科技大学 一种基于深度学习分步实现超分辨成像的方法
CN111461990B (zh) * 2020-04-03 2022-03-18 华中科技大学 一种基于深度学习分步实现超分辨成像的方法
CN111563843A (zh) * 2020-04-30 2020-08-21 苏州大学 一种图像超分辨重构方法、系统及相关装置
CN111563843B (zh) * 2020-04-30 2023-11-28 苏州大学 一种图像超分辨重构方法、系统及相关装置
CN111696034B (zh) * 2020-05-11 2023-10-27 青岛小鸟看看科技有限公司 图像处理方法、装置及电子设备
CN111696034A (zh) * 2020-05-11 2020-09-22 青岛小鸟看看科技有限公司 图像处理方法、装置及电子设备
CN112184555B (zh) * 2020-10-21 2022-07-19 天津大学 一种基于深度交互学习的立体图像超分辨率重建方法
CN112184555A (zh) * 2020-10-21 2021-01-05 天津大学 一种基于深度交互学习的立体图像超分辨率重建方法
CN112598575B (zh) * 2020-12-22 2022-05-03 电子科技大学 一种基于特征处理的图像信息融合及超分辨率重建方法
CN112598575A (zh) * 2020-12-22 2021-04-02 电子科技大学 一种基于特征处理的图像信息融合及超分辨率重建方法
CN112907441B (zh) * 2020-12-29 2023-05-30 中央财经大学 一种基于地下水卫星图像超分辨率的空间降尺度方法
CN112907441A (zh) * 2020-12-29 2021-06-04 中央财经大学 一种基于地下水卫星图像超分辨率的空间降尺度方法
CN115601242B (zh) * 2022-12-13 2023-04-18 电子科技大学 一种适用于硬件部署的轻量级图像超分辨率重建方法
CN115601242A (zh) * 2022-12-13 2023-01-13 电子科技大学（Cn） 一种适用于硬件部署的轻量级图像超分辨率重建方法
WO2025041222A1 (fr) * 2023-08-21 2025-02-27 Nec Corporation Appareil de mise en correspondance d'images, procédé de mise en correspondance d'images, appareil d'entraînement, procédé d'entraînement et support non transitoire lisible par ordinateur

Also Published As

Publication number Publication date
CN106796716B (zh) 2018-11-16
CN106796716A (zh) 2017-05-31

Similar Documents

Publication Publication Date Title
WO2016019484A1 (fr) Appareil et procédé pour fournir une super-résolution d'une image à basse résolution
Zhang et al. Designing a practical degradation model for deep blind image super-resolution
Lee et al. Local texture estimator for implicit representation function
Anwar et al. Densely residual laplacian super-resolution
Lan et al. MADNet: A fast and lightweight network for single-image super resolution
Dong et al. Learning a deep convolutional network for image super-resolution
Lai et al. Deep laplacian pyramid networks for fast and accurate super-resolution
CN111369440B (zh) 模型训练、图像超分辨处理方法、装置、终端及存储介质
Kim et al. Accurate image super-resolution using very deep convolutional networks
Zhang et al. Residual dense network for image super-resolution
Li et al. FilterNet: Adaptive information filtering network for accurate and fast image super-resolution
Brifman et al. Turning a denoiser into a super-resolver using plug and play priors
Shi et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
CN109146788B (zh) 基于深度学习的超分辨率图像重建方法和装置
CN114565528B (zh) 一种基于多尺度和注意力机制的遥感影像降噪方法及系统
CN116188272B (zh) 适用于多模糊核的两阶段深度网络图像超分辨率重建方法
CN113538246A (zh) 基于无监督多阶段融合网络的遥感图像超分辨率重建方法
CN111932480A (zh) 去模糊视频恢复方法、装置、终端设备以及存储介质
Liu et al. Multi-scale residual hierarchical dense networks for single image super-resolution
Deshpande et al. SURVEY OF SUPER RESOLUTION TECHNIQUES.
CN118628366A (zh) 基于自学习耦合扩散后验采样的高光谱和多光谱图像融合方法及系统
CN118967476A (zh) 一种基于深度先验和双注意力残差网络的高光谱图像融合方法、系统、设备及介质
Qin et al. Single-image super-resolution with multilevel residual attention network
CN106981046A (zh) 基于多层梯度约束回归的单幅图像超分辨率重建方法
Wang et al. Large receptive field convolutional neural network for image super-resolution

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14899412

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14899412

Country of ref document: EP

Kind code of ref document: A1