WO2017145752A1 - Système et dispositif de traitement d'image - Google Patents

Système et dispositif de traitement d'image Download PDF

Info

Publication number
WO2017145752A1
WO2017145752A1 PCT/JP2017/004512 JP2017004512W WO2017145752A1 WO 2017145752 A1 WO2017145752 A1 WO 2017145752A1 JP 2017004512 W JP2017004512 W JP 2017004512W WO 2017145752 A1 WO2017145752 A1 WO 2017145752A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
position shift
pixels
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2017/004512
Other languages
English (en)
Japanese (ja)
Inventor
影山　昌広
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Industry and Control Solutions Co Ltd
Original Assignee
Hitachi Industry and Control Solutions Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Industry and Control Solutions Co Ltd filed Critical Hitachi Industry and Control Solutions Co Ltd
Publication of WO2017145752A1 publication Critical patent/WO2017145752A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Definitions

  • the present invention relates to an image processing system and an image processing apparatus that perform processing related to image data size and image quality.
  • Images (moving images and still images) taken by a camera are simultaneously transmitted via a communication network such as the Internet, an intranet, or a public network, and are received by an image receiving terminal. It is generally done to display.
  • the number of pixels of a camera used for imaging has increased with the advance of technology, and at present, for example, a full HD size image composed of horizontal 1920 pixels ⁇ vertical 1080 pixels is generally used.
  • the received image may deteriorate or freeze due to an application communication error or data delay.
  • the transmitted image data is recorded and stored on a medium such as a hard disk (magnetic disk) or an optical disk, if the data size is large, it may be possible to record only a short time on a medium with a limited recording capacity.
  • there is a problem that the cost of the recording apparatus is increased due to the necessity of a large-capacity recording medium.
  • the image data size is reduced in accordance with the transmission band and recording capacity. For example, the number of pixels in the captured image frame is reduced to reduce the image size.
  • the data size is reduced by reducing the overall size and then encoding and compressing, and when the image is reproduced, the reduced image is enlarged and displayed.
  • Patent Document 1 discloses two series of digital data. A technique is disclosed in which a wideband output signal exceeding the Nyquist frequency of an input signal is obtained by canceling aliasing distortion in a one-dimensional direction using the signal.
  • Non-Patent Document 1 a plurality of image frames are combined into one frame, and a high-frequency component exceeding the Nyquist frequency of the input image is restored by performing a successive approximation process with iterative calculation, and high-resolution output is performed.
  • a technique for obtaining an image is disclosed.
  • Patent Document 1 Japanese Patent Laid-Open No. 10-280685
  • Non-patent document 1 S.M. C. Park, M.M. K. Park, and M.M. G. King, "Super-Resolution Image Reconstruction: A Technical Overview," IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, 2003.
  • a personal computer hereinafter abbreviated as “PC”
  • a mobile terminal etc. as means for realizing an application for transmitting, recording, and displaying an image of a video phone, a remote conference, a network surveillance camera, or the like.
  • signal processing is performed by performing software processing using a general-purpose CPU (Central Processing Unit), MPU (Micro Processing Unit), GPGPU (General Purpose Computing Computing on Graphics Processing Unit), DSP (Digital Signal Processor), etc.
  • CPU Central Processing Unit
  • MPU Micro Processing Unit
  • GPGPU General Purpose Computing Computing on Graphics Processing Unit
  • DSP Digital Signal Processor
  • the super-resolution processing exemplified in the above-described prior art requires a large amount of calculation for processing compared to edge enhancement processing that only amplifies high-frequency components included in an image.
  • a super-resolution technique for obtaining an output image by successive approximation processing using an input image of a plurality of frames as described in Non-Patent Document 1 a high-precision motion search in units of subpixels for each pixel is performed.
  • the accompanying inter-frame alignment (registration), image enlargement, enlargement image reduction, difference detection between the reduced image and the input image, and output image correction processing must be repeated many times (iteration).
  • the motion search and iteration for each pixel has a very large amount of calculation, so that it exceeds the processing limit of calculation resources such as CPU and memory, and frame dropping occurs that makes the motion of the image jerky. Or the entire process may stop, or user input from a mouse, keyboard, or the like may not be accepted.
  • the present invention has been made in view of the above, and provides an image processing system and an image processing apparatus capable of reducing the encoded data size while suppressing an increase in the amount of calculation related to image processing and a decrease in image quality. For the purpose.
  • the present invention provides a first image position shift unit that shifts an image position to any one of a plurality of predetermined shift positions for each of a plurality of temporally continuous images.
  • An image reduction unit that reduces the number of pixels of the image that has been position shifted by the first image position shift unit, and an encoding unit that encodes the image reduced by the image reduction unit to generate an encoded image
  • a decoding unit that decodes the encoded image transmitted via the communication network to generate a decoded image, and an image that is enlarged by increasing the number of pixels of the decoded image decoded by the decoding unit
  • An enlargement unit, a second image position shift unit that shifts the position of the image enlarged by the image enlargement unit so as to cancel the position shift performed by the first image position shift unit, and the second image position Position shift at shift section
  • the encoded data size can be reduced while suppressing an increase in the amount of computation related to image processing and a decrease in image quality, and high-speed image transmission and image sharpening processing can be performed even with relatively inefficient calculation resources.
  • FIG. 1 is a functional block diagram schematically showing an overall configuration of an image processing system according to a first embodiment. It is a figure which shows an example of an image position shift process roughly. It is a figure which shows schematically another example of an image position shift process. It is a functional block diagram which shows roughly an example of a structure of the image expansion and sharpening part of an image receiver. It is a functional block diagram which shows the structure of an image reduction part roughly. It is a functional block diagram which shows roughly the structure of the horizontal processing part and vertical processing part of an image reduction part.
  • FIG. 5 is a diagram schematically illustrating an example of a configuration for realizing a function of image enlargement / clearing processing in an image enlargement / clearing unit illustrated in FIG. 4.
  • FIG. 11 It is a figure which shows typically another example of the structure which implement
  • FIG. 10 is a block diagram illustrating an example of a configuration of a two-dimensional asymmetric filter included in the image enlargement / clearing unit in FIG. 9. It is a block diagram which shows the other example of a structure of the image expansion and sharpening part in the image receiver of FIG. It is a block diagram which shows an example of a structure in the motion detection part which the image expansion and sharpening part of FIG. 15 has. It is a figure which shows the structural example in the case of synchronizing using the frame phase information generation part of an image transmitter, and the frame phase information acquisition part of an image receiver.
  • FIG. 1 It is a figure which shows the structural example in the case of synchronizing using the frame phase information generation part of an image receiver, and the frame phase information acquisition part of an image transmitter. It is a figure which shows the structural example in the case of synchronizing using the frame phase information estimation part of an image receiving part. It is explanatory drawing which shows an example of the operation
  • the cut-off frequency of the low-pass filter is set higher than the Nyquist frequency
  • the reduced distortion component is mixed with the baseband component originally included in the pre-reduction image. Since the aliasing distortion appears to be moire (interference fringe) noise, the image quality may be greatly degraded.
  • the image enlargement / clearing unit (111) of the image receiving device (109) performs aliasing distortion using a plurality of image frames when enlarging the reduced transmission image.
  • a baseband component having a wide band is extracted, and the image is sharpened.
  • the property that the phase of aliasing distortion that occurs when the sampling phase of the same input signal is changed and the sampling is changed is used. This property is used in interlaced scanning. For example, in a general 2: 1 interlace, at the time of imaging, an image is scanned by scanning every scanning line from the top, and a first field consisting of an image of only odd-numbered scanning lines and only an even-numbered image are scanned.
  • the second field consisting of and transmitted.
  • conversion from interlaced scanning to progressive scanning is performed by signal processing, or the first field and the second field are synthesized and perceived in the brain by the afterimage effect of the human eye. , Getting clear frame images.
  • the image position shift unit (103) shifts the position of the image for each frame, and then the image reduction unit (104) reduces the image, thereby reducing the reduced image (transmission image).
  • the phase of the aliasing distortion can be uniquely determined from the image shift method (shift direction and shift amount). For example, if the position of an image of full HD size (horizontal 1920 pixels ⁇ vertical 1080 pixels) is shifted by 1 pixel in the horizontal direction, an image of D1 size (horizontal 704 pixels ⁇ vertical 480 pixels) after image reduction is 0 in the horizontal direction.
  • FIG. 1 is a functional block diagram schematically showing the overall configuration of the image processing system according to the present embodiment.
  • an image processing system includes an image transmission device (101), a communication network (107), an image storage device (108), an image reception device (109), an operation unit (114), an image reception device (109), and
  • the display unit (113) is a schematic configuration.
  • the image transmission device (101) includes a camera (102) that captures an image (still image, moving image), an image position shift unit (103) that performs image position shift processing on an image captured by the camera (102), and an image position An image reduction unit (104) that performs image reduction processing on an image that has undergone shift processing, an encoding unit (105) that performs encoding processing on an image that has undergone image reduction processing, and an image via a communication network (107) Based on the command and data transmitted from the reception device (109), the image transmission device (101) including the camera (102), the image position shift unit (103), the image reduction unit (104), and the encoding unit (105). And a control unit (106) for controlling the entire operation.
  • the image receiving device (109) performs the decoding process with the decoding unit (110) that performs decoding processing on the image transmitted from the image transmitting device (101) via the communication network (107).
  • An image enlargement / clearing unit (111) that performs image enlargement / clarification processing on the image and a control unit (112) that controls the operation of the entire image receiving device (109) are provided.
  • the transmission data size is reduced by reducing the number of pixels constituting one frame of the image in the image reduction unit (104) of the image transmission device (101), and the image reception device.
  • the image enlarging / clearing unit (111) included in (109) By enlarging the image in the image enlarging / clearing unit (111) included in (109), a clear image with less blur can be displayed.
  • the image having the third pixel number output from the image receiving device (109) is further enlarged or reduced using an image enlargement unit or image reduction unit (not shown) to obtain an image having the fourth pixel number (not shown). You may comprise so that it may display on a display part (113) after converting.
  • the camera (102) includes, for example, an imaging unit (not shown) composed of a photoelectric conversion element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) and a lens, signal level adjustment, contrast adjustment, brightness adjustment, and white balance.
  • An image (still image, moving image) captured by the camera (102) is sent to a subsequent image position shift unit (103).
  • the number of pixels of an image is represented by a set of the number of pixels in the horizontal direction (the number of horizontal pixels) and the number of pixels in the vertical direction (the number of vertical pixels).
  • the frame is assumed to be composed of the first number of pixels.
  • the image position shift unit (103) performs image position shift processing that shifts (shifts) the position of the image obtained by the camera (102) vertically and horizontally in units of integer pixels.
  • image position shift processing in the image position shift unit (103) will be described with reference to FIGS.
  • FIG. 2 is a diagram schematically showing an example of the image position shift process.
  • FIG. 2 an image in which three persons are subjects is schematically shown, the broken line is the original image frame (200) taken by the camera (102), and the solid line is shifted by image position shift processing.
  • the obtained image frames (201) to (204) are shown.
  • the arrow has shown the state transition for every frame time (for example, 1/30 second).
  • an image position shift process for shifting an image at a cycle of 4 frames is performed, which is referred to as “4-frame type image position shift”.
  • the shift direction and the shift amount in the image position shift unit (103) are controlled by the control unit (106).
  • FIG. 3 is a diagram schematically showing another example of the image position shift process.
  • the broken line indicates the original image frame (200) taken by the camera (102), and the solid line indicates the image frames (205) and (205) shifted by the image position shifting process. ing. Moreover, the arrow has shown the state transition for every frame time (for example, 1/30 second).
  • an image position shift process for shifting an image at a cycle of 2 frames is performed, which is referred to as “2-frame type image position shift”.
  • the image position shifting process is not limited to the examples shown in FIGS.
  • the position of the image frame (200) is set to one state of position shift, and “image frame (200) ⁇ position shift to right side ⁇ position shift to lower right ⁇ position shift to lower side ⁇ image frame (200) ⁇ ⁇ .. ”may be used to shift the position.
  • the position is not shifted (that is, the position is fixed as the image frame (200)).
  • the position may be shifted as follows: (200) ⁇ image frame (206) ⁇ image frame (200) ⁇ .
  • image position shift processing including position shift in the vertical direction and the oblique direction (not shown) may be performed.
  • a position shift of “9 frame periods” combining the position shift of 3 frames in the vertical direction may be performed.
  • the image sharpening state changes according to the direction of the position shift by the image position shift process.
  • the resolution (clearness) in the horizontal and vertical directions of the image after the image enlargement and sharpening processing is improved, as shown in FIG.
  • “two-frame position shift” is performed, only the resolution (clearness) in the horizontal direction is improved.
  • the resolution in a desired direction can be improved by performing an image position shift process that shifts the image frame in the direction in which the resolution is desired to be improved.
  • FIG. 5 is a functional block diagram schematically showing the configuration of the image reduction unit.
  • FIG. 6 is a functional block diagram schematically illustrating the configuration of the horizontal processing unit and the vertical processing unit of the image reduction unit.
  • an image reduction unit (104) performs an image reduction process for converting the number of pixels constituting one frame of an image so as to be a second number of pixels smaller than the first number of pixels.
  • This configuration is also referred to as an interpolation filter, and a horizontal processing unit (401-H) that performs horizontal image reduction processing on an image input from the image position shift unit (103) and vertical image reduction processing.
  • a vertical processing unit (401-V) Note that the processing order for the input image may be switched between the horizontal processing unit (401-H) and the vertical processing unit (401-V).
  • the horizontal processing unit (401-H) and the vertical processing unit (401-V) can be realized with the same configuration, and a pixel insertion unit (402) that performs m-times upsampling, A low-pass filter (403) for reducing and a pixel thinning unit (404) for performing 1 / n downsampling are provided.
  • m-times upsampling (zero insertion) is performed on the input image by the pixel insertion unit (402), and then the low-pass filter (403 ) To remove or reduce unnecessary high-frequency components, and 1 / n downsampling is performed by the pixel thinning unit (404), so that the number of pixels in the horizontal direction (or vertical direction) of the image is m / n times (where m , N can be increased or decreased to a positive integer).
  • image enlargement processing by the image enlargement unit (301) which will be described later, can be realized with the same configuration by setting the constants m and n of the image reduction unit 104.
  • the horizontal reduction (1920 pixels ⁇ 704 pixels) in the image reduction unit (104) can be realized.
  • Direction expansion (704 pixels ⁇ 1408 pixels) can be realized. The same applies to reduction / enlargement in the vertical direction.
  • the number of output pixels (second pixel number) in the image reduction unit (104) is controlled by setting m and n by the control unit.
  • the encoding unit (105) encodes and compresses the image input from the image reduction unit (104), and transmits the image to the image reception device (109) and the image storage device (108) via the communication network 107. is there.
  • the processing configuration for connecting the image transmission device (101) or the image reception device (109) to the communication network (107) uses a general technique and is not shown.
  • the decoding unit 110 of the image receiving device (109) performs a decoding process according to the encoding method used by the encoding unit (105).
  • Examples of encoding methods used in the encoding unit 105 include MPEG (Moving Picture Expert Group) -1, MPEG-2, MPEG-4, H.264, and the like. H.264, H.C. H.265, VC-1, JPEG (Joint Photographic Experts Group), Motion JPEG, JPEG-2000, or other standardized encoding may be performed, or non-standard encoding may be performed.
  • the encoding method in the encoding unit (105) is controlled by the control unit (106).
  • the communication network (107) may be either wired or wireless, and is a network for communicating digital data using a communication protocol such as a general IP (Internet Protocol).
  • a communication protocol such as a general IP (Internet Protocol).
  • FIG. 4 is a functional block diagram schematically showing an example of the configuration of the image enlargement / clearing unit of the image receiving apparatus.
  • an image enlargement / clearing unit (111) includes an image enlargement unit (301) that performs image enlargement processing on an image having the second number of pixels decoded by the decoding unit (110), and an image enlargement unit.
  • An image position shift unit (302) that performs an image position shift process on the processed image and an aliasing distortion reduction unit (303) that performs an aliasing distortion reduction process on the image that has undergone the image position shift process.
  • the same processing may be performed on all color image signals (RGB (red, green, blue), YUV (luminance Y, color difference UV), etc.).
  • the following signal processing may be performed only on the luminance signal Y and not on the color difference signal (UV), but only on the image enlargement processing.
  • the image enlargement unit (301) converts an image having the second number of pixels (for example, D1 size (horizontal 704 pixels ⁇ vertical 480 pixels)) input from the decoding unit (110) into an image having the third number of pixels.
  • the image enlargement process to be converted into the image reduction processing is performed, and can be realized by the same configuration as the image reduction unit (104) as described above.
  • the purpose of the image enlargement process is to increase the number of pixels of the image (that is, increase the sampling frequency) in order to prevent the aliasing distortion reduced in the subsequent processing block from being aliased again in the frequency domain.
  • the number of pixels may be larger than the second number of pixels. Therefore, for the sake of simplicity of explanation, the third pixel number is assumed to be twice the second pixel number (that is, horizontal 1408 pixels ⁇ vertical 960 pixels).
  • the image position shift unit (302) converts the image that has been subjected to the image enlargement process by the image enlargement unit (301) in the direction opposite to the image position shift process by the image position shift unit (103) of the image transmission device (101). By shifting the position (that is, shifting the position of the image so as to cancel the position shift performed by the image position shift unit (103)), image blur between frames is suppressed. For example, when image position shift processing is performed such that the image position shift unit (103) shifts one pixel in the left direction and one pixel in the upward direction, the image position shift unit (302) is set to 0. 0 in the right direction.
  • the image shift may be performed so as to cancel the image position shift process in the image position shift unit (103). For example, “4” illustrated in FIG.
  • the image position shift unit in order to shift all images to the center positions of the “frame type image position shift” and the “two frame type image position shift” shown in FIG. 3 (that is, the position of the image frame (200)). What is necessary is just to determine the shift direction and shift amount in 302).
  • the shift direction and shift amount in the image position shift unit (302) may be determined so that all images are shifted to fixed positions that are not the respective centers in the image position shift process of the image position shift unit (103). Good.
  • the shift direction and shift amount of the image position shift unit (302) are controlled by the control unit (112).
  • the aliasing distortion reduction unit (303) performs aliasing distortion reduction processing for reducing aliasing distortion of the image subjected to the image position shift processing by the image position shift unit (302), and outputs an image from the image enlargement / clearing unit (111). It is what. Note that details of the aliasing reduction processing will be described later.
  • FIG. 7 is a diagram schematically showing an example of a configuration for realizing the function of the image enlargement / clearing process in the image enlargement / clearing unit shown in FIG.
  • the image enlargement / clearing unit (501) obtains a wideband output signal exceeding the Nyquist frequency of the input signal by canceling the aliasing distortion in the one-dimensional direction using two series of digital signals.
  • each one-dimensional interpolation filter (503- # 0), (503- # 1) uses each position shift amount ( ⁇ 0, ⁇ 1).
  • the position of the image frame is adjusted, the output signal of the adder (504), the image signal after being multiplied by the coefficient K by the multiplier (507) after passing through the subtracter (505) and the Hilbert transformer (506) Are converted so that the phase of the aliasing distortion contained in both of them is 180 degrees (opposite phase), and then added by an adder (508) to correct the aliasing distortion in the image. Can be canceled.
  • the image enlargement unit (301) shown in FIG. 4 corresponds to the one-dimensional enlargement units (502- # 1) and (502- # 2) in FIG. 7, and the image position shift unit (302) This corresponds to the dimension interpolation filters (503- # 0) and (503- # 1).
  • the image enlargement unit (301) shown in FIG. 4 is equivalent to the interpolation filter (401) as described above, and the sinc function used to determine the coefficient of the low-pass filter (403) is sin (x ⁇ t).
  • the one-dimensional enlargement unit (502- #) 1) and (502- # 2) can be omitted.
  • 4 is realized by the adder (504), subtracter (505), Hilbert transformer (506), multiplier (507), and adder (508) in FIG. Therefore, the image enlargement / clearing unit (111) in FIG. 4 and the image enlargement / clearing unit (501) in FIG. 7 can be regarded as equivalent.
  • the configuration of the image enlargement / clearing unit (501) shown in FIG. 7 illustrates the case where the one-dimensional image enlargement process and the aliasing reduction process are performed.
  • the case of performing the direction image enlargement process and the aliasing reduction process will be described below with reference to FIG.
  • FIG. 8 is a diagram schematically showing another example of a configuration for realizing the function of the image enlargement / clearing process in the image enlargement / clearing unit shown in FIG.
  • processing blocks (501-HA), (501-HB), (501-V) having the same configuration as the image enlargement / clearing unit (501) shown in FIG.
  • the outputs of (501-HA) and (501-HB) are connected in series as the input of the processing block (501-V).
  • the horizontal processing unit (601) including processing blocks (501-HA) and (501-HB) performs horizontal image enlargement processing and aliasing distortion reduction processing, and vertical processing including processing blocks (501-V).
  • vertical image enlargement processing and aliasing reduction processing are performed in the part (602).
  • FIGS. 2 and 3 the position shift amount ( ⁇ 0, ⁇ 1) shown in FIG. 7, the horizontal position shift amount ( ⁇ 0, ⁇ 1, ⁇ 2, ⁇ 3) shown in FIG.
  • the relationship with the vertical position shift amount ( ⁇ 0, ⁇ 1) will be described.
  • an image having the first number of pixels (for example, horizontal 1920 pixels ⁇ vertical 1080 pixels) shown in FIG. 1 is the center position (image frame) of the rotational motion in the “4-frame position shift” shown in FIG.
  • an image having a first number of pixels (for example, horizontal 1920 pixels ⁇ vertical 1080 pixels) is reduced to a second number of pixels (for example, horizontal 704 pixels ⁇ vertical 480 pixels), and the communication network (107 ),
  • the aliasing distortion included in the image can be reduced, and the image can be sharpened without performing iteration.
  • FIG. 9 and 10 are diagrams schematically showing another example of a configuration for realizing the function of the image enlargement / clearing processing in the image enlargement / clearing unit shown in FIG.
  • the signal passing through the “one-dimensional asymmetric filter” in which the polarity (positive / negative) of the subtractor (505) is inverted may be added together at the end. The same output is obtained.
  • FIG. 9 the order of the “one-dimensional asymmetric filter” in the horizontal processing unit (601) and the “one-dimensional enlargement unit and one-dimensional interpolation filter” in the vertical processing unit (602) in FIG.
  • the vertical one-dimensional enlargement units are combined into a two-dimensional expansion unit (702)
  • the horizontal / vertical one-dimensional interpolation filters are combined into a two-dimensional interpolation filter (703)
  • the horizontal / vertical one-dimensional asymmetric filters are combined.
  • a two-dimensional processing unit (701) is configured as a two-dimensional asymmetric filter (704).
  • this two-dimensional processing unit (701) two-dimensional interpolation is performed in accordance with the horizontal position shift amounts ( ⁇ 0, ⁇ 1, ⁇ 2, ⁇ 3) and vertical position shift amounts ( ⁇ 0, ⁇ 1) corresponding to the inputs # 0 to # 3.
  • the coefficients of the filter (703) and the two-dimensional asymmetric filter (704) are set, and the two-dimensional processing units (701- # 0 to # 3) are set.
  • each signal passing through each two-dimensional processing unit (701- # 0 to # 3) is passed through each frame memory (705- # 0 to # 3), and then added to all signals by an adder (706). If added, an output equivalent to the configuration of FIG. 8 can be obtained.
  • each two-dimensional processing unit (701- # 0 to # 3) since there is no need for each two-dimensional processing unit (701- # 0 to # 3) for an image frame that is not input, only one two-dimensional processing unit (701) is installed as shown in FIG. In accordance with the amount ( ⁇ 0, ⁇ 1, ⁇ 2, ⁇ 3) and the vertical position shift amount ( ⁇ 0, ⁇ 1), the internal coefficient is set while changing according to the frame number, and the switch (707) is used.
  • the frame memory (705- # 0 to # 3) may be written while being selected for each frame.
  • the configuration of the image enlargement / clearing unit (111) in the case of “2-frame position shift” can be equivalently converted from the configuration of FIG. 7 to the configuration of FIG. That is, the one-dimensional enlargement units (502- # 0) and (502- # 1) in FIG. 7 are combined into the one-dimensional enlargement unit (801) in FIG. 11, and the one-dimensional interpolation filters (503- # 0) in FIG. (503- # 1) are combined into a one-dimensional interpolation filter (802) in FIG. 11, and “adder (504), subtracter (505), Hilbert transformer (506), multiplier (507)” in FIG. The adder (508) "is integrated into the one-dimensional asymmetric filter (803) in FIG.
  • the calculation amount for one one-dimensional interpolation filter can be reduced compared to the calculation amount in the configuration of FIG.
  • FIGS. 12 to 14 are a block diagram and an operation explanatory diagram showing an example of a configuration of an asymmetric filter included in the image enlargement / clearing unit of FIGS. 5 and 11.
  • FIG. 12 is a block diagram showing an example of the configuration of a one-dimensional asymmetric filter included in the image enlargement / clearing unit in FIG.
  • the input # 0 passes through the one-dimensional enlargement unit (502- # 0) and the one-dimensional interpolation filter (503- # 0), and then “signal that has passed through the adder (504)”. Then, "the signal that has passed through the subtracter (505), the Hilbert transformer (506), and the multiplier (507)” is added by the adder (508) to be an output.
  • the filter coefficient of the one-dimensional asymmetric filter (704) for the input # 0 is obtained by multiplying the signal obtained by multiplying the coefficient K through the Hilbert transformer (506) and the input signal by the adder (508). The added signal is output.
  • the filter coefficient of the one-dimensional asymmetric filter (704) for the input # 1 is a value obtained by inverting the polarity (positive / negative) of the coefficient K compared to the coefficient of the filter coefficient of the one-dimensional asymmetric filter (704) for the input # 0. It becomes.
  • the Hilbert transformer (506) is an odd symmetric filter.
  • t 2m (where m is an integer)
  • the filter coefficient (C (t)) shown in FIG. 13 is an example.
  • the coefficient C (t) (where t ⁇ 0) is obtained by multiplying the coefficient of the Hilbert transform by a general window function (such as Hanning window). ), The influence of the filter end may be reduced.
  • FIG. 14 is a block diagram illustrating an example of a configuration of a two-dimensional asymmetric filter included in the image enlargement / clearing unit in FIG.
  • the two-dimensional asymmetric filter (704) is obtained by connecting the above-described one-dimensional asymmetric filter (803) in series.
  • the horizontal processing unit (803-H) converts the number of pixels in the horizontal direction, and the vertical processing unit ( By converting the number of pixels in the vertical direction at 803-V), two-dimensional pixel number conversion can be realized.
  • the above-described configuration of the image enlargement / clearing unit (111) reduces the amount of calculation by eliminating the motion search and iteration for each pixel and realizes image sharpening capable of high-speed processing.
  • the position shift amounts ( ⁇ n, ⁇ n) used in the configurations of FIGS. 7 to 11 are fixed values for each frame. At this time, there is no problem if the subject in the image frame does not move, but there is a problem that multiple images are generated by calculation between the frames if the subject has movement. Therefore, a configuration example for solving such a problem will be described below.
  • FIG. 15 is a block diagram showing another example of the configuration of the image enlargement / clearing unit in the image receiving apparatus of FIG.
  • a processing unit (1001) including an image enlargement unit (301), an image position shift unit (302), and a aliasing distortion reduction unit (303) has the same configuration as that shown in FIG.
  • a signal in which image blur between frames is suppressed by the image position shift unit (302) is passed through a low-pass filter (1002) for suppressing unnecessary aliasing distortion, and an image pixel is detected by a motion detection unit (1003) described later.
  • a control signal (m, where 0 (no motion) ⁇ m ⁇ 1 (with motion)) corresponding to the magnitude of each motion is generated, and the low-pass filter (1002) is generated using the weighted mixing unit (1004).
  • the passed signal (p1) and the signal (p0) passed through the aliasing distortion reduction unit (303) are weighted and mixed in accordance with the control signal m to obtain an output signal.
  • the signal (p0) passed through the aliasing reduction unit (303) is output, and the region where the value of the control signal m is close to 1 (the motion is In the large region, control is performed so that the signal (p1) that has passed through the low-pass filter (1002) is output.
  • FIG. 16 is a block diagram illustrating an example of the configuration of the motion detection unit included in the image enlargement / clearing unit of FIG.
  • control signal m is obtained.
  • This normalization can be realized by a general technique in which a predetermined fixed value is subtracted or multiplied by the output signal of the maximum value unit (1106), and thus detailed illustration and description are omitted.
  • the motion detection unit (1003) configured in this way, an area in which all the values of the four-frame images (# 0, # 1, # 2, # 3) input to the motion detection unit (1003) match (that is, In the area where the image is still), the value of the control signal m is 0, and the area where the values of the images of 4 frames (# 0, # 1, # 2, # 3) do not match (that is, 1 out of 4 frames).
  • the control signal m has a value of 0 ⁇ m ⁇ 1 according to the magnitude of the absolute value of the difference signal described above.
  • the switcher (1101), frame memory (1102), average unit (1103), subtractor (1104) and the absolute value unit (1105) can be easily realized simply by changing from 4 frames to 2 frames.
  • 17 to 19 are block diagrams respectively showing an example of the configuration of the control unit included in the image transmission device and the image reception device in FIG.
  • the image position shift unit (103) included in the image transmission device (101) and the image position shift unit (302) included in the image reception device (109) operate in synchronization with each other accurately in units of frames and are in opposite directions. It is necessary to control to shift the position.
  • the frame phase information is, for example, one of the frame numbers “# 0, # 1, # 2, # 3” in the case of “4 frame type position shift”, and “2 frame type position shift”. In this case, the frame number is “# 0, # 1”.
  • FIG. 17 is a diagram illustrating a configuration example when synchronization is performed using the frame phase information generation unit of the image transmission device and the frame phase information acquisition unit of the image reception unit.
  • the frame phase information generation unit (1201) included in the control unit (106) of the image transmission apparatus (101) generates frame phase information
  • the multiplexing unit (1202) is used to encode the encoding unit (105). Is multiplexed with the image data output from.
  • the multiplexed data is transmitted to the image receiving device (109) via the communication network (107) and separated by the separating unit (1203), and then the image data is sent to the decoding unit (110) to receive frame phase information.
  • the frame phase information acquisition unit (1204) of the control unit (112) can obtain the same information as the frame phase information generated by the frame phase information generation unit (1201) of the image transmission device (101).
  • the frame phase information generation unit (1201), the multiplexing unit (1202), the separation unit (1203), and the frame phase information acquisition unit (1204) can be realized by a general technique, and thus detailed illustration is omitted.
  • FIG. 18 is a diagram illustrating a configuration example when synchronization is performed using the frame phase information generation unit of the image reception unit and the frame phase information acquisition unit of the image transmission apparatus.
  • the frame phase information generating unit (1206) included in the control unit (112) of the image receiving apparatus (109) generates frame phase information, which is transmitted to the image transmitting apparatus (101) via the communication network (107).
  • the frame phase information acquisition unit (1205) included in the control unit (106) of the image transmission device (101) is the same as the frame phase information generated by the frame phase information generation unit (1206) of the image reception device (109). Information can be obtained.
  • the frame phase information generation unit (1206) and the frame phase information acquisition unit (1205) can be realized by a general technique, and thus detailed illustration is omitted.
  • FIG. 19 is a diagram illustrating a configuration example when synchronization is performed using the frame phase information estimation unit of the image reception unit.
  • the frame phase information generation unit (1201) included in the control unit (106) of the image transmission apparatus (101) generates frame phase information, but this information is not transmitted to the image reception apparatus (109).
  • the encoded image data transmitted from the image transmitting device (101) is decoded by the decoding unit (110), and the frame included in the control unit (112) based on the decoded image.
  • Frame phase information is obtained by performing a frame phase information estimation process in the phase information estimation unit (1207).
  • FIG. 20 is an explanatory diagram showing an example of the operation of the frame phase information estimation process of the frame phase information estimation unit shown in FIG.
  • an image (1301) is a D1-size image obtained by reducing the image frame (201) shifted in the upper left direction in FIG. 2 by the image reduction unit (104) included in the image transmission apparatus (101) shown in FIG.
  • This is an image (horizontal 704 pixels ⁇ vertical 480 pixels), and is taken as an example for explanation.
  • FIG. 21 is a flowchart showing frame phase estimation processing in the frame phase information estimation unit.
  • step S1404 data (y ) Is incremented by one to set y ⁇ y + 1 (step S1404), and the processing of steps S1404 and S1402 is repeated until the determination result of step S1403 becomes YES. If the determination result in step S1403 is YES, the image shift direction is estimated according to the value of each counter value (L, R) (step S1405). That is, in step S1405, if “the value of the counter for the left end (L) ⁇ the value of the counter for the right end (R)”, it is estimated that the image is shifted leftward. Estimated to be shifted in the direction. When step S1405 ends, the process ends.
  • the image position shift processing in the image position shift unit (103) will be further considered. That is, depending on the type of image captured by the camera (102) of the image transmission device (101), it is necessary to consider the contents of the image position shift process. For example, in the case where the camera (102) includes image data of a single-plate image sensor on which an image Bayer array color filter is arranged, it is necessary to restrict the operation of the image position shift process of the image position shift unit (103).
  • FIG. 22 is a diagram showing a state of pixel arrangement in the Bayer array color filter.
  • R red, G: green, and B: blue color filters are regularly arranged with a two-pixel interval as one period in both the horizontal and vertical directions. ing. That is, when an image signal (hereinafter referred to as a Bayer image signal) obtained by photoelectrically converting light transmitted through the Bayer array color filter (1501) by a single-plate image sensor is input to the image position shift unit (103), If the position of either or both of the direction and the vertical direction is shifted in units of odd pixels, the positional relationship of R: red, G: green, B: blue changes, and the color of the image after the position shift is Will change.
  • a Bayer image signal obtained by photoelectrically converting light transmitted through the Bayer array color filter (1501) by a single-plate image sensor
  • the position shift unit (103) when the Bayer image signal is input to the image position shift unit (103), the position is shifted in units of two pixels (even number of pixels) in both the horizontal direction and the vertical direction, so that R: red, G: green, B : Image position shift processing is performed so that the positional relationship of blue is not different from that before the position shift.
  • R red
  • G green
  • B Image position shift processing is performed so that the positional relationship of blue is not different from that before the position shift.
  • the position shift in units of 2 pixels (even pixel units) as described above only the color signal (RGB, YUV, etc.) converted from the Bayer image signal or the luminance signal Y converted from the Bayer image signal.
  • FIG. 23 is a functional block diagram schematically showing a configuration example of the image storage device.
  • an image storage device (108) is a device for storing (recording) or reproducing encoded image data, and includes a communication interface (1601), a control unit (1602), a memory (1603). ), Storage (1604), output interface (1605), input interface (1606), and other processing blocks, and a bus (1607) connecting each processing block.
  • the image storage device (108) stores an application program in the storage (1604), the control unit (1602) expands the program from the storage (1604) to the memory (1603), and the control unit (1602) By executing the program, various functions such as recording, reproduction, and search can be realized.
  • various functions realized by the execution of each application program by the control unit (1602) are mainly realized by "various program function units" expanded in the memory (1603).
  • the application program may be stored in the storage (1604) in advance when the image storage device (108) is shipped, or a medium such as a CD (Compact Disc) / DVD (Digital Versatile Disc) or a semiconductor memory. And installed in the image storage device (108) via a medium connection unit (not shown). It is also possible to download and install from the communication network (107) via the communication interface (1601).
  • a medium such as a CD (Compact Disc) / DVD (Digital Versatile Disc) or a semiconductor memory.
  • a medium connection unit not shown
  • the communication interface (1601) is connected to the communication network (107), receives image data from the image transmission apparatus (101) shown in FIG. 109) also has a function of transmitting image data.
  • the control unit (1602) controls the communication interface (1601), the memory (1603) (various program function units), the storage (1604), and the input / output interfaces (1605, 1606).
  • the control unit (1602) also has a function of executing various signal processing according to a processing procedure described later.
  • the function part of the application program stored in the storage (1604) is expanded under the control of the control part (1602).
  • the storage (1604) accumulates image data from the image transmission apparatus (101), and stores application programs and various types of information created by the application programs.
  • the output interface (1605) has a function of outputting an image obtained as a result of signal processing by the control unit to an external device via the bus (1607).
  • the output image is displayed on an external display unit (1608).
  • the input interface (1606) has a function of receiving a signal from the operation unit (1609) and transmitting the signal to the control unit (1602) via the bus (1607).
  • the image storage device (108) configured in this manner follows the following operation sequence during recording. That is, at the time of recording, the image storage device (108) reads a recording application program (not shown) stored in the storage (1604) into the memory (1603), and follows the procedure described in the recording application program according to the procedure described in the recording application program. 1602) controls each part. First, a connection is established with the image transmission apparatus (101) shown in FIG. 1 via the communication interface (1601) and the communication network (107). Thereafter, the encoded image data transmitted from the image transmission device (101) is received via the communication interface (1601) and the communication network (107), and is encoded into the storage (1604) via the bus (1607). Accumulated image data is stored. At this time, the received encoded image data may be decoded by the control unit (1602), output an image via the output interface (1605), and displayed on the display unit (1608). Next, an operation sequence at the time of reproduction in the image storage device (108) will be described.
  • the image storage device (108) follows the following operation sequence during reproduction. That is, at the time of playback, the image storage device (108) reads a playback application program (not shown) stored in the storage (1604) into the memory (1603), and follows the procedure described in the playback application program according to the procedure described in the playback application program. 1602) controls each part. Thereafter, the encoded image data stored in the storage (1604) is read out via the bus (1607) and transmitted to the image receiving device (109) via the communication interface (1601) and the communication network (107). .
  • the encoded image data to be transmitted may be decoded by the control unit (1602), and the image may be output and displayed on the display unit (1608) via the output interface (1605).
  • the encoded image data stored in the storage (1604) is read out only by the image storage device (108) alone, decoded by the control unit (1602), and displayed via the output interface (1605) (1608). ) May be output and displayed.
  • the image position shift unit (103) of the image transmission apparatus (101) is displayed on the display unit (1608) as it is, the image is periodically blurred. Become.
  • the various program function units developed in the memory (1603) include the frame phase information estimation function (1610) having the same function as the frame phase information estimation unit (1207) and the same as the image position shift unit (302).
  • An image position shift function (1611) having a function is provided, and image blurring is canceled by performing a process of canceling out the effect of the image position shift process. That is, the image position shift function (1611) uses the third pixel number in the operation of the image position shift unit (302) in FIG. 4 as the pixel number of the encoded image data accumulated in the storage (1604) (second In this case, the operation of the image position shift unit (302) can be used as it is.
  • the image position shift function (1611) can be realized by using a convolution process with a linear filter having asymmetric coefficients.
  • FIG. 24 is a diagram for explaining an example of filter coefficients used in the image position shift function of FIG.
  • coefficients (a) to (d) indicate filter coefficients that are convoluted with the decoded image.
  • a symbol or a numerical value in parentheses that is, 0, ⁇ , 1- ⁇ , ⁇ , 1- ⁇ , etc.
  • the symbol “T” represents transposition.
  • the symbol “*” represents a convolution operation. That is, the coefficients (a) to (d) in FIG. 24 represent the coefficients of a two-dimensional filter of horizontal 3 taps ⁇ vertical 3 taps.
  • the center of gravity of the two-dimensional filter represented by the coefficient (a) is shifted in the lower right direction, and an image obtained by convolving this filter coefficient is in the lower right direction.
  • the image convolved with coefficient (b) is shifted in the lower left direction
  • the image convolved with coefficient (c) is shifted in the upper right direction
  • the image convolved with coefficient (d) is in the upper left. The position shifts in the direction.
  • an image having the first number of pixels in FIG. 1 is based on the two-dimensional coordinates of the center position of the rotational motion in “4-frame position shift” in FIG. 0,0), two-dimensional coordinates ( ⁇ h, ⁇ v) ⁇ (h, ⁇ v) ⁇ (h, v) ⁇ ( ⁇ h, v) ⁇ ( ⁇ h, ⁇ v) ⁇ .
  • blurring of the image can be suppressed by convolving it with the stored image. With the above configuration, image blurring can be suppressed when the image stored in the image storage device (108) is played back on the display unit (1608).
  • the “two-dimensional filter of horizontal 3 taps ⁇ vertical 3 taps” described above is an example for explaining the operation, and the present invention is not limited to this, and a two-dimensional filter having a different number of taps. Obviously, a filter may be used. In addition, it is obvious that the image in which blurring is suppressed in this way may be transmitted to the outside via the communication interface (1601) and the communication network (107).
  • the super-resolution processing exemplified in the prior art requires a large amount of calculation for processing compared to edge enhancement processing that only amplifies high-frequency components included in an image.
  • a super-resolution technique for obtaining an output image by successive approximation processing using an input image of a plurality of frames as described in Non-Patent Document 1 a high-precision motion search in units of subpixels for each pixel is performed.
  • the accompanying inter-frame alignment (registration), image enlargement, enlargement image reduction, difference detection between the reduced image and the input image, and output image correction processing must be repeated many times (iteration).
  • the motion search and iteration for each pixel has a very large amount of calculation, so that it exceeds the processing limit of calculation resources such as CPU and memory, and frame dropping occurs that makes the motion of the image jerky. Or the entire process may stop, or user input from a mouse, keyboard, or the like may not be accepted.
  • an image position shift unit (103) (103) that shifts the position of the image to any of a plurality of predetermined shift positions.
  • a first image position shift unit an image reduction unit (104) for reducing the number of pixels of the image whose position has been shifted by the image position shift unit (103), and an image reduced by the image reduction unit (104)
  • a decoding unit (105) that generates a decoded image by decoding the encoded image sent via the communication network (107)
  • An image enlargement unit (301) that increases the number of pixels of the decoded image decoded by the conversion unit (110) and enlarges the image
  • an image position shift unit 103)
  • An image position shift unit (302) (second image position shift unit) that shifts the position so as to cancel and cancel the shift, and an aliasing distortion of the image shifted by the image position shift unit (302) Since it is configured to include the aliasing distortion reduction unit (303) that performs aliasing distortion reduction processing that
  • FIG. 25 A second embodiment of the present invention will be described with reference to FIGS. 25 and 26.
  • FIG. 25 A second embodiment of the present invention will be described with reference to FIGS. 25 and 26.
  • the image transmission apparatus is configured to have a function of switching presence / absence of image position shift processing, and performs image position shift processing that cancels and cancels image position shift processing. Even when displayed via an image receiving device or an image storage device that does not have means for performing (that is, means for shifting the position in the reverse direction), an image that is not periodically blurred can be provided.
  • FIG. 25 is a functional block diagram schematically showing a configuration example of the image transmission apparatus according to the present embodiment.
  • the same members as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.
  • an image transmission device (1802) includes a camera (102) that captures an image (still image, moving image), and an image position shift unit (103) that performs an image position shift process on the image captured by the camera (102).
  • the image signal from the camera (102) is input to the image reduction unit (104), or the image signal from the camera (102) is passed through the image position shift unit (103) and then to the image reduction unit (104).
  • a switcher (1803) for switching between input and image, an image reduction unit (104) for performing an image reduction process on an image input via the switcher (1803), and an encoding process for the image subjected to the image reduction process
  • a control unit that controls the overall operation of the image transmission apparatus (1802) including the mela (102), the image position shift unit (103), the switch (1803), the image reduction unit (104), and the encoding unit (105).
  • FIG. 26 is a diagram illustrating an example of a menu display screen generated by the control unit included in the image receiving device or the image storage device and displayed on the display unit.
  • the menu display screen (1901) generated by the control unit (112, 1602) of the image receiving device (109) or the image storage device (108) and displayed on the display unit (113, 1608) is an image transmission.
  • a message portion (1902) indicating that the display is a menu display for selecting an operation related to the image position shift processing of the apparatus (1802), a message portion (1903) indicating that the image position shift processing is not performed, and an image position shift A selection unit (1905) for selecting not to perform processing, a message unit (1904) indicating that image position shift processing is performed, and a selection unit (1906) for selecting execution of image position shift processing And have.
  • FIG. 26 illustrates the case where the user of the image receiving device (109) or the image storage device (108) has selected to perform image position shifting.
  • the message portion (1903) indicating that the image position shift process is not performed may include a message indicating that resolution improvement cannot be expected.
  • the message portion (1904) indicating that the image position shift process is to be performed may include a message indicating that resolution improvement can be
  • the image transmission device (1802) When the image transmission device (1802) is connected to the image reception device (109) or the image storage device (108) having the image position shift unit (302) via the communication network (107), the user can display a menu. By selecting the selection unit (1906) of the screen (1901), the switch (1803) is switched to the lower side in the figure, and the image that has passed through the image position shift unit (103) is displayed by the image reduction unit (104). The image is reduced, encoded by the encoding unit (105), and transmitted to the image reception device (109) and the image storage device (108) via the communication network (107). As a result, the image receiving device (109) can output a high-quality image with reduced aliasing distortion.
  • the image transmission apparatus (1802) when the image transmission apparatus (1802) is connected to an image reception apparatus or image storage apparatus that does not have the image position shift unit (302) via the communication network (107), the user can display a menu display screen ( 1901) by selecting the selection unit (1905), the switch (1803) is switched to the upper side in the figure, and the image reduction unit (104) reduces the image without passing through the image position shift unit (103).
  • the encoding unit (105) encodes the data and transmits the data to the image reception device (109) and the image storage device (108) via the communication network (107).
  • the image receiving apparatus can output an image without blurring.
  • the switch (1803) is controlled in accordance with a control signal sent from the control unit (1804) according to the setting of the menu display screen (1901). At this time, the image is transmitted via the communication network (107). You may enable it to control according to the command automatically transmitted from the receiver (109) or the image storage device (108). That is, the image receiving device (109) or the image storage device (108) having the image position shift unit (302) is set in advance so as to send a command indicating that image position shift processing is performed, and this command is transmitted to the control unit ( When the signal is not received in 1804), it may be determined that an image receiving apparatus that does not have the image position shift unit (302) is connected, and the switch (1803) may be switched to the upper side in the drawing.
  • means for performing image position shift processing that cancels and cancels the image position shift process that is, means for position shifting in the reverse direction
  • the displayed image is periodically blurred.
  • any other image receiving device that does not include an image position shift unit, and the like can display a suitable image for each.
  • FIG. 27 is a functional block diagram schematically showing a configuration example of the image transmission apparatus according to the present embodiment.
  • FIG. 28 is a functional block diagram schematically showing a configuration example of the image receiving apparatus according to the present embodiment.
  • the same members as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.
  • an image transmission device includes a camera (102) that captures an image (still image, moving image), and an image position shift unit (103) that performs an image position shift process on the image captured by the camera (102).
  • the image reduction unit (104) that performs the image reduction process on the image that has been subjected to the image position shift process, and cancels out the position shift performed by the image position shift unit (103) on the image that has undergone the image reduction process.
  • An image position shift unit (2002) that shifts the position as described above, an encoding unit (105) that performs an encoding process on an image that has undergone image position shift processing by the image position shift unit (2002), and a communication network (107) Based on the command and data received via the camera, the camera (102), the image position shift unit (103), the image reduction unit (104), and the image position shift unit (2002) Control unit for controlling an image transmission apparatus (2001) the whole operations including the fine coding unit (105) and a (2003).
  • the image position shift unit (2002) is for shifting the image in the direction opposite to the image blur caused by the image position shift unit (103) under the control of the control unit (2003).
  • the asymmetric filter described above can also be used as the image position shift unit (2002).
  • an image receiving device (2004) performs a decoding process on the image sent from the image transmitting device (2001) via the communication network (107), and a decoding process.
  • An image position shift unit (2005) that performs the same image position shift process as the image position shift unit 103 on the performed image, and an image enlargement / clearing process on the image that has been subjected to the image position shift process by the image position shift unit (2005)
  • the image position shift unit (2005) operates under the control of the control unit (2006), and cancels the position shift performed by the image position shift unit (2002) of the image transmission device (2001) so as to cancel the position shift. To do.
  • FIG. 29 is a diagram illustrating an example of filter coefficients used in the image position shift unit in FIG.
  • coefficients (a) to (d) indicate filter coefficients that are convoluted with the decoded image.
  • a symbol or a numerical value in parentheses that is, 0, ⁇ , 1- ⁇ , ⁇ , 1- ⁇ , etc.
  • the right shoulder of the parenthesis “ ⁇ 1” of the symbol represents an inverse characteristic
  • the symbol “T” on the right shoulder of the parenthesis represents transposition.
  • the symbol “*” represents a convolution operation. That is, the coefficients (a) to (d) in FIG. 29 represent the coefficients of the inverse filter of the two-dimensional filter shown in FIG.
  • each coefficient of an inverse filter can be calculated
  • the image reception device (2004) can output a clear image by the same operation as the first embodiment.
  • the image transmission apparatus (2001) is connected to an image reception apparatus or an image storage apparatus that does not include an image position shift unit, it is possible to suppress image blurring that occurs during display.
  • the image storage device (108) in the first embodiment is used as the image receiving device (1612).
  • the image transmission device (101) is used as the image transmission device, and is described in an image reception application program (not shown) stored in the storage (1604) of the image storage device (108).
  • the control unit (1602) controls each unit in accordance with the processing (described later in detail with reference to FIG. 30), whereby the image storage device (108) is also used as the image reception device (1612) (see FIG. 23).
  • FIG. 30 is a flowchart showing an example of processing in the image receiving apparatus according to the present embodiment.
  • the image receiving device (1612) first acquires the encoded image data via the communication network (107) (step S2201), and the control unit (1602) decodes the image data. It stores in the memory (1603) (step S2202). Subsequently, frame phase information is acquired (step S2205). In parallel with step S2205, the image is enlarged (step S2203), and the position of the image is shifted (step S2204). These processes correspond to the processing contents of the image enlargement unit (301) and the image position shift unit (302) shown in FIG.
  • step S2206 aliasing distortion reduction processing is performed (step S2206).
  • step S2206 two-dimensional processing is performed (step S2207), the image is stored in a predetermined frame memory (memory (1603) in FIG. 23) (step S2208), and the values of all frame memories are added for each same pixel position. (Step S2209). Note that these processes correspond to the processing contents in the configuration shown in FIGS. 9 and 10.
  • step S2210 low-pass filter processing for suppressing unnecessary aliasing distortion is performed (step S2210).
  • step S2211 a motion detection process is performed to obtain a control signal m (step S2211). This processing corresponds to the processing content in the configuration shown in FIG.
  • Step S2212 weighted mixing is performed using the image after the aliasing reduction obtained in step S2206, the image after the low-pass filter obtained in step S2210, and the control signal m obtained in step S2211, thereby obtaining an output image ( Step S2212).
  • This processing corresponds to the operation of the weighted mixing unit (1004) shown in FIG.
  • the output image signal obtained in step S2212) is output via the output interface (1605) (step S2213), and the process ends.
  • steps S2201 to S2213 are controlled by the control unit (1602) so that it is executed each time image data is transmitted from the image transmission apparatus (101) to the image reception apparatus (1612). Good.
  • a command instructing transmission of image data may be transmitted from the image receiving apparatus (109) to the image transmitting apparatus (101).
  • the various program function units developed in the memory (1603) of FIG. 23 with another image position shift function (not shown)
  • the operation of the image position shift unit (2005) described in FIG. It is obvious that the operation shown in FIG. 29 can be realized, and it is obvious that the same operation as that shown in FIG. 28 can be realized using the configuration of the image receiving device (1612).
  • this invention is not limited to each above-mentioned embodiment, Various modifications are included.
  • the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to one having all the configurations described.
  • Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor.
  • the functions of the embodiments of the present invention can be realized by software program codes.
  • a storage medium in which the program code is recorded is provided to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads the program code stored in the storage medium.
  • the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.
  • a storage medium for supplying such program code for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. are used.
  • an OS operating system
  • the computer CPU or the like performs part or all of the actual processing based on the instruction of the program code.
  • the program code is stored in a storage means such as a hard disk or a memory of a system or apparatus, or a storage medium such as a CD-RW or CD-R
  • the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage means or the storage medium when used.
  • the described hardware may be implemented by ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc., and the described software is assembler, C / C ++, perl, Shell, PHP, Python. , Java (registered trademark) or a wide range of programs or script languages may be used.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the described software is assembler, C / C ++, perl, Shell, PHP, Python.
  • Java registered trademark
  • a wide range of programs or script languages may be used.
  • control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne des système et dispositif de traitement d'image comprenant : une unité de décalage de position d'image (103) destinée à décaler la position d'une image saisie en entrée ; une unité de réduction d'image (104) destinée à réduire la taille de l'image dont la position a été décalée, par découpage du nombre de ses pixels ; une unité d'agrandissement d'image (301) destinée à augmenter le nombre de pixels d'une image décodée afin de l'agrandir, l'image décodée étant l'image réduite ayant été codée, envoyée par l'intermédiaire d'un réseau de communication (107) et décodée ; une unité de décalage de position d'image (302) destinée à décaler la position de l'image agrandie de sorte à annuler et supprimer le décalage de position effectué par l'unité de décalage de position d'image (103) ; et une unité de réduction de distorsion de pliage (303) destinée à effectuer un traitement de réduction de distortion de pliage afin de réduire la distorsion de pliage de l'image dont la position a été décalée par l'unité de décalage de position d'image (302), au moyen d'informations relatives à une autre image dont la position a été décalée par l'unité de décalage de position d'image (302).
PCT/JP2017/004512 2016-02-24 2017-02-08 Système et dispositif de traitement d'image Ceased WO2017145752A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016033145A JP6516695B2 (ja) 2016-02-24 2016-02-24 画像処理システムおよび画像処理装置
JP2016-033145 2016-02-24

Publications (1)

Publication Number Publication Date
WO2017145752A1 true WO2017145752A1 (fr) 2017-08-31

Family

ID=59686142

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/004512 Ceased WO2017145752A1 (fr) 2016-02-24 2017-02-08 Système et dispositif de traitement d'image

Country Status (2)

Country Link
JP (1) JP6516695B2 (fr)
WO (1) WO2017145752A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111801948A (zh) * 2018-03-01 2020-10-20 索尼公司 图像处理装置和方法、成像元件和成像装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7350515B2 (ja) * 2019-05-22 2023-09-26 キヤノン株式会社 情報処理装置、情報処理方法およびプログラム
JP7583926B2 (ja) 2020-11-19 2024-11-14 ホアウェイ・テクノロジーズ・カンパニー・リミテッド 機械学習を基にしたピクチャコーディングにおけるクロマサブサンプリングフォーマット取り扱いのための方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007323480A (ja) * 2006-06-02 2007-12-13 Fuji Xerox Co Ltd 画像処理装置、画像符号化装置、画像復号化装置、画像処理システム及びプログラム
JP2009017425A (ja) * 2007-07-09 2009-01-22 Sony Corp 動画像変換装置、動画像復元装置、および方法、並びにコンピュータ・プログラム
JP2012195908A (ja) * 2011-03-18 2012-10-11 Hitachi Kokusai Electric Inc 画像転送システム、画像転送方法、画像受信装置、画像送信装置、及び、画像撮像装置
JP2013518463A (ja) * 2010-01-22 2013-05-20 トムソン ライセンシング サンプリングベースの超解像度ビデオ符号化および復号化方法並びに装置
WO2013191193A1 (fr) * 2012-06-20 2013-12-27 株式会社日立国際電気 Système de compression et de transmission vidéo

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007323480A (ja) * 2006-06-02 2007-12-13 Fuji Xerox Co Ltd 画像処理装置、画像符号化装置、画像復号化装置、画像処理システム及びプログラム
JP2009017425A (ja) * 2007-07-09 2009-01-22 Sony Corp 動画像変換装置、動画像復元装置、および方法、並びにコンピュータ・プログラム
JP2013518463A (ja) * 2010-01-22 2013-05-20 トムソン ライセンシング サンプリングベースの超解像度ビデオ符号化および復号化方法並びに装置
JP2012195908A (ja) * 2011-03-18 2012-10-11 Hitachi Kokusai Electric Inc 画像転送システム、画像転送方法、画像受信装置、画像送信装置、及び、画像撮像装置
WO2013191193A1 (fr) * 2012-06-20 2013-12-27 株式会社日立国際電気 Système de compression et de transmission vidéo

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111801948A (zh) * 2018-03-01 2020-10-20 索尼公司 图像处理装置和方法、成像元件和成像装置
CN111801948B (zh) * 2018-03-01 2023-01-03 索尼公司 图像处理装置和方法、成像元件和成像装置
US11765468B2 (en) 2018-03-01 2023-09-19 Sony Corporation Image processing device and method thereof, imaging element, and imaging device

Also Published As

Publication number Publication date
JP2017152902A (ja) 2017-08-31
JP6516695B2 (ja) 2019-05-22

Similar Documents

Publication Publication Date Title
US8315481B2 (en) Image transmitting apparatus, image receiving apparatus, image transmitting and receiving system, recording medium recording image transmitting program, and recording medium recording image receiving program
JP4356777B2 (ja) 画像処理装置、画像処理方法、プログラム、及び記録媒体
JP5529293B2 (ja) メタデータによる時間スケーリングのためのエッジエンハンスメントのための方法
US8319892B2 (en) Video signal processing apparatus, video displaying apparatus and high resolution method for video signal
JP4876048B2 (ja) 映像送受信方法、受信装置、映像蓄積装置
US20090034941A1 (en) Video signal recording/reproducing apparatus
US8089557B2 (en) Video signal processing apparatus, video signal processing method and video display apparatus
US20090009660A1 (en) Video displaying apparatus, video signal processing apparatus and video signal processing method
WO2017145752A1 (fr) Système et dispositif de traitement d'image
US8774552B2 (en) Image transfer system, image transfer method, image reception apparatus, image transmission apparatus, and image pickup apparatus
JP2009044538A (ja) 監視システムおよびこれに用いる撮像装置
JP2008294950A (ja) 画像処理方法、画像処理装置、及びこの画像処理装置を備えた電子機器
JP5113479B2 (ja) 画像信号処理装置及び画像信号処理方法
WO2009133403A2 (fr) Système de télévision
CN100583949C (zh) 图像处理装置及图像处理方法
JP5742048B2 (ja) カラー動画像構造変換方法及びカラー動画像構造変換装置
JP2009033582A (ja) 画像信号記録再生装置
CN1199448C (zh) 用于视频扫描速率转换的运动补偿上变频的方法和装置
CN101404733B (zh) 图像信号处理装置、图像信号处理方法及图像显示装置
US20120287313A1 (en) Image capture apparatus
JP6416255B2 (ja) 画像処理システム、及び、画像処理方法
JP2020021991A (ja) 画像処理装置、画像処理装置の制御方法およびプログラム
JP5686316B2 (ja) カラー動画像動き推定方法及びカラー動画像動き推定装置
JP2009278473A (ja) 画像処理装置、それを搭載した撮像装置、および画像再生装置
JP2009171370A (ja) 画像信号処理装置、画像表示装置、画像信号処理方法

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17756193

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17756193

Country of ref document: EP

Kind code of ref document: A1