WO2006084938A1 - Electronic device and method in an electronic device for processing image data - Google Patents

Electronic device and method in an electronic device for processing image data Download PDF

Info

Publication number
WO2006084938A1
WO2006084938A1 PCT/FI2005/050474 FI2005050474W WO2006084938A1 WO 2006084938 A1 WO2006084938 A1 WO 2006084938A1 FI 2005050474 W FI2005050474 W FI 2005050474W WO 2006084938 A1 WO2006084938 A1 WO 2006084938A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image data
focussing
blurring
imaging target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/FI2005/050474
Other languages
French (fr)
Inventor
Petri Ahonen
Simo Rossi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Inc
Original Assignee
Nokia Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Inc filed Critical Nokia Inc
Priority to EP05818842.6A priority Critical patent/EP1831841B1/en
Priority to CN2005800446897A priority patent/CN101088104B/en
Priority to JP2007548856A priority patent/JP4664379B2/en
Priority to KR1020077017173A priority patent/KR100890949B1/en
Priority to US11/792,120 priority patent/US8908080B2/en
Publication of WO2006084938A1 publication Critical patent/WO2006084938A1/en
Anticipated expiration legal-status Critical
Priority to US14/563,241 priority patent/US9552627B2/en
Priority to US15/392,403 priority patent/US9858651B2/en
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration using local operators
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/63Control of cameras or camera modules by using electronic viewfinders
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/67Focus control based on electronic image sensor signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70SSIS architectures; Circuits associated therewith
    • H04N25/71Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75Circuitry for providing, modifying or processing image signals from the pixel array
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2621Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/907Television signal recording using static stores, e.g. storage tubes or semiconductor memories
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details

Definitions

  • the present invention relates to an electronic device for per- forming imaging, including
  • - camera means for forming image data from an imaging target, the imaging target including at least one primary image object and at least one secondary image obj ect, - an image-processing chain arranged in connection with the camera means, for processing the image data formed from the imaging target, and
  • the invention also relates to a corresponding method, program product, and focussing module, for implementing the invention.
  • the depth of field is relatively wide, due to, among other factors, their short focal length.
  • This group of cameras is mobile stations equipped with a digital imaging capability.
  • the great depth of field makes it difficult to create a blurred background in the image .
  • One example of such an imaging application is por- traits . In them, it is only the primary image object that is desired to be shown sharply, the background, i . e . the secondary image obj ects , being desired to be blurred.
  • the present invention is intended to create a way of blurring non-desired imaging obj ects in digital imaging.
  • the characteristic features of the electronic device according to the invention are stated in the accompanying Claim 1 while the char- acteristic features of the method are stated in the accompanying Claim 9.
  • the invention also relates to a corresponding program product and a focussing module to be fitted for use in the device, the characteristic features of which are stated in the accompanying Claims 16 and 22.
  • blurring is performed using the information produced by focussing .
  • the invention is particularly suitable for application, for example, in such digital cameras, in which there is wide depth of field .
  • Such cameras are known, for example, from mobile stations .
  • the invention can be applied in both still and video imaging .
  • the information obtained from focussing of the camera is applied.
  • the one or more image obj ects in the image in which sharpness is to be retained, and correspondingly the image obj ects to be blurred, can be decided on the basis of this information . Focussing information is available immediately in the imaging situation, so that its application takes place very smoothly for achieving the purposes of the invention .
  • the blurring of inessential image obj ects can be performed, for example, by using filtering. There can even be pre- calculated filtering coefficients in the device for filtering, from which the most suitable group of coefficients can be selected for use in each situation. On the other hand, the sta- tistics formed for focussing can be used to calculate the filtering coefficients .
  • the end user can create a blurring effect in the image surprisingly already in the imaging stage using the camera .
  • One of the advantages achieved by the invention is that using small cameras , which generally are precisely those with wide depth of field, an such image can be achieved, in which the primary image obj ect is sharp and the background, or the secondary image obj ects in general, are blurred or otherwise made unclear .
  • Figure 1 shows an example in principle of an application of the electronic device according to the invention, as a schematic diagram
  • Figure 2 shows an example of an application of the program product according to the invention, for implementing blurring in an electronic device in the manner according to the invention
  • Figure 3 shows an example in principle of the method according to the invention, as a flow dia- gram
  • Figure 4 shows an example of an imaging target, to which the invention is applied.
  • Figure 1 shows an example in principle of an application of the electronic device 10 according to the invention, as a flow diagram, on the basis of which the invention is described in the following.
  • Figure 2 shows an example of the program product 30 according to the invention .
  • the program product 30 forms of a storage medium MEM and program code 31 stored on it, with reference to the code means 31.1 - 31.5 belonging to which program code 31 being made at suitable points in the following description, to connect them to the method and device 10 according to the invention .
  • the device 10 can be, for example, a digital camera equipped with a still and/or video imaging capability, a digital video camera, a mobile station equipped with a camera, or some other similar smart communicator ( PDA) , the components of which that are inessential from the point of view of the invention are not described in greater detail in this connection.
  • the inven- tion relates not only to the device 10, but equally to an im- aging-chain system 27 and a focussing module 28 , such as may be, for example, in the device 10.
  • a focussing module 28 is the Ai-AF system developed by Canon.
  • the device 10 according to the invention, and with it also the imaging system can include, as modular components, camera means 11, 14 and a digital image-processing chain 27 connected to it and a focussing circuit 28.
  • the camera means 11, 14 can include an image sensor totality 12 , 13, which is as such known, together with movable lenses 15, by means of which image data ID of the imaging target IT can be formed.
  • the imaging target IT which is converted by the camera sensor 12 in a known manner to form electrical signals , is converted into a digital form using an AD converter 13.
  • the focal length of the camera means 11, 14 may be less than 35 mm declared as a focal length equivalency with 35 mm film.
  • Some examples of the focal lengths of the camera means may be, declared as a focal length equivalency with 35 mm film, for example, 15 - 20 mm (special wide-angle) , 20 - 28 mm (wide- angle) or 28 - 35 mm (mild wide-angle) .
  • the use of the inven- tion achieves a particular advantage in devices 10 with an extensive depth of field, but the invention can of course also be applied in such devices with a narrow depth of field (for example, teleobjectives ) .
  • the focussing means 28 are in the device 10 for focussing the camera means 11, 14.
  • a solution that is, for example, as such, known, or which is still under development, can be applied as a focussing circuit 28.
  • the focussing circuit 28 at least one of the image obj ects II , 12 in the imaging target IT can be focussed to the camera means 11, 14 , more particularly to the sensor 12, prior to the performance of the imaging that it intended to be stored, or even during imaging to be stored, if the question is of, for example, a video imaging application .
  • the imaging target IT can include at least one primary image object II, relative to which it is wished to focus the camera means 11, 14 , and at least one secondary image object 12, which is an image subj ect that is of less interest from the point of view of the imaging . It can be, for example, the background to the primary image obj ect II .
  • focussing conventionally involves the collection of statistics from the image data ID.
  • the statistics can include, for example, a search for gradients for the detection of the edge of the primary image obj ect II .
  • the statistics can be formed of, for example, luminance information of the image data ID.
  • the focussing operations also include the movement of the lenses 15, in order to maximize the statistical image sharpness mathematically by comparing statistical information . Focussing can be performed automatically or also by the end user, who can manually adj ust the focus , if there is, for example, a manually adjustable focus disc (mechanical focus control) in the camera.
  • the focussing circuit 28 shown in Figure 1 can include an as such known autofocus control algorithm 24 , in which there can be a focus-point definition portion 24 as a sub-module .
  • the algorithm portion 24 receives AutoFocus AF-data from the calculating module 23 of AF statistics .
  • the statistics module 23 can process the image data ID coming directly from the AD converter 13, in ways that are, as such, known, and form from it, for example, the aforementioned gradient data .
  • the algorithm portion 24 can decide whether it images the selected first image object Il sharply to the sensor 12 in the set manner.
  • the algorithm portion 24 produces control data that is as such known, for the adjustment mechanism 14 of the set of lenses 15.
  • the control data is used to move the set of lenses 15, in such a way that the one or more image obj ects Il defined as primary by the focus-point sub- module 25 is imaged precisely and sharply to the sensor 12.
  • the image-processing chain 27 connected to the camera means 11 , 14 can include various modules in different implementation arrangements , which are used, for example, for processing, in the device 10 , the image data ID formed from the imaging target IT .
  • viewfinder imaging for which there can be a dedicated module VF in the device 10.
  • the viewfinder VF can be after colour-interpolation 16, or also after the blurring filter 17 according to the invention, which will be described in greater detail a little later.
  • the blurred background can, according to the invention, already be seen in the viewfinder image .
  • the image-processing chain IC can consist of one or more processing circuits/DSPs 16, 18 , which are, in terms of the inven- tion, entirely irrelevant components, and no further description of them is necessary in this connection .
  • the colour-interpolation 16 and image-data ID compression 18 of the image-processing chain 27 are shown.
  • the image data ID is stored, this can take place to some storage medium 19.
  • the technical implementation of these components, which are irrelevant in terms of the invention, will be obvious to one versed in the art and for this reason the invention is described in this connection at a very rough block-diagram level, for reasons of clarity.
  • hardware and software solu- tions, as well as combinations of them can be considered.
  • some of the operations of the modules 16, 18 , 23, 24 , 25 belonging to he image-processing and/or focussing chain 27 , 28 can be implemented even in a single module .
  • blurring means 17, 22 , 26, forming a blurring module 21 are arranged in the image-processing chain 27.
  • the sub-modules 17 , 22 , 26 belonging to the module 21 can also provide other tasks in the device 10 than those belonging to blurring, as will be demonstrated later (for example, focussing) .
  • the means 17 , 22, 26 can be used in a surprising manner to blur at least part of the secondary image objects 12 in the image data ID, which are not the primary object of interest in the imaging target IT, which the sensor 12 detects in its entirety.
  • the filtering module 17 of the blurring means is shown itself in the actual chain 27.
  • the other modules that implement blurring in the embodiment in question are the filtering-coefficient calculation module 22 and the focussed-area calculation/definition module 26.
  • the blurring means 17 , 22 , 26 use the information pro- pokerd by the focussing-module totality 28.
  • the focussing-area calculation module 26 can use the data obtained from the AF- statistics calculation portion 23 in the definition of the image area Il and now also the data obtained from the focussing point definition portion 25. Once the portion 26 has been cal- culated the focussed, i . e . the primary image object in the image data ID, its location in the image information formed by the image data ID, can be determined and also its shape, i . e . the location areas of the one or more primary image obj ects Il in the image IT .
  • the data obtained from the calculation portion 26 of the fo- cussed area can be sent to the filtering coefficient calculation module 22.
  • the final area which is used in the calculation of the filtering coefficients, can be selected / calculated. This area can even be pixel-accurate, thus delimiting the primary image obj ect Il very precisely .
  • the portrait area used can be entered, for instance manually, for example, by lassoing from a touch screen, if the device has one .
  • the module 22 can calculate the filtering coefficients, by using which the secondary image obj ects , i . e . the areas 12 are blurred.
  • the filtering coefficients calculated by the module 22 are provided to the filtering module 17 , which performs the blurring of the irrelevant image areas 12. This will be returned to in greater detail in the description of the method given next .
  • Figure 3 shows a flow chart of a schematic example in principle of the method according to the invention in digital imag- ing for blurring inessential image areas 12.
  • the image target IT consists of two people, a man with a briefcase and a woman reading a newspaper, as well as a short length of rolled-steel j oist .
  • the people are the primary image obj ects Il while the length of j oist is the secondary image obj ect 12 , which it is wished to blur.
  • the background area of the entire image area can, in connection with the invention, be understood as being such a secondary image obj ect 12.
  • stage 300 When imaging is started with the device 10, the imaging program is activated, as stage 300 , which in this case applied automatic focussing .
  • stage 301 the set of lenses 15 can be adjusted using the mechanism 14 to the initial focussing posi- tion .
  • image data ID is formed using the sensor 12 , i . e . viewfinder shots are formed, for example, for the view- finder VF.
  • the formation of the image data ID is performed continuously, the frequency being, for example, 15 - 30 frames per second.
  • stages 303 and 304 focussing operations that are, as such, known, can be performed. As such, all the blocks 301 - 304 , which are inside the block with a broken line around it, can be understood to be focussing sub-stages .
  • the actual focussing stages 303 and 304 can be taken care of automatically by the focussing module 28 , or allowance can also be made in them for operations made by the user .
  • the user can freely select the primary image obj ects Il or image obj ect areas FA, to be focussed, from the viewfinder VF. This automatic or manual selection also affects the obj ects to be blurred .
  • the user can, for example, from the image data ID formed by the sensor 12 for the viewfinder VF, to define at least one or even more image obj ects II, on which it is wished to focus the camera means 11, 14.
  • the selection made by the user can include, for example, the lassoing of an area, in which case even irregular obj ects can be set as primary image obj ects II .
  • the primary image obj ect Il can also be fitted to a predefined area with a rectangular or other shape .
  • Several different kinds of areas can be predefined in the memory MEM of the device 10.
  • focussing can be concentrated on, for example, one or more image areas (for example, on the centre of the imaging object) .
  • the focussing points can also be intelligently selected from the entire image area .
  • Canon' s Ai-AF system Artificial intelligence AF
  • stage 304 by the module 28 , more particularly, for example, its sub-module 24 , can be determined whether the image object Il has been focussed properly.
  • stage 304 If it is determined, in stage 304 , that the focussing is not correct, the automation 24 calculates new positions for the set of lenses 15. After that, a return is made to stage 301, in which the set of lenses 15 is moved to the new calculated positions . If, however, it is determined in stage 304 that the focussing is correct, the procedure moves to the actual imaging for storing, i . e . to stage 305.
  • stage 305 imaging for storing is performed, when the trigger button of the camera 10 can be pressed all the way down.
  • the image data ID captured using the sensor 12 is taken from the module 13 , which performs the AD conversion, to the image- processing chain 27.
  • colour interpolation using module 16, for example, can be performed as stage 306.
  • Other stages will also be obvious to one versed in the art and neither they, nor their order of performance are described here in greater detail .
  • this focussed image area II can also be used to blur the undesired image objects and their areas 12, in stage 307.
  • code means 31.1 in the program code 31 there are code means 31.1 in the program code 31.
  • the focussing point can be used to indicate one or more obj ects, i . e . in the context of the invention, a primary image object Il inside the imaging target IT .
  • the edges and shapes of the image obj ect Il can be identified, for example, in order to determine the size of the image obj ect Il and the position of the focussing point . In other words, this refers to the determining of the size of the primary image obj ect II .
  • This can be carried out by applying the statistical information from stages 303 and 304 , more generally produced by the focussing operation 28 obtained from the focussing stage indicated by the broken line .
  • the secondary image obj ects 12 are, of course, also defined, these being defined by the code means 31.5.
  • various filtering operations can be performed on the image, by means of which the background, i . e . the secondary image obj ects 12 are blurred, or made less sharp, as desired.
  • the code means 31.2 achieves this opera- tion .
  • the filtering can be of, for example, an evening type, such as spatial low-pass filtering.
  • spatial filtering coefficients for example, can be calculated using the module 22 (code means 31.3) .
  • the filtering coefficients can forms, for example, a mask that convolutes the image, which can be used to process the inessential image areas 12.
  • the convoluting mask, or in general the filtering coefficients can be defined, for example, from the luminance data formed by the focussing portion 28 , and even more particularly from the luminance data that refers to the secondary areas 12 that are to be blurred.
  • One criterion for the definition of the coefficients of the mask can then be, for example, that the blurring should be made to create "an even grey", thus avoiding the creation of a background that becomes too dark or too light .
  • the pixel values that are greater or less than the extremes of the luminance scale are generally cut to the extremes of the scale (for example, to zero, or to the value 255, if the depth is 8-bit) .
  • the coeffi- cients of the convoluting mask are attempted to be made to be adapted, using the focussing data, to be such that cutting of this kind does not occur .
  • blurring can also be made in such a way that noise is strongly mixed with the background area, or the secondary areas 12 in general, after which the background (i . e . noise) is low-pass filtered to become even. This too can be handled by the code 31.2.
  • the focus- sed area Il is made to remain untouched, i . e . sharp .
  • this is caused in such a way that the areas of the primary image obj ects Il are not processed at all, but only the inessential areas 12 are processed.
  • the sharpness of the areas Il remaining outside of the focussing area, i . e . the secondary areas in the context of the invention is then reduced.
  • suitable groups of filtering coefficients groups i . e . masks, using which filtering is then performed, can also be prearranged in the device 10.
  • the device 10 can provide filtering coefficients, either by calculating them on the fly, or by providing them from a "coefficient bank" prearranged in the memory MEM.
  • an area can also be taken into ac- count, for example, in stage 303 , in such a way that the edge and/or shape information of the primary image obj ect Il can be applied to select a non-rectangular shape .
  • the device 10 there can also be different kinds of precalculated area shapes . An attempt can be made to apply them to the selected primary image obj ect and then select/use the one that fits best . As some examples of these may be mention rectangular, circular, elliptical, and triangular areas FA.
  • the use of precalculated areas FA brings an advantage in the use of the processing power of the device 10 , because, in the case of area shapes that are frequently repeated, there is no need to perform the calculation again.
  • code means 31.4 in the program code 31, for performing this operation are code means 31.4 in the program code 31, for performing this operation .
  • the result is an image, which are limited depth of focus .
  • the image data ID is compressed and stored on the desired medium 19.
  • FIG. 2 shows a rough schematic diagram of one example of a program product 30 according to the invention .
  • the program product 30 can include a storage medium MEM and program code 31, written on the storage medium MEM, to be executed using the processor means CPU of the device 10 , for implementing blurring according to the method of the invention at least partly on a software level .
  • the storage medium MEM of the program code 31 can be, for example, a static or dynamic application memory in the device 10, or a blurring-circuit module totality being in the imaging chain IC, with which it can be di- rectly integrated.
  • the program code 31 can include several code means 31.1 - 31.5 to be executed by the processor means, the operation of which can be apply in the method descriptions given immediately above .
  • the code means 31.1 - 31.5 can consist of a group of processor commands to be performed consecutively, by means of which the functionalities desired, in terms of the invention, are created in the device 10 according to the invention.
  • a background blurring effect can be implemented in small digital cameras too, surprisingly already in the imaging stage, without any need for difficult postprocessing .
  • One example of an area of application of the invention can be the blurring of the background in portraits .
  • portrait applications may be additionally applied face recog- nition on basis of which the focus area and the background area to be blurred may be calculated.
  • face recog- nition on basis of which the focus area and the background area to be blurred may be calculated.
  • the color of the face which can usually be easily recognized by the algorithms known as such.
  • the case-specific calculation of filtering coefficients will achieve the most suitable background/blurring for each imaging target IT .
  • the invention is largely described above as a still- imaging application, it can, of course, also be applied video imaging, as well as to viewfinder imaging performed before the imaging for storing.
  • video imaging it should be understood that the flow chart of Figure 3 will then form a continuous loop, in which imaging, focussing, and blurring can be performed as a continuous process (from block 305 the procedure also moves to block 302 ) .
  • the imaging for storing is performed whether the focussing is then optimal or not (in stage 304 , the procedure moves in the directions of the yes and no arrows ) .
  • the focussing is iterated automatically to become correct by adj usting the optics 14 , without, however, interrupting imaging.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)
  • Focusing (AREA)
  • Automatic Focus Adjustment (AREA)

Abstract

The invention relates to an electronic device (10) for performing imaging, including: - camera means (11, 14) for creating image data (ID) from an imaging target (IT) , the imaging target (IT) including at least one primary image object (II) and at least one secondary image object (12), an image-processing chain (27) arranged in connection with the camera means, for processing the image data created from the imaging target, and - focussing means (28) for focussing the camera means on at least the primary image object. In addition, blurring means (17, 22, 26) are arranged in the image- processing chain, to blur at least some of the said secondary image objects in the image data, which blurring means are arranged to use the information produced by the focussing means.

Description

ELECTRONIC DEVICE AND METHOD IN AN ELECTRONIC DEVICE FOR PROCESSING IMAGE DATA
The present invention relates to an electronic device for per- forming imaging, including
- camera means for forming image data from an imaging target, the imaging target including at least one primary image object and at least one secondary image obj ect, - an image-processing chain arranged in connection with the camera means, for processing the image data formed from the imaging target, and
- focussing means for focussing the camera means on at least one primary image object . In addition, the invention also relates to a corresponding method, program product, and focussing module, for implementing the invention.
In small digital cameras, the depth of field is relatively wide, due to, among other factors, their short focal length. One example of this group of cameras is mobile stations equipped with a digital imaging capability. The great depth of field makes it difficult to create a blurred background in the image . One example of such an imaging application is por- traits . In them, it is only the primary image object that is desired to be shown sharply, the background, i . e . the secondary image obj ects , being desired to be blurred.
Solutions are known from the prior art, in which a shallow depth of field is implemented by using a large aperture (small F-number) and a long focal length . This arrangement is known, for example, from SLR ( Single Lens Reflex) cameras . Another possibility is blurring implemented by post-editing. This is a common functionality, for example, in still-image editors . US patent publication US-2002/0191100 Al (Casio Computer Co . Ltd . ) discloses one background-blurring method performed in a camera device in connection with imaging . It is based on capturing two images at the moment of imaging. The first image is focussed on the primary image object and before the second image is captured the focussing is altered to either the close or distant setting. After taking the shots, the first and second images are synthesized with each other . As a result of the synthesizing, a final image is obtained, in which the object is imaged sharply while the background is blurred.
Some other prior arts applying two or several images describe US 2002/0140823 Al , US 2002/0060739 Al, US 2003/0071905 Al and US 2002/0191100 Al .
The present invention is intended to create a way of blurring non-desired imaging obj ects in digital imaging. The characteristic features of the electronic device according to the invention are stated in the accompanying Claim 1 while the char- acteristic features of the method are stated in the accompanying Claim 9. In addition, the invention also relates to a corresponding program product and a focussing module to be fitted for use in the device, the characteristic features of which are stated in the accompanying Claims 16 and 22.
In the invention, blurring is performed using the information produced by focussing .
The invention is particularly suitable for application, for example, in such digital cameras, in which there is wide depth of field . Such cameras are known, for example, from mobile stations . The invention can be applied in both still and video imaging . In the invention, the information obtained from focussing of the camera is applied. The one or more image obj ects in the image in which sharpness is to be retained, and correspondingly the image obj ects to be blurred, can be decided on the basis of this information . Focussing information is available immediately in the imaging situation, so that its application takes place very smoothly for achieving the purposes of the invention .
The blurring of inessential image obj ects can be performed, for example, by using filtering. There can even be pre- calculated filtering coefficients in the device for filtering, from which the most suitable group of coefficients can be selected for use in each situation. On the other hand, the sta- tistics formed for focussing can be used to calculate the filtering coefficients .
In the invention, the end user can create a blurring effect in the image surprisingly already in the imaging stage using the camera . Thus, there is no need at all for a separate postediting operations that would take place outside the device after the imaging event .
One of the advantages achieved by the invention is that using small cameras , which generally are precisely those with wide depth of field, an such image can be achieved, in which the primary image obj ect is sharp and the background, or the secondary image obj ects in general, are blurred or otherwise made unclear .
The other characteristic features of the invention will become apparent from the accompanying Claims while additional advantages achieved are itemized in the description portion. The invention, which is not restricted to the embodiments described in the following, is examined in greater detail with reference to the accompanying figures, in which
Figure 1 shows an example in principle of an application of the electronic device according to the invention, as a schematic diagram,
Figure 2 shows an example of an application of the program product according to the invention, for implementing blurring in an electronic device in the manner according to the invention,
Figure 3 shows an example in principle of the method according to the invention, as a flow dia- gram, and
Figure 4 shows an example of an imaging target, to which the invention is applied.
Figure 1 shows an example in principle of an application of the electronic device 10 according to the invention, as a flow diagram, on the basis of which the invention is described in the following. In addition, Figure 2 shows an example of the program product 30 according to the invention . The program product 30 forms of a storage medium MEM and program code 31 stored on it, with reference to the code means 31.1 - 31.5 belonging to which program code 31 being made at suitable points in the following description, to connect them to the method and device 10 according to the invention .
The device 10 can be, for example, a digital camera equipped with a still and/or video imaging capability, a digital video camera, a mobile station equipped with a camera, or some other similar smart communicator ( PDA) , the components of which that are inessential from the point of view of the invention are not described in greater detail in this connection. The inven- tion relates not only to the device 10, but equally to an im- aging-chain system 27 and a focussing module 28 , such as may be, for example, in the device 10. One example of such a focussing module 28 is the Ai-AF system developed by Canon.
The device 10 according to the invention, and with it also the imaging system can include, as modular components, camera means 11, 14 and a digital image-processing chain 27 connected to it and a focussing circuit 28.
The camera means 11, 14 can include an image sensor totality 12 , 13, which is as such known, together with movable lenses 15, by means of which image data ID of the imaging target IT can be formed. The imaging target IT, which is converted by the camera sensor 12 in a known manner to form electrical signals , is converted into a digital form using an AD converter 13.
The focal length of the camera means 11, 14 may be less than 35 mm declared as a focal length equivalency with 35 mm film. Some examples of the focal lengths of the camera means may be, declared as a focal length equivalency with 35 mm film, for example, 15 - 20 mm (special wide-angle) , 20 - 28 mm (wide- angle) or 28 - 35 mm (mild wide-angle) . The use of the inven- tion achieves a particular advantage in devices 10 with an extensive depth of field, but the invention can of course also be applied in such devices with a narrow depth of field (for example, teleobjectives ) .
The focussing means 28 are in the device 10 for focussing the camera means 11, 14. A solution that is, for example, as such, known, or which is still under development, can be applied as a focussing circuit 28. Using the focussing circuit 28 , at least one of the image obj ects II , 12 in the imaging target IT can be focussed to the camera means 11, 14 , more particularly to the sensor 12, prior to the performance of the imaging that it intended to be stored, or even during imaging to be stored, if the question is of, for example, a video imaging application . This is because the imaging target IT can include at least one primary image object II, relative to which it is wished to focus the camera means 11, 14 , and at least one secondary image object 12, which is an image subj ect that is of less interest from the point of view of the imaging . It can be, for example, the background to the primary image obj ect II .
In cameras, focussing conventionally involves the collection of statistics from the image data ID. According to one embodiment , the statistics can include, for example, a search for gradients for the detection of the edge of the primary image obj ect II . The statistics can be formed of, for example, luminance information of the image data ID. The focussing operations also include the movement of the lenses 15, in order to maximize the statistical image sharpness mathematically by comparing statistical information . Focussing can be performed automatically or also by the end user, who can manually adj ust the focus , if there is, for example, a manually adjustable focus disc (mechanical focus control) in the camera.
If the focussing is implemented automatically in the device 10 , the focussing circuit 28 shown in Figure 1 can include an as such known autofocus control algorithm 24 , in which there can be a focus-point definition portion 24 as a sub-module . As input, the algorithm portion 24 receives AutoFocus AF-data from the calculating module 23 of AF statistics . The statistics module 23 can process the image data ID coming directly from the AD converter 13, in ways that are, as such, known, and form from it, for example, the aforementioned gradient data . On the basis of the data produced by the statistics mod- ule 23, the algorithm portion 24 can decide whether it images the selected first image object Il sharply to the sensor 12 in the set manner. As output, the algorithm portion 24 produces control data that is as such known, for the adjustment mechanism 14 of the set of lenses 15. The control data is used to move the set of lenses 15, in such a way that the one or more image obj ects Il defined as primary by the focus-point sub- module 25 is imaged precisely and sharply to the sensor 12.
The image-processing chain 27 connected to the camera means 11 , 14 can include various modules in different implementation arrangements , which are used, for example, for processing, in the device 10 , the image data ID formed from the imaging target IT . In both cases, whether imaging to be stored is being performed at that moment by the device 10 or not, it is possi- ble to perform so-called viewfinder imaging, for which there can be a dedicated module VF in the device 10. The viewfinder VF can be after colour-interpolation 16, or also after the blurring filter 17 according to the invention, which will be described in greater detail a little later. In that case, the blurred background can, according to the invention, already be seen in the viewfinder image .
The image-processing chain IC can consist of one or more processing circuits/DSPs 16, 18 , which are, in terms of the inven- tion, entirely irrelevant components, and no further description of them is necessary in this connection . In this case, the colour-interpolation 16 and image-data ID compression 18 of the image-processing chain 27 are shown. When the image data ID is stored, this can take place to some storage medium 19. The technical implementation of these components, which are irrelevant in terms of the invention, will be obvious to one versed in the art and for this reason the invention is described in this connection at a very rough block-diagram level, for reasons of clarity. In terms of the practical im- plementation of the invention, hardware and software solu- tions, as well as combinations of them, can be considered. Of course, some of the operations of the modules 16, 18 , 23, 24 , 25 belonging to he image-processing and/or focussing chain 27 , 28 can be implemented even in a single module .
As a surprisingly module, blurring means 17, 22 , 26, forming a blurring module 21, are arranged in the image-processing chain 27. Of course, the sub-modules 17 , 22 , 26 belonging to the module 21 can also provide other tasks in the device 10 than those belonging to blurring, as will be demonstrated later (for example, focussing) . The means 17 , 22, 26 can be used in a surprising manner to blur at least part of the secondary image objects 12 in the image data ID, which are not the primary object of interest in the imaging target IT, which the sensor 12 detects in its entirety.
In the embodiment of Figure 1, only the filtering module 17 of the blurring means is shown itself in the actual chain 27. The other modules that implement blurring in the embodiment in question are the filtering-coefficient calculation module 22 and the focussed-area calculation/definition module 26.
In order to blur the image obj ects 12 that are set to be secondary, the blurring means 17 , 22 , 26 use the information pro- duced by the focussing-module totality 28. The focussing-area calculation module 26 can use the data obtained from the AF- statistics calculation portion 23 in the definition of the image area Il and now also the data obtained from the focussing point definition portion 25. Once the portion 26 has been cal- culated the focussed, i . e . the primary image object in the image data ID, its location in the image information formed by the image data ID, can be determined and also its shape, i . e . the location areas of the one or more primary image obj ects Il in the image IT . The data obtained from the calculation portion 26 of the fo- cussed area can be sent to the filtering coefficient calculation module 22. On the basis of the data of the focussed focus area, i . e . in other words of the portrait area, the final area, which is used in the calculation of the filtering coefficients, can be selected / calculated. This area can even be pixel-accurate, thus delimiting the primary image obj ect Il very precisely . On the other hand, the portrait area used can be entered, for instance manually, for example, by lassoing from a touch screen, if the device has one . The module 22 can calculate the filtering coefficients, by using which the secondary image obj ects , i . e . the areas 12 are blurred. The filtering coefficients calculated by the module 22 are provided to the filtering module 17 , which performs the blurring of the irrelevant image areas 12. This will be returned to in greater detail in the description of the method given next .
Figure 3 shows a flow chart of a schematic example in principle of the method according to the invention in digital imag- ing for blurring inessential image areas 12. In the method description, reference is made to an example of an imaging situation, which is shown in Figure 4. In it, the image target IT consists of two people, a man with a briefcase and a woman reading a newspaper, as well as a short length of rolled-steel j oist . In this case, the people are the primary image obj ects Il while the length of j oist is the secondary image obj ect 12 , which it is wished to blur. In general, the background area of the entire image area can, in connection with the invention, be understood as being such a secondary image obj ect 12.
When imaging is started with the device 10, the imaging program is activated, as stage 300 , which in this case applied automatic focussing . In stage 301, the set of lenses 15 can be adjusted using the mechanism 14 to the initial focussing posi- tion . As stage 302, image data ID is formed using the sensor 12 , i . e . viewfinder shots are formed, for example, for the view- finder VF. In practice, the formation of the image data ID is performed continuously, the frequency being, for example, 15 - 30 frames per second. In stages 303 and 304 , focussing operations that are, as such, known, can be performed. As such, all the blocks 301 - 304 , which are inside the block with a broken line around it, can be understood to be focussing sub-stages .
The actual focussing stages 303 and 304 can be taken care of automatically by the focussing module 28 , or allowance can also be made in them for operations made by the user . When focussing is carried out manually by the end user, the user can freely select the primary image obj ects Il or image obj ect areas FA, to be focussed, from the viewfinder VF. This automatic or manual selection also affects the obj ects to be blurred . In stage 303, the user can, for example, from the image data ID formed by the sensor 12 for the viewfinder VF, to define at least one or even more image obj ects II, on which it is wished to focus the camera means 11, 14. The selection made by the user can include, for example, the lassoing of an area, in which case even irregular obj ects can be set as primary image obj ects II . On the other hand, the primary image obj ect Il can also be fitted to a predefined area with a rectangular or other shape . Several different kinds of areas can be predefined in the memory MEM of the device 10.
In the automatic focussing / image-obj ect selection applica- tion, focussing can be concentrated on, for example, one or more image areas (for example, on the centre of the imaging object) . The focussing points can also be intelligently selected from the entire image area . One example of this is Canon' s Ai-AF system (Artificial intelligence AF) . In stage 304 , by the module 28 , more particularly, for example, its sub-module 24 , can be determined whether the image object Il has been focussed properly.
If it is determined, in stage 304 , that the focussing is not correct, the automation 24 calculates new positions for the set of lenses 15. After that, a return is made to stage 301, in which the set of lenses 15 is moved to the new calculated positions . If, however, it is determined in stage 304 that the focussing is correct, the procedure moves to the actual imaging for storing, i . e . to stage 305.
In stage 305, imaging for storing is performed, when the trigger button of the camera 10 can be pressed all the way down. The image data ID captured using the sensor 12 is taken from the module 13 , which performs the AD conversion, to the image- processing chain 27. In the image-processing chain 27 , colour interpolation, using module 16, for example, can be performed as stage 306. Other stages will also be obvious to one versed in the art and neither they, nor their order of performance are described here in greater detail .
Besides specific one or more image obj ects Il being able to be focussed in the previous stages 303 and 304 in a manner that is , as such, known, this focussed image area II, or more particularly its position, size, and/or shape can also be used to blur the undesired image objects and their areas 12, in stage 307. For this purpose, there are code means 31.1 in the program code 31. In stage 303 , the focussing point can be used to indicate one or more obj ects, i . e . in the context of the invention, a primary image object Il inside the imaging target IT . Around the selected focussing point, for example, the edges and shapes of the image obj ect Il can be identified, for example, in order to determine the size of the image obj ect Il and the position of the focussing point . In other words, this refers to the determining of the size of the primary image obj ect II . This can be carried out by applying the statistical information from stages 303 and 304 , more generally produced by the focussing operation 28 obtained from the focussing stage indicated by the broken line . As a result of the operation, the secondary image obj ects 12 are, of course, also defined, these being defined by the code means 31.5.
Once the information concerning the position and size of the focussed primary area Il of the imaging target IT has been obtained, various filtering operations , for example, can be performed on the image, by means of which the background, i . e . the secondary image obj ects 12 are blurred, or made less sharp, as desired. The code means 31.2 achieves this opera- tion . The filtering can be of, for example, an evening type, such as spatial low-pass filtering. According to one embodiment, in the invention spatial filtering coefficients, for example, can be calculated using the module 22 (code means 31.3) . The filtering coefficients can forms, for example, a mask that convolutes the image, which can be used to process the inessential image areas 12.
The convoluting mask, or in general the filtering coefficients can be defined, for example, from the luminance data formed by the focussing portion 28 , and even more particularly from the luminance data that refers to the secondary areas 12 that are to be blurred. One criterion for the definition of the coefficients of the mask can then be, for example, that the blurring should be made to create "an even grey", thus avoiding the creation of a background that becomes too dark or too light . As is known, as a result of convolution, the pixel values that are greater or less than the extremes of the luminance scale are generally cut to the extremes of the scale (for example, to zero, or to the value 255, if the depth is 8-bit) . In order to avoid this kind of cutting to the extremes , the coeffi- cients of the convoluting mask are attempted to be made to be adapted, using the focussing data, to be such that cutting of this kind does not occur . According to a second embodiment, blurring can also be made in such a way that noise is strongly mixed with the background area, or the secondary areas 12 in general, after which the background (i . e . noise) is low-pass filtered to become even. This too can be handled by the code 31.2. When applying coefficients in the filtering, the focus- sed area Il is made to remain untouched, i . e . sharp . In part, this is caused in such a way that the areas of the primary image obj ects Il are not processed at all, but only the inessential areas 12 are processed. Correspondingly, as a result of the coefficients, the sharpness of the areas Il remaining outside of the focussing area, i . e . the secondary areas in the context of the invention, is then reduced.
On the other hand, suitable groups of filtering coefficients groups, i . e . masks, using which filtering is then performed, can also be prearranged in the device 10. Stated in more gen- eral terms, the device 10 can provide filtering coefficients, either by calculating them on the fly, or by providing them from a "coefficient bank" prearranged in the memory MEM.
Even more particularly, an area can also be taken into ac- count, for example, in stage 303 , in such a way that the edge and/or shape information of the primary image obj ect Il can be applied to select a non-rectangular shape . In the device 10 there can also be different kinds of precalculated area shapes . An attempt can be made to apply them to the selected primary image obj ect and then select/use the one that fits best . As some examples of these may be mention rectangular, circular, elliptical, and triangular areas FA. Among other things, the use of precalculated areas FA brings an advantage in the use of the processing power of the device 10 , because, in the case of area shapes that are frequently repeated, there is no need to perform the calculation again. There are code means 31.4 in the program code 31, for performing this operation .
The result , after the operations according to the invention, is an image, which are limited depth of focus . As final stages 308 - 310, the image data ID is compressed and stored on the desired medium 19.
Figure 2 shows a rough schematic diagram of one example of a program product 30 according to the invention . The program product 30 can include a storage medium MEM and program code 31, written on the storage medium MEM, to be executed using the processor means CPU of the device 10 , for implementing blurring according to the method of the invention at least partly on a software level . The storage medium MEM of the program code 31 can be, for example, a static or dynamic application memory in the device 10, or a blurring-circuit module totality being in the imaging chain IC, with which it can be di- rectly integrated.
The program code 31 can include several code means 31.1 - 31.5 to be executed by the processor means, the operation of which can be apply in the method descriptions given immediately above . The code means 31.1 - 31.5 can consist of a group of processor commands to be performed consecutively, by means of which the functionalities desired, in terms of the invention, are created in the device 10 according to the invention.
Owing to the invention, a background blurring effect can be implemented in small digital cameras too, surprisingly already in the imaging stage, without any need for difficult postprocessing . One example of an area of application of the invention can be the blurring of the background in portraits . In portrait applications may be additionally applied face recog- nition on basis of which the focus area and the background area to be blurred may be calculated. For the recognition may be used the color of the face which can usually be easily recognized by the algorithms known as such. The case-specific calculation of filtering coefficients will achieve the most suitable background/blurring for each imaging target IT .
Though the invention is largely described above as a still- imaging application, it can, of course, also be applied video imaging, as well as to viewfinder imaging performed before the imaging for storing. In video imaging, it should be understood that the flow chart of Figure 3 will then form a continuous loop, in which imaging, focussing, and blurring can be performed as a continuous process (from block 305 the procedure also moves to block 302 ) . In any event, the imaging for storing is performed whether the focussing is then optimal or not (in stage 304 , the procedure moves in the directions of the yes and no arrows ) . The focussing is iterated automatically to become correct by adj usting the optics 14 , without, however, interrupting imaging.
It must be understood that the above description and the related figures are only intended to illustrate the present invention . The invention is thus in no way restricted to only the embodiments disclosed or stated in the Claims , but many different variations and adaptations of the invention, which are possible within the scope on the inventive idea defined in the accompanying Claims , will be obvious to one versed in the art .

Claims

CIAIMS
1. An electronic device ( 10 ) for performing imaging, including
- camera means ( 11, 14 ) for forming image data ( ID) from an imaging target ( IT) , the imaging target ( IT) including at least one primary image obj ect (II) and at least one secondary image obj ect (12 ) ,
- an image-processing chain (27 ) arranged in connection with the camera means (11, 14 ) , for processing the image data ( ID) formed from the imaging target
(IT) , and
- focussing means (28 ) for focussing the camera means ( 11, 14 ) on at least one primary image object ( II) , characterized in that, in addition, blurring means ( 17 , 22 , 26) are arranged in the image-processing chain (27 ) , to blur at least some of the said secondary image obj ects ( 12 ) in the image data ( ID) , which blurring means ( 17 , 22 , 26) are arranged to use the information produced by the focussing means (28 ) .
2. A device ( 10 ) according to Claim 1 , characterized in that the focal length equivalency with 35 mm film of the camera means ( 11 , 14 ) of the device ( 10 ) is <= 35 mm.
3. A device ( 10 ) according to Claim 1 or 2 , characterized in that the said information produced by the focussing means ( 28 ) is arranged to be available for defining the said secondary image obj ects ( 12 ) to be blurred .
4. A device ( 10 ) according to any of Claims 1 - 3 , characterized in that the blurring is arranged to take place by filtering the image data ( ID) .
5. A device ( 10 ) according to any of Claims 1 - 4 , character- ized in that the device ( 10 ) is arranged to provide filtering coefficients , such as , for example, spatial filtering coefficients, by using which the blurring is arranged to be performed .
6. A device ( 10 ) according to any of Claims 1 - 5 , characterized in that the primary image obj ect ( II ) is arranged to be fitted to an area ( FA) with a set shape, a collection of coefficients corresponding to which areas ( FA) being prearranged in the device ( 10 ) .
7. A device ( 10 ) according to any of Claims 1 - 6, characterized in that the blurring is arranged to be performed in connection with the formation of the image data (ID) .
8. A device ( 10 ) according to any of Claims 1 - 7 , characterized in that the secondary image obj ects ( 12 ) are arranged to be blurred by applying noise to the areas corresponding to them, which area is then arranged to be equalized by low-pass filtering .
9. A method in digital imaging for processing image data ( ID) , in which method
- camera means ( 11 , 14 ) are used to form image data ( ID) from an imaging target ( IT) , the imaging target ( IT) including at least one primary image obj ect ( II ) and at least one secondary image obj ect ( 12 ) ,
- the camera means ( 11 , 14 ) are focussed on at least one primary image obj ect ( II ) , and
- the camera means ( 11 , 14 ) are used to form focussed image data ( ID) , which image data ( ID) is processed in the device ( 10 ) , in order to achieve the desired changes in the image data ( ID) , characterized in that, in the processing, at least some of the said secondary image obj ects ( 12 ) in the image data ( ID) are blurred using the information produced in the focussing .
10. A method according to Claim 9, characterized in that the said secondary image objects ( 12 ) to be blurred are defined using the said information produced in the focussing.
11. A method according to Claim 9 or 10 , characterized in that the blurring is performed by filtering the image data ( ID) .
12. A method according to any of Claims 9 - 11 , characterized in that filtering coefficients , such as, for example, spatial filtering coefficients , are calculated from the information produced in the focussing, using which the blurring is performed .
13. A method according to any of Claims 9 - 12 , characterized in that the primary image obj ect ( II ) is fitted to an area ( FA) with a set shape, the areas outside of which area ( FA) are blurred .
14. A method according to any of Claims 9 - 13 , characterized in that the blurring is performed in connection with the formation of the image data ( ID) .
15. A method according to any of Claims 9 - 14 , characterized in that the secondary image obj ects ( 12 ) are blurred by arranging noise in the areas corresponding to them, which area is then low-pass filtered to equalize it .
16. A program product ( 30 ) for processing image data ( ID) in an electronic device ( 10 ) , which program product ( 30 ) forms of a storage medium (MEM) and program code (31 ) written on the storage medium (MEM) to be executed using processor means (CPU) , the electronic device ( 10 ) including
- camera means ( 11 , 14 ) for forming image data (ID) from an imaging target ( IT ) , the imaging target ( IT) including at least one primary image object ( II) and at least one secondary image object (12) ,
- an image-processing chain (27 ) arranged in connection with the camera means (11, 14 ) , for processing the image data (ID) formed from the imaging target
(IT) , and
- focussing means (28 ) for focussing the camera means (11, 14 ) on at least one primary image object (II) , characterized in that the program code (31) includes - first code means (31.1) configured to blur at least some of the said secondary image objects (12 ) in the image data ( ID) , using the information produced by the focussing means (28 ) .
17. A program product (30 ) according to Claim 16, characterized in that the program code ( 31 ) includes additionally second code means ( 31.2 ) configured to blur the image data ( ID) by filtering.
18. A program product (30 ) according to Claim 16 or 17 , characterized in that the program code (31 ) includes additionally third code means (31.3 ) configured to calculate filtering coefficients , such as, for example, spatial filtering coefficients, from the information produced by the focussing means (28 ) , using which the blurring is arranged to be performed.
19. A program product (30) according to any of Claims 16 - 18 , characterized in that the program code (31) includes additionally fourth code means (31.4 ) configured to fit the primary image obj ect (II) to an area (FA) with a set shape, a collection of filtering coefficients corresponding to which areas being prearranged in the device (10) .
20. A program product (30) according to any of Claims 16 - 19, characterized in that the program code (31) is arranged to be executed in connection with the formation of the image data (ID) .
21. A program product ( 30 ) according to any of Claims 16 - 20 , characterized in that the program code (31 ) includes additionally fifth code means (31.5 ) configured to define the said secondary image obj ects (12 ) to be blurred, by using the said information produced by the focussing means (28 ) .
22. A focussing module (28 ) , which can be arranged in an electronic device ( 10 ) for performing imaging, the device including
- camera means (11, 14 ) for forming image data (ID) from an imaging target (IT) , the imaging target (IT) including at least one primary image object ( II ) and at least one secondary image object (12 ) ,
- an image-processing chain (27 ) arranged in connection with the camera means (11, 14 ) , for processing, in the device (10) , the image data ( ID) formed from the imaging target ( IT) , and in which
- the said focussing module (28 ) can be arranged in the device ( 10 ) at least for focussing the camera means (11, 14 ) on at least one primary image object
(H) , characterized in that, in addition, blurring means (17 , 22, 26) are arranged in the image-processing chain (21 ) , to blur at least some of the said secondary image obj ects (12) in the image data (ID) , which blurring means (17 , 22, 26) are arranged to use the information produced by the said focussing module (28 ) .
PCT/FI2005/050474 2004-12-29 2005-12-22 Electronic device and method in an electronic device for processing image data Ceased WO2006084938A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
EP05818842.6A EP1831841B1 (en) 2004-12-29 2005-12-22 Electronic device and method in an electronic device for processing image data
CN2005800446897A CN101088104B (en) 2004-12-29 2005-12-22 Electronic device and method in electronic device for processing image data
JP2007548856A JP4664379B2 (en) 2004-12-29 2005-12-22 Electronic device and image data processing method for image data processing
KR1020077017173A KR100890949B1 (en) 2004-12-29 2005-12-22 Electronic device and method in an electronic device for processing image data
US11/792,120 US8908080B2 (en) 2004-12-29 2005-12-22 Electronic device and method in an electronic device for processing image data
US14/563,241 US9552627B2 (en) 2004-12-29 2014-12-08 Electronic device and method in an electronic device for processing image data
US15/392,403 US9858651B2 (en) 2004-12-29 2016-12-28 Electronic device and method in an electronic device for processing image data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FI20045511A FI117265B (en) 2004-12-29 2004-12-29 Electronic apparatus and method for processing image data in an electronic apparatus
FI20045511 2004-12-29

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US11/792,120 A-371-Of-International US8908080B2 (en) 2004-12-29 2005-12-22 Electronic device and method in an electronic device for processing image data
US14/563,241 Continuation US9552627B2 (en) 2004-12-29 2014-12-08 Electronic device and method in an electronic device for processing image data

Publications (1)

Publication Number Publication Date
WO2006084938A1 true WO2006084938A1 (en) 2006-08-17

Family

ID=33548107

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/FI2005/050474 Ceased WO2006084938A1 (en) 2004-12-29 2005-12-22 Electronic device and method in an electronic device for processing image data

Country Status (7)

Country Link
US (3) US8908080B2 (en)
EP (1) EP1831841B1 (en)
JP (1) JP4664379B2 (en)
KR (1) KR100890949B1 (en)
CN (1) CN101088104B (en)
FI (1) FI117265B (en)
WO (1) WO2006084938A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101426093B (en) * 2007-10-29 2011-11-16 株式会社理光 Image processing device, image processing method
EP2487645A1 (en) * 2011-02-09 2012-08-15 Research In Motion Limited Method of controlling the depth of field for a small sensor camera using an extension for EDOF

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7606417B2 (en) * 2004-08-16 2009-10-20 Fotonation Vision Limited Foreground/background segmentation in digital images with differential exposure calculations
US20090067805A1 (en) * 2007-09-10 2009-03-12 Victor Company Of Japan, Limited Recording and reproducing device
US9179060B2 (en) * 2007-09-27 2015-11-03 Qualcomm Incorporated Method and apparatus for camera shake effect image stabilization
KR101491594B1 (en) 2007-11-05 2015-02-09 삼성전자주식회사 Handheld terminal having a touch screen and image processing method thereof
JP5124372B2 (en) * 2008-07-10 2013-01-23 株式会社リコー Image processing apparatus, image processing method, and digital still camera
US8725751B1 (en) * 2008-08-28 2014-05-13 Trend Micro Incorporated Method and apparatus for blocking or blurring unwanted images
JP2010226558A (en) 2009-03-25 2010-10-07 Sony Corp Image processing apparatus, image processing method, and program
JP5460173B2 (en) * 2009-08-13 2014-04-02 富士フイルム株式会社 Image processing method, image processing apparatus, image processing program, and imaging apparatus
JP5493789B2 (en) 2009-12-07 2014-05-14 株式会社リコー Imaging apparatus and imaging method
KR101643613B1 (en) * 2010-02-01 2016-07-29 삼성전자주식회사 Digital image process apparatus, method for image processing and storage medium thereof
JP5300756B2 (en) * 2010-02-05 2013-09-25 キヤノン株式会社 Imaging apparatus and image processing method
US8355039B2 (en) 2010-07-06 2013-01-15 DigitalOptics Corporation Europe Limited Scene background blurring including range measurement
KR101896026B1 (en) * 2011-11-08 2018-09-07 삼성전자주식회사 Apparatus and method for generating a motion blur in a portable terminal
CN102932541A (en) * 2012-10-25 2013-02-13 广东欧珀移动通信有限公司 Mobile phone photographing method and system
WO2014193377A1 (en) * 2013-05-30 2014-12-04 Nokia Corporation Image refocusing
CN104424640B (en) * 2013-09-06 2017-06-20 格科微电子(上海)有限公司 The method and apparatus for carrying out blurring treatment to image
CN105279752A (en) * 2014-07-25 2016-01-27 王辉 Digital image overall artistic effect processing method
CN104219445B (en) * 2014-08-26 2017-07-25 小米科技有限责任公司 Screening-mode method of adjustment and device
CN105141858B (en) * 2015-08-13 2018-10-12 上海斐讯数据通信技术有限公司 The background blurring system and method for photo
WO2017132074A1 (en) * 2016-01-26 2017-08-03 Russell David Wayne System and method for targeted imaging from collection platforms
CN105979165B (en) * 2016-06-02 2019-02-05 Oppo广东移动通信有限公司 Virtual photo generation method, device and mobile terminal
CN110022430A (en) * 2018-01-10 2019-07-16 中兴通讯股份有限公司 Image weakening method, device, mobile terminal and computer readable storage medium
CN110022392B (en) * 2019-05-27 2021-06-15 Oppo广东移动通信有限公司 Camera control method and related products
US11948276B2 (en) 2020-01-16 2024-04-02 Samsung Electronics Co., Ltd. Apparatus and method for enhancing videos

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4998126A (en) * 1988-11-04 1991-03-05 Nikon Corporation Automatic focus adjustment camera
US20020060739A1 (en) 2000-11-17 2002-05-23 Minolta Co., Ltd. Image capture device and method of image processing
US20020140823A1 (en) 2001-03-30 2002-10-03 Mikio Sakurai Image processing method, image processing apparatus and image processing program
US20020191100A1 (en) 2001-06-19 2002-12-19 Casio Computer Co., Ltd. Image pick-up apparatus, image pick-up method, and storage medium that records image pick-up method program
US20030071905A1 (en) 2001-10-12 2003-04-17 Ryo Yamasaki Image processing apparatus and method, control program, and storage medium
US20030117511A1 (en) * 2001-12-21 2003-06-26 Eastman Kodak Company Method and camera system for blurring portions of a verification image to show out of focus areas in a captured archival image

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3093781B2 (en) * 1990-11-09 2000-10-03 オリンパス光学工業株式会社 Focus position detection device
JP2983763B2 (en) * 1992-05-26 1999-11-29 三洋電機株式会社 Video camera autofocus device
JPH1020392A (en) * 1996-07-02 1998-01-23 Olympus Optical Co Ltd Camera used for silver salt photographing and electronic image pickup
JPH10248068A (en) 1997-03-05 1998-09-14 Canon Inc Imaging device and image processing device
JPH11136568A (en) * 1997-10-31 1999-05-21 Fuji Photo Film Co Ltd Touch panel operation-type camera
JPH11252427A (en) 1998-03-02 1999-09-17 Fuji Photo Film Co Ltd Touch panel operation type camera
JP4612750B2 (en) * 1998-03-17 2011-01-12 キヤノン株式会社 Digital camera, photographing method, and storage medium
US7106376B1 (en) * 1998-10-22 2006-09-12 Flashpoint Technology, Inc. Method and system for improving image quality of portrait images using a focus zone shift
JP4154053B2 (en) * 1998-12-25 2008-09-24 キヤノン株式会社 Image recording / reproducing system, image recording apparatus, and image reproducing apparatus
US20010013895A1 (en) * 2000-02-04 2001-08-16 Kiyoharu Aizawa Arbitrarily focused image synthesizing apparatus and multi-image simultaneous capturing camera for use therein
JP2002027425A (en) * 2000-07-04 2002-01-25 Canon Inc Video distribution apparatus and method
JP2002077591A (en) * 2000-09-05 2002-03-15 Minolta Co Ltd Image processing device and imaging device
JP4565784B2 (en) 2001-09-20 2010-10-20 Hoya株式会社 Digital camera
JP3781016B2 (en) * 2002-06-18 2006-05-31 カシオ計算機株式会社 Electronic camera, photographing direction acquisition method and program
US7110575B2 (en) 2002-08-02 2006-09-19 Eastman Kodak Company Method for locating faces in digital color images
CN100423021C (en) * 2002-10-17 2008-10-01 精工爱普生株式会社 Method and device for image segmentation with low depth of field
KR101086404B1 (en) 2004-10-27 2011-11-25 삼성전자주식회사 Control method of digital photographing apparatus for out-focusing operation, and digital photographing apparatus employing this method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4998126A (en) * 1988-11-04 1991-03-05 Nikon Corporation Automatic focus adjustment camera
US20020060739A1 (en) 2000-11-17 2002-05-23 Minolta Co., Ltd. Image capture device and method of image processing
US20020140823A1 (en) 2001-03-30 2002-10-03 Mikio Sakurai Image processing method, image processing apparatus and image processing program
US20020191100A1 (en) 2001-06-19 2002-12-19 Casio Computer Co., Ltd. Image pick-up apparatus, image pick-up method, and storage medium that records image pick-up method program
US20030071905A1 (en) 2001-10-12 2003-04-17 Ryo Yamasaki Image processing apparatus and method, control program, and storage medium
US20030117511A1 (en) * 2001-12-21 2003-06-26 Eastman Kodak Company Method and camera system for blurring portions of a verification image to show out of focus areas in a captured archival image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP1831841A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101426093B (en) * 2007-10-29 2011-11-16 株式会社理光 Image processing device, image processing method
EP2487645A1 (en) * 2011-02-09 2012-08-15 Research In Motion Limited Method of controlling the depth of field for a small sensor camera using an extension for EDOF

Also Published As

Publication number Publication date
US20080106615A1 (en) 2008-05-08
FI20045511L (en) 2006-06-30
US9858651B2 (en) 2018-01-02
FI20045511A0 (en) 2004-12-29
FI117265B (en) 2006-08-15
US9552627B2 (en) 2017-01-24
KR100890949B1 (en) 2009-03-27
CN101088104A (en) 2007-12-12
EP1831841A4 (en) 2011-06-08
JP4664379B2 (en) 2011-04-06
EP1831841A1 (en) 2007-09-12
JP2008529322A (en) 2008-07-31
US20170109868A1 (en) 2017-04-20
US20150146043A1 (en) 2015-05-28
CN101088104B (en) 2010-05-12
US8908080B2 (en) 2014-12-09
KR20070097550A (en) 2007-10-04
EP1831841B1 (en) 2016-02-24

Similar Documents

Publication Publication Date Title
US9858651B2 (en) Electronic device and method in an electronic device for processing image data
CN113888437B (en) Image processing method, device, electronic device and computer readable storage medium
US8023000B2 (en) Image pickup apparatus, image processing apparatus, image pickup method, and image processing method
CN101998053B (en) Image processing method, image processing apparatus, and imaging apparatus
JP5313127B2 (en) Video composition method, video composition system
EP1231778A2 (en) Method and system for motion image digital processing
DE112013004507T5 (en) Image processing apparatus, image capturing apparatus, image processing method, program and recording medium
EP4546767A1 (en) Video processing method and apparatus, and device and medium
US7800664B2 (en) Digital photographic instrument, method for adjusting focus of digital photographic instrument, and program for digital photographic instrument
KR20150032764A (en) Method and image capturing device for generating artificially defocused blurred image
JP6116436B2 (en) Image processing apparatus and image processing method
KR20070041552A (en) Electronic device for forming image information and method and corresponding program by electronic device
CN110784642B (en) Image processing apparatus, control method thereof, storage medium, and imaging apparatus
KR100647955B1 (en) Image processing device to correct defocus phenomenon
JP2009118434A (en) Blur correction device and imaging device
CN113763524B (en) Dual-stream bokeh rendering method and system based on physical optics model and neural network
JP4934992B2 (en) Image processing apparatus, electronic camera, and image processing program
JP4807623B2 (en) Imaging apparatus, imaging method, and imaging program
CN113691731B (en) Processing method and device and electronic equipment
CN113542625B (en) Image processing method, device, equipment and storage medium
JP7695102B2 (en) IMAGING APPARATUS, CONTROL METHOD FOR IMAGING APPARATUS, PROGRAM, AND RECORDING MEDIUM
JP5251546B2 (en) Image processing program, image processing apparatus, and imaging apparatus
CN119603553A (en) Photographic sequence focusing method and system for continuous snapshot function
CN120640126A (en) Shooting method, device, electronic device and readable storage medium
CN119788970A (en) Moon shooting method and device and electronic equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 11792120

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 200580044689.7

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2007548856

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2005818842

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 1020077017173

Country of ref document: KR

WWP Wipo information: published in national office

Ref document number: 2005818842

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 11792120

Country of ref document: US