EP4659125A1 - Capteurs, systèmes et procédés pour appareils de prise de vues à polarisation d'identification de visage compacts - Google Patents

Capteurs, systèmes et procédés pour appareils de prise de vues à polarisation d'identification de visage compacts

Info

Publication number
EP4659125A1
EP4659125A1 EP24703633.8A EP24703633A EP4659125A1 EP 4659125 A1 EP4659125 A1 EP 4659125A1 EP 24703633 A EP24703633 A EP 24703633A EP 4659125 A1 EP4659125 A1 EP 4659125A1
Authority
EP
European Patent Office
Prior art keywords
image sensor
polarization
pixel
image
color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP24703633.8A
Other languages
German (de)
English (en)
Inventor
Michael Scherer
Jaeseong Kim
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Corephotonics Ltd
Original Assignee
Corephotonics Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Corephotonics Ltd filed Critical Corephotonics Ltd
Publication of EP4659125A1 publication Critical patent/EP4659125A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters
    • G02B5/201—Filters in the form of arrays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B1/00—Optical elements characterised by the material of which they are made; Optical coatings for optical elements
    • G02B1/002—Optical elements characterised by the material of which they are made; Optical coatings for optical elements made of materials engineered to provide properties not available in nature, e.g. metamaterials
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters
    • G02B5/208—Filters for use with infrared or ultraviolet radiation, e.g. for separating visible light from infrared and/or ultraviolet radiation
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/30—Polarising elements
    • G02B5/3025—Polarisers, i.e. arrangements capable of producing a definite output polarisation state from an unpolarised input state
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
    • H04W12/06—Authentication

Definitions

  • Embodiments disclosed herein relate in general to biometric security methods and in particular to unlocking mobile devices with biometric security methods.
  • Mobile handheld electronic devices such as smartphones, tablets, headsets, smartwatches, smart glasses etc., which are configured to use biometric methods to unlock (or to open) a mobile device are known.
  • a popular biometric method is face unlock (or face ID). It uses image data of a face of an owner of the mobile device to extract biometric face features that are unique (or characteristic) to the owner. Based on the biometric face features, the mobile device is unlocked when the owner approaches.
  • FIG. 1A shows an example of a known mobile device 100 that is configured to support face unlock.
  • Mobile device 100 has a rear surface 102 which is in general pointed away from a user.
  • Mobile device 100 has a front surface 104 which is in general pointed towards the user.
  • Front surface 104 includes a first selfie (or front) camera 110 having a first selfie camera field- of-view (FOVi) 112.
  • First selfie camera 110 may be a color (or C) camera as known in the art, including for example a C image sensor such as a RGB image sensor 150 (FIG. 1C).
  • C camera may include a C image sensor such as a Red, Green, Blue, White (RGBW), Cyan, Magenta, Yellow (CMY), Red, Blue, White (RBW), etc.
  • first selfie camera 110 as “selfie C camera” 110.
  • Front surface 104 may also include a second selfie camera 120 having a second selfie camera FOV2 122. As shown, FOVi 112 and FOV2 122 overlap.
  • second selfie camera 120 may be a depth camera as known in the art that provides depth image data.
  • second selfie camera 120 may be a C camera that captures C image data.
  • First selfie camera 110 and second selfie camera 120 form a stereo camera that is operational to provide depth image data.
  • second selfie camera 120 as “selfie depth camera” 120.
  • Front surface 104 may include a screen 124.
  • Rear surface 102 may include a multi-camera 126 as known in the art. The various cameras have dimensions (or “sizes”) compatible with slim and compact mobile devices.
  • FIG. IB shows a known method for face unlock numbered 130.
  • a user approaches a mobile device (e.g. a smartphone) like mobile device 100, configured to perform face unlock.
  • the mobile device captures a selfie image (or a video stream of selfie images) including a face such as a face 114 (FIG. 1A) of the user.
  • a selfie image or a video stream of selfie images
  • a face such as a face 114 (FIG. 1A) of the user.
  • a first face unlock example only selfie C camera 110 is used to capture a selfie C image.
  • the first face unlock example we refer to the first face unlock example as “single-camera face unlock”.
  • selfie C camera 110 and selfie depth camera 120 are used to capture a selfie C image and a selfie depth image respectively.
  • the mobile device analyses the selfie C image to identify an owner of the mobile device.
  • the mobile device analyses the selfie depth image to identify the owner of the mobile device.
  • the selfie depth image is fused (or “merged”) with the selfie C image to obtain a fused “C-depth” image. The C-depth image is then analysed to identify the owner of the mobile device.
  • a set of face features is extracted, and the set of face features is compared to known set of face features of the owner, i.e. a set of face features that were previously captured by, and stored on, the mobile device.
  • known set of face features of the owner i.e. a set of face features that were previously captured by, and stored on, the mobile device.
  • the mobile device is unlocked.
  • the mobile device is not unlocked.
  • An advantage of single-camera face unlock is that it can be implemented using relatively simple (and/or inexpensive) and compact camera hardware.
  • An advantage of dualcamera face unlock over single-camera face unlock is that it provides a higher degree of security to an owner of the mobile device.
  • a higher degree of security means here that a probability of incorrectly unlocking the mobile device is lower for dual-camera face unlock than for single-camera face unlock.
  • Incorrectly unlocking the mobile device means for example that the mobile device is unlocked even for a person not being an owner of the mobile device, or that the mobile device is unlocked although the owner’s face is not present in the scene (referred to as “face spoofing”).
  • an image of the owner or a face mask resembling the owner may be positioned into FOVi and FOV2.
  • the higher degree of security is achieved by using additional image information.
  • the additional image information is depth image information.
  • a disadvantage of dual-camera face unlock is that its implementation requires relatively complex (or expensive) and large camera hardware. Specifically, it requires two cameras (instead of one camera), which is not beneficial in terms of cost, and it covers a larger area of the screen of the mobile device, which is not beneficial in terms of industrial design.
  • EER error rate
  • TPR true positive rate
  • FPR False Positive Rate
  • AUC Area under the ROC Curve
  • FIG. 1C shows a known RGB image sensor 150 in a top view. Included in a RGB camera, the optical axis of the RGB camera is oriented perpendicular to RGB image sensor 150.
  • RGB image sensor 150 includes a plurality of single pixels (red-R, green-G and blue-B) like single R pixel 152. In general, in a top view a single pixel has a square shape.
  • RGB image sensor 150 comprises a plurality of pixel units like a pixel unit 154.
  • pixel unit 154 includes one R pixel, one B pixel and two G pixels.
  • Pixel unit 154 represents the smallest pixel unit (or “assembly”) that can be used as a building block to construct (“assemble”) RGB image sensor 150.
  • RGB image sensor 150 can be formed by repeating pixel units such as pixel unit 154, as shown for pixel units 156, 158, 160 and 162.
  • FIG. ID shows a known polarization image sensor 170 in a top view. Included in a polarization camera, an optical axis of the polarization camera is oriented perpendicular to polarization image sensor 170.
  • Polarization image sensor 170 includes a plurality of single pixels like single pixel 172.
  • Polarization image sensor 170 comprises a plurality of pixel units like a pixel unit 174.
  • Pixel unit 174 includes one pixel measuring incoming light having a 90 degree linear polarization (marked “90°”), one pixel measuring incoming light having a 45 degree linear polarization (marked “45°”), one pixel measuring incoming light having a 135 degree linear polarization (marked “135°”) and one pixel measuring incoming light having a 0 linear degree polarization (marked “0°”).
  • a polarization filter is applied on top of each pixel.
  • Pixel unit 174 represents the smallest pixel unit that can be used as a building block to construct polarization image sensor 170.
  • polarization image sensor 170 can be formed by repeating pixel units such as pixel unit 174, as shown for pixel units 175, 176, 177 and 178.
  • a polarization image sensor such as polarization image sensor 170 included in a polarization camera can provide additional image information.
  • the additional image information is polarization image information.
  • the polarization image information can be used in a dual-camera face unlock (instead of the depth image information) to provide a higher degree of security.
  • Face features represent characteristic shapes, distances between shapes, etc. that are present in color images and polarization images respectively.
  • contrast in the color image that allows for identification of a particular face feature may be based on a color information
  • contrast in the polarization image that allows for identification of a particular face feature may be based on a degree of linear polarization (“DOLP”) or on an angle of linear polarization (“AOLP”).
  • Contrast in the polarization image may also be based on a degree of circular polarization (“DOCP”) or on an angle of circular polarization (“AOCP”).
  • DOLP degree of linear polarization
  • AOLP angle of linear polarization
  • Contrast in the polarization image may also be based on a degree of circular polarization (“DOCP”) or on an angle of circular polarization (“AOCP”).
  • DOCP degree of circular polarization
  • AOCP angle of circular polarization
  • polarization image data can be used to determine surface normals of a surface of a captured object. This fact can be used to prevent particular examples of face spoofing.
  • face spoofing attack using a photo of a face of an owner printed on a piece of paper.
  • C image it is sometimes very hard (both for humans and algorithms) to distinguish between an image of an actual face of the owner and an image of a photo of the face of the owner.
  • polarization image data can be used to distinguish between a real face and a photo of a face as shown on a screen.
  • a method comprising: providing a mobile device comprising a sparse color-polarization (SCP) camera that includes a single SCP image sensor configured and operational to capture SPC images, wherein the SCP image sensor includes a first plurality Ni of color pixel units and a second plurality N2 of polarization pixel units, and wherein Ni > 4N2; capturing a SCP image of a user; analysing the captured SCP image to determine whether the user is an owner of the mobile device; and if the user is identified as the owner, unlocking the mobile device.
  • SCP sparse color-polarization
  • the polarization pixel units are operational to capture linear polarization information. In some examples, the polarization pixel units are operational to capture circular polarization information.
  • the SCP images are used to create a color image and a polarization image.
  • the analysing of the captured SCP image includes analysing the color image and analysing the polarization image.
  • the polarization image has a pixel resolution of at least 1200 pixels. In some examples, the polarization image has a pixel resolution of at least 4800 pixels. In some examples, the polarization image has a pixel resolution of at least 30000 pixels.
  • the mobile device has a front surface and a rear surface, the front surface includes a screen, and the SCP camera is located at the front surface.
  • the rear surface includes a multi-camera.
  • the SCP camera includes a lens having an effective focal length (EFL) in the range of 2-10mm.
  • EFL is in the range of 3-6mm.
  • the single SCP image sensor has a full image sensor diagonal (SD) in the range of 3- 17mm. In some examples, the SD is in the range of or 4- 10mm.
  • the single SCP image sensor has a pixel resolution in the range of 5 megapixel (MP) to 400MP. In some examples, the pixel resolution in the range of 8MP to 50MP. In some examples, the pixel resolution is 12MP. In some examples, the pixel resolution is 48MP.
  • the mobile device includes a processor configured to perform the analysing of the captured SCP image.
  • the mobile device includes a memory for storing a first set of face features of the owner.
  • the mobile device includes a communication module configured to for retrieve a first set of face features of the owner.
  • the analysing of the captured SCP image includes calculating a second set of face features and comparing the second set of face features with the first set of face features of the owner.
  • a color-polarization (CP) image sensor comprising: a first plurality Ni of color pixel units, each color pixel unit including at least a first color pixel and a second color pixel; a second plurality N2 of polarization pixel units, each polarization pixel unit including at least a first polarization pixel and a second polarization pixel; and a single metasurface layer, wherein the single metasurface layer covers a photo-conversion area of both the color pixel units and of the polarization pixel units, wherein the single metasurface layer is operational to route light of a first color into a first color pixel, and light of a second color into a second color pixel, and wherein the single metasurface layer is operational to route light of a first polarization into a first polarization pixel, and light of a second polarization into a second polarization pixel.
  • CP color-polarization
  • the single metasurface layer includes nano-pillars made of at least two different materials having two different refractive indices.
  • the CP image sensor is operational to detect three different colors.
  • the three detected colors are red, green and blue.
  • the CP image sensor is operational to detect two different polarization directions. In some examples, the CP image sensor is operational to detect four different polarization directions.
  • the CP image sensor is operational to detect 0-degree linear polarization, 45-degree linear polarization, 90-degree linear polarization and 135-degree linear polarization.
  • the CP image sensor is operational to detect left-handed circular polarization and right-handed circular polarization.
  • the single metasurface layer has a height H in the range of 50nm - 5pm. In some examples, H is in the range of 200nm - 2um. In some examples, a lower refractive index n is in the range of 1 to 2.5, and a higher refractive index nu is in the range of 1.25 to 5. In some examples, n is in the range of 1 to 1.75, and nn is in the range of 2 to 4.
  • the color pixel units include color pixels and the polarization pixel units include polarization pixels, the color pixels and the polarization pixels having an identical pixel pitch P in the range of 0.25pm - 5pm. In some examples, P is in the range of 0.35pm - 2pm.
  • a pixel resolution of the SCP image sensor is in the range of 5MP to 400MP. In some examples, the pixel resolution is in the range of 8MP to 50MP. In some examples, the pixel resolution is 12MP. In some examples, the pixel resolution is 48MP.
  • the CP image sensor is included in a camera that has a lens with an EFL in the range of 2- 10mm. In some examples, EFL is in the range of 3-6mm.
  • the CP image sensor has a full image SD in the range of 3-15mm. In some examples, SD is in the range of or 4- 10mm.
  • the CP image sensor is included in a mobile device.
  • the mobile device has a front surface including a screen, and the CP image sensor is included in a camera located at the front surface.
  • the mobile device is a smartphone. In some examples, the mobile device is a tablet.
  • FIG. 1A shows schematically a mobile device configured to perform face unlock as known in the art
  • FIG. IB shows a method to perform face unlock as known in the art
  • FIG. 1C shows a RGB image sensor as known in the art
  • FIG. ID shows a polarization image sensor as known in the art
  • FIG. IE shows a sparse color-polarization (SCP) image sensor as known in the art
  • FIG. 2A shows an example of a SCP selfie image for performing single-camera face unlock as disclosed herein;
  • FIG. 2B shows an example of a SCP image sensor as disclosed herein in a top view
  • FIG. 2C shows the SCP image sensor of FIG. 2B in a cross-sectional side view
  • FIG. 2D shows a metasurface layer included in the SCP image sensor of FIG. 2B in a cross-sectional side view
  • FIG. 2E shows parts of the metasurface layer of FIG. 2D in a top view
  • FIG. 2F shows other parts of the metasurface layer of FIG. 2D in a top view
  • FIG. 3 shows a method for single-camera face unlock as disclosed herein
  • FIG. 4 shows schematically an embodiment of a mobile device configured to perform single-camera face unlock disclosed herein.
  • FIG. IE shows a known sparse color-polarization (SCP) image sensor 180.
  • SCP image sensor 180 includes single pixels like a single pixel 182 arranged in a first plurality of color pixel units 184 (similar to unit 154) and a second plurality of polarization pixel units 186 (similar to unit 174).
  • color pixel units 184 are RGB pixel units.
  • Polarization pixel units 186 are “sparsely” integrated between color pixel units 184. “Sparsely integrated” refers here to the fact that SCP image sensor 180 includes more color pixel units 184 than polarization pixel units 186.
  • a polarization pixel unit like unit 186 may be also be configured to capture color image data.
  • such polarization pixel units may also be covered with color filters, in addition to polarization filters.
  • a single pixel (not shown) can measure both a polarization signal and a color signal. That is, a single pixel can measure one polarization direction and one color.
  • FIG. 2A illustrates schematically an example of a captured SCP selfie image 200 for performing single-camera face unlock as disclosed herein.
  • SCP selfie image 200 is captured by a sparse CP selfie camera including a SCP image sensor such as SCP image sensor 180.
  • the SCP selfie camera may be a front camera in a mobile device.
  • Captured SCP selfie image 200 may include a face of the user taking the selfie.
  • SCP selfie image 200 includes color (e.g. RGB) image data 202 captured by color pixel units such as units 184 and polarization image data 204 captured by polarization pixel units such as units 186.
  • RGB color
  • polarization image data represented by rectangles 204 is available at only nine locations in SCP selfie image 200. In general, the number of such locations will be much higher, e.g. at least hundreds, or thousands, or even hundreds of thousands of locations in a SCP image.
  • a pixel unit like polarization pixel unit 186 may be considered to be one, single “larger” pixel.
  • pixel unit including one larger pixel, four values (one for each 90°, 45°, 135° and 0°) are available after performing polarization demosaicing.
  • two separate images can be created from SCP selfie image 200, for example, a polarization image that includes only polarization image data, and a color image that includes only color image data.
  • a relatively high sparsity is beneficial for achieving a relatively high image quality (“IQ”), as more color pixel units contribute to the output image.
  • IQ image quality
  • a relatively high sparsity of e.g. N > 64, or N > 256, or N > 1024 or even N > 4096 may be sufficient to achieve a relatively high level of security.
  • a CP image sensor such as SCP image sensor 180 or SCP selfie image sensor 220 or CP image sensor 225 (see below) may have a pixel (or “spatial”) resolution in the range of 2.5 megapixel (MP) to 400MP, typically in the range of 8MP to 50MP, for example 12MP or 48MP.
  • a width : height ratio of the SCP image sensor may be 4:3 or 16:9.
  • a polarization image that includes only polarization image data may have a pixel resolution in the range of about 500 pixel to 15MP.
  • a pixel resolution may be about 40x30 (1200 pixel) or about 80x60 (4800 pixel) or about 200x150 (30000 pixel) etc.
  • a camera including a CP or a SCP image sensor may include a lens having an EFL in the ranges of 2mm - 15mm or 3mm - 6mm, and the CP or the SCP image sensor may have a (full) image sensor diagonal SD in the ranges of 2mm - 17mm or 4mm - 10mm.
  • FIG. 2B shows an example of a SCP selfie image sensor disclosed herein and numbered 220 in a top view.
  • SCP selfie image sensor 220 includes color image sensor regions 222 which include color pixel units as disclosed herein, and polarization image sensor regions 224 which include polarization pixel units as disclosed herein.
  • FIG. 2C shows exemplary a color-polarization image sensor 225 as disclosed herein in a cross-sectional side view.
  • the cross section shows a first color pixel unit 226, a polarization pixel unit 228 and a second color pixel unit 230.
  • the pixel units include a metasurface layer 232, a (optional) spacer layer 234, a (optional) filter layer 236, and a pixel layer 238.
  • Color image sensor regions 222 and polarization image sensor regions 224 are both covered by metasurface layer 232.
  • metasurface layer 232 includes a first color-router 240 in first color pixel unit 226, a polarization-router 242 in polarization pixel unit 228 and a second color-router 244 in second color pixel unit 230.
  • First color-router 240 and second color-router 244 may be “metasurface color-routers” (or “metasurface nano-prisms”) as known in the art. Metasurface color-routers are operational to route color to a particular pixel region or into a particular pixel.
  • Metasurface color-routers are described for example in “Optical design of dispersive metasurface nano-prism structure for high sensitivity CMOS image” by Chulsoo Choi et al., IEEE International Electron Devices Meeting (IEDM), December 9- 13,.2022023. Metasurface color-routers may be used instead of, or in conjunction with, color filters, which is beneficial, since a higher amount of light signal can be harvested by a CP selfie image sensor 225 with metasurface color-filters when compared to color image sensor regions using (only) color filters.
  • Metasurface lenses that route particular polarization directions into a particular image sensor region are known, for example as used in the product “Polar ID” by company Metalenz.
  • Polarization-router 242 may include “metasurface polarization-routers” (or simply “polarization-routers”), i.e. polarization- sensitive metalenses, that route particular polarization directions into a particular pixel region or into a particular pixel.
  • Polarizationrouters may be used instead of, or in conjunction with, polarization filters, which is beneficial as a higher amount of light signal can be harvested by a SCP selfie image sensor 220 with polarization-router when compared to polarization image sensor regions using (only) polarization filters. Both color-routers and polarization-routers may be referred to as “nanorouters”.
  • CP image sensor 225 may be a sparse CP image sensor, e.g. like SCP image sensor 220.
  • CP image sensor 225 may include a first plurality Ni of color pixel units and a second plurality N2 of polarization pixel units, and wherein Ni > N2.
  • CP image sensor 225 may not be a sparse CP image sensor.
  • CP image sensor 225 may be included in a camera located at a front surface such as front surface 104. In other examples, CP image sensor 225 may be included in a camera located at a rear surface of a mobile device such as rear surface 102.
  • a CP image sensor such as SCP image sensor 180 or SCP image sensor 220 or CP image sensor 225 may be operational to harvest visible light.
  • a SCP image sensor may be operational to harvest light in the near infrared (“NIR”) range, i.e. light in a wavelength range of about 750nm - lOOOnm.
  • NIR near infrared
  • a CP image sensor may be operational to harvest light in the short-wave infrared (“SWIR”) range, i.e. light in a wavelength range of about lOOOnm - 1800nm.
  • a CP image sensor may be operational to harvest visible light, IR light and SWIR light, i.e. light in a wavelength range of about 450nm - 1800nm.
  • “color” may refer to a particular wavelength band within the visible and NIR and SWIR wavelength region respectively.
  • Optional spacer 234 may be used to allow sufficient spatial splitting of, respectively, the spectral and polarization components of the light.
  • optional filter 236 may be a known color filter that compensates for a non-ideal spatial splitting of the light’s spectral components.
  • optional filter 236 may be a known polarization filter that compensates for a non-ideal spatial splitting of the light’ s polarization components.
  • optional filter 236 may be a color filter in color pixel units, and it may be a polarization filter in polarization pixel units. Photon-electron conversion (or “photon harvesting”) takes place in pixel layer 238, as known.
  • FIG. 2C shows schematically rays of green light (G) and of red light (R) impinging on CP image sensor 225 from a direction parallel to the z-axis.
  • first color pixel unit 226 and second color pixel unit 230 include a G pixel 246 and a G pixel 254 as well as a R pixel 248 and a R pixel 256 respectively.
  • polarization pixel unit 228 includes a 90-degree linear polarization (“90°”) pixel 250 as well as a 45-degree linear polarization (“45°”) pixel 252.
  • any G light (solid line) impinging on first color pixel unit 226 and second color pixel unit 230 is routed (or “directed”) respectively into G pixel 246 and G pixel 254, and any R light (dotted line) impinging on first color pixel unit 226 and second color pixel unit 230 is routed respectively into R pixel 248 and R pixel 256.
  • any 90-degree linearly polarized light (solid line) impinging on polarization pixel unit 228 is routed into 90° pixel 250, and any 45-degree linearly polarized light (dotted line) impinging on polarization pixel unit 228 is routed into 45° pixel 252.
  • a pixel pitch (or “pixel width”) of a single pixel is indicated by “P” and may be in the range of 0.25pm - 5pm, preferably in the range of 0.35pm - 2pm, e.g. 0.5pm, or 0.6pm or 0.7pm etc.
  • FIG. 2D shows exemplarily a metasurface layer 232 disclosed herein and operational to be used in a CP image sensor like CP image sensor 225 in a cross-sectional side view.
  • metasurface layer 232 has a symmetry axis which is oriented parallel to the x-axis.
  • metasurface layer 232 is created (or manufactured) by forming nanostructures (also referred to as “nano-pillars” or “nano-rods”) of two or more different materials.
  • a first type of nanostructure 258 may be made of a first material having a relatively low refractive index (“HL”) and a second type of nanostructure 260 may be made of a second material having a relatively high refractive index (“UH”).
  • HL relatively low refractive index
  • UH relatively high refractive index
  • each white region of metasurface layer 232 represents the first type of nanostructure 258, and each black region of metasurface layer 232 represents the second type of nanostructure 260. Two regions, each including both types of nanostructures 258 and 260 are indicated as an example.
  • a height (“H”) of metasurface layer 232 (and of the nanostructure) may be in the range of 50nm - 5p m, preferably in the range of 200nm - 2pm.
  • a width (“W”) of an individual nanostructure may be in the range of 20nm - 1pm, preferably in the range of 50nm - 500nm.
  • both color-routers 240 and 244 as well as polarizationrouter 242 are made (or “manufactured”) from the same two or more different materials.
  • the functionality of color-routing and polarization-routing is primarily obtained by a geometry of the nanostructure (FIG. 2E), as well as by an interaction of the nano-pillars with each other, and not necessarily by the material combination.
  • metasurface layer 232 can provide color-routing as well as polarization-routing capabilities.
  • this is beneficial for manufacturing a CP image sensor 225 disclosed herein, because it allows the manufacturing of color-routers 240 and 244 as well as of polarization-router 242 in one or more identical process steps required to manufacture the single metasurface layer. That is, identical process steps can be used to manufacture both color-routers 240 and 244 as well as polarization-router 242.
  • one layer of nanostructures is used in metasurface layer 232.
  • a metasurface layer such as metasurface layer 232 may include two or more layers of nanostructures, i.e. a stack of nanostructures. Also to these other examples, we refer as “single” or “continuous” metasurface layer, if the functionality of color-routing and polarization-routing is introduced in identical process steps, and in particular while using the same two or more different materials.
  • FIG. 2E shows exemplarily color-router 240 in a top view.
  • color-router 240 may spatially split light into three different colors, R, G and B.
  • FIG. 2F shows exemplarily polarization-router 242 in a top view.
  • polarization-router 242 may spatially split light into four different linear polarizations, into 90°, 45°, 0° and 135°.
  • nanostructures included in color-router 240 and in polarization-router 242 respectively show structural differences.
  • nanostructures included in color-router 240 may predominantly include geometries that are rotational- symmetric along the z-axis.
  • Nanostructures included in polarization-router 242 may predominantly include geometries which are not rotational-symmetric along the z-axis, but have a preferred directionality in the x-y plane.
  • Predominantly means here that in a colorrouter there may be a larger share of rotational-symmetric geometries than in a polarizationrouter, and vice versa, that in a polarization-router there may be a larger share of preferred directionality geometries than in a color-router. Still, preferred directionality geometries may be included in the color-router, and rotational-symmetric geometries may be included in the polarization-router.
  • FIG. 3 shows a method for single-camera face unlock disclosed herein and numbered 300.
  • a user approaches a mobile device configured to perform single-camera face unlock.
  • the mobile device captures a SCP selfie image 200 (or a stream of selfie SCP selfie images) including a face such as face 114 of the user.
  • the mobile device analyses a color image created from SCP selfie image 200 to identify the face of an owner of the mobile device.
  • the mobile device analyses a polarization image created from SCP selfie image 200 to identify the face of an owner of the mobile device.
  • a set of face features may be extracted from SCP selfie image 200, and the set of face features may be compared to the face features of the owner.
  • the face features extracted in step 306 and 308 are different, which is beneficial for achieving a relatively high level of security.
  • a detection of face spoofing may be performed. Performing the detection of face spoofing may include analysing a polarization image to distinguish between a real face and a photo of a face as shown on a screen or a photo of a face printed on a piece of paper.
  • step 310 in case the mobile device found that a set of face features extracted in steps 306 and 308 is identical with face features of the owner of the mobile device, the mobile device is unlocked. In case the mobile device found that a set of face features extracted in steps 306 and 308 differs from face features of the owner of mobile device, the mobile device is not unlocked.
  • the mobile device analyses SCP selfie image 200 as is.
  • color image data and polarization image data may be analyzed in one single step.
  • the mobile device creates a fused CP image in a first sub-step.
  • color image data is used to estimate (or artificially generate) polarization image data which is actually not present in SCP selfie image 200.
  • polarization image data is used to estimate color image data that is not actually present in SCP selfie image 200, after which a fused CP image is created.
  • seven values (one for each R, G, B, 90°, 45°, 135° and 0°) may be available for each pixel of the fused CP image. One may refer to the latter as a “dense CP image”.
  • the mobile device analyses the created image to identify a face of the owner of the mobile device.
  • image processing methods known in the art such as “super-resolution” or “upsampling” are used to create a “high-resolution color image”, i.e. a color image with elevated pixel resolution.
  • the created color image may have a pixel resolution identical to the SCP selfie image pixel resolution.
  • the mobile device analyses the high- resolution color image to identify a face of an owner of the mobile device.
  • the created polarization image may have a pixel resolution identical to the SCP selfie image pixel resolution.
  • the mobile device analyses the high-resolution polarization image to identify a face of an owner of the mobile device.
  • An advantage of method 300 over known single-camera face unlock methods is that it provides a higher degree of security.
  • the higher degree of security is achieved by using additional image information.
  • the additional image information is polarization image information.
  • An advantage of method 300 over the known dual-camera face unlock is that it can be implemented using relatively simple (or inexpensive) and compact camera hardware. Specifically, only one camera is required (instead of two cameras), which is beneficial in terms of cost, and it covers a smaller area of a screen such as screen 124, which is beneficial in terms of industrial design.
  • FIG. 4 shows schematically an embodiment of a mobile device (for example, a smartphone) numbered 400 and configured to perform single-camera face unlock disclosed herein.
  • Mobile device 400 comprises a selfie camera 410 having a selfie camera FOVs and including an image sensor 412.
  • Image sensor 412 is a sparse CP image sensor such as SCP image sensor 180 or SCP image sensor 220 or CP image sensor 225 as defined above.
  • mobile device 400 further comprise a second camera, e.g. a multi-camera as known in the art.
  • a sparse CP image sensor may be integrated in the multi-camera.
  • Mobile device 400 further includes an application processor (AP) 420.
  • AP 420 includes a face detector 422, e.g. configured to perform face detection, a color face identifier 424, e.g. configured to identify a face of an owner of mobile device 400 in a color image, a polarization face identifier 426, e.g. configured to identify a face of an owner of mobile device 400 in a polarization image and/or to detect face spoofing, and an image sensor controller 428.
  • Mobile device 400 includes in addition a screen 430 and a memory 440. Memory 440 may e.g. be used to store a set of face features of an owner of mobile device 400.
  • mobile device 400 may in addition include a communication module, operational to retrieve a set of face features of an owner of mobile device 400.
  • the communication module may retrieve the set of face features of an owner of mobile device 400 from the internet, from a server, from a cloud etc.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Optics & Photonics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Toxicology (AREA)
  • Color Television Image Signal Generators (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne des procédés d'identification du propriétaire d'un appareil mobile dans le but de déverrouiller l'appareil mobile, consistant à : inclure dans le dispositif mobile un appareil de prise de vues à polarisation de couleurs clairsemées (SCP) qui possède un seul capteur d'image SCP configuré et opérationnel pour capturer des images SCP, le capteur d'image SCP comprenant une première pluralité N1 d'unités de pixels de couleur et une deuxième pluralité N2d'unités de pixels de polarisation, avec N1 ≥ 4N2 ; utiliser l'appareil de prise de vues SCP pour capturer une image SCP d'un utilisateur ; analyser l'image SCP capturée pour déterminer si l'utilisateur est le propriétaire du dispositif mobile ; et si l'utilisateur est identifié comme étant le propriétaire, déverrouiller le dispositif mobile.
EP24703633.8A 2023-02-02 2024-01-31 Capteurs, systèmes et procédés pour appareils de prise de vues à polarisation d'identification de visage compacts Pending EP4659125A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363482847P 2023-02-02 2023-02-02
PCT/IB2024/050895 WO2024161329A1 (fr) 2023-02-02 2024-01-31 Capteurs, systèmes et procédés pour appareils de prise de vues à polarisation d'identification de visage compacts

Publications (1)

Publication Number Publication Date
EP4659125A1 true EP4659125A1 (fr) 2025-12-10

Family

ID=89845342

Family Applications (1)

Application Number Title Priority Date Filing Date
EP24703633.8A Pending EP4659125A1 (fr) 2023-02-02 2024-01-31 Capteurs, systèmes et procédés pour appareils de prise de vues à polarisation d'identification de visage compacts

Country Status (4)

Country Link
EP (1) EP4659125A1 (fr)
KR (1) KR20250126122A (fr)
CN (1) CN120641897A (fr)
WO (1) WO2024161329A1 (fr)

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6904431B2 (ja) * 2017-11-09 2021-07-14 ソニーグループ株式会社 情報処理装置、プログラム及び情報処理方法
US11841522B2 (en) 2020-04-03 2023-12-12 Samsung Electronics Co., Ltd. On-chip simultaneous full stokes polarization (linear + circular) and (multi/hyper) spectral imaging

Also Published As

Publication number Publication date
CN120641897A (zh) 2025-09-12
WO2024161329A1 (fr) 2024-08-08
KR20250126122A (ko) 2025-08-22

Similar Documents

Publication Publication Date Title
US7855786B2 (en) Single camera multi-spectral imager
US11747533B2 (en) Spectral sensor system using optical filter subarrays
EP2721828B1 (fr) Capture d'image multispectrale de haute résolution
US8786741B2 (en) Digital multi-spectral camera system having at least two independent digital cameras
TWI566391B (zh) 影像感測器
US20130222603A1 (en) Imaging systems for infrared and visible imaging
US10313643B2 (en) Imaging system having four image sensors
US20090147112A1 (en) Method and apparatus for achieving panchromatic response from a color-mosaic imager
EP3133812A1 (fr) Appareil photo-détecteur, appareil de capture d'image à semi-conducteurs et leurs procédés de fabrication
CN105988215B (zh) 一种多光谱模组成像系统及其制造方法和应用
CN105021282B (zh) 快照式偏振高光谱相机及成像方法
TW201721113A (zh) 多點光譜系統
TWI514894B (zh) 攝取數位影像中的白平衡校正技術
WO2020155739A1 (fr) Capteur d'image, procédé d'acquisition de données d'image à partir d'un capteur d'image, et dispositif de caméra
JP2018513964A5 (ja) スナップショット型偏光ハイパースペクトルカメラ及び画像化方法
US11696043B2 (en) White balance compensation using a spectral sensor system
CN113973197B (zh) 像素结构、像素阵列、图像传感器及电子设备
US20150070466A1 (en) Camera Devices And Systems Based On A Single Image Sensor And Methods For Manufacturing The Same
JP2020120204A (ja) 画像処理装置、画像処理方法およびプログラム
WO2024161329A1 (fr) Capteurs, systèmes et procédés pour appareils de prise de vues à polarisation d'identification de visage compacts
US9110293B2 (en) Prismatic image replication for obtaining color data from a monochrome detector array
CN101036380A (zh) 创建彩色图像的方法、成像设备和成像模块
WO2017181381A1 (fr) Procédé et dispositif photographique permettant d'acquérir des informations de profondeur
US20080173795A1 (en) Image sensor
JP2010276469A (ja) 画像処理装置及び測距装置の画像処理方法

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250806

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR