WO2022250986A1 - Selective obfuscation of objects in media content - Google Patents
Selective obfuscation of objects in media content Download PDFInfo
- Publication number
- WO2022250986A1 WO2022250986A1 PCT/US2022/029288 US2022029288W WO2022250986A1 WO 2022250986 A1 WO2022250986 A1 WO 2022250986A1 US 2022029288 W US2022029288 W US 2022029288W WO 2022250986 A1 WO2022250986 A1 WO 2022250986A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- objects
- media data
- data
- media
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09C—CIPHERING OR DECIPHERING APPARATUS FOR CRYPTOGRAPHIC OR OTHER PURPOSES INVOLVING THE NEED FOR SECRECY
- G09C5/00—Ciphering apparatus or methods not provided for in the preceding groups, e.g. involving the concealment or deformation of graphic data such as designs, written or printed messages
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
- H04L2209/16—Obfuscation or hiding, e.g. involving white box
Definitions
- Techniques are provided herein for providing automatic obfuscation of one or more objects identified within media content (e.g., an image or video). Such techniques may be performed on media data (e.g., audio data, video data, still images, etc.) received from a number of data sources (e.g., public cameras, body-mounted cameras utilized by law enforcement, vehicle- mounted cameras, etc.) to automatically obfuscate objects within that media data.
- media data e.g., audio data, video data, still images, etc.
- data sources e.g., public cameras, body-mounted cameras utilized by law enforcement, vehicle- mounted cameras, etc.
- a method is disclosed as being performed by a media obfuscation platform, the method comprising receiving, from a data source, a media data comprising a depiction of a number of objects, identifying, within the media data, a set of objects associated with the media data, storing an indication of one or more locations of the objects in the set of objects within the media data with respect to time, receiving a request for the media data, updating the media data by applying an obfuscation effect to the one or more locations with respect to time, and providing the updated media data in response to the request.
- An embodiment is directed to a computing device comprising: a processor; and a memory including instructions that, when executed with the processor, cause the computing device to receive, from a data source, a media data comprising a depiction of a number of objects, identify, within the media data, a set of objects associated with the media data, store an indication of one or more locations of the objects in the set of objects within the media data with respect to time, receive a request for the media data, update the media data by applying an obfuscation effect to the one or more locations with respect to time, and provide the updated media data in response to the request.
- An embodiment is directed to a non-transitory computer-readable media collectively storing computer-executable instructions that upon execution cause one or more computing devices to perform acts comprising receiving, from a data source, a media data comprising a depiction of a number of objects, identifying, within the media data, a set of objects associated with the media data, storing an indication of one or more locations of the objects in the set of objects within the media data with respect to time, receiving a request for the media data, updating the media data by applying an obfuscation effect to the one or more locations with respect to time, and providing the updated media data in response to the request.
- Embodiments of the disclosure provide several advantages over conventional techniques.
- embodiments of the proposed system provide for automatic obfuscation of objects within media data.
- Systems in which manual obfuscation of objects in media content (e.g., by an administrator) is performed can be very expensive from a resource perspective, requiring extensive review of each media data to be obfuscated. This is especially true when the media data is lengthy and needs to be reviewed several times.
- Embodiments of the current system remove this requirement, freeing up valuable resources.
- FIG. 1 illustrates a computing environment configured to provide automated obfuscation of objects within media content in accordance with at least some embodiments
- FIG. 2 is a block diagram showing various components of a computing system architecture that supports obfuscation of objects within media data in accordance with some embodiments
- FIG. 3 depicts a block diagram showing an example process flow for correlating media data with events in accordance with embodiments
- FIG. 4 depicts an illustrative example of a listing of objects that may be maintained with respect to a media data in accordance with at least some embodiments
- FIG. 5 depicts an illustrative example of a mapping of objects to a media data in accordance with at least some embodiments
- FIG. 6 depicts an illustrative example of various elements of a graphical user interface (GUI) that may be implemented in accordance with at least some embodiments; and [0010]
- FIG. 7 depicts a block diagram showing an example process flow for automatically identifying and obfuscation objects in a media data in accordance with embodiments.
- GUI graphical user interface
- Described herein are techniques that may be used to automatically identify and obfuscate objects within media data. Such techniques may be applied to various types of media data received from a number of external data sources. In some embodiments, one or more object recognition techniques may be used to identify and track objects depicted within the media data. Information about the identified objects (e.g., times and corresponding locations) may be stored with respect to a particular media data.
- an obfuscation process may be performed. To do this, a determination may be made as to which objects should be obfuscated and which objects should remain unobfuscated, i.e., in the clear. In some embodiments, such a determination may be made based on a role or the requesting entity, a role of the object (with respect to the media data or an event), a geographic location associated with the media data, or any other suitable factor. For each object determined to require obfuscation, an obfuscation effect may be applied at times and locations at which that object is depicted within the media data.
- the media data may be encoded (e.g., using a codec) to prevent the obfuscation effect from being removed.
- the obfuscated media data may then be provided to the requesting entity in response to the received request.
- FIG. 1 illustrates a computing environment configured to provide automated obfuscation of objects within media content in accordance with at least some embodiments.
- a media obfuscation platform 102 is depicted in communication with a number of electronic device components, such as one or more independent data sources 106, and one or more third-party devices 108.
- the media obfuscation platform 102 can include any computing device configured to perform at least a portion of the operations described herein.
- Media obfuscation platform 102 may be composed of one or more general purpose computers, specialized server computers (including, by way of example, PC (personal computer) servers, UNIXTM servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other appropriate arrangement and/or combination.
- Service provider computer 108 can include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for the computer.
- the media obfuscation platform 102 may maintain a number of components, including an object obfuscation engine 110 capable of identifying and obfuscating objects identified within media content as well as a request processing engine 112 capable of receiving and responding to requests for media data.
- the object obfuscation engine 110 may maintain access to a database of information related to one or more identified objects and corresponding media (e.g., object / media data 114).
- the object obfuscation engine 110 may be configured to identify a number of objects depicted within a video or image using one or more object recognition techniques. The object obfuscation engine 110 then determines which of the identified objects should, or should not, be obfuscated within the media data. In some embodiments, such a determination may be made based on an identity of an entity from which the request was received. Once a determination is made to obfuscate a particular object depicted in the media data, the object may be tracked within the media data with respect to time. One or more obfuscation techniques (e.g., blurring) may be used to obfuscate the tracked object.
- obfuscation techniques e.g., blurring
- a data source 106 may comprise any suitable electronic device capable of being used to record information about a person and/or vehicle’s surroundings.
- the recording device may be a camera mounted within a vehicle.
- the recording device may be a device that is capable of being worn or otherwise mounted or fastened to a person.
- the data source 106 may include at least one input device, such as a microphone or camera, and a number of sensors capable of obtaining data about an environment in which the recording device is located.
- such data sources may include wearable recording devices, vehicle-mounted recording devices, private security network devices, public network cameras, or any other suitable data collection device.
- Each of the data sources 106 may be configured to collect a variety of media information, such as images, audio, and/or video.
- one or more of the data sources may be operated by an entity unaffiliated with the media obfuscation platform 102.
- the data sources may have a number of sensors that include a temperature sensor, a real-time clock (RTC), an inertial measurement unit (IMU), or any other suitable sensor.
- An IMU may be any electronic device that measures and reports a body's specific force, angular rate, and sometimes the orientation of the body, using a combination of accelerometers, gyroscopes, and magnetometers.
- the computing environment may be in communication with one or more third-party computing devices 108.
- a third-party computing device may be a computing device operated on behalf of an entity unaffiliated with the media obfuscation platform.
- the third-party computing device may submit a request 118 to the media obfuscation platform for media data related to an event.
- such a request may be a request that complies with a law or legal requirement granting information access rights (e.g., the Freedom of Information Act).
- the media obfuscation platform may identify any relevant media data, obfuscate one or more objects depicted within the media data, and provide such media data 120 to the third-party device with access to that that event data.
- video data i.e., media data
- the video data may be associated with an event, such as an event generated in association with a request submitted for emergency services.
- the media obfuscation platform may use object recognition techniques to identify and classify objects detected within the video.
- a request may be received for information related to the event.
- Such a request may be received from a third-party entity that is unaffiliated with the media obfuscation platform.
- the media obfuscation platform may obscure one or more objects within the video and reencode the obscured video. The video with obscured objects is then provided to the entity from which the request was received.
- FIG. 1 For clarity, a certain number of components are shown in FIG. 1. It is understood, however, that embodiments of the disclosure may include more than one of each component. In addition, some embodiments of the disclosure may include fewer than or greater than all of the components shown in FIG. 1. In addition, the components in FIG. 1 may communicate via any suitable communication medium (including the Internet), using any suitable communication protocol.
- any suitable communication medium including the Internet
- FIG. 2 is a block diagram showing various components of a computing system architecture that supports obfuscation of objects within media data in accordance with some embodiments.
- the system architecture 200 may include a media obfuscation platform 102 that comprises one or more computing devices.
- the media obfuscation platform 102 may include a communication interface 202, one or more processors 204, memory 206, and hardware 208.
- the communication interface 202 may include wireless and/or wired communication components that enable the media obfuscation platform 102 to transmit data to, and receive data from, other networked devices.
- the hardware 208 may include additional user interface, data communication, or data storage hardware.
- the user interfaces may include a data output device (e.g., visual display, audio speakers), and one or more data input devices.
- the data input devices may include, but are not limited to, combinations of one or more of keypads, keyboards, mouse devices, touch screens that accept gestures, microphones, voice or speech recognition devices, and any other suitable devices.
- the media obfuscation platform 102 can include any computing device or combination of computing devices configured to perform at least a portion of the operations described herein.
- the media obfuscation platform 102 may be composed of one or more general purpose computers, specialized server computers (including, by way of example, PC (personal computer) servers, UNIX® servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other appropriate arrangement and/or combination.
- the media obfuscation platform 102 can include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for the computer.
- the media obfuscation platform 102 may include virtual computing devices in the form of virtual machines or software containers that are hosted in a cloud.
- the memory 206 may be implemented using computer-readable media, such as computer storage media.
- Computer-readable media includes, at least, two types of computer- readable media, namely computer storage media and communications media.
- Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.
- Computer storage media includes, but is not limited to, RAM, DRAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device.
- communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanisms.
- a modulated data signal such as a carrier wave, or other transmission mechanisms.
- the 102 may implement functionality from one or more software modules and data stores. Such software modules may include routines, program instructions, objects, and/or data structures that are executed by the processors 204 to perform particular tasks or implement particular data types.
- the memory 206 may include at least a module for identifying, tracking, and obfuscating objects within media data (e.g., object obfuscation engine 110) as well as a module for receiving and responding to requests (e.g., request processing engine 112). Additionally, the memory 206 may further include a data store 210 that contains a number of database tables or other data storage schemas.
- the data store 210 may include at least a database of media data received from various data sources (media data 212) and a database of objects detected with respect to the media data.
- the object obfuscation engine 110 may be configured to, in conjunction with the processor 204, identify one or more objects depicted within a media file (e.g., a video file).
- a media file e.g., a video file
- the identified objects may be classified and/or tracked in that a location of the object within the media file may be indicated with respect to time. For example, multiple objects may be identified as being depicted within a particular media file.
- an identifier may be stored for each object as well as an indication of a series of locations (e.g., within a graphical user interface presentation) at which the respective object is depicted at a corresponding series of times.
- an object may be identified using one or more object recognition algorithms, such as a facial recognition algorithm, a pattern recognition algorithm, a template matching algorithm, and/or so forth.
- the object recognition algorithms may implement the use of machine-learning models.
- the identified object may be tracked across multiple video frames of video data using a video tracking algorithm.
- the video tracking algorithm may include a kernel-based tracking function that performs an iterative localization procedure based on the maximization of a similarity measure for the video frames.
- the video tracking algorithm may include a contour tracking function that detects object contour boundaries and iteratively evolves an initial contour from a previously video frame to its new position in a current video frame.
- the request processing engine 112 may be configured to, in conjunction with the processor 204, respond to a request received from a third-party computing device.
- the request processing engine may retrieve media data that has been obfuscated to protect the privacy of more or more individuals associated with the media.
- the request processing engine 112 may be configured to perform additional processing on the media data. For example, before responding to a request for media data related to a particular event, the request processing engine 112 may be configured to call the object obfuscation engine 110 in order to obfuscate images of innocent (or at least non-relevant) parties within a video file.
- the request processing engine 112 may provide a data file to the requesting entity that includes a separate copy of any obfuscated media data. In other embodiments, the request processing engine 112 may provide the requesting entity with a link (or other suitable location reference) that the requesting entity can use to access the obfuscated media data.
- the communication interface 202 may include wireless and/or wired communication components that enable the media obfuscation platform to transmit or receive data via a network, such as the Internet, to a number of other electronic devices (e.g., data source 106). Such a communication interface 202 may include access to both wired and wireless communication mechanisms. In some cases, the media obfuscation platform transmits data to other electronic devices over a long-range communication channel, such as a data communication channel that uses a mobile communications standard (e.g., long-term evolution (LTE)).
- LTE long-term evolution
- FIG. 3 depicts a block diagram showing an example process flow for correlating media data with events in accordance with embodiments.
- the process 300 involves interactions between various components of the architecture 100 described with respect to FIG. 1. More particularly, the process 300 involves interactions between at least a media obfuscation platform 102, at least one data source 106, and one or more third-party devices 108.
- media data is received from one or more data sources. Such data sources may include street cameras, body-mounted recording devices, vehicle-mounted cameras, or any other suitable recording device in communication with a media obfuscation platform.
- the received media data may include videos, images, and/or audio data.
- one or more objects may be identified as being depicted within the media data. In some embodiments, this may involve the use of one or more object recognition techniques to identify and classify objects depicted within the media data. In some cases, the object may be classified as belonging to a broad category of object. In some cases, the object may be classified as being a particular object. By way of illustration, in some cases an object may be identified as being a person whereas in other cases the object may be identified as being a particular person (e.g., using facial recognition). In some embodiments, trained machine learning models may be used to perform object recognition.
- a first trained machine learning model may be used to determine a broad category into which the object belongs and a second trained machine learning model may be used to determine a specific type/identity of the object within the broad category.
- the one or more identified objects may be tracked throughout the media data. For example, for each object identified in the media data, a series of locations and times associated with that object may be stored. In this example, each of the series of locations and times may correspond to a frame in a video (where the media data is a video). In some embodiments, such as when the media data is a still image, the object may be associated with a single location within the media data.
- each of the identified objects is assigned a role based on information stored in relation to the media data. For example, a number of people may be depicted in a video that relates to a crime that was committed. With respect to that crime in this example, some of the people may be assigned a role as a bystander, some of the people may be assigned a role as a victim, and some of the people may be assigned a role as a suspect.
- a request may be received for information related to a particular object (e.g., a particular person) or related to a particular event or incident.
- the request may be categorized based on a type of the received request and/or a role of the entity from which the request was received (e.g., reporter, prosecutor, defense attorney, etc.).
- a role of the entity from which the request was received e.g., reporter, prosecutor, defense attorney, etc.
- one or more obfuscation rules may be determined.
- such obfuscation rules may be received from a user (e.g., via the request for media at 308).
- the user may be presented with an indication of each of the objects that have been identified within the media data and may be provided the ability to select whether at least a portion of those objects are obfuscated within the media data that is to be provided in response to the request.
- a number of rules may be maintained by a media obfuscation platform that provide for automatic obfuscation of one or more objects based on a role of the requesting entity and/or a role of an object with respect to the media data. For example, one rule may dictate that if the requesting entity is a member of the public that is not in law enforcement, then any person (i.e., an object) that is not a suspect (e.g., a victim or a bystander) should be obfuscated automatically.
- An obfuscation process may then be performed for each of the objects identified with respect to a media data.
- a role or category for each object in the media may also be identified.
- a determination may be made with respect to each of the identified objects as to whether the respective object should be obfuscated. In some embodiments, such a determination may be made based on an identified role of the object with respect to the media. For example, all objects that are identified as people may be categorized as victims, bystanders, or suspects. In this example, objects identified as victims or bystanders may be obfuscated whereas all objects identified as suspects may remain unobfuscated.
- one or more rules may be maintained that dictate circumstances under which a number of non-people objects (e.g., drivers’ licenses, vehicle license plates, addresses, etc.) should be obfuscated.
- a determination as to whether a particular object should be obfuscated may be made based on a selection received from a user (e.g., an administrator or the requesting entity).
- the respective object may be obfuscated at 314. In some cases, this comprises creating an obfuscation effect at a location associated with the identified object.
- Such an obfuscation effect may be any suitable effect that can obfuscate the object.
- a suitable obfuscation effects may include blur effects such as stack blur, gaussian blur, motion blur, box blur, radial blur, or heavy radial blur effects.
- Other examples of obfuscation effects may include pixilation, of the object, replacing the object with another image (black bar, dot, etc.), or otherwise distorting the object at the location.
- the media data may be encoded at 318.
- this is done using a codec (e.g., a video codec) that is configured to compress the media data that includes the obfuscation effects.
- a video codec may be used to compress an altered video data into a format that conforms to a standard video coding format. Compressing the media data in this manner can prevent a downstream user from removing an applied obfuscation effect from the obfuscated objects.
- FIG. 4 depicts an illustrative example of a listing of objects that may be maintained with respect to a media data in accordance with at least some embodiments.
- one or more object recognition techniques may be used to identify a number of objects 402 that are depicted throughout that media.
- Each of the number of objects 402 may be assigned an identifier 404 that can be used identify that object.
- such an identifier may comprise an identifier that correlates the object within other media data received by a media obfuscation platform.
- the media obfuscation platform may maintain a set of information for each object identified within various media data that it receives.
- a determination may be made during an object recognition process that characteristics associated with an identified object match characteristics associated with an object that has previously been identified in relation to another media data.
- the identified object may then be assigned the identifier of the object previously identified in relation to the other media data.
- a media data may be associated with one or more events or incidents 406.
- such an event may correspond to a case file maintained by a law enforcement agency or a medical care provider.
- a request submitted to the media obfuscation platform for information may be submitted in relation to a specified event.
- the media obfuscation platform then identifies and performs obfuscation techniques with respect to each of the media data that relate to the specified event.
- each of the identified objects may be assigned a role 408.
- an object’s role may be specific to a particular media data and/or a particular incident.
- an object identified within both a first media data and a second media data may be associated with a first role with respect to the first media data and a second role with respect to the second media data.
- a person identified within both a first video and a second video may be assigned a role as a suspect in the first video and a role as a bystander in the second video.
- rules that are used to determine whether the object should be obfuscated in that media data may make such a determination based on a role of the object with respect to that media data.
- the media obfuscation platform may identify a set of objects that are not to be obfuscated in relation to a piece of media data.
- object recognition may be performed to identify a number of different objects depicted within the media data of a particular type (e.g., people).
- characteristics of each of the identified objects may be compared to corresponding characteristics of the objects in the set of objects. If a match is identified between the characteristics of an identified object and the characteristics of at least one object in the set of objects, then no obfuscation is applied to that object.
- FIG. 5 depicts an illustrative example of a mapping of objects to a media data in accordance with at least some embodiments.
- a timeline 502 may be maintained with respect to a media data.
- a number of objects e.g., Object l and Object_2
- each object may be associated with at least a beginning point in time and an ending point in time.
- the beginning point in time may correspond to a point on the timeline at which the respective object is first determined to be depicted within the media data and the ending point in time may correspond to a point on the timeline at which the respective object is last determined to be depicted within the media data.
- Each object may be associated with a different beginning point in time and ending point in time.
- Object l may be associated with a beginning point in time 504 and an ending point in time 506 whereas Object_2 may be associated with a beginning point in time 508 and an ending point in time 510.
- a mapping 512 may be maintained with respect to the media data that stores information about one or more objects identified within that media data.
- the mapping may include a series of times 514 and corresponding locations 516 of an object at the respective times.
- the series of times 514 may include any suitable indication of a position within a timeline associated with the media data.
- a time within the series of times 514 may be representative of a particular point in time (e.g., a timestamp) that corresponds to the timeline 502 associated with the media data.
- a time within the series of times 514 may be representative of a particular frame within a video.
- the locations 516 may include any suitable indication of a position within the content of the media.
- the location In the case of a video (i.e., a series of images) or still image, the location may be representative of horizontal (e.g., X-axis) and vertical (e.g., Y-axis) coordinates of the object as identified within an image. In some embodiments, the location may be representative of a particular pixel or set of pixels within an image.
- the mapping 512 may be used by an object obfuscation engine (e.g., object obfuscation engine 110 of FIG. 1) during an obfuscation process.
- the object obfuscation engine may generate an obfuscation effect of a size roughly corresponding to that of the object to be obfuscated.
- the obfuscation effect is then positioned at each of the locations 516 within the media data during each of the respective times in the series of times 514, causing the portion of the media data at the respective position to be obfuscated.
- FIG. 6 depicts an illustrative example of various elements of a graphical user interface (GUI) that may be implemented in accordance with at least some embodiments. Such a GUI may be instantiated upon execution of a software application in communication with the media obfuscation platform.
- GUI graphical user interface
- a GUI may include a media player element 602 that enables consumption (e.g., viewing) of the media data.
- the media player element may provide the ability for a user (e.g., an administrator) to view either an obfuscated version of the media data or an unobfuscated version of the media data.
- the user may be provided an illustrative example of an obfuscation effect 606 that may be applied to the media data.
- the user may be given the ability to change a location or size of the obfuscation effect in order to more effectively obfuscate an object.
- the user may be provided with the ability to manually add an obfuscation effect at a specified location or locations. The user may be further provided with the ability to move the obfuscaton effect within the media data with respect to time.
- the GUI may further include an object selection element 604.
- An object selection element may provide a user with an indication of each of the objects that have been identified within the media data.
- each object may be identified via an identifier associated with that object.
- each object may be identified via a graphical representation of the object, such as a thumbnail image.
- the indication of each of the objects identified within the media data may include one or more selectors 608 (1 - 3) that can be selected by a user to indicate to the media obfuscation platform that the object associated with that selector should be obfuscated.
- a user may make a selection as to which objects should be obfuscated.
- a user may make a selection as to which objects should not be obfuscated and a determination may be made to obfuscate all non-selected objects.
- FIG. 7 depicts a block diagram showing an example process flow for automatically identifying and obfuscation objects in a media data in accordance with embodiments.
- the process 700 may be performed by components within a system 100 as discussed with respect to FIG. 1 above.
- the process 700 may be performed by a media obfuscation platform 102 in communication with a number of data sources 106, and one or more third-party devices 108.
- the process 700 comprises receiving media data from a data source.
- the media data comprises one or more of audio data, video data, or image data.
- the data source may comprise any suitable electronic device capable of capturing media data in relation to an environment in which the data source is located.
- the data source comprises at least one of a street camera, body-mounted camera, or vehicle-mounted camera.
- the process 700 comprises identifying a set of objects associated with the media data. In some embodiments, this comprises performing one or more object recognition (e.g., pattern recognition) techniques on the media data.
- the set of objects may comprise a set of persons or items having personally-identifiable information. For example, a set of objects may include depictions of drivers’ licenses, vehicle license plates, addresses, or any other identifying items.
- the process 700 comprises storing information about the identified objects.
- Such information may include an indication of times and corresponding locations at which the object is detected within the media data.
- the indication of times at which the object is detected within the media data comprises a timestamp.
- the indication of times at which the object is detected within the media data comprises an indication of a video frame.
- the indication of locations at which the object is detected within the media data comprises an indication of a pixel within an image.
- the indication of locations at which the object is detected within the media data comprises horizontal and vertical coordinates.
- the process 700 comprises receiving a request for the media data.
- a request may be a FOIA request that relates to information about a particular event or events.
- the request specifies an event and the media data is determined to be associated with the event.
- the event may correspond to a request for emergency services made via a phone call.
- the process 700 comprises updating the media data to include an obfuscation effect at one or more times and corresponding locations associated with an object depicted within the media data.
- the process may further comprise determining, for each of the object in the set of objects, whether to apply the obfuscation effect. In some cases, a determination as to whether to apply the obfuscation effect to an object in the set of objects is made based at least in part on a role of an entity from which the request was received. In some cases, a determination as to whether to apply the obfuscation effect to an object in the set of objects is made based at least in part on a role of the object with respect to the media data.
- the process 700 comprises providing the updated media data to the requesting entity in response to the request.
- providing the updated media data in response to the request comprises providing a set of data files for download.
- providing the updated media data in response to the request comprises providing a link to a location in memory (e.g., a Uniform Resource Locator (URL) link) at which the updated media data may be accessed.
- URL Uniform Resource Locator
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Bioethics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Databases & Information Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Storage Device Security (AREA)
- Alarm Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AU2022282193A AU2022282193B2 (en) | 2021-05-24 | 2022-05-13 | Selective obfuscation of objects in media content |
| CA3220189A CA3220189C (en) | 2021-05-24 | 2022-05-13 | Selective obfuscation of objects in media content |
| EP22811853.5A EP4352967B1 (en) | 2021-05-24 | 2022-05-13 | Selective obfuscation of objects in media content |
| JP2023572762A JP7547709B2 (en) | 2021-05-24 | 2022-05-13 | SELECTIVE OBFUSCATION OF OBJECTS IN MEDIA CONTENT - Patent application |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/328,804 | 2021-05-24 | ||
| US17/328,804 US11706381B2 (en) | 2021-05-24 | 2021-05-24 | Selective obfuscation of objects in media content |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022250986A1 true WO2022250986A1 (en) | 2022-12-01 |
Family
ID=84103002
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2022/029288 Ceased WO2022250986A1 (en) | 2021-05-24 | 2022-05-13 | Selective obfuscation of objects in media content |
Country Status (6)
| Country | Link |
|---|---|
| US (1) | US11706381B2 (en) |
| EP (1) | EP4352967B1 (en) |
| JP (1) | JP7547709B2 (en) |
| AU (1) | AU2022282193B2 (en) |
| CA (1) | CA3220189C (en) |
| WO (1) | WO2022250986A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12452490B2 (en) * | 2021-10-18 | 2025-10-21 | Apprentice FS, Inc. | Method for distributing censored videos of manufacturing procedures performed within a facility to remote viewers |
| CN113906765B (en) * | 2019-05-29 | 2024-05-31 | 苹果公司 | Method and apparatus for obfuscating location-specific data associated with a physical environment |
| EP4599350A1 (en) * | 2022-10-05 | 2025-08-13 | Bumble IP Holdco LLC | Privacy protection of digital image data on a social network |
| US20250131538A1 (en) * | 2023-10-24 | 2025-04-24 | Genetec Inc. | Dynamic privacy protection from a calibrated camera system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160171244A1 (en) * | 2012-08-15 | 2016-06-16 | Empire Technology Development Llc | Digital media privacy protection |
| US20160218872A1 (en) * | 2015-01-27 | 2016-07-28 | Arris Enterprises, Inc. | Obfuscation for protection of streaming media and other data flows |
| US20160365974A1 (en) * | 2008-05-12 | 2016-12-15 | New Bis Safe Luxco S.À.R.L. | Data obfuscation system, method, and computer implementation of data obfuscation for secret databases |
| EP3301937A1 (en) | 2013-04-12 | 2018-04-04 | Panasonic Intellectual Property Management Co., Ltd. | Communication system |
| US10311446B2 (en) * | 2008-12-05 | 2019-06-04 | Nokia Technologies Oy | Method and apparatus for obfuscating context information |
| US20190392194A1 (en) * | 2018-06-22 | 2019-12-26 | Apical Limited | Data processing |
Family Cites Families (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090025063A1 (en) * | 2007-07-18 | 2009-01-22 | Novell, Inc. | Role-based access control for redacted content |
| US20090251311A1 (en) * | 2008-04-06 | 2009-10-08 | Smith Patrick W | Systems And Methods For Cooperative Stimulus Control |
| US10354689B2 (en) * | 2008-04-06 | 2019-07-16 | Taser International, Inc. | Systems and methods for event recorder logging |
| US9396319B2 (en) * | 2013-09-30 | 2016-07-19 | Laird H. Shuart | Method of criminal profiling and person identification using cognitive/behavioral biometric fingerprint analysis |
| US10438631B2 (en) * | 2014-02-05 | 2019-10-08 | Snap Inc. | Method for real-time video processing involving retouching of an object in the video |
| US9679194B2 (en) * | 2014-07-17 | 2017-06-13 | At&T Intellectual Property I, L.P. | Automated obscurity for pervasive imaging |
| EP3262833B1 (en) * | 2015-02-24 | 2023-02-22 | Axon Enterprise, Inc. | Systems and methods for bulk redaction of recorded data |
| US10242282B2 (en) * | 2017-03-20 | 2019-03-26 | Conduent Business Services, Llc | Video redaction method and system |
| US10535120B2 (en) * | 2017-12-15 | 2020-01-14 | International Business Machines Corporation | Adversarial learning of privacy protection layers for image recognition services |
| US10846151B2 (en) * | 2018-04-13 | 2020-11-24 | safeXai, Inc. | Notifying entities of relevant events removing private information |
| JP7259838B2 (en) | 2018-03-13 | 2023-04-18 | ソニーグループ株式会社 | Information processing device, information processing method, and recording medium |
| US11122237B2 (en) * | 2018-06-05 | 2021-09-14 | Axon Enterprise, Inc. | Systems and methods for redaction of screens |
| US10839104B2 (en) * | 2018-06-08 | 2020-11-17 | Microsoft Technology Licensing, Llc | Obfuscating information related to personally identifiable information (PII) |
| CN108848382B (en) | 2018-06-12 | 2020-04-03 | 浙江大华技术股份有限公司 | Encoding and decoding method and device |
| CA3105060A1 (en) * | 2018-06-29 | 2020-01-02 | Alarm.Com Incorporated | Object tracking using disparate monitoring systems |
| US10846573B2 (en) * | 2018-07-31 | 2020-11-24 | Triangle Digital Ventures Ii, Llc | Detecting, redacting, and scoring confidential information in images |
| JP7171406B2 (en) * | 2018-12-12 | 2022-11-15 | キヤノン株式会社 | Electronic device and its control method |
| US10824897B1 (en) * | 2019-03-26 | 2020-11-03 | Facebook, Inc. | Detecting an identity of a brand and/or an obfuscated identity of the brand within an image based on embeddings corresponding to regions of interest within the image |
| US11386229B2 (en) * | 2019-07-04 | 2022-07-12 | Blackberry Limited | Filtering personally identifiable information from vehicle data |
| JP7287667B2 (en) | 2019-07-16 | 2023-06-06 | i-PRO株式会社 | Surveillance cameras and information processing equipment |
| US20210051294A1 (en) * | 2019-08-12 | 2021-02-18 | Microsoft Technology Licensing, Llc | Content aware automatic background blurring |
| US11698928B2 (en) * | 2019-11-26 | 2023-07-11 | Motorola Solutions, Inc. | System and method for intelligent prioritization of media related to an incident |
| US11281803B2 (en) * | 2019-12-11 | 2022-03-22 | Sap Se | Obfuscating content based on requesting user roles |
| US11557182B2 (en) * | 2020-10-13 | 2023-01-17 | Trax Technology Solutions Pte Ltd. | Regaining frictionless status of shoppers |
| CN112887799B (en) | 2021-03-11 | 2023-07-25 | 北京小米移动软件有限公司 | Image resource sharing method, device and medium |
-
2021
- 2021-05-24 US US17/328,804 patent/US11706381B2/en active Active
-
2022
- 2022-05-13 JP JP2023572762A patent/JP7547709B2/en active Active
- 2022-05-13 EP EP22811853.5A patent/EP4352967B1/en active Active
- 2022-05-13 WO PCT/US2022/029288 patent/WO2022250986A1/en not_active Ceased
- 2022-05-13 AU AU2022282193A patent/AU2022282193B2/en active Active
- 2022-05-13 CA CA3220189A patent/CA3220189C/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160365974A1 (en) * | 2008-05-12 | 2016-12-15 | New Bis Safe Luxco S.À.R.L. | Data obfuscation system, method, and computer implementation of data obfuscation for secret databases |
| US10311446B2 (en) * | 2008-12-05 | 2019-06-04 | Nokia Technologies Oy | Method and apparatus for obfuscating context information |
| US20160171244A1 (en) * | 2012-08-15 | 2016-06-16 | Empire Technology Development Llc | Digital media privacy protection |
| EP3301937A1 (en) | 2013-04-12 | 2018-04-04 | Panasonic Intellectual Property Management Co., Ltd. | Communication system |
| US20160218872A1 (en) * | 2015-01-27 | 2016-07-28 | Arris Enterprises, Inc. | Obfuscation for protection of streaming media and other data flows |
| US20190392194A1 (en) * | 2018-06-22 | 2019-12-26 | Apical Limited | Data processing |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP4352967A4 |
Also Published As
| Publication number | Publication date |
|---|---|
| CA3220189C (en) | 2025-05-13 |
| CA3220189A1 (en) | 2022-12-01 |
| JP7547709B2 (en) | 2024-09-10 |
| US20220377277A1 (en) | 2022-11-24 |
| JP2024520418A (en) | 2024-05-24 |
| AU2022282193B2 (en) | 2024-08-08 |
| EP4352967A1 (en) | 2024-04-17 |
| AU2022282193A1 (en) | 2024-01-18 |
| EP4352967B1 (en) | 2026-04-22 |
| EP4352967A4 (en) | 2025-01-29 |
| US11706381B2 (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3220189C (en) | Selective obfuscation of objects in media content | |
| Ardabili et al. | Understanding policy and technical aspects of ai-enabled smart video surveillance to address public safety | |
| EP3479270B1 (en) | Incident response analytic maps | |
| US12314423B2 (en) | Predictive response-generation systems to facilitate timely compliance with information-disclosure laws | |
| US11468617B2 (en) | Selective redaction of images | |
| US11024169B2 (en) | Methods and systems for utilizing vehicles to investigate events | |
| US20210233371A1 (en) | Automatic video privacy | |
| US10332227B2 (en) | Multiple source watermarking for surveillance | |
| US12293000B2 (en) | Device and method for redacting records based on a contextual correlation with a previously redacted record | |
| US12175848B2 (en) | Computer implemented method, apparatus and computer program for privacy masking video surveillance data | |
| US11546141B1 (en) | Cryptographic protection for portions of media | |
| US20220171969A1 (en) | Sentiment analysis for situational awareness | |
| EP4214622A1 (en) | Systems and methods for configuring and operating de-identification systems | |
| CN119577814A (en) | A smart glasses control method and system based on multimodal privacy protection | |
| US20240143645A1 (en) | Item analysis and linking across multiple multimedia files | |
| US11818507B2 (en) | Automated correlation of media data to events | |
| US11558550B1 (en) | Privacy-protecting multi-pass street-view photo-stitch | |
| US12619776B2 (en) | Anonymizing personally identifiable information in stored data | |
| KR102859585B1 (en) | Data generating device and method for processing personal information method using the same | |
| US20230057547A1 (en) | Anonymizing personally identifiable information in stored data | |
| CN121334413A (en) | A business video processing method, apparatus, equipment, medium, and product | |
| KR20250104956A (en) | Ai application system having a function of priacy protection and ai application method for non-recognition | |
| CN121456857A (en) | Application watermark realization method, device, system, equipment, medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22811853 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2023572762 Country of ref document: JP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 3220189 Country of ref document: CA |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2022282193 Country of ref document: AU Ref document number: AU2022282193 Country of ref document: AU |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2022811853 Country of ref document: EP |
|
| ENP | Entry into the national phase |
Ref document number: 2022811853 Country of ref document: EP Effective date: 20240102 |
|
| ENP | Entry into the national phase |
Ref document number: 2022282193 Country of ref document: AU Date of ref document: 20220513 Kind code of ref document: A |
|
| WWG | Wipo information: grant in national office |
Ref document number: 2022811853 Country of ref document: EP |