WO2025040468A1

WO2025040468A1 - Augmented reality method and system

Info

Publication number: WO2025040468A1
Application number: PCT/EP2024/072534
Authority: WO
Inventors: Alan FENWICK; Lauren BATH
Original assignee: Exquisite Gaming Ltd
Current assignee: Exquisite Gaming Ltd
Priority date: 2023-08-22
Filing date: 2024-08-08
Publication date: 2025-02-27
Anticipated expiration: 2026-02-22

Abstract

A computer-implemented Augmented Reality (AR) method and system for implementing the method. The method comprises the steps of capturing, on one or more object comprising an AR marker recognisable by an image processing means, an image with image capturing means of a data processing system. The data processor system, such as a smartphone or computer, further comprises, image processing means, and display means such as a camera and screen. The captured image is processed with the image processing means to detect at least a presence of an AR marker within the captured image. A database of AR images is accessed, responsive to the detected AR marker and an AR image associated with that AR marker is selecting from the database. The captured image with the AR image associated with that AR marker is augmented or replaced on the display means and the AR image associated with the AR marker within the database is changed dynamically over time.

Description

AUGMENTED REALITY METHOD AND SYSTEM

[0001] This invention is an Augmented Reality (AR) method and system.

[0002] Augmented reality is a technology that allows for a user to see, through the screen of an appropriate device such as a cell phone (often referred to as a smartphone), camera, computer or tablet, a combination of computer generated and real-world imagery. A processing resource linked to the camera and the screen is able to create a composite image of both the real-world image and a computer-generated image, which may be stored on the device or accessed from a database elsewhere. Such computer-generated images may be still or animated, and may be in some way induced by a trigger (sometimes referred to as an AR marker) that is recognised by the user’s device.

[0003] Examples exist in the prior art, for example, US2014/0378023 is directed to an augmented reality system that uses a video camera to identify markers on toy blocks in its field of view and to then incorporate an animation in that location. US2014/0188669 describes an augmented reality system that helps people virtually try on clothes, and in US2015/0206349 an augmented reality system describes introduction of advertisements into scenes. US11188755 describes the use of wearable pin badges as AR markers.

[0004] US2015265934 describes interactive toys that provide a physical representation of an avatar created by a user within a virtual or on-line gaming environment. The interactive toy in this case comprises a unique identifier tag and an electrical circuit. The electrical circuit comprises an identifier tag sensor and an interface that provides a means for the toy to communicate with a computer system. The inclusion of the unique identifier tag allows for a portable toy to be produced that can act as both an identifier and a scanner.

[0005] The prior art methods and systems suffer from the drawback of a poor user experience, since after the initial excitement of viewing an AR image associated with a particular AR marker, the image will remain the same for each interaction of the device with the AR marker.

[0006] The present invention aims to provide an improved user experience. [0007] Therefore, according to a first aspect of the present invention there is provided a computer-implemented Augmented Reality (AR) method, the method comprising: capturing, on one or more object comprising an AR marker recognisable by an image processing means, an image with image capturing means of a data processing system, the data processing system further comprising image processing means, and display means; processing the captured image with the image processing means to detect at least a presence of an AR marker within the captured image; accessing a database of AR images, responsive to the detected AR marker; selecting an AR image associated with that AR marker; augmenting or replacing the captured image with the AR image associated with that AR marker on the display means; changing dynamically within the database the AR image associated with the AR marker over time.

[0008] Advantageously, the AR image that is displayed to a user when an AR marker is identified by the data processing system, will not always be the same, and thus provide an improved user experience. The database could be refreshed to include new AR images which are associated with each AR marker, or the AR markers associated with each AR image could be randomised or reallocated. AR images could be manually created and uploaded, or be Al generated.

[0009] When a user points the image capturing means at an object and views it through the display means, the image presented to them will be augmented or replaced with an AR image and appear differently. Over time, the AR image that the user will see will change when the data processing system is presented to the object. Such objects may be collectable discs or plates, which may be a planar and discoid or rectangular in shape, or could comprise a device display stand or a device charger and be used in association with peripherals and accessories such as display stands and chargers for games console handsets, headphones and the like. The objects may be based on characters from media such as graphic novels, books, computer games, televisions shows or motion pictures and the like, and the images associated with the AR marker may comprise still or animated images associated with such media, to either give the illusion that the object is moving, or may be links to trailers or movie clips associated with the object. The discs or plates may be made from any of metal, plastic, word or cardboard, and may include two-dimensional artwork, three- dimensional artwork or a hologram, be textured or be smooth. The object may comprise a rectangular card made from metal, and of a size that is comparable to so-called trading cards as are well known in the art.

[0010] The AR marker may comprise a machine-readable element. The machine-readable element may comprise a barcode, and RFID or NFC. Preferably, the machine-readable element comprises a two-dimensional barcode such as a QR code. Alternatively, or additionally, the AR marker may include text or artwork, including part or all of an image on the object, the shape of the object, a colour of the object or a combination thereof.

[0011] The data processing system may comprise a smartphone, a tablet computer, a laptop computer or a desktop computer. The image capturing means may comprise a camera and the display means may comprise a screen. Data processing systems such as tablets and smartphones will generally comprise an integrated screen and camera, and desktop computers will generally comprise an external camera, such as a webcam, and screen or monitor. Generally, the data processing systems will be connected to the internet via Wi-Fi, ethernet or cellular networks.

[0012] The database may be stored on a memory on the data processing system or on a remote server. The computer program which implements the present method may be stored on a remote server through a web-based system, or may be stored on the data processing system in the form of an app.

[0013] The image processing means may use any known means of processing the image and turning it into digital data, as is known in the field of computer vision, image recognition, machine vision and the like.

[0014] The present method may further include the step of authenticating, when an AR marker is first detected by the image processing means, the owner of the object, and creating a digital version of the object with the data processing system. Initially, the owner of the object may create an account on a system, which may include any of name, address, telephone number, security information and billing information. When the data processing system first encounters an AR marker the owner’s account will be authenticated and link the AR marker and the object to the owner. At this point, a digital twin of the object will be created and stored by the data processing system. The digital twin can then be interacted with and used by the owner separately from the previous AR experience, and for example can be placed on social media or used in computer games. Such digital twins may also be tradable with different users

[0015] The object may include a plurality of AR markers, including a machine- readable marker such as a QR code and a separate image, such as artwork. The object may be a planar card, and have a machine readable element on one side and artwork on the other. The QR code might be to authenticate the object, and the artwork might separately be detectable by the image processing means and subsequently be augmented with an IR image.

[0016] The present method may further comprise the step of determining a geolocation of the data processing system with a global positioning system component of the data processing system; assigning geolocation data to AR images in the database; selecting from the database an image associated with the AR marker and the geolocation.

[0017] Advantageously, the AR image that is selected in association with the AR marker will be associated with a particular geolocation. For example, the owner could visit a particular location, such as a shop, convention or place associated with a particular object, and the AR image will be unique to that location. This will create an enhanced user experience.

[0018] The present method may further comprise the step of capturing, on two or more objects comprising one or more AR marker, with the image capturing means an image; selecting from the database an AR image associated with combined AR markers of the two or more objects. Advantageously, the present method permits for a greater variety of AR images based upon combinations of AR markers on two or more objects. The two or more objects could be in the same field of view of the image capturing means, and could for example comprise a first object, such as a charging station for a computer peripheral, and a second object, which could be a collectable plate attachable to the charging station. There could be a first AR image drawn from the database that is associated with the first object, and second AR image drawn from the database that is associated with the second object, and then a third AR image that is drawn from the database that is associated with the combination of the first and second objects. Therefore, there will be an enhanced user experience in terms of the AR experience which can be changed with different combinations of objects, which will also change dynamically over time.

[0019] An object may further comprise two or more AR markers.

[0020] The present method may further comprise the step of processing the captured image with the image processing means to detect at least a presence of an AR marker within the captured image, wherein the image processing means processes the orientation of the AR marker relative to the image capturing means; accessing a database of AR images, responsive to the orientation of the detected AR marker; selecting an AR image associated with the orientation of that AR marker. Therefore, there may be one or more AR image in the database associated with each AR marker due to the number of possible orientations of the AR marker relative to the image capturing means. Most prior art systems and methods allow for one AR image per AR marker, which will normally require a image capturing means such as a camera to be pointed directly at the AR marker. The present methos allows for the image capturing means, such as a camera, to be pointed at the AR marker on the object at a range of different angles, and this will result in a different AR experience. For example, if the image capturing means are tilted with respect to the marker, the AR image may be of a character disposed at a different angle to one where the image capturing means is disposed directly above the AR marker on the object. This may allow for a three- dimensional AR rendering of a character about an object, or simply provide a range of different AR still images and animations. The orientations of the image capturing means relative to the AR marker could be in bands, such as 0-10 degrees, 10-20 degrees, 20-30 degrees, 30-40 degrees, 40-50 degrees, 60-70 degrees 70-80 degrees and 80-90 degrees relative to the perpendicular.

[0021] According to the present invention there is also provided an Augmented Reality (AR) system for carrying out the present method, the system comprising: one or more object comprising an AR marker recognisable by an image processing means; a data processing system comprising image capturing means, image processing means, and display means, and a database of AR images.

[0022] The present invention is differentiated with respect to the prior art at the application layer. The specific use of AR systems anchor the image to devices designed to hold/stand or charge electronic devices such as gaming controllers and cellular/mobile/smart phones. Most of the prior art AR methods and systems either focus on how the AR markers interact with the AR system to generate the AR scene including the way the camera image (which may be still or animated/video) interacts with the marker or what the AR image does. For example this may relate to advertising, or relate to the placement of AR markers on the article itself. In the case of US11188755 there is required the use of an App/application software to connect the marker on a wearable pin badge to a database of imagery, where each pin is related to a particular image in the database.

[0023] The present invention is directed to the manner in which multiple AR marker activations are combined within the objects which could also include the use of unique identifier tags and other electronic circuits/short range communication devices and other mechanisms that will allow for the first time a way of linking a specific product to a specific AR activation that is unique to that user experience allowing for personalised AR experiences.

[0024] The prior art is directed to singular AR experiences - a user points a camera at a toy brick, item of clothing or a pin badge and a singular AR experience is activated - and this AR experience is the same experience every time the AR marker is activated (even if the AR image itself is different between products i.e. a still image, an animated image, video clip or live feed) each experience is the same each time. The present invention allows every experience to be linked to an ever-changing dynamic experience - this is designed to build product lifetime sustainability. The purpose of a product can therefore be changed by adapting or enhancing the AR experience for the consumer: i.e. the invention of the use of the AR is not fixed to the marker or the solution but can be dynamically changed in real time to change the experience the user has when utilising the AR component of the product which means the usefulness of the product is extended. This is not just because of the function of the product but the way in which the dynamic AR can be deployed to change or enhance the purpose of the product itself.

[0025] The prior art AR experience does not change when used in different situations. In some examples the AR image or animation changes when two triggers are brought together e.g. Card a = event 1 card b = event 2 and when card a and b are brought into the same camera view then a new AR experience such as an animation is triggered. The present use of AR triggers/markers may change the actual AR experience without needing to have a separate database or even a new combined AR marker.

[0026] AR images include in the first instance the appearance of a character, image or animation, in a second instance content from a creator such as advertisements or trailers for new games, in a third instance reward for customer loyalty, and in a fourth instance gamification where a user must compete in a game to gain a reward.

[0027] The present invention will now be described in detail, but by way of example only, with reference to the following figures, wherein:

[0028] Figure 1 shows a flow chart of the method of the present invention;

[0029] Figure 2 shows a flow diagram of a further method of the present invention;

[0030] Figure 3 shows a flow diagram of a further method of the present invention;

[0031] Figure 4 shows a flow diagram of a further method of the present invention;

[0032] Figure 5 shows a flow diagram of a further method of the present invention;

[0033] Figure 6 shows a schematic representation of a system according to the present invention;

[0034] Figure 7 shows a schematic representation of an object according to the present invention;

[0035] Figure 8 shows a schematic representation of an alternative object according to the present invention; and [0036] Figure 9 shows a schematic representation of two objects according to the present invention.

[0037] Turning first to Figure 1 , there is shown a computer-implemented Augmented Reality (AR) method, the method comprising: at Step 110 capturing, on one or more object comprising an AR marker recognisable by an image processing means, an image with image capturing means of a data processing system, the data processing system further comprising, image processing means, and display means.

[0038] At Step 120, there is shown the step of processing the captured image with the image processing means to detect at least a presence of an AR marker within the captured image.

[0039] At Step 130, there is shown the step of accessing a database of AR images, responsive to the detected AR marker.

[0040] At Step 140, there is shown the step of selecting from the database an AR image associated with that AR marker.

[0041] At Step 150, there is shown the step of augmenting or replacing the captured image with the AR image associated with that AR marker on the display means.

[0042] At Step 160, there is shown the step of changing dynamically within the database the AR image associated with the AR marker over time.

[0043] Turning now to Figure 2, there is shown an alternative computer implemented Augmented Reality method which includes the further Step 170 of authenticating, when an AR marker is first detected by the image processing means, the owner of the object, and creating a digital version of the object with the data processing system, which occurs between Step 120 and Step 130. The AR marker for authenticating the object is a machine-readable element, such as a QR code. The object may further comprise additional AR markers such as artwork or text.

[0044] Turning now to Figure 3, there is shown an alternative computer implemented Augmented Reality method which comprises additional initial Step 180 of assigning geolocation data to AR images in the database, then at Step 190 determining a geolocation of the data processing system with a global positioning system component of the data processing system, which occurs between Step 120 and Step 130, and then at Step 200 selecting from the database an AR image associated with the AR marker and the geolocation. Step 200 replaces Step 140.

[0045] Turning now to Figure 4 there is shown an alternative computer implemented Augmented Reality method which comprises at Step 210, which replaces Step 110, capturing, on two or more objects comprising one or more AR marker, with the image capturing means an image; selecting from the database an AR image associated with combined AR markers of the two or more objects. At Step 220 there is shown the further step of processing the captured image with the image processing means to detect at least a presence of the AR markers within the captured image. At Step 230 there is shown the step of accessing a database of AR images, responsive to the detected AR markers. At Step 240 there is shown the step of selecting from the database an AR image associated with combined AR markers of the two or more objects. At Step 250 there is shown the step of augmenting or replacing the captured image with the AR images associated with the AR markers on the display means. At Step 260 there is shown the step of changing dynamically within the database the AR image associated with the AR markers over time.

[0046] Turning to Figure 5, there is shown a computer-implemented Augmented Reality (AR) method which differs from that of Figure 1 in that Step 120 is replaced with Step 270 which is the step of processing the captured image with the image processing means to detect at least a presence of an AR marker within the captured image, wherein the image processing means processes the orientation of the AR marker relative to the image capturing means; accessing a database of AR images, responsive to the orientation of the detected AR marker. Step 140 is replaced with Step 280 which is the step of selecting from the database an AR image associated with the orientation of that AR marker.

[0047] The methods of Figures 1 to 5 can be combined in any combination to provide a user with an enhanced AR experience.

[0048] Turning now to Figure 6 there is shown a schematic representation of the system according to the present invention comprising one or more object, generally indicated 500 comprising a first AR marker 510 on a first side of the object and a second AR marker 520 on a second side of the object. The first AR marker 510, which is this example is a QR code, may be any other form of machine-readable element such as a one-dimensional, two-dimensional barcode, an RFID or NFC element. The first AR marker is recognisable by an image processing means and is used to authenticate the object 500 with a particular user. The second AR marker 520 comprises artwork and is also recognisable by an image processing means

[0049] The system further comprises a data processing system, generally indicated 530, which may comprise a smartphone, tablet computer, laptop computer, desktop computer or the like. The data processing system 530 comprises image capturing means 540, which may comprise a camera, image processing means 550, and display means 560, which may comprise a screen. The image capturing means 540 of the data processing system 530 captures, in use, an image of the object 500. The image processing means 550 then processes the captured image to detect at least a presence of an AR marker 510,520 within the captured image. Responsive to the detected marker element 510,520, the image processing means 550 access a database of AR images 570 and select an AR image 580 associated with that AR marker 510,520. The image processing means 550 then augment or replace the captured image with the image associated with that AR marker 580 on the display means 560. In order to maintain a positive user experience, the image associated with the AR marker dynamically changes over time.

[0050] As shown in Figure 6, the database 550 is stored on a memory on the data processing means 530. Alternatively, the database 570 could be stored on a remote web or cloud-based server, and the data processing means 530 could include an internet connection such as WiFi or ethernet or via a cellular telephone network, to access the remote server.

[0051] Figures 7, 8 and 9 show a range of different objects. Figures 7 and 8 show objects 600 and 610 respectfully, which include QR codes 620 and 630. Objects 600 and 610 are planar collectable cards which as well as having QR codes 620 and 630 on one side thereof include a picture 640 such as a drawing, photograph or hologram on the other side, which also comprises an AR marker. Each QR code 610, 620 may be associated with AR images that could cause the augmented or replaced image to be an animation of a character on the objects, or could be a link to a video that is in relation to the character. [0052] Figure 9 shows object 600 in combination with a further object generally indicated 650. In this case, object 610 is a collectable card, and object 650 is a display stand. Display stand 650 also includes an AR marker 660 which is associated with a particular AR image. AR marker 660 is readable at the same time as the picture 640 of object 610, by the data processing means 520. The combination of the two AR markers 640,660 being processed simultaneously will result in a different AR image than if AR markers 640,660 were processed individually.

[0053] Generally, the AR markers could be centrally placed on the front of the object, or could be disposed about the rim thereof. Where two objects are provided in combination, one object could obscure one of or part of an AR marker on the other object, and the obscured AR marker could then comprise a new AR marker.

Claims

1. A computer-implemented Augmented Reality (AR) method, the method comprising: capturing, on one or more object comprising an AR marker recognisable by an image processing means, an image with image capturing means of a data processing system, the data processing system further comprising, image processing means, and display means; processing the captured image with the image processing means to detect at least a presence of an AR marker within the captured image; accessing a database of AR images, responsive to the detected AR marker; selecting from the database an AR image associated with that AR marker; augmenting or replacing the captured image with the AR image associated with that AR marker on the display means; changing dynamically within the database the AR image associated with the AR marker over time.

2. A computer-implemented Augmented Reality (AR) method according to claim 1 , further comprising the step of: authenticating, when an AR marker is first detected by the image processing means, the owner of the object, and creating a digital version of the object with the data processing system.

3. A computer-implemented Augmented Reality (AR) method according to claim 1 or claim 2, further comprising the steps of: determining a geolocation of the data processing system with a global positioning system component of the data processing system; assigning geolocation data to AR images in the database; selecting from the database an image associated with the AR marker and the geolocation.

4. A computer-implemented Augmented Reality (AR) method according to any of claims 1 to 3, further comprising the steps of: capturing, on two or more objects comprising one or more AR marker, with the image capturing means an image; selecting from the database an AR image associated with combined AR markers of the two or more objects.

5. A computer-implemented Augmented Reality (AR) method according to any of claims 1 to 4, further comprising the steps of: processing the captured image with the image processing means to detect at least a presence of an AR marker within the captured image, wherein the image processing means processes the orientation of the AR marker relative to the image capturing means; accessing a database of AR images, responsive to the orientation of the detected AR marker; selecting from the database an AR image associated with the orientation of that AR marker.

6 An Augmented Reality (AR) system for carrying out the method of any of claims 1 to 5, the system comprising: one or more object comprising an AR marker recognisable by an image processing means; a data processing system comprising image capturing means, image processing means, and display means, and a database of AR images.

7. An Augmented Reality (AR) system according to claim 6, wherein the object comprises a planar element which is discoid or rectangular in shape, a device display stand or a device charger.

8. An Augmented Reality (AR) system according to claim 6 or claim 7, wherein the AR marker further comprises a machine-readable element or text or a picture of a combination thereof.

9. An Augmented Reality (AR) system according to claim 8, wherein the machine-readable element comprises a 2-dimensional barcode.

10. An Augmented Reality (AR) system according to any of claims 6 to 9, wherein the data processing system comprises a smartphone, a tablet computer, a laptop computer or a desktop computer.

11. An Augmented Reality (AR) system according to any of claims 6 to 10, wherein the image capturing means is a camera and the display means is a screen.

12. An Augmented Reality (AR) system according to any of claims 6 to 11 , wherein the database is stored on a memory on the data processing system or on a remote server.