WO2018096430A1

WO2018096430A1 - Automatic optimisation of touch signal

Info

Publication number: WO2018096430A1
Application number: PCT/IB2017/057201
Authority: WO
Inventors: Tomas Christiansson; Henrik Wall; Nicklas OHLSSON; Mattias KRUS; Tomas Svensson
Original assignee: FlatFrog Laboratories AB
Current assignee: FlatFrog Laboratories AB
Priority date: 2016-11-24
Filing date: 2017-11-17
Publication date: 2018-05-31
Anticipated expiration: 2019-05-24
Also published as: EP3545392A1; EP3545392A4; US20210089164A1; US20190317640A1; US10761657B2; CN110100226A

Abstract

A method of operating an optical touch apparatus is described wherein, the optical touch apparatus comprising a panel, a plurality of emitters and a plurality of detectors arranged with respect to the panel, the plurality of detectors configured to receive light from the plurality of emitters thereby defining detection samples of light transmitted between pairs of emitters and detectors. The optical touch apparatus being further configured to operate in an ambient light evaluation mode comprising the steps of: generating a first set of samples over a first integration time and a second set of samples over a second integration time using at least one detector, and applying a filter to at least the first and second set of samples to generate a residual ambient light value indicative of ambient light noise, wherein the touch apparatus is configured to perform the following steps: operating according to the ambient light evaluation mode for a plurality of repetitions.

Description

AUTOMATIC OPTIMISATION OF TOUCH SIGNAL

Technical Field

The present invention relates to a method and apparatus for reducing the effects of ambient light noise resulting from incident ambient light on an optical touch system.

Background Art

Touch sensing systems ("touch systems") are in widespread use in a variety of applications. Typically, the touch sensing systems are actuated by a touch object such as a finger or stylus, either in direct contact, or through proximity (i.e. without contact), with a touch surface. Touch sensing systems are for example used as touch pads of laptop computers, in control panels, and as overlays to displays on e.g. hand held devices, such as mobile telephones and interactive whiteboards. A touch panel that is overlaid on or integrated in a display is also denoted a "touch screen". Many other applications are known in the art.

WO2010/064983 discloses a multi-touch system, which is based on frustrated total internal reflection (FTIR). Light sheets are coupled into a panel to propagate inside the panel by total internal reflection (TIR). When an object comes into contact with a touch surface of the panel, the propagating light will be attenuated at the point of touch. Arrays of light sensors are located around the perimeter of the touch surface to detect the received light for each light sheet and generate a signal value for each light path across the touch surface. The signal values of the light sensors are normalized by respective reference values and input into an image

reconstruction algorithm that generates a two-dimensional distribution of attenuation values across the touch surface. This enables repeated determination of current position/size/shape of touches while one or more users interact with the touch surface. In FTIR systems, the touches need to be detected against a background of interferences, e.g. originating from fingerprints and other types of smear on the touch surface. The influence of interferences may vary not only over time but also across the touch surface, making it difficult to properly detect the touches on the touch surface at all times. WO2011/028169 proposes manipulating the reference values to compensate for the influence of contaminations in the attenuation pattern, and W0201 1/049512 proposes tracking a two-dimensional background pattern of the contamination contribution and using the background pattern to generate an attenuation pattern that is compensated for contaminations.

Optical touch systems often need to be designed with a high sensitivity, since a touching object may result in only a small attenuation of the propagating light, e.g. less than 1 %. Certain systems may therefore be designed to detect attenuations on the order of 0.1 % - 0.01 %. To ensure proper touch detection, WO2011/078769 proposes monitoring the temporal variability of the signal values to identify potential causes for reduced performance of the touch system, e.g. due to component failure, and implementing corrective measures to account for unreliable signal values.

WO2013/115710 describes processing the output signal of an optical touch system to generate a set of data samples comprises ON energy values when the light signal is being transmitted and OFF energy value when no light signal is being transmitted. An estimated ambient value is determined from the OFF energy value. An ambient-compensated energy value is generated by subtracting an estimated ambient value from the ON energy value for each actual detection line. However, this technique only provides significant improvements for light noise sources that are unmodulated (e.g. Sunlight) or modulated with a low frequency (e.g. Incandescent lamps). This technique provides poor results with respect to modulated noise sources like fluorescent lighting.

Therefore, what is needed is a way of improving performance of an optical touch system in an environment that comprises various types of ambient light noise.

Summary

It is an objective of the invention to at least partly overcome one or more limitations of the prior art.

A first embodiment of the invention describes a method of operating an optical touch apparatus, the optical touch apparatus comprising: a panel (10), a plurality of emitters (30a, 31 a, 32a) and a plurality of detectors (30b, 31 b, 32b) arranged with respect to the panel, the plurality of detectors configured to receive light from the plurality of emitters thereby defining detection samples (D) of light transmitted between pairs of emitters and detectors, a controller coupled to the plurality of emitters to generate the signals and wherein each detector is configured to generate an output signal, the optical touch apparatus being configured to operate in a data sampling mode operating in a sequence of repetitions, each repetition comprising the steps of: turning the emitters ON and OFF according to a timing configuration during each repetition such that the output signal for each detector is generated to comprise ON energy values for the detection samples (D) and at least one OFF energy value, processing the output signals to generate a set of data samples, each data sample being generated to represent detected energy for one of the detection samples (D), and processing the set of data samples to detect an interaction with the panel, the optical touch apparatus being further configured to operate in an ambient light evaluation mode comprising the steps of: generating a first set of samples over a first integration time and a second set of samples over a second integration time using at least one detector, and applying a filter to at least the first and second set of samples to generate a residual ambient light value indicative of ambient light noise, wherein the method comprises the steps of: operating the optical touch apparatus according to the ambient light evaluation mode for a plurality of repetitions, wherein the length of at least one of the first integration time and second integration time vary between repetitions, selecting the first integration time and/or second integration time that generated an optimal residual ambient light value, and operating the optical touch apparatus in the data sampling mode with a timing configuration

corresponding to the selected first integration time and/or second integration time. The filter may be configured to generate a residual ambient light value indicative of ambient light in dependence on the difference between the second set of samples and the first set of samples. The optimal residual ambient light value may correspond to the residual ambient light value having the lowest magnitude. The first integration time may be the same length as the second integration time. A first settling period may occur between the first integration time and the second integration time.

The ambient light evaluation mode may further comprising the steps of:

generating a first set of samples over a first integration time, a second set of samples over a second integration time, and a third set of samples over a third integration time using at least one detector (30b'), and applying an ambient light filter to the first, second, and third set of samples to generate a residual ambient light value indicative of ambient light noise, wherein the touch apparatus is configured to operate in the data sampling mode with a timing configuration corresponding to the first, second, and third integration time of the determined the ambient light evaluation mode repetition. The first integration time may be the same length as the third integration time. A second settling period may occur between the second integration time and the third integration time. The second integration time may correspond to the integration time of the ON signal, the first integration time corresponds to integration time of the OFF signal preceding the ON signal, and the third integration time corresponds to integration time of the OFF signal following the ON signal. The ambient light filter may be configured to generate a residual ambient light value indicative of ambient light in dependence on the difference between a signal sampled over the second integration time and an average of the signal sampled over the first integration time and the signal sampled over the third integration time. The ambient light filter may be configured to generate a residual ambient light value indicative of ambient light in dependence on the difference between a signal sampled over the second integration time and a linear interpolation of the signal sampled over the first integration time and the signal sampled over the third integration time. The steps of the ambient light evaluation mode may be performed once per frame, more than once per frame, or less often than every frame. The ambient light evaluation mode steps may be performed at the beginning, middle or end of every frame. The residual ambient light value may be determined for a plurality of the detectors of the touch apparatus and wherein the optimal residual ambient light value is determined as a function of the plurality of residual ambient light values. The function may be an average or maximum of the residual ambient light values. A total of number of different configurations of the first, second, and third integration times may be greater than 2 and wherein the apparatus being configured to cycle through the total of number of configurations in subsequent ambient light evaluation mode repetitions. Preferably, between 1 and 10 configurations of the total number of configurations may be tested each frame. Preferably, the first second or third integration time may have a length of between 3ps to 30ps. Preferably, the first second or third integration times are changed in steps of between 0.1 μβ and 20MS each ambient light evaluation mode repetition. The optimal residual ambient light value may also be selected as a function of the first and second integration times as well as the residual ambient light value. The residual ambient light value exceeding a performance threshold value and having the shortest first and second integration times may be selected as the optimal residual ambient light value. The system may be configured to not alter the first, second and third integration times used by the optical touch apparatus whilst operating in the data sampling mode whilst a touch interaction with the touch system is occurring.

A second embodiment of the invention describes An optical touch apparatus (100) comprising: a panel (10), a plurality of emitters (30a, 31 a, 32a) and a plurality of detectors (30b, 31b, 32b) arranged with respect to the panel, the plurality of detectors configured to receive light from the plurality of emitters thereby defining detection samples (D) of light transmitted between pairs of emitters and detectors, a controller coupled to the plurality of emitters to generate the signals and wherein each detector is configured to generate an output signal, the optical touch apparatus being configured to operate in a data sampling mode operating in a sequence of repetitions, each repetition comprising the steps of: turning the emitters ON and OFF according to a timing configuration during each repetition such that the output signal for each detector is generated to comprise ON energy values for the detection samples (D) and at least one OFF energy value, processing the output signals to generate a set of data samples, each data sample being generated to represent detected energy for one of the detection samples (D), and processing the set of data samples to detect an interaction with the panel, the optical touch apparatus being further configured to operate in an ambient light evaluation mode comprising the steps of: generating a first set of samples over a first integration time and a second set of samples over a second integration time using at least one detector, and applying a filter to at least the first and second set of samples to generate a residual ambient light value indicative of ambient light noise,

wherein the touch apparatus is configured to perform the following steps: operating according to the ambient light evaluation mode for a plurality of repetitions, wherein the length of at least one of the first integration time and second integration time vary between repetitions, selecting the first integration time and/or second integration time that generated an optimal residual ambient light value, and operating the touch apparatus in the data sampling mode with a timing configuration corresponding to the selected first integration time and/or second integration time.

Still other objectives, features, aspects and advantages of the present invention will appear from the following detailed description, from the attached claims as well as from the drawings.

Brief Description of Drawings Embodiments of the invention will now be described in more detail with reference to the accompanying schematic drawings.

Figure 1 is a top plan view of a touch sensing apparatus with light beams.

Figure 2 is a side view of the apparatus in figure 1 in which light propagates from emitters to detectors by total internal reflection through a glass panel.

Figure 3 is a side view of the apparatus in figure 1 in which light travels from emitters to detectors along a path largely above a glass panel.

Figures 4a to 4e show a portion of the scan scheme sequence.

Figure 5 shows a signal detected at a single detector. An uncompensated signal with ambient disturbances and an uncompensated signal without ambient disturbances is shown.

Figure 6 shows typical power spectrum received by a detector with no ambient electrical light source present.

Figure 7 shows a typical ambient filter kernel.

Figure 8 shows an ambient light filter response function where N=44.

Figure 9 shows an ambient light filter response function where N=32.

Figure 10 shows an ambient light filter response function where N=20.

Figure 11 shows a normalised ambient performance magnitude.

Figure 12 shows typical power spectrum received by a detector with a compact fluorescent lamp light source present.

Figure 13a shows an ambient light filter response function where N=20 with respect to the compact fluorescent lamp light source.

Figure 13b shows an ambient light filter response function where N=32 with respect to the compact fluorescent lamp light source.

Figure 14 shows an ambient light filter response function where N=44 with respect to the compact fluorescent lamp light source.

Figure 15 shows a normalised ambient performance magnitude with a compact fluorescent lamp light source present.

Figure 16 is a flow chart of an embodiment of the ambient light evaluation mode and data sampling mode.

Figure 17 is a flow chart of another embodiment of the ambient light evaluation mode and data sampling mode. Detailed Description of Example Embodiments

Below follows a description of example embodiments of a technique for enabling extraction of touch data for objects in contact with a touch surface of a touch- sensitive apparatus. Throughout the following description, the same reference numerals are used to identify corresponding elements.

Terms

Before describing embodiments of the invention, a few definitions will be given.

A "touch object" or "touching object" is a physical object that touches, or is brought in sufficient proximity to, a touch surface so as to be detected by one or more sensors in the touch system. The physical object may be animate or inanimate.

An "interaction" occurs when the touch object affects a parameter measured by the sensor.

An "interaction strength" is a relative or absolute measure of the degree of interaction. The "interaction strength" may be represented as an "attenuation".

An "interaction map" or "interaction pattern" is a two-dimensional (2D) distribution of interaction strength across the touch surface, or a portion thereof. If the interaction is presented as attenuation, the interaction map/pattern is also denoted an

"attenuation map" or "attenuation pattern".

A "touch" denotes a point of interaction as seen in the interaction pattern.

A "frame" or "iteration" denotes a repeated event comprising an ambient light evaluation, data collection, and determination of touch data.

As used herein, a "trace" is information about the temporal history of an interaction. Touches from an interaction detected in a sequence of frames, i.e. at different points in time, are collected into a trace.

Embodiments of the inventions relate to signal processing in relation to a touch- sensitive apparatus which is based on the concept of transmitting energy of some form across a touch surface, such that an object that is brought into close vicinity of, or in contact with, the touch surface causes a local decrease in the transmitted energy. The apparatus may be configured to permit transmission of energy in one of many different forms. The emitted signals may be any radiation or wave energy that can travel in and/or across the touch surface including, without limitation, light waves in the visible or infrared or ultraviolet spectral regions. Example embodiments of the invention will be described in relation to a touch-sensitive apparatus 100, which is shown in Figures 1-3 and is based on the concept of transmitting the light through a glass panel via TIR (Total Internal Reflection) or transmission of the light above the touch surface. The apparatus 100 operates by transmitting light across a panel 10, from light emitters 30a, 31a, 32a to light sensors or detectors 30b, 31 b, 32b. In the embodiment shown in figure 2, the transmitted light illuminates a touch surface 20 from within the panel 10. The panel 10 is made of solid material in one or more layers and may have any shape. The panel 10 defines an internal radiation propagation channel, in which light propagates by internal reflections. The

propagation channel is defined between the boundary surfaces 5, 6 of the panel 10, where the top surface 5 allows the propagating light to interact with touching objects 7 and thereby defines the touch surface 20. This is achieved by injecting the light into the panel 10 such that the light is reflected by total internal reflection (TIR) in the touch surface 20 as it propagates through the panel 10. The light may be reflected by TIR in the bottom surface 6 or against a reflective coating thereon. It is also conceivable that the propagation channel is spaced from the bottom surface 6, e.g. if the panel comprises multiple layers of different materials. The apparatus 100 may be designed to be overlaid on or integrated into a display device or monitor.

The apparatus 100 allows an object 7 that is brought in contact with the touch surface 20 to interact with the propagating light at the point of touch. In this interaction, part of the light may be scattered by the object 7, part of the light may be absorbed by the object 7, and part of the light may continue to propagate in its original direction across the panel 10. Thus, the touching object 7 causes a local frustration of the total internal reflection, which leads to a decrease in the energy (or, equivalently, power or intensity) of the transmitted light.

In the embodiment shown in figure 3, light is passed above the surface of the panel instead of within the panel. Such a configuration of an optical touch system may be described as an 'above surface' optical touch system. In this configuration, the apparatus 100 allows an object 7 that is brought in contact with the touch surface 20 to interact with the light travelling above the surface of the panel at the point of touch on the panel. In this interaction, a portion of the light travelling above the surface of the panel will be interrupted by the object, which leads to a decrease in the energy (or, equivalently, power or intensity) of the transmitted light received at detectors downstream of the touching objects 7. The emitters 30a, 31 a, 32a are distributed along the perimeter of the touch surface 20 to generate a corresponding number of light sheets inside or above the panel 10. In the example of figure 2, each emitter 31 a generates a beam of light that expands in the plane of the panel 10 while propagating in the panel 10. Each beam propagates from one or more entry or incoupling points on the panel 10. The detectors 31 b are distributed along the perimeter of the touch surface 20 to receive the light from the emitters 31 a at a number of spaced-apart outcoupling points on the panel 10. It should be understood that the incoupling and outcoupling points merely refer to the position where the beam enters and leaves, respectively, the panel 10. Thus, although not shown in figure 2, one emitter/detector may be optically coupled to a number of incoupling/outcoupling points.

While various configurations for an above surface system exist, figure 3 describes an embodiment in which a portion of the light emitted by emitters 32a is transmitted through transmissive panel 100. The light exits transmissive panel 100 through touch surface 20 and is reflected by reflector surface 80 of edge reflector 70 to travel along paths 90a and 90b in a plane parallel with touch surface 20. The light will then continue until deflected by reflector surface 80 of the edge reflector 70 at an opposing edge of the transmissive panel 100, wherein the light will be deflected back down through transmissive panel 100 and onto detectors 32b. In another

embodiment, the emitters and detectors may be positioned to emit and receive light directly above the surface of the panel. Alternatively, the light from emitters positioned below the panel may be directed around the edge of the panel by means of a light guide or other optical arrangement. A corresponding arrangement may convey the light from above the surface of the panel to the detectors positioned beneath the panel.

The detectors 30b, 31 b, 32b collectively provide an output signal, which is received and sampled by a signal processor 130. The output signal contains a number of sub-signals, also denoted "projection signals", each representing the energy of light emitted by a certain light emitter and received by a certain detector. Depending on implementation, the signal processor 130 may need to process the output signal for separation of the individual projection signals. Conceptually, the touch apparatus 100 is considered to define a grid of detection lines D on the touch surface 20, where each detection line D corresponds to a light propagation path from an emitter to a detector, as projected onto the touch surface 20. Thus, the projection signals represent the received energy or power of light on the individual detection lines D. It is realised that the touching object 7 results in a decrease (attenuation) of the received energy on one or more detection lines D.

As will be explained below, the signal processor 130 may be configured to process the projection signals so as to determine a distribution of attenuation values (for simplicity, referred to as an "attenuation pattern") across the touch surface 20, where each attenuation value represents a local attenuation of light. The attenuation pattern may be represented in many different ways, e.g. as attenuation values arranged in a regular x-y-g rid, such as in an ordinary digital image, although other types of grids are conceivable, e.g. hexagonal patterns or triangular meshes. The attenuation pattern may be further processed by the signal processor 130 or by a separate device (not shown) for touch determination, which may involve extraction of touch data, such as a position (e.g. x,y coordinates), a shape or an area of each touching object. In the following, a "frame" denotes a repeated event starting with ambient light evaluation and data collection and ending with determination of touch data.

The signal processor 130 may be configured to process the projection signals so as to determine a property of the touching objects, such as a position (e.g. in an x,y coordinate system), a shape, or an area. This determination may involve a straight-forward triangulation based on the attenuated detection lines, e.g. as disclosed in US7432893 and WO2010/015408, or a more advanced processing to recreate a distribution of attenuation values (for simplicity, referred to as an

"attenuation pattern") across the touch surface 20, where each attenuation value represents a local degree of light attenuation. The attenuation pattern may be further processed by the signal processor 130 or by a separate device (not shown) for determination of a position, shape or area of touching objects. The attenuation pattern may be generated e.g. by any available algorithm for image reconstruction based on projection signal values, including tomographic reconstruction methods such as Filtered Back Projection, FFT-based algorithms, ART (Algebraic

Reconstruction Technique), SART (Simultaneous Algebraic Reconstruction

Technique), etc. Alternatively, the attenuation pattern may be generated by adapting one or more basis functions and/or by statistical methods such as Bayesian inversion. Examples of such reconstruction functions designed for use in touch determination are found in WO2009/077962, WO201 1/04951 1 , WO2011/139213, WO2012/050510, and WO2013/062471.

In the illustrated example, the apparatus 100 also includes a controller 120 (also called a which is connected to selectively control the activation of the emitters 30a, 31a, 32a and, possibly, the readout of data from the detectors 30b, 31 b, 32b. The signal processor 130 and the controller 120 may be configured as separate units, or they may be incorporated in a single unit. One or both of the signal processor 130 and the controller 120 may be at least partially implemented by software executed by a processing unit 140.

In one embodiment, the different emitters 30a, 31 a, 32a are activated in sequence within a frame, such that each individual emitter is activated separately, whereby the transmitted light energy can be measured at each detector that receives light from the activated emitter. Thereby, an energy value is measured for each detection line during the frame.

In an alternative embodiment, groups of emitters 30a, 31 a, 32a are activated concurrently, wherein each emitter is controlled to transmit a code by way of the emitted light such that the code identifies the respective emitter. Based on the transmitted codes, the energy measured by each detector in the light detection arrangement can be separated into energy values for each detection line. Such an emitter activation scheme is denoted "multiplexing" and is further described in PCT publication WO2010/064983.

Embodiments of the invention utilise the energy values measured by the detectors 30b, 31 b, 32b while the emitters 30a, 31 a, 32a are OFF and ON. In the following, an "ON value" denotes detected energy when a signal is conducted from the emitter to the detector, and an "OFF value" denotes detected energy without a signal being conducted.

The OFF values are conventionally used for so-called ambient compensation, which aims at removing or at least suppressing signal components originating from ambient light and detector dark level, including any analog-to-digital conversion (ADC) offset. Ambient light refers to light that is received by a detector while not originating from the emitters. The ON/OFF activation scheme of the emitters may enable each detector to measure an OFF value one or several times during a frame. As described in WO2013115710A2, an ambient-compensated energy value each for detection line may then be obtained. The ON value or an average of ON values is recorded for each detection line, i.e. the energy measured by the detector associated with detection line when the emitter associated with detection line is ON, and OFF value or an average of OFF values, i.e. the energy measured by the detector associated with detection line is recorded when all emitters are OFF. The OFF values, obtained during one or plural frames, are processed to generate a current estimate of the noise level for each individual detector, and hence for each detection line. The ambient-compensated energy value of each detection line is then determined to be the difference between the ON value and OFF value. By using the ambient measurements for validation, all influence from touching objects on the estimated noise level is avoided. Furthermore, the validation process may be designed to operate on the same data as the ambient compensation. In one embodiment, the noise estimate is continuously updated using the latest OFF values. In another embodiment, the noise estimate is given by an exponential forget estimation of the variance (standard deviation) of the OFF values.

It was realised that the above technique was insufficient to handle ambient noise having a variety of different power spectra. Although unmodulated or modulated light at low frequencies may be adequately addressed by the above technique, light modulated at a frequency higher than the frequency at which the emitters are turned ON and OFF is effectively unfiltered and may cause significant detrimental effects to performance of the touch system. Therefore, the following text describes a dynamic ambient light filter configured to respond to the characteristics of ambient light sources and filter to the ambient light source in an optimal way, taking into account restraints including touch accuracy, frame rate, and power usage.

Figures 4a-4e and figure 5 provide an example of the measurement of OFF and ON samples. Figures 4a-4e show a portion of a scan scheme sequence in which a sequence of emitters are used to transmit a signal with rest periods in between. Figure 5 shows an uncompensated signal 210 with ambient disturbances and uncompensated signal 200 without ambient disturbances. Figure 5 shows a received signal at detector 30b' over a portion of the scan scheme sequence. In figure 4a, no signal is being transmitted by any emitter for a first period of time. Figure 5 shows a corresponding received signal at detector 30b' over the first period of time marked AO. In figure 4b, emitter 30a' emits a signal for a second period of time. Figure 5 shows a corresponding received signal at detector 30b' over the second period of time marked S'. In figure 4c, no signal is being transmitted by any emitter for a third period of time. Figure 5 shows a corresponding received signal at detector 30b' over the third period of time marked A1 . In figure 4d_: emitter 30a" emits a signal for a fourth period of time. Figure 5 shows a corresponding received signal at detector 30b' over the fourth period of time marked S". In figure 4e, no signal is being transmitted by any emitter for a fifth period of time. Figure 5 shows a corresponding received signal at detector 30b' over the third period of time marked A2.

Figure 6 shows typical power spectrum received by a detector with no ambient electrical light source present and no emitter turned on.

Figure 16 shows a flow chart for a system according to an embodiment of the present invention. The flow chart shown in figure 16 may describe a sequence of steps performed by the optical touch System of the present invention in each frame. In one embodiment, the steps of the ambient light evaluation mode are performed once per frame. In alternative embodiments, the ambient light evaluation mode steps may be performed more than once per frame or once every number of frames. The ambient light evaluation mode steps may be performed at the beginning, middle or end of every frame. In the following embodiment, the ambient light evaluation mode steps are executed once per frame and at the start of every frame.

According to the embodiment shown in figure 16, the optical touch system begins the frame in an ambient light evaluation mode. Whilst in the ambient light evaluation mode, the system executes the following steps:

1 - A sample AO is recorded over a first integration time from a single detector 30b'.

2 - A sample S' is recorded over a second integration time, subsequent to the first integration time, from detector 30b'.

3 - A sample A1 is recorded over a third integration time, subsequent to the second integration time, from detector 30b'.

4 - An ambient light filter function is applied to samples AO, A1 and S' to generate a residual ambient light value.

5 - Record the first, second, and third integration time and corresponding residual ambient light value in a memory as a 'filter configuration'.

6 - (Repeat at least once) Modify the length of the first, second, and/or third integration time and repeat steps 1-5. 7 - Determine the filter configuration in the memory having the optimal residual ambient light value.

8 - Operating the optical touch system in a data sampling mode using the first, second, and third integration time of the filter configuration having the optimal residual ambient light value.

The above steps will now be described in detail.

The sample AO may be a single data value recorded from a detector over the first integration time. In this case, the first integration time describes a period of time over which an analogue filtering and/or integration of the received signal is generated. Alternatively, AO may be a series of data samples.

The sample S' may be a single data value recorded from a detector over the second integration time. In this case, the second integration time describes a period of time over which an analogue filtering and/or integration of the received signal is generated. Alternatively, S' may be a series of data samples.

Preferably, a settling time is used between the first integration time and second integration time in order to minimise distortions resulting from the rise and/or fall times of the emitter and detector components between the first integration time and second integration time.

Preferably, the same settling time is used between the second integration time and third integration time as that used between the first integration time and second integration time.

The sample A1 may be a single data value recorded from a detector over the third integration time. In this case, the third integration time describes a period of time over which an analogue filtering and/or integration of the received signal is generated. Alternatively, A1 may be a series of data samples.

In one embodiment, instead of sampling values from a single detector 30b', a plurality or all of the detectors are sampled and an average of the sampled values is used for each of AO, A1 , and S'. 4 - An ambient light filter function is applied to samples AO, A1 and S' to generate a residual ambient light value.

In a preferred embodiment, the ambient light filter function calculates the residual ambient light value according to the following equation:

A = Average of AO and A1

5 = Average of S'

Residual ambient light value = Magnitude of difference between A and S.

In an embodiment, the ambient light filter function calculates the residual ambient light value according to a linear interpolation of AO and A1 evaluated at the centre of S'. If S' is a collection of data samples, S' may be digitally integrated and normalised by the number of samples in S' to generate S.

The ambient light filter function may be executed on signal processor 12. Other digital filters as known in the art may be used.

Figure 7 shows a typical ambient filter kernel. The ambient filter kernel is applied to the samples recorded at a detector 30b'. Settling times are used in which samples are set to 0 in order to minimise distortions resulting from any signal changes between the first integration time and second integration time. Detector and emitter components typically have a rise and fall time in which output or detection stabilises to a correct value.

Figure 8 shows an ambient light filter response function where N=44 and P=15. Figure 8 is a Fourier transform of the ambient filter kernel shown in figure 7, wherein N is the number of digitally accumulated samples used for each of the first, second, and third integration times and P is the number of samples used during the settling times. The ADC in this example is running at 2.5 MHz, i.e. each data sample is 0.4 microseconds long. The left side lobe of figure 8 shows that low frequency components are somewhat filtered by filter. At the first singularity, frequencies are not filtered at all. The multiple lobes on the right of the graph at the higher frequencies are a result of the settling times used in between the integration times.

It is clear that from figure 8 that shorter integration times (i.e. first, second, and third integration times with fewer samples) can compensate low-frequencies better, i.e. moving the main lobe of figure 8 higher up in frequency. It should be noted that the ambient filter may actually worsen the residual for some frequencies. If the filter response magnitude is 1.0, the ambient filter does not reduce the ambient light impact at all. If it is above 1 .0 it will make matters worse. A desired outcome is one with the lowest filter output values possible. It is important that the main and side lobes do not line up with peaks in the power spectral density of high-frequency noise from lamps.

Figure 9 shows an ambient light filter response function where N=32 and P=15.

Figure 10 shows an ambient light filter response function where N=20 and P=15.

The data may be stored in a datastore on signal processor 120. Each filter configuration comprises a first integration time, a second integration time, a third integration time and a residual ambient light value.

In one embodiment, steps 1 -3 are repeated a multiple number of times using the same first, second, and third integration times, with the recorded residual ambient light value being the average value of the residual ambient light values calculated each repetition.

6 - (Repeat at least once) Modify the length of the first, second, and/or third integration time and repeat steps 1 -5.

Preferably, the first integration time, second integration time, and third integration time are all of the same length. Alternatively, both the first and third integration time may be the same length whilst varying in length in proportion to the second integration time. For example, if the first and third integration time are made longer by 5%, the second integration time is made longer by 5% also. However, in some embodiments, the length of the first, second, and third integration times may all be varied independently of each other.

The number of repeats of steps 1 -4 may be chosen in dependence on the number of different residual ambient light values that are needed.

In one embodiment, a number of different configurations of the first integration time and second integration time are tested to determine an optimal residual ambient light value. Depending on the number of variables, the number of configurations may be between 5 and 5000. In a preferred embodiment, 3 configurations of 300 possible configurations are tested each frame. This way, all configurations of the first integration time and second integration time may be tested over 100 frames. For an optical touch system operating at 120 frames per second, this allows all 300 configurations to be tested in under a second. Once all 300 configurations have been tested, the system preferably restarts the cycle. This allows the optical touch system to rapidly react to changes in the ambient lighting conditions.

Figure 11 shows a normalised ambient performance magnitude with no significant ambient light noise. The normalisation is achieved by dividing the value by the integration time to compensate for the fact that the actual signal becomes higher when more samples are accumulated. In figure 11 , where the number of

accumulation samples (n) is 20, the residual ambient light value is 1.3. White noise causes the performance magnitude to curve downwards to the right of the graph.

In one embodiment, the residual ambient light value of each filter configuration may be formed from an averaging of residual ambient light values over a number of cycles. In one example, the residual ambient light values of a particular filter configuration may be calculated as the average of the residual ambient light values of the filter configuration of the previous three cycles. As this average is for samples taken at different times with respect to the ambient noise, the samples are more likely to be recorded at different phases of the modulated ambient noise, reducing the effect and likelihood of aliasing between the high-frequency component of the lamp and the ambient filter kernel. Doing an average over several values make sure that the difference in phase between lamp and filter gets averaged.

In a preferred embodiment where the first integration time and second integration time are the same length, the system is configured to test a second integration time over a range of 6ps to 30ps, and incremented in steps of between 0.4με and 20με.

In one embodiment of the invention, the length of the settling times can be varied in a similar manner to the first, second, and third integration times.

7 - Determine the filter configuration in the memory having the optimal residual ambient light value.

In an embodiment, the system is configured to review the filter configurations stored in the datastore to determine an optimal filter configuration. In one

embodiment, the optimal filter configuration is determined to be the filter

configuration having the residual ambient light value with the lowest magnitude. The lowest magnitude indicates that any modulated ambient light is modulated at a frequency which has less effect on the sampled signal when sampled according to the first and second integration times of the filter configuration. Figure 12 shows typical power spectrum received by a detector with a compact fluorescent lamp light source present. We can see that the lamp has a very strong modulation frequencies around 45 kHz, including overtones at 90, 135, 180, 225, 265 kHz.

Figure 13a shows an ambient light filter response function where N=20 with respect to the compact fluorescent lamp light source. We can see that the 45 kHz signal falls within the main lobe and that the first overtone at 90 kHz lines up with one of side lobes. This filter configuration provides a bad response with a substantial amount of the ambient noise remaining unfiltered.

Figure 13b shows an ambient light filter response function where N=32 with respect to the compact fluorescent lamp light source. This filter configuration provides a similarly bad response the N=20 filter configuration with a substantial amount of the ambient noise remaining unfiltered.

Figure 14 shows an ambient light filter response function where N=44 with respect to the compact fluorescent lamp light source. In this figure we see that the 45 kHz peak lies between the main lobe and the first side lobe. The 90 kHz peak also lies between two side lobes. This means that the ambient filter is very good at reducing the ambient residual error from this lamp.

Figure 15 shows a normalised ambient performance magnitude with a compact fluorescent lamp light source present. For short integration times (N < 25) we see that the 45 kHz peak become amplified by the filter since it falls within the region where the frequency response of the filter is greater than 1.0, see new figure. As the integration time increases we see that we have an optimal alignment between power spectral density of the current lamp and ambient filter response when N is around 44.

The optimal filter configuration may also be selected as a function of the first and second integration times as well as the residual ambient light value. In one embodiment shown in figure 15, a threshold value 300 exists wherein all filter configurations having a residual ambient light value below the threshold qualify as suitable candidates for usage, i.e. The touch accuracy of the optical touch system when operating according to the first and second integration times of each of the candidates filter configurations is likely to be adequate. In this embodiment, the system may therefore select which filter configuration to use in dependence on characteristics other than the residual ambient light values. Preferably, the filter configuration having the shortest first and second integration times may be selected in order to allow the optical touch system to perform at the fastest frame rate in order to reduce touch latency. Alternatively, whilst the frame rate is maintained at a standard rate (e.g. 150 frames per second), the filter configuration having the shortest first and second integration times may be selected in order to allow the touch system to operate using the lowest power consumption possible. In figure 15, the system may select between a value of N at 44 for optimal performance, or N at 29 for shortest sampling periods (for optimal power consumption or shortest frame length).

In the data sampling mode, the system is configured to activate each of the emitters 30a in sequence, whereby the transmitted light energy can be measured at each detector that receives light from the activated emitter. Thereby, an energy value is measured for each detection line during the frame. According to the present embodiment, the system is configured to activate each emitter ON for a period of time corresponding to the second integration time. The period of time for which the emitter is activated ON is typically slightly longer (e.g. 1-20MS) than the second integration time to allow for rise and/or fall times of the emitter and detector. During the ON period, the system generates an ON sample value at each detector over the second integration time. The system is then configured to ensure that all emitters are OFF for a period of time corresponding to the first/third integration times between ON activations. During the OFF period, the system generates an OFF sample value at each detector over the first integration time. Preferably, a settling time is used between the ON and OFF periods corresponding to the rest period used between the sampling periods of the ambient light evaluation mode. An ambient-compensated energy value of each detection line is then determined to be the difference between the ON sample value and an average of the OFF sample value preceding the ON sample value and the OFF sample value recorded after the ON sample value.

Alternatively, ambient-compensated energy value of each detection line may be determined according to a linear interpolation of AO and A1 evaluated at the centre of S'. In an alternative embodiment, the system is configured to activate each emitter ON for a period of time corresponding to but shorter than the second integration time. The period of time for which the emitter is activated ON is slightly shorter (e.g. 1-20με) than the second integration time to allow for sampling the complete pulse including the rise and/or fall time of the pulse.

In one embodiment, the length of the first, second and third integration times affect the length of the frame, as the longer the first, second and third integration times are, the longer it takes to complete activation of all the emitters and the interspaced OFF periods. In some embodiments, the frame comprises a period of time reserved as slack, so that the variation of the first, second and third integration times does not require the frame length to be altered. Preferably, the third integration time can be reused as the first integration for the next detection line recorded for the same single detector.

In a preferred embodiment, the system is configured to not alter the first, second and third integration times used by the optical touch system whilst in the data sampling mode whilst a touch interaction with the touch system is occurring. As any touch interaction will necessarily significantly alter the detected touch signal during the interaction, the results of the ambient light evaluation mode can be disregarded or postponed for use until the touch interaction has finished.

In an embodiment in which the touch system utilises multiplexing of several emitters, the main difference is that the signal captured during the ON period comes from several emitters. The ambient reduction filter works in the same way as for non- multiplexing systems. In a multiplexing system there is an extra de-multiplexing step between ambient compensation and the transmission/interpolation/reconstruction steps.

Figure 17 shows a flow chart for a system according to a simpler variation of the embodiment shown in figure 16 in which two samples are recorded instead of three. According to this embodiment, the system executes the following steps whilst in ambient light evaluation mode:

2 - A sample S' is recorded over a second integration time, subsequent to the first integration time, from detector 30b'. 3 - An ambient light filter function is applied to samples AO and S' to generate a residual ambient light value.

4 - Record the first integration time and second integration time and

corresponding residual ambient light value in a memory as a 'filter configuration'.

5 - (Repeat at least once per frame) Modify the length of the first integration time and/or second integration time and repeat steps 1 -4.

6 - Determine the filter configuration in the memory having the optimal residual ambient light value.

7 - Operating the optical touch system in a data sampling mode using the first integration time and second integration time of the filter configuration having the optimal residual ambient light value.

The above steps will now be described in detail.

1 - A sample AO is recorded over a first integration time from a single detector 30b' substantially as described in the previous embodiment.

2 - A sample S' is recorded over a second integration time, subsequent to the first integration time, from detector 30b' substantially as described in the previous embodiment.

3 - An ambient light filter function is applied to samples AO and S' to generate a residual ambient light value.

A = Average of AO

S = Average of S'

Residual ambient light value = Magnitude of difference between A and S.

4 - Record the first integration time and second integration time and corresponding residual ambient light value in a memory as a 'filter

configuration'.

The data may be stored in a datastore on signal processor 12. Each filter configuration comprises a first integration time, a second integration time, and a residual ambient light value.

5 - (Repeat at least once) Repeat steps 1 -4 whilst modifying the length of the first integration time and/or second integration time.

Preferably, the first integration time and second integration time are the same length. Alternatively, both the first integration time and second integration time are varied in length in proportion to each other. For example, if the first integration time is made longer by 5%, the second integration time is made longer by 5% also.

However, in some embodiments, the length of the first integration time may be varied independently of the length of the second integration time. Alternatively, the length of the second integration time may be varied independently of the length of the first integration time.

As with the previous embodiment, the optimal filter configuration may also be selected as a function of the first, second and third integration times as well as the residual ambient light value.

According to the present embodiment, the system is configured to activate each emitter ON for a period of time corresponding to the second integration time. During the ON period, the system generates an ON sample value at each detector over the second integration time. The system is then configured to ensure that all emitters are OFF for a period of time corresponding to the first integration time between ON activations. During the OFF period, the system generates an OFF sample value at each detector over the first integration time. Preferably, a settling time is used between the ON and OFF periods corresponding to the rest period used between the sampling periods of the ambient light evaluation mode. An ambient- compensated energy value of each detection line is then determined to be the difference between the ON sample value and OFF sample value.

The touch determination process and the validation process according to the various embodiments disclosed herein may be implemented by a data processing device which is connected to sample measurement values from the detectors 30b, 31 b, 32b. The device 140 may include an input for receiving the output signal. The device 140 may further include a data collection element (or means) for obtaining the current projection values, a generation element (or means) for generating absolute values, an interpolation element (or means) for generating a matched sinogram with absolute values, an error correction element for maintaining the relative signal transmission of affected interpolation points, a conversion element (or means) for converting the absolute values of the interpolation points into relative values, a reconstruction element (or means) for generating a current interaction pattern, and an output for outputting the current interaction pattern. The device 140 may include an input for receiving the output signal. The device 140 may further include an element (or means) for obtaining the ON values, an element (or means) for obtaining the OFF values, a compensation element (or means) for generating ambient- compensated ON values, a noise estimation element (or means) for computing a noise estimate, and a datastore.

The device 140 may be implemented by special-purpose software (or firmware) run on one or more general-purpose or special-purpose computing devices. In this context, it is to be understood that each "element" or "means" of such a computing device refers to a conceptual equivalent of a method step; there is not always a one- to-one correspondence between elements/means and particular pieces of hardware or software routines. One piece of hardware sometimes comprises different means/elements. For example, a processing unit may serve as one element/means when executing one instruction, but serve as another element/means when executing another instruction. In addition, one element/means may be implemented by one instruction in some cases, but by a plurality of instructions in some other cases. Naturally, it is conceivable that one or more elements (means) are

implemented entirely by analog hardware components.

The software controlled device 140 may include one or more processing units, e.g. a CPU ("Central Processing Unit"), a DSP ("Digital Signal Processor"), an ASIC ("Application-Specific Integrated Circuit"), discrete analog and/or digital components, or some other programmable logical device, such as an FPGA ("Field Programmable Gate Array"). The device 140 may further include a system memory and a system bus that couples various system components including the system memory to the processing unit. The system bus may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system memory may include computer storage media in the form of volatile and/or non-volatile memory such as read only memory (ROM), random access memory (RAM) and flash memory. The special-purpose software, the exclusion data, the reference values, and any other data needed during execution, may be stored in the system memory, or on other removable/non-removable volatile/non-volatile computer storage media which is included in or accessible to the computing device, such as magnetic media, optical media, flash memory cards, digital tape, solid state RAM, solid state ROM, etc. The data processing device 140 may include one or more communication interfaces, such as a serial interface, a parallel interface, a USB interface, a wireless interface, a network adapter, etc, as well as one or more data acquisition devices, such as an AID converter. The special-purpose software may be provided to the device 140 on any suitable computer-readable medium, including a record medium, and a readonly memory.

While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and the scope of the appended claims.

Claims

1. A method of operating an optical touch apparatus (100), the optical touch apparatus comprising:

a panel (10),

a plurality of emitters (30a, 31a, 32a) and a plurality of detectors (30b, 31 b, 32b) arranged with respect to the panel, the plurality of detectors configured to receive light from the plurality of emitters thereby defining detection samples (D) of light transmitted between pairs of emitters and detectors,

• a controller coupled to the plurality of emitters to generate the signals and wherein each detector is configured to generate an output signal,

the optical touch apparatus being configured to operate in a data sampling mode operating in a sequence of repetitions, each repetition comprising the steps of: turning the emitters ON and OFF according to a timing configuration during each repetition such that the output signal for each detector is generated to comprise ON energy values for the detection samples (D) and at least one OFF energy value,

processing the output signals to generate a set of data samples, each data sample being generated to represent detected energy for one of the detection samples (D), and

processing the set of data samples to detect an interaction with the panel,

• the optical touch apparatus being further configured to operate in an ambient light evaluation mode comprising the steps of:

generating a first set of samples over a first integration time and a second set of samples over a second integration time using at least one detector, and applying a filter to at least the first and second set of samples to generate a residual ambient light value indicative of ambient light noise,

wherein the method comprises the steps of:

• operating the optical touch apparatus according to the ambient light

evaluation mode for a plurality of repetitions, wherein the length of at least one of the first integration time and second integration time vary between repetitions,

selecting the first integration time and/or second integration time that generated an optimal residual ambient light value, and • operating the optical touch apparatus in the data sampling mode with a timing configuration corresponding to the selected first integration time and/or second integration time.

2. The method of claim 1 , wherein the filter is configured to generate a residual ambient light value indicative of ambient light in dependence on the difference between the second set of samples and the first set of samples.

3. The method of claim 2, wherein the optimal residual ambient light value corresponds to the residual ambient light value having the lowest magnitude.

4. The method of any preceding claim, wherein the first integration time is the same length as the second integration time.

5. The method of any preceding claim, wherein a first settling period occurs between the first integration time and the second integration time.

6. The method of any preceding claim, wherein the ambient light evaluation mode comprising the steps of:

generating a first set of samples over a first integration time, a second set of samples over a second integration time, and a third set of samples over a third integration time using at least one detector (30b'), and

applying an ambient light filter to the first, second, and third set of samples to generate a residual ambient light value indicative of ambient light noise, wherein the touch apparatus is configured to operate in the data sampling mode with a timing configuration corresponding to the first, second, and third integration time of the determined the ambient light evaluation mode repetition.

7. The method of claims 6, wherein the first integration time is the same length as the third integration time.

8. The method of claims 6-7, wherein a second settling period occurs between the second integration time and the third integration time.

9. The method of claims 6-8, wherein the second integration time corresponds to the integration time of the ON signal, the first integration time corresponds to integration time of the OFF signal preceding the ON signal, and the third integration time corresponds to integration time of the OFF signal following the ON signal.

10. The method of claims 6-9, wherein the ambient light filter is configured to generate a residual ambient light value indicative of ambient light in dependence on the difference between a signal sampled over the second integration time and an average of the signal sampled over the first integration time and the signal sampled over the third integration time.

11. The method of claims 6-10, wherein the ambient light filter is configured to generate a residual ambient light value indicative of ambient light in dependence on the difference between a signal sampled over the second integration time and a linear interpolation of the signal sampled over the first integration time and the signal sampled over the third integration time.

12. The method of any preceding claim wherein the steps of the ambient light evaluation mode are performed once per frame, more than once per frame, or less often than every frame.

13. The method of any preceding claim wherein the ambient light evaluation mode steps may be performed at the beginning, middle or end of every frame.

14. The method of any preceding claim wherein the residual ambient light value is determined for a plurality of the detectors of the touch apparatus and wherein the optimal residual ambient light value is determined as a function of the plurality of residual ambient light values.

15. The method of claim 14 wherein the function is an average or maximum of the residual ambient light values.

16. The method of any preceding claim wherein a total of number of different configurations of the first, second, and third integration times is greater than 2 and wherein the apparatus being configured to cycle through the total of number of configurations in subsequent ambient light evaluation mode repetitions.

17. The method of claim 16 wherein between 1 and 10 configurations of the total number of configurations are tested each frame.

18. The method of any preceding claim wherein the first, second or third integration time have a length of between 3με to 30μ3

19. The method of any preceding claim wherein first second or third integration times are changed in steps of between 0.1 με and 20με each ambient light evaluation mode repetitions.

20. The method of any preceding claim wherein the optimal residual ambient light value may also be selected as a function of the first and second integration times as well as the residual ambient light value.

21. The method of claim 20 wherein the residual ambient light value exceeding a performance threshold value and having the shortest first and second integration times may be selected as the optimal residual ambient light value.

22. The method of any preceding claim wherein the system is configured to not alter the first, second and third integration times used by the optical touch apparatus whilst operating in the data sampling mode whilst a touch interaction with the touch system is occurring.

23. An optical touch apparatus (100) comprising:

a panel (10),

a plurality of emitters (30a, 31a, 32a) and a plurality of detectors (30b, 31 b, 32b) arranged with respect to the panel, the plurality of detectors configured to receive light from the plurality of emitters thereby defining detection samples (D) of light transmitted between pairs of emitters and detectors, a controller coupled to the plurality of emitters to generate the signals and wherein each detector is configured to generate an output signal, the optical touch apparatus being configured to operate in a data sampling mode operating in a sequence of repetitions, each repetition comprising the steps of: turning the emitters ON and OFF according to a timing configuration during each repetition such that the output signal for each detector is generated to comprise ON energy values for the detection samples (D) and at least one OFF energy value,

• processing the set of data samples to detect an interaction with the panel, the optical touch apparatus being further configured to operate in an ambient light evaluation mode comprising the steps of:

• generating a first set of samples over a first integration time and a second set of samples over a second integration time using at least one detector, and applying a filter to at least the first and second set of samples to generate a residual ambient light value indicative of ambient light noise,

wherein the touch apparatus is configured to perform the following steps:

operating according to the ambient light evaluation mode for a plurality of repetitions, wherein the length of at least one of the first integration time and second integration time vary between repetitions,

selecting the first integration time and/or second integration time that generated an optimal residual ambient light value, and

operating the touch apparatus in the data sampling mode with a timing configuration corresponding to the selected first integration time and/or second integration time.