WO2011078682A1

WO2011078682A1 - Method for removing flickering

Info

Publication number: WO2011078682A1
Application number: PCT/NO2010/000448
Authority: WO
Inventors: Petter Schatvet
Original assignee: Tandberg Telecom AS
Current assignee: Tandberg Telecom AS
Priority date: 2009-12-23
Filing date: 2010-12-07
Publication date: 2011-06-30
Anticipated expiration: 2012-06-23
Also published as: US20110205393A1; CN102812700B; NO331527B1; EP2517458B1; US8294784B2; EP2517458A1; EP2517458A4; NO20093588A1; CN102812700A

Abstract

The present invention discloses a method providing removal of the distractive illuminative flickering due to discrepancy between line frequency, screen updating rate and camera exposure time in video conferencing and video recording by primarily adjusting the screen updating rate and secondary the camera exposure time to achieve a flickering free experience of the video captured by the camera.

Description

Method for removing flickering

Field of invention

The present invention discloses a method and a video conference terminal or video recording unit adapted to remove flickering in video pictures of objects illuminated by ambient lighting and screen light.

Technical background

In order to have a meeting involving participants not lo^¬ cated in the same area, a number of technological systems are available. These systems may include video conferencing, web conferencing and audio conferencing.

The most realistic substitute for real meetings is high-end video conferencing systems. Conventional video conferencing systems comprise a number of endpoints communicating real- time video, audio and/or data streams over WAN, LAN and/or circuit switched networks. The endpoints include one or more monitors, cameras, microphones and/or data capture devices and a codec, which encodes and decodes outgoing and incoming streams, respectively. Video conference systems are used throughout the business community for point to point audio and visual communication between individuals. The users of video conferencing may sit in workplace environments such as personal office or cubical, small or large meeting rooms or board rooms. Video conference applications installed on multi purpose

computers has contributed to widespread the use of personal video conferencing even more.

Figure 1 illustrates a customary situation where a person as an object captured by a camera sits close to the display which shows the far side. As screens become bigger and brighter, the screen will illuminate the object which can lead to artefacts such as colour and intensity variations on objects. The artefacts can visually manifest itself in flickering or colour variations that are modulated by whatever is seen on the screen. The flickering is caused by the camera' s exposure time aka integration time to record each individual frame in a video picture combined with the frequencies of the ambient lighting and the refresh rate/frequency of the screen illuminating objects captured by the camera. A screen contains thousands of illuminative pixels in array. Visual content is displayed by successively applying voltage on each pixel in a pixel line from left to right for each line from the top of the screen and downwards. A screen's refresh rate is therefore the frequency of illuminating the pixels in the whole screen from top left to bottom right.

Depending on the refresh rate of the screen in question, it may cause its own flickering even if the camera is

optimized for the normal ambient environment. Furthermore, with the advent of larger and high intensity emissive displays, indirect illumination caused by such emitters may cause flickering in all or portions of the image seen by the camera.

It is usually desirable for the integration time to be as long as possible i.e. maximally responding to the frame rate ( 1/framerate ) to maximize signal to noise (S/N) ratio in resulting footage. However, due to line frequency of 50Hz (60Hz in US, 400Hz in airplanes), lamps operating at mains AC exhibit intensity variations at double the line frequency. To avoid flickering (i.e. phantom intensity variations over time, looks like flickering or line rolling) , the normal approach is to select an appropriate integration time for the environment with the aim of overcoming the problem during capture. The general

relationship that ensures cameras do not beat against a global cyclical stimuli f is: Integration time = n/(2f), n= 1, 2, 3, ....

The result of adjusting the integration time according to this equation is that the camera will capture integer numbers of half light wavelengths thereby avoiding

variations of light intensity among captured frames.

As already mentioned, it is not only ambient light causing flickering. Especially, in situations as discussed above when captured objects is positioned closed to relatively big screens, the illumination contribution from the screen may be considerable, and if the integration time is adapted to ambient light only, flickering may occur due to the screen's refresh rate.

Figure 2 shows the effect of a scene optimized for 50Hz ambient at 10ms integration time, with the intensity variations of various common screen refresh rates/

frequencies when lit 100% by single sinusoidal intensity source. The X-axis is complete 10ms integration windows. As can be seen, the intensity variations increase as the refresh rate moves away from 50 Hz. This is experienced as flickering by people watching the video pictures captured by the camera.

It is therefore a need for a method removing flickering illumination contribution from both ambient lighting and screen light.

Brief description of the object of the invention

The present invention provides method and a correspondingly adjusted video conferencing or video recording unit for removing flickering from video of objects illuminated by a screen operating on a refresh rate (f_sc) among a first value set including screen compatible refresh rate values and an ambient lighting operating on a certain line

frequency (fi) captured by a camera with a certain exposure time (t_e) among a second value set including camera

compatible exposure time values, by executing the steps of; determining whether a non-zero integer number n exists

I. a. if said non-zero integer number n does not exist, then removing current value of t_e from the second value set, assigning t_e to a value in the second value set fulfilling t_e= n/2fi and repeating step I;

I.b. if said non-zero integer number n does exist, then determining whether non-zero integer numbers for k and 1 exist so that kf_scr= lfj;

.1. if said non-zero integer numbers for k and 1 exist, then implementing t_e as the camera's exposure time and f_sc as the screen's refresh rate; I.b.2. if said non-zero integer numbers for k and 1 does not exist, then calculating a third value set including values for f_scr fulfilling kf_scr= lfi where k and 1 are non-zero integer numbers, and

determining whether one or more values in the third value set are equal to one or more values in the first value set;

I.b.2.1. if not one or more values in said third value set are equal to one or more values in the first value set, then executing step I. a; I.b.2.2. if one or more values in said third value set are equal to one or more values in the first value set, then assigning f_scr to one of the one or more values in the third set being equal to one or more values in the first value set, and I. a.1.1.1. implementing t_e as the camera's exposure time and _st ^as the screen' s refresh rate.

Brief description of the drawings The present invention will be described in detail with reference to a preferred embodiment as shown in the following drawings :

Figure 1 is an illustration of a situation where an object captured by a camera is illuminated both by a screen and ambient lighting.

Figure 2 is a graph illustrating the intensity variations of light captured by a camera caused by various common screen update rates,

Figure 3 is a flow chart illustrating an example of a method according to the present invention.

Detailed description of the invention

In the following, the present invention will be discussed by describing a preferred embodiment, and by referring to the accompanying drawings. However, people skilled in the art will realize other applications and modifications within the scope of the invention as defined in the

enclosed independent claims.

According to the present invention, the distractive

illuminative flickering due to discrepancy between line frequency, screen updating rate and camera exposure time in video conferencing and video recording is removed by an inventive method of primarily adjusting the screen updating rate and secondary the camera exposure time to achieve a flickering free experience of the video captured by the camera . It is therefore assumed that both the screen refresh rate and camera exposure time at least to a certain extent are adjustable from data routines through driving devices of the screen and the camera, respectively. This is typically the case in videoconferencing between general purpose computers having a software videoconferencing client installed, and a camera either plugged into a USB-port or integrated in the computer's screen. It could likewise also be the situation for video recording units, when the recorded objects are positioned closes a screen.

Some web cameras have APIs (Application Programming

Interface) enabling interaction with other software, much, in the same way that a user interface facilitates

interaction between humans and computers. This also applies for both external and internal screens. For example, by entering into the Control Panel of Microsoft Windows operating system, one can manually select the refresh rate of the screen or the exposure time of the camera, normally between fixed compatible values. Likewise, through an API associated with the screen or the camera, a software routine can make the same selection by directing certain requests or instructions interpretable for the APIs.

The line frequency of the ambient lighting is predefined, but has a great impact to the selection of screen refresh rate and camera exposure time according to the present invention. However, it can vary between countries, regions, and settings so it should be measured by e.g. the camera before the actual selection occurs. Optical measurement of line frequency from ambient lighting has become a trivial task well known in the art as e.g. described in US Patent 5394217 - "Method and apparatus for determining line frequency and detecting variable frequency light sources", Gaboury et al. Alternatively, it the frequency could also be measured directly from the power line input of the video conferencing device, or simply predefined. Figure 3 is a flow chart illustrating an example of the process of reaching a flickering free video experience wherein a camera captures objects that are at least partly illuminated by light from a screen where the display are being line wised refreshed with a certain screen refresh rate .

The example starts by some preparation steps. The screen refresh rate is normally set to a default rate, as well as the camera exposure time normally is set to a default time. These default values are in step 1 and 2 captured and allocated to f_sc and t_e respectively so that the default values becomes the selected values right away, if they in combination with the line frequency provides flickering free video captured by the camera without any

modifications. The line frequency itself is not necessarily known and should be measured e.g. optically by the camera before assigning it to fi. As already indicated, screens are normally compatible with a limited numbers of allowed refresh rates,, and these are in a preparation step 4 captured from the driver unit of the screen and put in a set of possible refresh rate values ω.

In the decision step 5, it is determined whether the flickering free equation t_e= n/2fx where n is a non-zero integer number has a solution with the current value for t_e. If not, the process proceeds to step 6, where t_e is changed to an alternative value which is not tested in the process before. The selected value for t_e should preferably be as high -as possible, but the selection could also be based on other considerations, provided that it picks an potential exposure time that not has been tested earlier in the process.

The process is then looped back to decision step 5.

If in decision step it appears that t_e= n/2fi has a

solution, decision step 7 then determines whether there is one or more solution of the equation kf_scr =lfi where k and 1 are non-zero integer numbers with current f_scr. This test simply reveals if a least common multiple of l/f_scr and 1/fi exists, and if so, it is quite trivial to deduct that the general flickering free equation: integration time = n/2f n= 1,2,3,... has a solution for both f_scr and fi, and there will be no flickering contribution either from the screen or the ambient light with the corresponding exposure time

(integration time) .

If decision step 7 is positive, the process is terminated, and current f_sc and t_e are selected to be the screen refresh rate and camera exposure time respectively. If the values are not equal to the default values, they must be modified in the associated driver units, by e.g. sending a request to the associated API for changing to the selected

value ( s ) .

If decision step 7 is negative, the process proceeds to step 8 where a set (φ) of possible values for f_sc

fulfilling the equation kf_scr =lfi where k and 1 are nonzero integer numbers is determined. The process then proceeds to decision step 9 investigating whether there are one or more values in φ corresponding to one or more values of compatible screen refresh rates (φ) already fetched in preparation step 4. If hot, the process turns back to step 6 to select an alternative exposure time (t_e) . However, if it appears that φ and φ have common values, the process proceeds to step 10 where the highest common value of the sets φ and ω is allocated as f_SCr- In this example the highest value is selected assuming that a highest possible screen refresh rate is preferred. Alternatively, this selection could also be based on other considerations, but the basic thing is to reach a value being present in both sets of values. The process is then finally terminated, and current f_sc and t_e are selected to be the screen refresh rate and camera exposure time, respectively. They are then modified in the associated driver units, by e.g. sending a request to the associated API for changing to the selected value (s)

Claims

C L A I M S

1. A method for removing flickering from video recording c objects illuminated by a screen operating on a refresh ra (f_sc) among a first value set including screen compatible refresh rate values and an ambient lighting operating on certain line frequency (fi) captured by a camera^' with a certain exposure time (t_e) among a second value set

including camera compatible exposure time values,

characterized in determining whether a non-zero integer number n exists so that t_e= n/2f_1;

I.b. if said non-zero integer number n does exist, then determining whether non-zero integer numbers for k and 1 exist so that kf_scr= lfi;

I.b.l. if said non-zero integer numbers for k and 1 exist, then implementing t_e as the camera's exposure time and f_sc as the screen's refresh rate;

I.b.2. if said non-zero integer numbers for k and 1 does not exist, then calculating a third value set including values for f_scr fulfilling kf_scr= lfi where k and 1 are non-zero integer numbers, and determining whether one or more values in the third value set are equal to one or more values in the first value set;

I.b.2.1. if not one or more values in said third value set are equal to one or more values in the first value set , then executing step I. a; I.b.2.2. if one or more values in said third value set are equal to one or more values in the first value set, then assigning f_scr to one of the one or more values in the third set being equal to one or more values in the first value set, and

I.b.2.3. implementing t_e as the camera's exposure time and fsc as the screen' s refresh rate.

2. A method according to claim 1,

characterized in executing the following

preparation step prior to step I:

Optically measuring the line frequency from the camera captured lighting and assigning the measured line frequency to fi.

3. A method according to claim 1 or 2,

characterized in executing the following

preparation step prior to step I :

Fetching a default screen refresh rate value from a screen driver device and assigning the default screen refresh rate value to f_sc.

4. A method according to claim 1 or 2,

characterized in executing the following

preparation step prior to step I:

Fetching a default exposure time value from a camera driver device and assigning the default exposure time value to t_e.

5. A method according to claim 1 or 2,

characterized in that step I . b .1 and I.b.2.3 further includes the following steps: Instructing a camera driver device associated API set the camera exposure time to t_e, and

Instructing a screen driver device associated API set the camera screen refresh rate to f_sc.

5 6. A method according to claim 1 or 2,

characterized in that, in step I . a . , the value in the second value set fulfilling t_e= n/2fi is the highest value in the second value set.

7. A method according to claim 1 or 2,

o characterized in that, in step I.b.2.2, assigning f_scr to the highest one of the one or more values in the third set being equal to one or more values in the first value set.

8. A video conference terminal or a video recording units with a camera and a screen incorporated therein or

connected thereto, the screen is adapted to operate on a refresh rate (f_sc) among a first value set including screen compatible refresh rate values, the camera is adapted to capture video pictures with a certain exposure time (t_e)o among a second value set including camera compatible

exposure time values of objects illuminated by the screen and an ambient lighting operating on a certain line frequency ( fi) ,

characterized in that the video conference unit5 or video recording unit is adapted to:

I. determining whether a non-zero integer number n exists

I. a. if said non-zero integer number n does not exist, then removing current value of t_e from the secondo value set, assigning t_e to a value in the second

value set fulfilling t_e= n/2fi and repeating step I; b. if said non-zero integer number n does exist, then determining whether non-zero integer numbers for k and 1 exist so that kf_scr= Ifi;

I.b.l. if said non-zero integer numbers for k and 1 exist, then implementing t_e as the camera's exposure time and f_sc as the screen's refresh rate; b.2. if said non-zero integer numbers for k and 1 does not exist, then calculating a_. third value set including values for f_scr fulfilling kf_scr= lfi where k and 1 are non-zero integer numbers, and

I.b.2.1. if not one or more values in said third value set are equal to one or more values in the first value set, then executing step I. a;

I. b.2.2. if one or more values in said third value set are equal to one or more values in the first value set, then assigning f_scr to one of the one or more values in the third set being equal to one or more values in the first value set, and

I. b.2.3. implementing t_e as the camera' s exposure time and f_sc as the screen' s refresh rate.