JPH0738706B2 - Video display system - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video display system, and more particularly to a video display system capable of reducing the amount of information transmitted between a video signal generator and a display device.

[0002]

2. Description of the Related Art Conventionally, when band compression of a video signal or an image signal is performed, redundancy such as DPCM, orthogonal transformation, and subsampling, that is, correlation of image data, The height is used to perform uniform processing on the image data of the entire screen. Therefore, when an image based on the image signal that has been band-compressed and transmitted is displayed on the screen, the observer can see the image displayed on the screen at a uniform resolution. That is, the image displayed on the screen has a uniform resolution in every part, and the observer can see the image having the same resolution everywhere on the screen.

However, since the resolution of the image is uniform, when the image is viewed in detail in the observer's gaze area,
There may be cases where the resolution is insufficient due to coarse pixels. In order to prevent this, it is necessary to widen the transmission band and improve the resolution, but generally it is not preferable to widen the transmission band.

On the other hand, in the technical field of broadcasting and image communication, from the conventional form in which the same image or image is shown to many observers under the same conditions, each observer can individually adjust a desired image portion to his or her preference. Is observing and observing. Therefore, uniform band compression of video signals or image signals cannot meet this demand, and the development of a technique for individually band compression according to the observer's need has become an important issue.

The present invention has been made to solve the above problems and provides a video display system capable of reducing the amount of information transmitted between a video signal generator and a display device. To aim.

[0006]

A video display system according to the present invention includes a video signal generating means, a video display means for displaying a video on a screen based on a video signal generated by the video signal generating means, and a display. Gaze state detecting means for detecting a gaze area and gaze time length on the screen of an observer observing the captured image, and determining that the detected gaze time length exceeds a predetermined time length And a video information reducing means for reducing the video information amount in the gazing area of the video signal generated by the video signal generating means in response to the watching time length determining means.

[0007]

In the image display system according to the present invention, the gaze time length determining means determines that the observer's gaze time length detected by the gaze state detecting means exceeds a predetermined time length. . When the gaze time length exceeds a predetermined time length, the video information reducing means responds to the gaze time length determining means, and the video information of the gaze area of the video signal generated from the video signal generating means. Reduce the amount. The image display means displays on the screen an image based on the image signal whose information amount has been reduced by the image information reducing means. Therefore, the amount of video information of the video signal transmitted to the video display means is reduced.

[0008]

FIG. 1 is a block diagram of a video display system showing an embodiment of the present invention. Referring to FIG. 1, this video display system includes a video signal generator 1 for generating a video signal of a video to be displayed on the screen of a display device 10, and a low pass filter (LPF) 2 for limiting a band of the video signal.
And an image processing device 8 that performs image processing on a video signal according to a change in the gaze area on the screen of the observer, an eye movement detection device 5 that detects the movement of the eyeball of the observer, and an observation using a magnetic sensor. A head movement detecting device 6 for detecting a person's head movement,
A gazing point processing device 7 that obtains the line-of-sight data of the observer by processing the detected head movement data and eye movement data, and a gaze area separation device 3 that obtains the gaze area data of the observer based on the gaze point data. Including and The image processing device 8 generates a control signal based on the switching unit 81 that receives the video signal Sa generated from the video signal generation device 1 and the video signal Sb provided through the low pass filter 2, and the observer's gaze area data. The control signal generation unit 82 and the gaze time determination unit 83 that determines that the observer's gaze time length exceeds a predetermined time length and generates the band control signal BW are included.

In operation, the eye movement and head movement of the observer are detected by the eye movement detecting device 5 and the head movement detecting device 6, respectively. Based on the detected motion data, the gazing point processing device 7 performs calculation processing to obtain line-of-sight data indicating the movement of the line of sight of the observer. The obtained line-of-sight data is given to the gaze area separation device 3, where the coordinates of the gaze area on the screen on which the observer is gazing and the coordinates of the surrounding area are obtained. The coordinates of the gaze area and its peripheral area are given to the control signal generation unit 82 in the image processing device 8. The above processing will be described in detail later.

The video signal Sa generated from the video signal generator 1 is directly supplied to the switching unit 81 in the image processing device 8.
Is also given to the low-pass filter 2. The band-limited video signal Sb is output via the low-pass filter 2 and given to the switching unit 81.
The control signal generation unit 82 generates a control signal required to control the switching unit 81 based on the gaze area data provided from the gaze area separation device 3. The processing in the switching unit 81 and the gaze time determination unit 83 will be described below.

FIG. 10 is a graph showing the relationship between the human gaze time and the correct answer rate. The relationship between the gaze time and information processing in the brain will be described below with reference to FIG. Details are described in a document entitled “Gaze time control presentation device and visual search experiment using the same” by Hongo, Yamada et al. The experiment result shown in FIG. 10 is to give subjects A, B, and C a task of counting the number of circles from the figure in which patterns such as circles, squares, and triangles are scattered, and various gaze time is given. It shows how the correct answer rate of the count changes. Therefore,
The abscissa shows the gaze time (msec), and the ordinate shows the correct answer rate (%). From this figure, with a gaze time of 270 ms,
You can get the same correct answer rate as in the normal case where there is no limit on the time of gazing. That is, given such a task,
270m gaze time required for correct counting
It is s, and it can be seen that it is not necessary to present the image to the subject over a longer period of time. From various other visual experiments, the time Tv required for visual information processing in the human brain
Is known to be around 300 ms (± 100 ms). Therefore, in the video display system shown in FIG. 1, by utilizing the above experimental results, the video signal of the gazing portion is band-limited after the time Tv has elapsed from the start of gazing on the screen, thereby observing. It is intended to reduce the amount of transmission information of a video signal without reducing the visual information processing capability of a person's image.

FIG. 2 is a timing chart for explaining the processing in the image processing apparatus 8 shown in FIG. Figure 2
As shown in (A), the gaze area on the screen is C1, C2,
It is assumed that the vehicle moves in the order of C3. That is, as shown in FIGS. 2B and 2C, the observer is
Pay attention to the area C1 on the screen in the period from 1 to t2,
It is assumed that the region C2 is watched during the period from time t3 to t6 and the region C3 is watched during the period from time t7 to t8. The vertical axis of FIG. 2B indicates a change in the state of the observer, that is, whether the observer is in the gaze state or the non-gaze state. The vertical axis of FIG. 2C indicates the band BW of the gaze area component of the video signal supplied to the display device 10.
Shows the change.

The gaze time determination unit 83 shown in FIG. 1 stops on the basis of the gaze area data generated from the gaze area separating device 3 within the certain area where the line of sight of the observer is longer than the time length Tv. Is determined. That is, the period in which the observer is gazing at the gazing region C1 is between time t1 and t2, and the time length of this period is shorter than Tv.
Therefore, the gaze time determination unit 83 outputs the signal BWW for controlling the video signal to have a wide band during this period, and supplies this to the control signal generation unit 82. Next, the observer gazes at the region C2 during the period from time t3 to time t6. When the gaze time exceeds t4, the gaze time length is Tv
Therefore, the gaze time determination unit 83 outputs a signal BWN for controlling to narrow the band of the video signal, and supplies this to the control signal generation unit 82. In response to the signal BWN, the control signal generation unit 82 gives a control signal to the switching unit 81 so that the video signal component of the suspension area C2 has a narrow band. After the lapse of a predetermined time length Tm from time t4, the band of the video signal in the gazing area C2 is widened again. That is, since the gaze time determination unit 83 outputs the signal BWW after the elapse of this time Tm, the control signal generation unit 8
2 widens the band of the video signal in the gazing area C2.

The observer gazes at the region C3 during the period from time t7 to t8, but since this time length does not exceed Tv, the band of the video signal in the gaze region C3 is kept wide.

Next, the operation of the control signal generator 82 shown in FIG. 1 will be described. The switching unit 81 does not limit the band, that is, the video signal S having a wide band.
a and a video signal Sb band-limited by the low-pass filter 2, that is, having a narrow band. The control signal generation unit 82 outputs a control signal necessary for controlling the band in the gazing area of the video signal to the switching unit 81.
Give to. That is, the control signal generation unit 82 responds to the band control signal BW given from the gaze time determination unit 83, generates a control signal necessary for controlling the band of the video signal in the gaze area, and switches it. Give to 81. The switching unit 81 synthesizes a video signal based on the received video signals Sa and Sb in response to the applied control signal. As a result, the combined video signal Sc in which the band of the video signal in the gaze area is controlled is the switching unit 81.
And the composite video signal is transmitted to the display device 10. Since the band of the transmitted composite video signal Sc is reduced in the video signal component of the gazing area according to the gaze state of the observer, the amount of transmission information is reduced.

In the embodiment shown in FIG. 1, the case where the band of the video signal is limited by using the low-pass filter 2 has been described, but not limited to this, for example, the resolution of the color information included in the video signal is lowered. It is pointed out that it is also effective to change the transmission band to monochrome in some cases.

Sight V for determining the radius of the gaze area on the screen
The angle θ is preferably 3 ° to 5 °, including Parafovea, which has the next highest resolution after the fovea and is also processed in parallel. For the time length Tm, for example, after the time Tv has elapsed, 200 ms (± 100 ms), which is the time between the following gazing points, is suitable. This is because after the time Tv has passed, the human eye generally
This is based on the fact that a decline in visual function called Saccadic Suppression has been reported at this time in order to start a flying movement called e. Note that the time lengths Tv and Tm and the visual Vθ change depending on the observation target and are set arbitrarily through visual experiments.

In the following description, the processing for detecting the movement of the line of sight of the observer and extracting the gaze area based on the movement of the observer in the image display system shown in FIG. 1 will be described.

FIG. 5 shows the head movement detecting device 6 shown in FIG.
It is a block diagram of. Referring to FIG. 5, the head movement detecting device 6 includes an exciting coil 61 that generates a reference magnetic field, and a drive circuit 65 that AC-drives the exciting coil 61.
A sensor coil 62 mounted on the observer's head, a detection circuit 63 for detecting a voltage induced in the sensor coil 62 according to the movement of the observer's head, and a voltage detected by the detection circuit 63. And a CPU 64 that obtains the observer's head movement by arithmetic processing based on the above. The CPU 64 also controls the drive circuit 65. The exciting coil 61 is composed of three coils wound in three axis (X, Y, Z) directions. Each coil is supplied with driving power supply voltage generated from the drive circuit 65 and having different frequencies. Therefore, the exciting coil 61 generates a magnetic field based on the three axes, and the observer is present in the magnetic field. The sensor coil 62 also includes three detection coils wound in the (X, Y, Z) directions. When the observer's head moves, the sensor coil 6 is moved according to the movement.
An electromotive force is induced in each coil in 2. By detecting the electromotive force generated in each coil by the detection circuit 63, a signal corresponding to the movement of the observer's head can be obtained. CPU64
Calculates the head movement data H indicating the movement of the observer's head by performing arithmetic processing based on this signal, and outputs it.

FIG. 3 is a schematic diagram showing the directions of motion parameters of the observer's head motion. The above-mentioned sensor coil 62 is attached to the observer's forehead. The parameters Hx, Hy, Hz, Hθ, Hφ and Hψ shown in FIG. 3 respectively indicate the movement of the head movement in the directions shown in this figure.
That is, Hx indicates a horizontal parallel movement of the head, Hy indicates a front-back movement, and Hz indicates a vertical parallel movement. Further, Hθ indicates a motion in which the neck is tilted to the left and right, Hφ indicates a vertical rotational motion of the neck, and Hψ indicates a horizontal rotational motion of the head.

FIG. 4 is a schematic view showing how the observer looks at the display device 10. Referring to FIG. 4, the observer
Eyeglasses such as goggles are attached to observe the screen of the display device 10. The above-mentioned exciting coil 61 is placed at a predetermined position. Sensor coil 62 (not shown)
Are attached to the eyeglasses 66. When the observer's head moves, the positional relationship between the excitation coil 61 and the sensor coil 62 relatively changes, and head movement detection is performed.

FIG. 6 shows an eye movement detecting device 5 shown in FIG.
2 is a schematic diagram showing the detection principle in FIG. With this device,
An example is one using a scleral reflection method that utilizes the difference in reflectance between the white and black eyes of the cornea surface. As another method, a corneal reflection method or a search coil method can be applied.
Referring to FIG. 6, sensors 71 and 72 are arranged in front of left and right eyes E1 and E2, respectively. Each sensor 7
A light emitting element 73 is arranged at the center of 1 and 72.
As the light emitting element 73, a light emitting diode for infrared projection with a relatively wide directivity of about ± 21 ° is used. Light receiving elements 74 are provided on both sides of the light emitting element 73. As the light receiving element 74, a photodiode having a sharp directivity of about ± 10 ° is used. The light projected from the light emitting element 73 toward the eyeball is reflected by the white and black eyes. Since the reflectance differs between the white and black eyes, the difference in the reflectance is amplified, and if the difference is taken, the horizontal (left and right) eye movement is obtained as output, and if the sum is taken, the vertical (up and down) eyeballs are obtained. Is obtained as an output.

The positions of the sensors 71 and 72 with respect to the eyeball differ between the horizontal direction and the vertical direction. The horizontal sensor 71 detects reflected light at the center with respect to the upper and lower sides of the eyeball, and the vertical sensor 72 moves downward. Are arranged so as to detect reflected light. A horizontal sensor 71 is arranged on one eye E1, and a vertical sensor 72 is arranged on the other eye E2. By using these simultaneously, it is possible to detect a two-dimensional eye movement. Using such a detection principle, the moving speed, moving direction, moving distance,
A method for obtaining the gaze time is disclosed in, for example, Japanese Patent Laid-Open No. 60-1
It is disclosed in Japanese Patent No. 26140. Therefore, the data E indicating the eye movement of the observer is obtained from the eye movement detecting device 5 shown in FIG. 1 and given to the gazing point processing device 7.

FIG. 7 is a schematic view of the spectacles 66 worn by the observer. The glasses 66 are provided with a horizontal sensor 71 for detecting eye movement and a sensor coil 62 for detecting head movement.

FIG. 8 is a schematic diagram for explaining the movement of the line of sight due to the head movement. Referring to FIG. 8, generally, when a visual target moves, the eyeball moves following the visual target. Instead of this movement of the eyeball, it is possible to follow the visual target by moving the head. Usually, both movements are used together. The head movement includes parallel movement depending on the orientation of the foot and spine, and rotational movement realized by the neck, spine, hip, foot, and the like. This head movement is converted into an eyeball rotation angle by the method described later, and the head movement correction eyeball rotation angle α shown in FIG. 8 is obtained. In the gazing point processing device 7 shown in FIG. 1, the line-of-sight data is expressed by a combination of the head movement correction eyeball rotation angle α and the rotation angle θ of the eyeball itself. Here, the sum of the rotation angles α and θ is generically defined as the line of sight.

The processing in the gazing point processing device 7 shown in FIG. 1 will be described below. The gazing point processing device 7 receives the head movement data H and the eye movement data E. It is assumed that (Hx, Hy, Hz) is given as the horizontal motion component of the data H, and (Hφ, Hθ, Hψ) is given as the rotational motion component. The line of sight changes depending on the horizontal component (Hx, Hy, Hz) of the head movement, which is the eyeball rotation angle (Ex, Ey).
The following formula is used to convert to. Here, D is the distance between the subject and the observation target.

[0027]

[Equation 1] E _x = 180 / π · tan ⁻¹ Hx / (D + Hy) …… (1) E _y = 180 / π · tan ⁻¹ Hz / (D + Hy) …… (2)

When the neck is tilted toward the left or right shoulder by Hθ, the coordinates of the eye movement system rotate. Therefore, H
It is necessary to convert the eye movement coordinate system (Xe, Ye) inclined by θ to the coordinate system (Xe ′, Ye ′) orthogonal to the original observation target. Therefore, the following equation is used.

[0029]

[Equation 2] Xe ′ = Xe · cosHθ + Ye · sinHθ (3) Ye ′ = − Xe · sinHθ + Ye · cosHθ (4) Movement of the line of sight (Xh, Yh) realized by head movement
Is expressed by the following equation from the equations (1) and (2).

[0030]

## EQU3 ## Xh = Ex + Hφ (5) Yh = Ey + Hφ (6) Therefore, the line-of-sight movement (Xv, Y) considering the head movement is calculated.
v) is expressed by the following equation from the equations (3) to (6).

[0031]

## EQU00004 ## Xv = Xe '+ Xh (7) Yv = Ye' + Yh (8) By using equations (7) and (8), head movement and eye movement are combined. It is possible to reproduce the normal movement of the line of sight. The processing described above is shown in FIG.
Is performed in the gazing point processing device 7 shown in FIG. 1 and the line-of-sight data indicating the movement of the line of sight is given to the gaze area separating device 3.

The gaze area separation device 3 obtains data indicating the gaze area on the screen of the observer based on the given line-of-sight data. Specifically, as shown in FIG. 2A, the gazing area C3 has its radius Dth determined by the product of a predetermined threshold velocity Vth and the sampling interval Ts. That is, with the position of the current viewpoint as P1, the radius D
An area determined by th is a gaze area C3 on the screen.

Since the coordinates of the data obtained by the line-of-sight detection are different from the coordinates on the screen of the display device 10, the following coordinate conversion is performed. FIG. 9 is a coordinate diagram showing the correspondence between the coordinates in line-of-sight detection and the coordinates on the screen. Referring to FIG. 9A, it is assumed that the screen of display device 10 is placed at a distance D from the observer, and the size of the screen is vertical Dv and horizontal Dh. Therefore, the coordinates on the screen are represented by (1, 1) ... (Px, Py). Therefore, the viewing angle of the screen viewed from the observer is represented by the following equation, as shown in FIG.

[0034]

[Formula 5] a = 180 / π · tan ⁻¹ Dh / (2 · D) …… (9) b = 180 / π · tan ⁻¹ Dv / (2 · D) …… (10) where the line of sight Assuming that the movement angle is Xv, Yv, the equation for converting this to the coordinates Xp, Yp on the screen is expressed by the following equation.

[0035]

## EQU6 ## Xp = f (Xv) = p + (rp) .Xv / (a-Xv) (11) Yq = g (Yv) = q + (sq) .Yv / (b-Yv) (12) Therefore, by using the above formula, the coordinate data of the gaze area and its surrounding area obtained based on the sight line detection can be converted into the coordinate data on the screen. As a result, the control signal generator 82 shown in FIG.
A control signal necessary for performing image processing can be generated by distinguishing the gaze area and its peripheral area.

[0036]

As described above, according to the present invention, the video information reducing means for reducing the video information amount in the gazing area of the video signal to be transmitted is provided according to the watching time length of the observer. Therefore, a video display system capable of reducing the information amount of the video signal transmitted to the display device is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram of a video display system showing an embodiment of the present invention.

FIG. 2 is a timing chart for explaining processing in the image processing apparatus shown in FIG.

FIG. 3 is a schematic view showing directions of respective motion parameters of the observer's head motion.

FIG. 4 is a schematic diagram showing how an observer is looking at a display device.

5 is a block diagram of the head movement detecting device shown in FIG. 1. FIG.

FIG. 6 is a schematic diagram showing a detection principle in the eye movement detecting device shown in FIG.

FIG. 7 is a schematic view of glasses worn by an observer.

FIG. 8 is a schematic diagram for explaining the movement of the line of sight in head movement.

FIG. 9 is a coordinate diagram showing correspondence between coordinates in line-of-sight detection and coordinates on the screen.

FIG. 10 is a graph showing a relationship between a human gaze time and a correct answer rate.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 video signal generator 2 low-pass filter 7 gazing point processing device 8 image processing device 10 display device 81 switching unit 82 control signal generating unit 83 gazing time determination unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Keiichi Ueno 5 Seiji-cho, Seika-cho, Soraku-gun Kyoto Prefecture

Claims (2)

[Claims] 1. A video signal generating means, a video display means for displaying a video on a screen based on the video signal generated by the video signal generating means, and an observation for observing the video displayed by the video display means. A gaze state detecting means for detecting a gaze area and a gaze time length on a person's screen, and determining that the gaze time length detected by the gaze state detecting means exceeds a predetermined time length Gaze time length determining means, and in response to the gaze time length determining means, video information reducing means for reducing the video information amount of the gaze area of the video signal generated from the video signal generating means, A video display system, wherein the video display means displays a video on a screen based on the video signal whose information amount has been reduced by the video information reduction means. 2. The predetermined time length is determined based on a human gaze time length-correct answer rate characteristic, and the image information reducing means responds to the gaze time length determining means, The video display system according to claim 1, further comprising video band reduction means for reducing a band of a gaze area component of the video signal generated from the video signal generation means.