JPH10136263A - Thermal picture recording device/method - Google Patents

Thermal picture recording device/method

Info

Publication number
JPH10136263A
JPH10136263A JP8282394A JP28239496A JPH10136263A JP H10136263 A JPH10136263 A JP H10136263A JP 8282394 A JP8282394 A JP 8282394A JP 28239496 A JP28239496 A JP 28239496A JP H10136263 A JPH10136263 A JP H10136263A
Authority
JP
Japan
Prior art keywords
thermal image
projection
image recording
infrared
global
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP8282394A
Other languages
Japanese (ja)
Inventor
Tetsushi Matsunaga
徹志 松永
Akira Umeboshino
晁 梅干野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
N II C SANEI KK
Original Assignee
N II C SANEI KK
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by N II C SANEI KK filed Critical N II C SANEI KK
Priority to JP8282394A priority Critical patent/JPH10136263A/en
Publication of JPH10136263A publication Critical patent/JPH10136263A/en
Pending legal-status Critical Current

Links

Landscapes

  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

PROBLEM TO BE SOLVED: To grasp the thermal balance of an object as three-dimensional space distribution by executing one-dimensional scanning with a thermal picture recording point as a center and recording a picture as a global picture obtained by panning, tilting and globally enlarging it. SOLUTION: An infrared detection camera 3 is placed on the revolving panhead 19 of an electric stretching pole 18 erected on an outdoor environment observation vehicle 17. A display device containing a panhead control part 20 and the control part of a thermal picture recording device and the like are provided in the observation vehicle 17. The electric stretching pole 18 can be extended to the height of 7m at maximum for avoiding the visual obstacles of vehicles and fences at the time of observation and overlooking/photographing much more ground faces and the wall faces of buildings and it is extended by an electric remote controller at 10mm pitches. The revolving panhead 19 can be panned and tilted. The maximum rotary angle is set to be 330 degrees, maximum rotary speed to be 90 deg./sec and maximum load weight to be 10kg. The angle display resolution of the panhead control part 20 is 0.01 deg., angle display precision is ±0.08 and angle reproducibility is ±0.1 deg..

Description

【発明の詳細な説明】DETAILED DESCRIPTION OF THE INVENTION

【0001】[0001]

【発明の属する技術分野】本発明は地表面や建物表面等
の3次元的空間から熱放射される赤外線を検出し、画像
化する為の熱画像収録装置及び熱画像収録方法に関す
る。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a thermal image recording device and a thermal image recording method for detecting and imaging infrared rays radiated from a three-dimensional space such as a ground surface or a building surface.

【0002】[0002]

【従来の技術】従来から、都市、建築の熱的居住性を向
上させる為に、これら建物や地表面が空間構成に及ぼす
影響の放射温度解析が広く行われている。この様な放射
温度解析時に建物の壁面や地表面の温度等をグローブ温
度計、或いは純放射を計測する放射収支計等を用いて温
度計測しても、これら対象物の温度は周囲の気温による
放射、対流、伝導の影響を受け、壁面や地表面の純放射
温度を計測することが困難である。
2. Description of the Related Art Conventionally, in order to improve the thermal livability of cities and buildings, radiation temperature analysis of the effects of these buildings and the ground surface on the spatial composition has been widely performed. In such radiation temperature analysis, even if the temperature of the wall or ground surface of the building is measured using a globe thermometer or a radiation balance meter that measures pure radiation, the temperature of these objects depends on the surrounding air temperature. Due to the effects of radiation, convection, and conduction, it is difficult to measure the net radiation temperature of the wall or ground surface.

【0003】上述のグローブ温度計、或いは放射収支計
等では点的に温度計測を行なう為に3次元的熱環境の計
測は極めて困難であった。
With the above-described globe thermometer or radiation balance meter, it is extremely difficult to measure a three-dimensional thermal environment because the temperature is measured pointwise.

【0004】[0004]

【発明が解決しようとする課題】建物の壁面等の対象物
の表面温度分布計測には赤外線放射カメラが有効である
が、従来一般に使用されている赤外線放射カメラでは計
測視野が狭すぎるため熱環境を包括的に3次元的に捕ら
える手段としては限界が生ずる問題があった。
An infrared radiation camera is effective for measuring the surface temperature distribution of an object such as a wall surface of a building. However, a conventionally used infrared radiation camera has a measurement field of view that is too narrow to provide a thermal environment. However, there is a problem in that there is a limit as a means for capturing three-dimensionally comprehensively.

【0005】本発明は叙上の問題点を解消した熱画像収
録装置及び熱画像収録方法を提供しようとするもので、
その課題とするところは、対象物の熱的収支を3次元的
な空間分布として捕らえることが出来る熱画像計測シス
テムを得ようとするものである。
The present invention aims to provide a thermal image recording device and a thermal image recording method which have solved the above-mentioned problems.
An object of the present invention is to provide a thermal image measurement system capable of capturing the thermal balance of an object as a three-dimensional spatial distribution.

【0006】[0006]

【課題を解決するための手段】本発明の熱画像収録装置
は対象物から放射される赤外線を検出し、画像化する熱
画像収録装置に於いて、上記対象物から放射される赤外
線を検出するための赤外線検出カメラを、対象物の熱画
像収録点を中心に一次元走査すると共にパン及びチルト
させて全球に拡大した全球熱画像として収録して成るも
のである。
SUMMARY OF THE INVENTION A thermal image recording apparatus according to the present invention detects infrared rays emitted from an object, and detects infrared rays emitted from the object in a thermal image recording apparatus for imaging. An infrared detection camera is one-dimensionally scanned around the thermal image recording point of the object, and is panned and tilted and recorded as a global thermal image enlarged to the whole world.

【0007】本発明の熱画像収録方法は対象物としての
地表面や建物表面の温度分布を3次元的な空間分布とし
て赤外線検出カメラで赤外線全球熱画像として検出し、
赤外線全球熱画像を平面的熱画像として表示して成るも
のである。
According to the thermal image recording method of the present invention, a temperature distribution on a ground surface or a building surface as an object is detected as a three-dimensional spatial distribution as an infrared global thermal image by an infrared detection camera.
The infrared thermal image is displayed as a planar thermal image.

【0008】本発明の熱画像収録装置及び熱画像収録方
法によれば点又は狭い範囲での熱環境情報でなく、観測
点を中心に全球の放射温度環境計測が可能となる。
According to the thermal image recording apparatus and the thermal image recording method of the present invention, it is possible to measure the radiation temperature environment around the observation point instead of the thermal environment information at a point or a narrow range.

【0009】[0009]

【発明の実施の形態】以下、本発明の熱画像収録装置及
び熱画像収録方法を図面を用いて詳記する。
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A thermal image recording apparatus and a thermal image recording method according to the present invention will be described below in detail with reference to the drawings.

【0010】先ず、本発明の熱画像収録装置の全体的な
系統図を図3を用いて説明する。
First, an overall system diagram of the thermal image recording apparatus of the present invention will be described with reference to FIG.

【0011】図3で動植物、建物等の対象物1から放射
する赤外線2の強弱は赤外線検出カメラ3を介して赤外
線検出カメラ3内の赤外線検出器3aで検出される。
In FIG. 3, the intensity of infrared rays 2 radiated from an object 1 such as animals and plants and buildings is detected by an infrared detector 3 a in the infrared detection camera 3 via the infrared detection camera 3.

【0012】赤外線2はウィンド3bを介してガルバノ
ミラー等の走査光学系3c、フォーカス用の対物レンズ
3d、チョッパ3e、リレーレンズ3f並びに冷却剤
(液体窒素やアルゴン等)や冷却素子で冷却されたHg
CdTe等の赤外線検出器3aを有し、走査光学系3c
はガルバノドライブ3gで、対物レンズ3dはフォーカ
スドライブ3hで、チョッパ3eはチョッパモータ3j
で夫々駆動され、対象物1は通常ガルバノドライバ3g
で走査光学系3cを介して2次元的に線順次走査され赤
外線の温度分布画像信号の強弱を検出するが、本例では
一次元的に走査する様に成される。
The infrared light 2 is cooled via a window 3b by a scanning optical system 3c such as a galvanometer mirror, a focusing objective lens 3d, a chopper 3e, a relay lens 3f, a coolant (such as liquid nitrogen or argon), or a cooling element. Hg
A scanning optical system 3c having an infrared detector 3a of CdTe or the like;
Is a galvano drive 3g, an objective lens 3d is a focus drive 3h, and a chopper 3e is a chopper motor 3j.
And the object 1 is usually a galvano driver 3g.
The two-dimensional line-sequential scanning is performed through the scanning optical system 3c to detect the intensity of the infrared temperature distribution image signal. In this example, the scanning is performed one-dimensionally.

【0013】この赤外線検出カメラ3内の赤外線検出器
3aで検出した温度分布画像信号はプリアンプ4に供給
され、図示しないが放射率補正、リニアライズイング、
レベル調整、センス調整、等を介してアナログ−デジタ
ル変換器(ADC)5で標本化及び符号化が行なわれ
て、デジタル−データに変換される。
The temperature distribution image signal detected by the infrared detector 3a in the infrared detection camera 3 is supplied to a preamplifier 4, and although not shown, emissivity correction, linearization,
Sampling and encoding are performed in an analog-to-digital converter (ADC) 5 through level adjustment, sense adjustment, and the like, and converted into digital-data.

【0014】ADC5からのデジタル−データは画像メ
モリ6a及び6bに供給され、赤(R)、緑(G)、青
(B)の信号を発生させる疑似カラー発生回路7のカラ
ーパレット(メモリ)7R,7G,7Bに供給されて、
デジタル−データのレベル(温度)に応じて定められた
色付けがなされる。
The digital data from the ADC 5 is supplied to image memories 6a and 6b, and a color palette (memory) 7R of a pseudo color generation circuit 7 for generating red (R), green (G), and blue (B) signals. , 7G, 7B,
The coloring is determined according to the level (temperature) of the digital data.

【0015】これらカラーパレット7R,7G,7Bか
らの各データはデジタル−アナログ変換回路(DAC)
8(8R,8G,8B)でアナログの温度分布カラー画
像信号に変換されて陰極線管(CRT)等のモニタ用の
表示装置9に供給されて画面上に対象物1の画像が色別
された温度分布パターンとして表示される。
Each data from the color pallets 7R, 7G, 7B is converted into a digital-analog conversion circuit (DAC).
8 (8R, 8G, 8B), which is converted into an analog temperature distribution color image signal, supplied to a monitor display device 9 such as a cathode ray tube (CRT), and the image of the object 1 is color-coded on the screen. It is displayed as a temperature distribution pattern.

【0016】マイクロコンピュータ(CPU)10はA
DC5、画像メモリ6a及び6b、疑似カラー発生器
7、DAC8を制御すると共にインタフェース11を介
して赤外線カメラ3のガルバノドライブ3g等を検出カ
メラコントローラ12等を介して制御する様に成されて
いる。
The microcomputer (CPU) 10 is A
The DC 5, the image memories 6a and 6b, the pseudo color generator 7, and the DAC 8 are controlled, and the galvano drive 3g and the like of the infrared camera 3 are controlled via the interface 11 via the detection camera controller 12 and the like.

【0017】ROM13及びRAM14等はCPU10
が通常有しているワーク用のメモリであり操作部15は
キーボード等の入力部である。
The ROM 13 and the RAM 14 are the CPU 10
Is a work memory normally provided, and the operation unit 15 is an input unit such as a keyboard.

【0018】上述の様に構成した熱画像収録装置16の
赤外線検出カメラ3を図1に示す屋外環境観測車17に
植立した電動伸縮ポール18の旋回雲台19上に載置す
る。
The infrared detection camera 3 of the thermal image recording device 16 configured as described above is mounted on a rotating pan head 19 of an electric telescopic pole 18 erected on an outdoor environment observation vehicle 17 shown in FIG.

【0019】雲台制御部20及び熱画像収録装置16の
制御部を含む表示装置9等も観測車17内に配設されて
いる。更に通風筒付乾湿球白金温度計、プロペラ式微風
向風速計、ネオ日射計等の気象観測装置21が観測車1
7の屋根に配設されている。
A display device 9 including a pan head control unit 20 and a control unit of the thermal image recording device 16 and the like are also provided in the observation vehicle 17. In addition, a meteorological observation device 21 such as a dry-wet bulb platinum thermometer with a ventilation tube, a propeller-type micro-wind anemometer, and a neo-pyrometer is provided by the observation vehicle
7 are located on the roof.

【0020】赤外線検出カメラ3の光学走査系は目的に
応じて三種類の波長帯3〜5μm、5.5〜7.9μ
m、8〜13μmが用いられ、波長3〜5μm帯では赤
外線検出器3aはInSbが用いられ、波長5.5〜
7.9μm及び8〜13μm帯ではHgCdTeが用い
られる。
The optical scanning system of the infrared detection camera 3 has three wavelength bands 3 to 5 μm and 5.5 to 7.9 μ according to the purpose.
m, 8 to 13 μm is used. In a wavelength band of 3 to 5 μm, InSb is used for the infrared detector 3 a and a wavelength of 5.5 to 5.5 μm.
HgCdTe is used in the 7.9 μm and 8 to 13 μm bands.

【0021】又、視野角も30°、50°,80°の3
種類を選択出来る。分解能は0.1℃〜0.15℃で温
度測定精度は±1.0℃、水平解像度344本以上、走
査角は水平30°×垂直28.5°、使用温度範囲0℃
〜40℃、瞬時視野角1.5mmradであり、使用す
る波長帯は測定する対象物1を構成する材料の分光放射
特性や、反射成分の性質、大気の影響等によって決定す
る。
The viewing angles are 30 °, 50 ° and 80 °.
You can select the type. Resolution is 0.1 ° C to 0.15 ° C, temperature measurement accuracy is ± 1.0 ° C, horizontal resolution is 344 or more, scanning angle is 30 ° horizontal x 28.5 ° vertical, and operating temperature range is 0 ° C
The instantaneous viewing angle is 1.5 mmrad, and the wavelength band to be used is determined by the spectral radiation characteristics of the material constituting the object 1 to be measured, the properties of the reflection components, the influence of the atmosphere, and the like.

【0022】電動伸縮ポール18は観測時に車や垣根等
の視覚的障害を回避し、より多くの地表面や建物の壁面
を俯瞰撮影するためにポールは最長で地上7mの高さま
で伸長可能にし、10mmピッチで電動リモコンで伸縮
し、昇降速度0.06m/secとし、ポールの振れ角
度は最長時±0.5°以内に抑えた。
The electric telescopic pole 18 avoids visual obstacles such as cars and fences at the time of observation, and allows the pole to be extended up to a height of 7 m above the ground in order to take a bird's eye view of more ground surfaces and building walls. It was expanded and contracted by an electric remote controller at a pitch of 10 mm, the elevating speed was 0.06 m / sec, and the swing angle of the pole was kept within ± 0.5 ° at the longest.

【0023】赤外線検出カメラ3を載置し、全球熱画像
を収録するために旋回雲台19はパン(水平回転)及び
チルト(鉛直回転)可能となし、最大回転角度は、33
0°とし、最大回転速度は90°/secで最大搭載重
量を10kgとした。
The rotating head 19 is capable of panning (horizontal rotation) and tilting (vertical rotation) in order to mount the infrared detection camera 3 and record a global thermal image.
0 °, the maximum rotation speed was 90 ° / sec, and the maximum loading weight was 10 kg.

【0024】又、雲台制御部20は角度表示分解能は
0.01°、角度表示精度は±0.08、角度再現性は
±0.1°の性能を持っている。
The pan head controller 20 has an angle display resolution of 0.01 °, an angle display accuracy of ± 0.08, and an angle reproducibility of ± 0.1 °.

【0025】この様に観測車17に熱画像収録装置を載
置しているので屋外熱環境計測の為に、例えば住宅地、
緑地帯、工業地帯等に自由に移動計測出来る。
Since the thermal image recording device is mounted on the observation vehicle 17 as described above, for example, in a residential area,
It can be freely moved and measured in green areas and industrial areas.

【0026】図2は上述の屋外環境観測車17を用いて
測定対象物1の全球熱画像の収録、表示及び応用例を示
す屋外熱環境計測システムのフロー図を示すものであ
る。
FIG. 2 is a flow chart of an outdoor thermal environment measuring system showing recording, display and application examples of a global thermal image of the object 1 to be measured using the outdoor environment observation vehicle 17 described above.

【0027】先ず、収録段階では第1ステップST1
測定対象物1の決定が行なわれ、第2ステップST2
熱画像収録方法の選択が成される。通常全球熱画像を得
るために赤外線検出カメラ3の載置された旋回雲台19
の回転方法は二通りあり、一つは赤外線検出カメラのガ
ルバノメータ等の走査光学系3cを図4Bの様に水平方
向に所定角度走査24しながらチルト方向25に回転
し、その動作をパン方向26に一定角度ずつ移動しなが
ら撮像を行なうチルト回転法及び走査光学系3cを図6
Bの様に鉛直方向に鉛直走査27しながらパン方向26
に回転し、チルト方向25に一定角度ずつ移動しながら
撮像を行なうパン回転法がある。
[0027] First, in the recording step determines the measuring object 1 is performed in a first step ST 1, the selection of the thermal image recording method is performed in a second step ST 2. Usually, a swivel head 19 on which the infrared detection camera 3 is mounted to obtain a global thermal image.
There are two rotation methods. One is to rotate a scanning optical system 3c such as a galvanometer of an infrared detection camera in a tilt direction 25 while scanning a predetermined angle 24 in the horizontal direction as shown in FIG. FIG. 6 shows a tilt rotation method and a scanning optical system 3c for performing imaging while moving at a constant angle to each other.
Panning 26 while vertical scanning 27 in the vertical direction as shown in B
There is a pan rotation method in which the image is rotated while moving at a constant angle in the tilt direction 25.

【0028】旋回雲台19の回動範囲がパン及びチルト
共に±180°以上あれば立体角として4πSrを包括
する。
If the rotation range of the revolving head 19 is ± 180 ° or more for both pan and tilt, 4πSr is included as a solid angle.

【0029】全球熱画像の収録は、どちらの回転方法で
も可能である。即ち、図2で第3ステップST3 でパン
回転法を選択するか、第4ステップST4 でチルト回転
法を選択することで収録された熱画像は第5又は第6ス
テップST5 又はST6 の様に正距円筒図法及び横正距
円筒図法となる。
The recording of the global thermal image is possible by either rotation method. That is, choose the pan rotation method in the third step ST 3 in FIG. 2, thermal image was recorded by selecting the tilt rotation method in the fourth step ST 4 is the fifth or sixth step ST 5 or ST 6 The equirectangular projection and the horizontal equirectangular projection are as follows.

【0030】図4(A)及び(B)は旋回雲台19のチ
ルト回転法を示すものであり、本例では前記した様にパ
ン、チルト共に可動範囲が330°で一方向の旋回角度
が±165°であり、この場合、図4(A)及び(B)
並びに図6(A)及び(B)に示す様に死角23となる
範囲(斜線部)がチルト方向(下方向)とパン方向の左
右端に生ずるが、下方向は観測車17の屋根であるとみ
てよく、又、左右はパン方向回転角を±180°以上に
すれば生ずることはない。
FIGS. 4 (A) and 4 (B) show the tilt rotation method of the swivel pan head 19. In this embodiment, as described above, the movable range of both pan and tilt is 330 °, and the swivel angle in one direction is as described above. ± 165 °, in which case FIGS. 4 (A) and (B)
As shown in FIGS. 6A and 6B, a range (shaded area) that forms a blind spot 23 occurs at the left and right ends in the tilt direction (downward) and the panning direction, and the downward direction is the roof of the observation vehicle 17. It can be considered that the left and right sides do not occur when the panning rotation angle is set to ± 180 ° or more.

【0031】図5はチルト回転法による全球原画像の1
例を、図7はパン回転法による全球原画像を示すもので
あり、チルト回転法では光学走査系(以下スキャナーと
記す)3Cの視野角は水平方向(走査方向)に50°と
し、パン方向に45°ずつ8等分して収録している。
FIG. 5 shows one of the global original images obtained by the tilt rotation method.
For example, FIG. 7 shows a global original image by the pan rotation method. In the tilt rotation method, the viewing angle of an optical scanning system (hereinafter referred to as a scanner) 3C is 50 ° in the horizontal direction (scanning direction), and the pan direction is And divided into 8 equal parts at 45 °.

【0032】この様な熱原画像のままでは目視判断、ポ
リゴン描画等に利用して行く上で不適当であるので広角
レンズによる歪補正を行った後に、地図投影法を利用し
た幾何変換を行なう。収録された全球熱画像は幾何補正
され、使用目的に適応した地図投影法で表現される。全
球熱画像は本来球画像であるため、収録された画像は、
それが平面投影されたものとみなせる。そこで水平線を
赤道とし、天頂を北極とした、各画素の位置を緯度、経
度とした地図投影法で第7ステップST7 の様に表現す
る。
Since such an original heat image is unsuitable for use in visual judgment, polygon drawing, etc., distortion correction by a wide-angle lens is performed, and then geometric transformation using a map projection method is performed. . The recorded global thermal image is geometrically corrected and represented by a map projection method adapted to the purpose of use. Since the global thermal image is originally a spherical image, the recorded image is
It can be regarded as a plane projection. Therefore the horizontal line and the equator, the zenith and the Arctic, expressed as a seventh step ST 7 on map projection of the position of each pixel and latitude, and longitude.

【0033】次に、第8ステップST8 では熱画像から
得たい情報は何かを考えて、第9ステップST9 の様に
ある対象物の方位角、天頂角を知りたい時は第10ステ
ップST10の様に正距円筒図法で表示する。
[0033] Next, the information to be obtained from the eighth step ST 8 in the heat image thinking of something, the azimuth angle of the object is in the way of the ninth step ST 9, when you want to know the zenith angle is the 10th step as of ST 10 to display in equirectangular projection.

【0034】図6及び図7で示したパン回転法による原
画像の投影法でもある正距円筒図法は経線と緯線が碁盤
目となっており、各画素の測定点からの高度と水平角の
目視判断が可能である。よってこの図法を用いれば、測
定対象建物等が面している方位と高度を推定することが
できる。
In the equirectangular projection, which is also a projection method of the original image by the pan rotation method shown in FIGS. 6 and 7, the meridians and the latitude lines are grids, and the height and horizontal angle of each pixel from the measurement point are measured. Visual judgment is possible. Therefore, by using this projection, it is possible to estimate the azimuth and altitude where the measurement target building or the like faces.

【0035】第11ステップST11はある対象物1の焦
点からの立体角等を収録したい時には第12ステップS
12の様にモルワイデ(MOLLWEIDE)図法或いはサンソン
−フラムスチード(SANSON-FLAMSTEED)図法で表示す
る。
[0035] when the 11 step ST 11 to be recorded solid angle like from the focal point of the object 1 in the twelfth step S
Mollweide like the T 12 (MOLLWEIDE) projection or Sanson - Furamusuchido (SANSON-FLAMSTEED) to display in the projection.

【0036】モルワイデ図法は図8に示す様に中央経線
28と赤道対応緯線29を長軸とする楕円で表したもの
であり、サンソン−フラムスチード図法は中央経線28
を基準として、各緯線を正長に分割し、対応する分割点
を曲線で連ねたもので収録熱画像は図9の様になる。
The Mollweide projection is represented by an ellipse having a central meridian 28 and an equator-corresponding parallel 29 as major axes as shown in FIG. 8, and the Sanson-Flamsteed projection is represented by a central meridian 28.
Each of the parallel lines is divided into regular lengths with reference to, and the corresponding division points are connected by a curve. The recorded thermal image is as shown in FIG.

【0037】サンソン−フラムスチード図法は、全球熱
画像の球面を1ピクセルを微小平面とする多面体で近似
し、その表面における全画素を平面上に展開した状態に
ほぼ等しい。これは、緯線・経線ともに連続した図法と
しては、球面に最も近いものといえる。従って原画像を
幾何変換しても、画素の補間や重複によりデータとして
の信頼性を低下してはならない場合は、図9に示すサン
ソン−フラムスチード図法を用いる。
The Sanson-Flamsteed projection approximates the spherical surface of a global thermal image with a polyhedron having one pixel as a small plane, and is almost equivalent to a state in which all pixels on the surface are developed on a plane. This can be said to be the closest to a spherical surface as a projection in which both latitude and longitude lines are continuous. Therefore, if the reliability of the data must not be reduced by interpolation or duplication of pixels even when the original image is geometrically transformed, the Sanson-Flamsteed projection shown in FIG. 9 is used.

【0038】サンソン−フラムスチード図法は等積図法
であるが、中央経線28及び赤道対応緯線29より遠い
位置の対象物ほど形状の歪みが著しいため、画像上の被
写体へのポリゴン描画等を行い、測定点における立体角
を計測したい場合には図8に示すモルワイデ図法の方が
適切といえる。
The Sanson-Flamsteed projection is an equal-area projection. However, since an object located farther from the central meridian 28 and the equator-corresponding parallel line 29 has a remarkably distorted shape, a polygon is drawn on a subject on an image and measurement is performed. When it is desired to measure a solid angle at a point, the Mollweide projection shown in FIG. 8 is more appropriate.

【0039】又、上述のサンソン−フラムスチード図法
とモルワイデ図法の長所をとったグード(GOODE)図法の
分裂図等を用いてもよい。
Alternatively, a division diagram of the GOODE projection, which has the advantages of the above-mentioned Sanson-Flamsteed projection and Mollweide projection, may be used.

【0040】更に、第13ステップST13に示す様に測
定点の天空率を考慮した。天空からの熱放射等を知りた
い時には第14ステップST14で示す様に正射図法(図
10)を用いて表示する様にする。即ち、正射図法(正
射投影)による全球熱画像上では、対象物1の面積は測
定点に仮定した円形の微小平面に対する形態係数を表し
ているからである。
[0040] Further, considering sky view factor of the measurement points as shown in the thirteenth step ST 13. When you want to know the thermal radiation and the like from the sky to as displayed using the orthographic projection as shown by the 14 step ST 14 (FIG. 10). That is, on a global thermal image by the orthographic projection (orthogonal projection), the area of the object 1 represents a view factor with respect to a circular minute plane assumed to be a measurement point.

【0041】上述した様な各ステップでモルワイデ図法
で全球熱画像の収録表示を行った場合の応用一例を図2
の第15ステップST15より説明する。
FIG. 2 shows an example of an application in which a global thermal image is recorded and displayed by the Mollweide projection in each step as described above.
It is described than the 15 step ST 15 of.

【0042】第15ステップST15では全球熱画像によ
る放射温度分布は3次元幾何情報と連係される。即ち、
収録対象物1の3次元幾何情報はCADにより入力され
て、表面温度情報等に付加される。
The radiometric temperature distribution by Zentamanetsu image In a 15 step ST 15 is associated with 3D geometric information. That is,
The three-dimensional geometric information of the recording target 1 is input by CAD and added to surface temperature information and the like.

【0043】次に第16ステップST16や第17ステッ
プST17に示す様に測定対象物1からの純放射量の算
出、或いは人体のMRT(平均放射温度)分布への加工
等が行われる。
Next processing, etc. to the net radiation amount calculation, or human MRT (mean radiant temperature) distribution from the measured object 1 as shown in the sixteenth step ST 16 and 17 step ST 17 is performed.

【0044】上記した放射収支の算出時には正確な情報
を得るために放射の大気中の伝搬による減衰や、対向面
からの反射を考慮しなければならないが、いずれも3次
元幾何情報から計算可能である。又、測定地域内の人体
に対するMRTの空間分布を、他の温度要素の実測デー
タと併用し、人体への熱負荷や体感温度の空間分布へと
加工する。その際、地上の人体に対して、全球熱画像の
死角となる部位からの放射が大きく影響すると解析に支
障を来すため、いくつかの地点から全球熱画像を収録
し、この問題を回避する必要がある。
At the time of calculating the radiation balance described above, in order to obtain accurate information, it is necessary to consider attenuation due to the propagation of radiation in the atmosphere and reflection from the opposing surface, but all of them can be calculated from three-dimensional geometric information. is there. In addition, the spatial distribution of the MRT for the human body in the measurement area is processed into the spatial distribution of the heat load on the human body and the sensible temperature by using the measured data of other temperature elements together. At this time, if the radiation from the blind spot of the global thermal image greatly affects the human body on the ground, it will hinder the analysis, so global thermal images are recorded from several points to avoid this problem There is a need.

【0045】全球原画像を幾何変換して地図投影法で表
示する際、補間法には最近隣法を採用した。そのため、
画像上には補間された画素が多く生じ、2次元の画像と
してデータを解析する場合には無視できなくなる。図1
1はサンソン図法の被補間画素を示し、図12は正距円
筒図法の被補間画素を示している。図11及び図12で
30と示されたものが補間画素である。
When the global original image was geometrically transformed and displayed by the map projection method, the nearest neighbor method was adopted as the interpolation method. for that reason,
Many interpolated pixels occur on the image, and cannot be ignored when analyzing data as a two-dimensional image. FIG.
1 shows a pixel to be interpolated by the Sanson projection, and FIG. 12 shows a pixel to be interpolated by the equirectangular projection. What is indicated by 30 in FIGS. 11 and 12 is an interpolation pixel.

【0046】そこで各図法における画素の補間状況を検
討した。下表1に補間されている画素の全画素数に対す
る割合(被補間率と呼ぶ)を示す。画素サイズは赤道・
子午線の長さで定義している(原画像のサイズは204
8×1024である)。
Therefore, the state of pixel interpolation in each projection was examined. The following Table 1 shows the ratio of interpolated pixels to the total number of pixels (referred to as an interpolated ratio). Pixel size is equator
Defined by the length of the meridian (the size of the original image is 204
8 × 1024).

【0047】[0047]

【表1】 [Table 1]

【0048】当然の結果ではあるが、今回検討対象とし
た4種類の図法のうち、被補間率が最大であったのは、
極付近の間延びが著しい正距円筒図法で47.9%(等
倍)、最小地を示したのは、近似的に球面自体を表すサ
ンソン図法で18.5%であった。また、サンソン図法
においては、被補間画素がほぼ均等に分布しており、画
像の一部に誤差が集中して存在する危険がない。以上の
検討結果より、全球熱画像をサンソン図法で表示するこ
とが、2次元の画像データとして最も信頼性の高い表示
方法であることを確認した。
As a matter of course, the largest interpolated rate among the four projections studied in this study is as follows.
The equirectangular projection, which has a remarkable elongation near the pole, is 47.9% (1: 1), and the minimum is 18.5% in the Sanson projection, which roughly represents the spherical surface itself. Further, in the Sanson projection, pixels to be interpolated are distributed almost uniformly, and there is no danger that errors are concentrated in a part of the image. From the above examination results, it was confirmed that displaying the global thermal image by the Sanson projection is the most reliable display method as two-dimensional image data.

【0049】上述の第15ステップST15での全球熱画
像と3次元幾何情報とを関連付けるには全球熱画像を3
次元幾何情報内の該当する座標内に取り付け、スキャン
ライン法により、テクスチャマッピングをしながら2次
元画像に投影することによって行った。
[0049] The Zentamanetsu image to associate the total Tamanetsu images and 3D geometric information in the fifteenth step ST 15 described above 3
The measurement was performed by attaching the image to the corresponding coordinates in the dimensional geometric information and projecting it onto a two-dimensional image while performing texture mapping by the scan line method.

【0050】図13は図9に示された全球熱画像を三次
元幾何情報に投影したもので図13で中庭に見える白い
部分は全球熱画像の収録地点からみて死角スペース31
である。三角形の方は、広場の中央の彫刻によってつく
られた死角スペース32で、白い円の方は、観測車17
の位置に相当する。観測車17の位置が、死角を表す白
い円になっているのは、観測車の屋根面にあたり、熱画
像を収録する必要がないためである。死角スペースがM
RT分析に支障をきたすならば、計測点の数は増やされ
なければならない。なお、図13では図14の実例概念
図中手前の33の建物は除いて示してある。
FIG. 13 is a projection of the global thermal image shown in FIG. 9 onto three-dimensional geometric information. In FIG. 13, the white portion visible in the courtyard is a blind spot space 31 when viewed from the recording point of the global thermal image.
It is. The triangle is the blind spot space 32 created by sculpture in the center of the square, and the white circle is the observation vehicle 17
Position. The position of the observation vehicle 17 is a white circle representing a blind spot, because it corresponds to the roof surface of the observation vehicle and there is no need to record a thermal image. Blind spot space is M
If the RT analysis is hindered, the number of measurement points must be increased. Note that FIG. 13 excludes 33 buildings in the front in the conceptual diagram of the actual example in FIG. 14.

【0051】次に測定対象区域(屋外)の人体に対する
MRTの空間分布の算出方法を詳記する。本例では測定
対象区域のMRTの空間分布の算出は、人体モデルの形
状を決定させてから行なう様にする。図15は、この様
な方法を用いたMRT分布算出結果を示す。
Next, a method of calculating the spatial distribution of the MRT for the human body in the measurement target area (outdoor) will be described in detail. In this example, the spatial distribution of the MRT in the measurement target area is calculated after the shape of the human body model is determined. FIG. 15 shows an MRT distribution calculation result using such a method.

【0052】即ち、図13のビルの中庭での実測日は冬
季(1月)で天候は晴れ、周囲の外気温は9.3℃であ
った。計算を簡単にするため、人体モデル形状は微小6
面体を採用した。等形態係数に分割された放射ベクトル
の数は、1点につき98×6である。ある地点で必要な
対象物が死角に隠れ、全球熱画像からの放射温度の抽出
が不可能な場合、測定点を増やす代わりに補間で十分な
場合もある。今回は、地物が見えない場合は周辺気温
を、天空が見えない場合は別の方位における天空放射温
度を補間に充てた。
That is, the actual measurement day in the courtyard of the building shown in FIG. 13 was winter (January), the weather was fine, and the surrounding outside temperature was 9.3 ° C. In order to simplify the calculation, the shape of the human body model is
A facepiece was adopted. The number of radiation vectors divided into isomorphic coefficients is 98 × 6 per point. When a required object is hidden in a blind spot at a certain point and it is impossible to extract the radiation temperature from the global thermal image, interpolation may be sufficient instead of increasing the number of measurement points. This time, the surrounding air temperature is used for interpolation when the feature is not visible, and the sky radiation temperature in another direction is used for the interpolation when the sky is not visible.

【0053】図15は、MRT分布の算出結果である。
中庭中央付近の地は比較的均一であるが、MRTは建物
の近くでは均一ではない。中庭全体で4℃の分布があ
る。夏季においてはMRTの空間分布のばらつきは更に
大きくなるものと予想される。
FIG. 15 shows the calculation result of the MRT distribution.
The ground near the center of the courtyard is relatively uniform, but the MRT is not uniform near the buildings. There is a 4 ° C distribution throughout the courtyard. It is expected that the variation in the spatial distribution of the MRT will be even greater in summer.

【0054】任意の場所におけるMRTの値を正確に計
算するためには、固体表面の放射温度から表面温度を推
定したり、天空の放射温度を、全赤外域(0.75μm
〜1mm)の放射量を反映した値に補正する必要があ
る。
In order to accurately calculate the MRT value at an arbitrary location, the surface temperature is estimated from the radiation temperature of the solid surface, or the radiation temperature of the sky is measured in the all infrared region (0.75 μm
(1 mm) must be corrected to a value reflecting the radiation amount.

【0055】上述の全球熱画像収録の赤外線検出カメラ
や気象観測装置は観測車17上で行ったが、計測点が固
定された場所では計測点を固定し得ることは当然であ
る。
Although the above-described infrared detection camera and weather observation device for recording a global thermal image were carried out on the observation vehicle 17, it is obvious that the measurement points can be fixed at places where the measurement points are fixed.

【0056】更に赤外線検出カメラの視野角、旋回雲台
19の回転角は1例として述べた各値に限定されるもの
ではなく、計測の精度、空間分解能によって適宜選択し
得る。
Further, the viewing angle of the infrared detection camera and the rotation angle of the revolving pan head 19 are not limited to the values described as one example, but can be appropriately selected depending on the measurement accuracy and the spatial resolution.

【0057】[0057]

【発明の効果】本発明の熱画像収録装置及び熱画像収録
方法によれば屋外環境での観測点を中心とする全球の放
射環境計測が可能となり、地物や人体等の熱収支を地物
表面分布や人体の存在する空間分布として捕らえること
の出来るものが得られる。
According to the thermal image recording device and the thermal image recording method of the present invention, it is possible to measure the radiation environment of the entire sphere centering on an observation point in an outdoor environment, and to measure the heat balance of a terrestrial object or a human body. What can be grasped as a surface distribution or a spatial distribution where a human body exists can be obtained.

【図面の簡単な説明】[Brief description of the drawings]

【図1】本発明の熱画像収録装置を搭載した観測車の略
線図である。
FIG. 1 is a schematic diagram of an observation vehicle equipped with a thermal image recording device of the present invention.

【図2】本発明の熱画像収録装置の屋外熱環境計測シス
テムのフロー図である。
FIG. 2 is a flowchart of an outdoor thermal environment measuring system of the thermal image recording device of the present invention.

【図3】本発明の熱画像収録装置の系統図である。FIG. 3 is a system diagram of the thermal image recording device of the present invention.

【図4】本発明の熱画像収録装置に用いる旋回雲台の回
転方法(チルト回転法)の説明図である。
FIG. 4 is an explanatory diagram of a method of rotating a rotating head (tilt rotation method) used in the thermal image recording apparatus of the present invention.

【図5】本発明に用いるチルト回転法による全球原画像
例である。
FIG. 5 is an example of a global original image by the tilt rotation method used in the present invention.

【図6】本発明の熱画像収録装置に用いる旋回雲台の回
転方法(パン回転法)の説明図である。
FIG. 6 is an explanatory diagram of a method of rotating a pan head (pan rotation method) used in the thermal image recording apparatus of the present invention.

【図7】本発明に用いるパン回転法による全球原画像例
である。
FIG. 7 is an example of a global original image by the pan rotation method used in the present invention.

【図8】本発明に用いるモルワイデ図法による全球熱画
像例である。
FIG. 8 is an example of a global thermal image by the Mollweide projection method used in the present invention.

【図9】本発明に用いるサンソン−フラムスチード図法
による全球熱画像例である。
FIG. 9 is an example of a global thermal image by the Sanson-Flamsteed projection used in the present invention.

【図10】本発明に用いる正射図法による全球熱画像例
である。
FIG. 10 is an example of a global thermal image by the orthographic projection method used in the present invention.

【図11】サンソン図法の被補間画素例である。FIG. 11 is an example of a pixel to be interpolated by the Sanson projection.

【図12】正距離円筒図法の被補間画素例である。FIG. 12 is an example of a pixel to be interpolated by the equidistant cylindrical projection.

【図13】全球熱画像と3次元幾何情報の連係説明図で
ある。
FIG. 13 is an explanatory diagram of a link between a global thermal image and three-dimensional geometric information.

【図14】図13の実測概念図である。FIG. 14 is a conceptual diagram of actual measurement in FIG.

【図15】本発明によるMRT分布算出結果を示すパタ
ーンである。
FIG. 15 is a pattern showing an MRT distribution calculation result according to the present invention.

【符号の説明】[Explanation of symbols]

3 赤外線検出カメラ 17 屋外環境観測車 18 電動伸縮ポール 19 螺旋雲台 20 雲台制御部 3 Infrared detection camera 17 Outdoor environment observation vehicle 18 Electric telescopic pole 19 Spiral head 20 Head control unit

Claims (5)

【特許請求の範囲】[Claims] 【請求項1】 対象物から放射される赤外線を検出し、
画像化する熱画像収録装置に於いて、 上記対象物から放射される赤外線を検出するための赤外
線検出カメラを該対象物の熱画像収録点を中心に一次元
走査すると共にパン及びチルトさせて全球に拡大した全
球熱画像として収録して成ることを特徴とする熱画像収
録装置。
1. detecting infrared light radiated from an object,
In the thermal image recording device for imaging, the infrared detection camera for detecting the infrared radiation emitted from the object is one-dimensionally scanned around the thermal image recording point of the object, and panned and tilted, and is globally. A thermal image recording device characterized in that the thermal image recording device is recorded as a global thermal image magnified in FIG.
【請求項2】 前記赤外線検出カメラで収録した全球熱
画像を地図投影法により平面熱画像に変換して表示して
成ることを特徴とする請求項1記載の熱画像収録装置。
2. The thermal image recording apparatus according to claim 1, wherein a global thermal image recorded by the infrared detection camera is converted into a planar thermal image by a map projection method and displayed.
【請求項3】 前記地図投影法が正距円筒図法、正射図
法、サンソン図法、モルワイデ図法のいずれか1つであ
ることを特徴とする請求項2記載の熱画像収録装置。
3. The thermal image recording apparatus according to claim 2, wherein the map projection method is one of an equirectangular projection, an orthographic projection, a Sanson projection, and a Mollweide projection.
【請求項4】 対象物としての地表面や建物表面の濃度
分布を3次元的な空間分布として赤外線検出カメラで赤
外線全球熱画像として検出し、該赤外線全球熱画像を平
面的熱画像として表示して成ることを特徴とする熱画像
収録方法。
4. An infrared global thermal image is detected by an infrared detection camera as a three-dimensional spatial distribution of a density distribution on a ground surface or a building surface as an object, and the infrared global thermal image is displayed as a planar thermal image. A thermal image recording method, comprising:
【請求項5】 前記赤外線全球熱画像を平面的熱画像に
幾何変換時の被補間率が小さな投影法を用いたことを特
徴とする請求項4記載の熱画像収録方法。
5. The thermal image recording method according to claim 4, wherein a projection method with a small interpolated ratio at the time of geometric transformation of the infrared global thermal image into a planar thermal image is used.
JP8282394A 1996-10-24 1996-10-24 Thermal picture recording device/method Pending JPH10136263A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP8282394A JPH10136263A (en) 1996-10-24 1996-10-24 Thermal picture recording device/method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP8282394A JPH10136263A (en) 1996-10-24 1996-10-24 Thermal picture recording device/method

Publications (1)

Publication Number Publication Date
JPH10136263A true JPH10136263A (en) 1998-05-22

Family

ID=17651843

Family Applications (1)

Application Number Title Priority Date Filing Date
JP8282394A Pending JPH10136263A (en) 1996-10-24 1996-10-24 Thermal picture recording device/method

Country Status (1)

Country Link
JP (1) JPH10136263A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003038752A1 (en) * 2001-10-29 2003-05-08 Sony Corporation Non-flat image processing apparatus, image processing method, recording medium, and computer program
JP2007148784A (en) * 2005-11-28 2007-06-14 D Link Corp Image composition system and image composition method
JP2011034279A (en) * 2009-07-31 2011-02-17 Toyota Motor Corp Three-dimensional temperature distribution display device, texture creation device and three-dimensional temperature distribution display method
JP2013169338A (en) * 2012-02-21 2013-09-02 Sakura Rubber Co Ltd Luminaire mounting tool for fire hose
KR101456519B1 (en) * 2013-02-28 2014-11-07 조선대학교산학협력단 A Stand Supporting Equipments for Diagnosis Using Thermal Imaging Techniques
CN105004933A (en) * 2015-08-11 2015-10-28 王佳莺 Monitoring and analysis device of urban electromagnetic radiation
WO2016087903A1 (en) * 2014-12-05 2016-06-09 Audi Ag Observation device, vehicle with an observation device and method for operating an observation device
CN109782775A (en) * 2019-03-13 2019-05-21 刘乐 A kind of automobile obstacle avoidance system based on thermal image
JPWO2022259296A1 (en) * 2021-06-07 2022-12-15

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003038752A1 (en) * 2001-10-29 2003-05-08 Sony Corporation Non-flat image processing apparatus, image processing method, recording medium, and computer program
CN100380403C (en) * 2001-10-29 2008-04-09 索尼公司 Image processing device and image processing method for non-planar image
US7483590B2 (en) 2001-10-29 2009-01-27 Sony Corporation Image processing apparatus and image processing method for non-planar image, storage medium, and computer program
JP2007148784A (en) * 2005-11-28 2007-06-14 D Link Corp Image composition system and image composition method
JP2011034279A (en) * 2009-07-31 2011-02-17 Toyota Motor Corp Three-dimensional temperature distribution display device, texture creation device and three-dimensional temperature distribution display method
JP2013169338A (en) * 2012-02-21 2013-09-02 Sakura Rubber Co Ltd Luminaire mounting tool for fire hose
KR101456519B1 (en) * 2013-02-28 2014-11-07 조선대학교산학협력단 A Stand Supporting Equipments for Diagnosis Using Thermal Imaging Techniques
WO2016087903A1 (en) * 2014-12-05 2016-06-09 Audi Ag Observation device, vehicle with an observation device and method for operating an observation device
CN105004933A (en) * 2015-08-11 2015-10-28 王佳莺 Monitoring and analysis device of urban electromagnetic radiation
CN109782775A (en) * 2019-03-13 2019-05-21 刘乐 A kind of automobile obstacle avoidance system based on thermal image
CN109782775B (en) * 2019-03-13 2020-04-14 刘乐 Automobile obstacle avoidance system based on thermal image
JPWO2022259296A1 (en) * 2021-06-07 2022-12-15

Similar Documents

Publication Publication Date Title
CN112254637B (en) Tunnel excavation surface scanning device and detection method based on multiple fusion data
US10116920B2 (en) Balancing colors in a scanned three-dimensional image
JP5816778B2 (en) Laser scanner with additional detector
CN114743021B (en) Fusion method and system of power transmission line image and point cloud data
US7697126B2 (en) Three dimensional spatial imaging system and method
US9599715B2 (en) Scanner display
CN106123798B (en) A kind of digital photography laser scanning device
AU2005200937A1 (en) Imaging system
CN112907647B (en) A three-dimensional spatial dimension measurement method based on a fixed monocular camera
JPH10136263A (en) Thermal picture recording device/method
US5045699A (en) Heat imaging camera with a cooled detector mosaic
CN104732557A (en) Color point cloud generating method of ground laser scanner
Asawa et al. Portable recording system for spherical thermography and its application to longwave mean radiant temperature estimation
US20070058044A1 (en) Systems and methods for measuring spatial and angular performance of a visual display
JP4161025B2 (en) Global thermal image recording apparatus, global thermal image recording method and system
Lin et al. Thermal texture selection and correction for building facade inspection based on thermal radiant characteristics
CN118553055B (en) Three-dimensional electronic fence early warning method based on thunder integrated machine
CN118230173B (en) Real-time judgment method and system for star sky ornamental suitability
Scaioni et al. High-resolution texturing of building facades with thermal images
Asano et al. Development of an urban thermal environment measurement system using a new spherical thermography technique
CN117541114A (en) Method for rapidly evaluating high spatial resolution thermal comfort of urban outdoor scene
JP2004191195A (en) Illuminance measuring instrument
CN118741309B (en) Variable-focus multi-path camera fusion device and method
EP4222427B1 (en) Heliostat calibration
JP3356356B2 (en) X-ray equipment

Legal Events

Date Code Title Description
A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20031202

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 4

Free format text: PAYMENT UNTIL: 20071212

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20081212

Year of fee payment: 5

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 6

Free format text: PAYMENT UNTIL: 20091212

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 7

Free format text: PAYMENT UNTIL: 20101212

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101212

Year of fee payment: 7

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 8

Free format text: PAYMENT UNTIL: 20111212

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121212

Year of fee payment: 9

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 9

Free format text: PAYMENT UNTIL: 20121212

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131212

Year of fee payment: 10

LAPS Cancellation because of no payment of annual fees