WO2020021949A1 - Système d'imagerie pour véhicule ferroviaire - Google Patents

Système d'imagerie pour véhicule ferroviaire Download PDF

Info

Publication number
WO2020021949A1
WO2020021949A1 PCT/JP2019/025429 JP2019025429W WO2020021949A1 WO 2020021949 A1 WO2020021949 A1 WO 2020021949A1 JP 2019025429 W JP2019025429 W JP 2019025429W WO 2020021949 A1 WO2020021949 A1 WO 2020021949A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
area
railway vehicle
image
obstacle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/025429
Other languages
English (en)
Japanese (ja)
Inventor
広幸 小林
雄介 高橋
拓也 二神
勝大 堀江
直人 瀬戸
世支明 山崎
陽平 服部
博章 伊藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toshiba Corp
Toshiba Infrastructure Systems and Solutions Corp
Original Assignee
Toshiba Corp
Toshiba Infrastructure Systems and Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp, Toshiba Infrastructure Systems and Solutions Corp filed Critical Toshiba Corp
Priority to SG11202012917RA priority Critical patent/SG11202012917RA/en
Priority to EP19840166.3A priority patent/EP3829155B1/fr
Priority to CN201980048961.0A priority patent/CN112470456B/zh
Publication of WO2020021949A1 publication Critical patent/WO2020021949A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B61—RAILWAYS
    • B61K—AUXILIARY EQUIPMENT SPECIALLY ADAPTED FOR RAILWAYS, NOT OTHERWISE PROVIDED FOR
    • B61K9/00—Railway vehicle profile gauges; Detecting or indicating overheating of components; Apparatus on locomotives or cars to indicate bad track sections; General design of track recording vehicles
    • B61K9/08—Measuring installations for surveying permanent way
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B61—RAILWAYS
    • B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L23/00—Control, warning or like safety means along the route or between vehicles or trains
    • B61L23/04—Control, warning or like safety means along the route or between vehicles or trains for monitoring the mechanical state of the route
    • B61L23/041—Obstacle detection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more two-dimensional [2D] image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G03B35/08—Stereoscopic photography by simultaneous recording
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images

Definitions

  • the embodiment of the present invention relates to an imaging system for a railway vehicle.
  • an imaging apparatus for imaging a region in a traveling direction of a railway vehicle for the purpose of inspecting a railway line and detecting an obstacle (a gazing object that requires gazing) existing in the railway line while the railway vehicle is traveling.
  • Various systems using a stereo camera have been proposed.
  • Such a stereo camera is mainly used for assisting detection of an obstacle and determination of whether or not the obstacle affects the operation of the railway vehicle in a railway vehicle on which a driver actually rides and actually drives. By doing so, it is used as a system for improving the safety of operation.
  • the railway vehicle imaging system includes a first imaging unit and a second imaging unit.
  • the first imaging unit is provided at the front end of the leading railway vehicle, and can acquire a stereo image in a first area in the traveling direction of the railway vehicle.
  • the second imaging unit is provided at the front end, is capable of imaging at least a second region including a region farther than the first region, and is capable of acquiring a high-resolution image with a higher resolution than the first imaging unit.
  • FIG. 1 is an exemplary and schematic diagram illustrating a configuration of a leading portion of a railway vehicle including the railway vehicle imaging system according to the embodiment.
  • FIG. 2 illustrates a positional relationship between a first camera and a second camera as a first imaging unit and a third camera as a second imaging unit included in the imaging unit in the railway vehicle imaging system according to the embodiment.
  • FIG. 2 is an exemplary schematic diagram for performing the operation.
  • FIG. 3 is an exemplary and schematic block diagram for explaining the configuration of the railway vehicle imaging system according to the embodiment.
  • FIG. 4 is a diagram illustrating a positional relationship between a first region imaged by the first imaging unit and a second region imaged by the second imaging unit in the railway vehicle imaging system according to the embodiment.
  • FIG. 3 is an exemplary and schematic diagram of FIG.
  • FIG. 5 is an exemplary and schematic flowchart showing a flow of a detection process of an obstacle or the like in the railway vehicle imaging system according to the embodiment.
  • FIG. 6 is an exemplary schematic flow chart showing
  • FIG. 1 is an exemplary schematic diagram showing a configuration of the railway vehicle imaging system 100 according to the embodiment.
  • the railway vehicle imaging system 100 is mounted on a railway vehicle RV that travels along a traveling direction D on a track R formed by a pair of rails.
  • the railway vehicle imaging system 100 includes an imaging unit 10, a camera control unit 12, an image processing unit 14, and an output unit 16.
  • the imaging unit 10 is provided at a front end portion of the railway vehicle RV (for example, a driver's seat of the leading railway vehicle RV) and captures an image of a region in the traveling direction D (front region) of the railway vehicle RV.
  • the line R is included in the captured image obtained by the imaging unit 10.
  • the imaging unit 10 of the present embodiment includes a first imaging unit 18 (stereo camera) capable of acquiring a stereo image in a first area in the traveling direction D of the railway vehicle RV, and at least an area farther than the first area.
  • a second imaging unit 20 single camera capable of imaging the second region including the second region.
  • FIG. 2 is a diagram schematically illustrating an example of an arrangement relationship between the first imaging unit 18 and the second imaging unit 20 included in the imaging unit 10.
  • the first imaging unit 18 and the second imaging unit 20 are fixed by, for example, a support member 22 (a support plate, a storage box, or the like) so that the mutual positional relationship does not change due to vibration or the like of the railroad vehicle RV during traveling. ing.
  • the first imaging unit 18 includes a first camera 10a and a second camera 10b that are arranged at a predetermined interval in the vehicle width direction of the railway vehicle RV.
  • the first camera 10a and the second camera 10b are digital cameras having built-in image pickup devices such as a CCD (Charge Coupled Device) and a CIS (CMOS image sensor), for example, so-called “HD image quality” moving image data (captured image data). ) Can be output at a predetermined frame rate.
  • CCD Charge Coupled Device
  • CIS CMOS image sensor
  • the second imaging unit 20 is, for example, a first camera 10a and a second camera 10b that are arranged in the vehicle width direction.
  • a first imaging unit 20 is disposed above an intermediate position between the first camera 10a and the second camera 10b.
  • the image pickup section 18 (the first camera 10a and the second camera 10b) includes a third camera 10c capable of capturing a high-resolution image with a higher resolution.
  • the third camera 10c is, for example, a digital camera having a built-in image sensor such as a CCD or CIS, and can output so-called “4K image quality” moving image data (captured image data) at a predetermined frame rate.
  • FIG. 3 is an exemplary and schematic block diagram for explaining the configuration of the railway vehicle imaging system 100.
  • the functional modules such as the camera control unit 12 and the image processing unit 14 can be configured as a computer having hardware such as a processor and a memory. More specifically, each functional module is realized by the processor of the railway vehicle imaging system 100 reading and executing a program stored in a memory such as a storage unit. Note that functional modules such as the camera control unit 12 and the image processing unit 14 may be realized by dedicated hardware (circuit).
  • the first camera 10a, the second camera 10b, and the third camera 10c are controlled by the camera control unit 12.
  • the camera control unit 12 provides, for example, a signal for controlling the exposure, shutter speed, white balance, and the like to the first camera 10a, the second camera 10b, and the third camera 10c, in addition to a synchronization signal. I do.
  • FIG. 4 shows a first region 24 imaged by the first imaging unit 18 (first camera 10a, second camera 10b) and a second area imaged by the second imaging unit 20 (third camera 10c).
  • FIG. 3 is an exemplary and schematic diagram for explaining a positional relationship of a region 26 of FIG.
  • FIG. 4 shows a state in which a railroad vehicle RV (not shown) traveling on the track R is heading for the platform 28 of the station located ahead.
  • the first imaging unit 18 captures a part of the second area 26 captured by the second imaging unit 20.
  • the directions of the optical axes of the first imaging unit 18 and the second imaging unit 20 are set so as to overlap a part of the first region 24.
  • the second area 26 includes the far area 26 a (far area) farther than the first area 24 that is not included in the first area 24 in the imaging range, and the first area 26 is located in front of the far area.
  • the near area 26b (near area) overlapping with 24 is also included in the imaging range.
  • the image processing unit 14 performs various types of image processing on captured image data (image signals) acquired from the first camera 10a, the second camera 10b, and the third camera 10c. Then, the image processing unit 14 detects the line R included in the first area 24, detects the presence or absence of an attention object (obstacle, etc.) within a predetermined range on or around the line R, and detects the presence of the attention object. The distance is detected. Similarly, the image processing unit 14 detects the line R included in the distant region 26 a of the second region 26, detects the presence or absence of a target object (obstacle, etc.) within a predetermined range on or around the line R, and An object detection process such as a process of estimating the distance to the attention target object is executed.
  • the first camera 10a and the second camera 10b are arranged so as to be separated from each other in the vehicle width direction of the railroad vehicle RV, are set so that the first region 24 is set as an imaging range, and are synchronized by a synchronization signal. It is configured to execute imaging of the same first region 24 (the same object included in the first region 24) at the same timing.
  • the image processing unit 14 can acquire a stereo image (first stereo image) of the same scene by the first camera 10a and the second camera 10b. Therefore, the image processing unit 14 can calculate the distance to the object included in the first region 24 based on the parallax of the acquired first stereo image using a well-known stereo matching technique or triangulation technique. it can.
  • the first area 24 is, for example, a processing target range up to about 200 m ahead of the railway vehicle RV. Distance measurement that can withstand practical use.
  • the image processing unit 14 can detect the line R from the image captured by the first camera 10a or the second camera 10b by using a known line R detection technique. For example, the image processing unit 14 divides the captured image into a plurality of pixel groups (detection areas, search areas), and for each of the pixel groups, a feature amount (for example, The line R is detected based on the characteristic relating to the luminance.
  • the fixed positions of the first camera 10a and the second camera 10b in the railcar RV are known. Further, the railway vehicle RV runs on the track R. Therefore, the position of the track R in the first region 24 imaged by the first camera 10a and the second camera 10b can be estimated.
  • the image processing unit 14 may generate a distance image reflecting the distance information based on the stereo images acquired from the first camera 10a and the second camera 10b. Then, the image processing unit 14 may set a detection area with respect to the generated distance image and detect the track R.
  • the third camera 10c includes the first area 24 in the imaging area, it functions as a single camera, and therefore cannot calculate the distance to the attention target included in the distant area 26a, but exists in the distant area 26a.
  • the presence of the attention target object can be recognized.
  • the third camera 10c is a “4K camera” capable of capturing images with “4K image quality”
  • the distant area 26a of the second area 26 is practically usable, for example, in a range up to about 400 m in front of the railway vehicle RV. It is possible to realize the detection of an object to be noted (hereinafter, referred to as an “obstacle or the like”).
  • accurate distance information can be obtained in the first region 24 imaged by the first camera 10a and the second camera 10b. Then, for example, the distance to the object M existing in the farthest part 30 in the first area 24 and the size of the object M can be accurately obtained.
  • the size of the object N existing in the nearest part 32 in the far area 26 a contacting the farthest part 30 is substantially the same as the size of the object M.
  • the size of the object of interest existing in the distant region 26a for example, the size of an obstacle or the like existing around the track R is known.
  • the sizes of power transmission poles installed along the line R and structures around the line R are known.
  • the length (size) corresponding to one pixel in the far area 26a can be calculated. That is, the first distance to the first position (for example, the object M) existing in the first area is calculated, and based on this first distance, the second area 26 (distant area 26a) is set. It is possible to estimate a second distance to an existing second position (object P). In this case, since the third camera 10c is the captured image data of 4K image quality, it is possible to sufficiently secure the reliability of the distance estimation based on the number of pixels.
  • the image processing unit 14 detects a target object (an obstacle such as an object M or an object P) in the first area 24 and the second area 26 (distant area 26a) by detecting a well-known pattern matching or the like. This is possible by applying the method to the line R and the area around the line R.
  • a substantial detection range of an obstacle or the like can be expanded.
  • the braking distance is, for example, about 400 m or less.
  • the braking force in the high-speed region increases as the maximum speed of the railway vehicle RV increases, so that the braking distance when the initial braking speed is 100 km / h is further reduced. Therefore, when the position of an obstacle or the like up to 400 m ahead of the railway vehicle RV can be estimated, or when an obstacle or the like exists, braking control can be performed from, for example, about 400 m before the obstacle or the like.
  • the railway vehicle imaging system 100 can be applied to obstacle detection and automatic braking control in an automatically traveling railway vehicle RV.
  • the image processing unit 14 obtains a first stereo image using the first camera 10a and the second camera 10b, and calculates a distance to an obstacle or the like in the first area 24.
  • the second area 26 which is the imaging range of the third camera 10c partially overlaps the first area 24 which is the imaging range of the first camera 10a and the second camera 10b. Therefore, the image processing unit 14 uses the second camera 10b and the third camera 10c to capture a stereo image (second image) of the overlapping area of the first area 24 and the second area 26, for example, using the second camera 10b and the third camera 10c. Stereo image).
  • the image processing unit 14 uses the third camera 10c and the first camera 10a, for example, to obtain a stereo image (second image) of the overlapping area of the first area 24 and the second area 26 using the third camera 10c and the first camera 10a. 3 stereo images). Therefore, distance measurement can be performed on obstacles and the like in the overlapping area for which distance measurement has been performed using the first stereo image, using the second stereo image and the third stereo image.
  • the distance measurement result based on the first stereo image, the distance measurement result based on the second stereo image, and the distance measurement result based on the third stereo image for example, by using an average value or the like of each distance measurement result.
  • the distance accuracy to an obstacle or the like in the overlapping area can be improved. That is, the distance measurement accuracy when the railway vehicle RV approaches an obstacle or the like can be improved. Further, the accuracy of distance estimation to an obstacle or the like in the distant region 26a can be improved.
  • FIG. 5 and FIG. 6 are flow charts showing the detection processing of a target object (for example, an obstacle) by the railway vehicle imaging system 100, that is, the flow of stereo image processing. It will be described using FIG. It is assumed that the processing of the flowchart shown in FIG. 5 is repeatedly executed at a predetermined processing cycle.
  • a target object for example, an obstacle
  • the camera control unit 12 of the railway vehicle imaging system 100 captures an image of the first area 24 by the first camera 10a and the second camera 10b when the railway vehicle RV is in an operable state (for example, when the power is on). Then, an image of the second area 26 is taken by the third camera 10c (S100).
  • the camera control unit 12 controls the exposure, shutter speed, white balance, and the like, in addition to the synchronization signal, so that the first camera 10a, the second camera 10b, and the third camera 10c capture the same scene under the same conditions. Provide a signal.
  • the first camera 10a, the second camera 10b, and the third camera 10c are configured to individually receive a control signal, a power supply, and the like. Therefore, even if a failure occurs in any one of the cameras, the configuration is such that normal imaging of the camera and output of the captured image data are not affected.
  • the image processing unit 14 executes the stereo image processing when the captured image data of all the cameras (the first camera 10a, the second camera 10b, and the third camera 10c) are sequentially obtained (Yes in S102). (S104). Details of the stereo image processing will be described with reference to the flowchart in FIG.
  • the image processing unit 14 determines that the captured images (first stereo images) captured by the first camera 10a and the second camera 10b have not been processed (No in S200), the first camera 10a and the second camera 10a A parallel equalization conversion process is performed on the captured image of 10b (S202). That is, the image processing unit 14 sets the first stereo image to a state where parallax detection by stereo matching processing is possible. Then, the image processing unit 14 generates a first distance image including distance information based on the captured images (first stereo images) captured by the first camera 10a and the second camera 10b based on the detected parallax (S204). ).
  • the image processing unit 14 applies a known technique for detecting the line R to the first distance image by using features related to luminance and the like, and detects the line R (S206). Note that, as described above, the location of the track R in the captured image is substantially determined. Therefore, the processing load on the image processing unit 14 can be reduced by setting the detection area of the track R to an expected existing position. Subsequently, the image processing unit 14 sets a recognition area for detecting an obstacle or the like on the detected track R (S208). Then, the image processing unit 14 performs detection of an obstacle or the like using a known detection technique (for example, pattern matching or the like) in the set recognition area (S210). The purpose of detecting an obstacle or the like is mainly to ensure safe traveling of the railway vehicle RV.
  • the obstacle or the like is set in a limited range of a predetermined range on and around the line R.
  • the processing load of the image processing unit 14 for detecting an obstacle or the like can be reduced.
  • the image processing unit 14 returns to the processing of S200.
  • the image processing unit 14 has processed the captured images (first stereo images) captured by the first camera 10a and the second camera 10b (Yes in S200), and captures the images of the second camera 10b and the third camera 10c. If it is determined that the captured image (second stereo image) has not been processed (No in S214), parallel equalization conversion processing is performed on the captured images of the second camera 10b and the third camera 10c (S216). That is, the image processing unit 14 sets the second stereo image in a state where parallax detection by stereo matching processing is possible. Then, the image processing unit 14 generates a second distance image including distance information based on the captured images (second stereo images) captured by the second camera 10b and the third camera 10c based on the detected parallax (S218). ).
  • the image processing unit 14 detects the line R for the second distance image in the same manner as the processing for the first distance image (S206), and sets the recognition area (S208). Then, the image processing unit 14 executes detection of an obstacle or the like in the recognition area set in the second distance image (S210). In S212, whether the detection of the obstacle or the like by the combination of the cameras has been completed again. It is determined whether or not it is (S212).
  • the image processing unit 14 detects the line R for the third distance image in the same manner as the processing for the first distance image and the second distance image (S206), and sets the recognition area (S208). Then, the image processing unit 14 detects an obstacle or the like in the recognition area set in the third distance image (S210), and again determines in S212 whether the detection of the obstacle or the like by the combination of the cameras has been completed. It is determined whether or not it is (S212).
  • step S212 the image processing unit 14 detects an obstacle or the like by each combination of the first camera 10a, the second camera 10b, and the third camera 10c (detection of the first distance image, the second distance image, and the third distance image). If it is determined that the processing has been completed (Yes in S212), the distance to the obstacle or the like, which is the object of attention, is corrected based on each detection result, and the obstacle detection position is corrected (increase in distance accuracy). ) Is performed (S224). Then, the image processing unit 14 determines the detected obstacle or the like and the distance to the obstacle or the like as the detection result of the near area (the first area 24 or the overlapping area of the first area 24 and the second area 26). Is temporarily stored in the storage unit (S226).
  • the distance measurement to an obstacle or the like is individually performed by the three stereo cameras in which the first camera 10a, the second camera 10b, and the third camera 10c are combined and reflected on the obstacle detection position.
  • the accuracy of distance measurement to an obstacle or the like in an area where the first area 24 and the second area 26 overlap can be improved.
  • the first camera 10a, the second camera 10b, and the third camera 10c operate normally (when the captured image data is normally output)
  • the second distance image, the second The generation of the three-distance image may be omitted.
  • distance measurement to an obstacle or the like in a near area can be realized by distance measurement based on a stereo image.
  • the process returns to the flowchart of FIG. 5, and the image processing unit 14 adds the distant area 26a captured by the third camera 10c to the near area, thereby detecting the track R (Setting of an extended recognition area) is performed (S106). Then, the image processing unit 14 detects an obstacle or the like in the extended recognition area (S108). Prior to the detection of an obstacle or the like in the extended recognition area, the image processing unit 14 applies a well-known technique for detecting the line R using features related to luminance and the like, similarly to the first area 24. May be detected to set a detection area such as an obstacle.
  • the line R in the distant region 26a is located on the extension of the line R detected in the first region 24. May be regarded as existing, and an obstacle or the like may be detected in the extension area.
  • the image processing unit 14 performs distance estimation in the extended area (far area 26a) (S110).
  • the size of the object M at the farthest part 30 in the near area (the first area 24) and the size of the object N at the closest part 32 of the distant area 26a substantially match, and the distance to the object M is: It can be obtained with high accuracy by processing the stereo image in the first area 24. Therefore, based on a comparison between the size of the object N in the distant area 26a and an obstacle or the like existing farther from the distant area 26a, and based on the distance to the object M (the object N), an obstacle or the like detected in the distant area 26a. (Distance to an obstacle or the like) can be estimated.
  • the process of S110 may be omitted. In this case, the processing load on the image processing unit 14 in the distant region 26a can be reduced.
  • the image processing unit 14 outputs the detection result of the obstacle or the like to the output unit 16 (S112), and temporarily ends the processing around the processing. That is, when an obstacle or the like is detected in the first area 24 (or the overlapping area of the first area 24 and the second area 26) in the stereo image processing of S104, the image processing unit 14 The presence / absence and the position (distance) of the railway vehicle RV are provided to the traveling system of the railway vehicle RV and an external system that manages and monitors the traveling of the railway vehicle RV via the output unit 16 and are reflected in the control of the railway vehicle RV. Let it.
  • the image processing unit 14 when an obstacle or the like is detected in the distant area 26a, the image processing unit 14 outputs information indicating that the obstacle is detected via the output unit 16 to the traveling system of the railway vehicle RV or an external system. To be reflected in the control of the railway vehicle RV. When the position (distance) is estimated when an obstacle or the like is detected in the distant area 26a, the image processing unit 14 may output the information via the output unit 16 together. When providing the detected obstacle or the like to an external system (monitoring center) or the like via the output unit 16, the image processing unit 14 may add an emphasis signal or an alarm signal for emphasizing the obstacle or the like. .
  • the image processing unit 14 performs the detection of the line R by applying a known detection technique of the line R to the generated distance image by using a characteristic related to luminance or the like (S122). Subsequently, the image processing unit 14 sets a recognition area for detecting an obstacle or the like on the detected track R (S124). Then, the image processing unit 14 executes detection of an obstacle or the like using a known detection technique (for example, pattern matching or the like) in the set recognition area (S126). The image processing unit 14 outputs the detection result of the obstacle or the like in the near area via the output unit 16 (S128), and temporarily ends the processing around the processing. That is, the image processing unit 14 detects an obstacle or the like using two cameras that can output normal captured image data at present.
  • a known detection technique for example, pattern matching or the like
  • the detection area for obstacles and the like is limited to the first area 24 where normal imaging by the first camera 10a and the second camera 10b is possible.
  • the second camera 10b or the second camera 10b it is possible to detect the presence or absence and the position of an obstacle or the like in the overlapping area of the first area 24 and the second area 26.
  • the image processing unit 14 when there are no two cameras that can normally output captured image data (No in S114), the image processing unit 14 cannot detect the position (distance) even if an obstacle or the like exists. It is determined that there is, and error processing is executed (S130). That is, the image processing unit 14 provides the error information to the traveling system of the railway vehicle RV or an external system via the output unit 16 and reflects the error information in the traveling control of the railway vehicle RV. For example, a temporary traveling stop process or the like is executed.
  • the first camera 10a, the second camera 10b, and the third camera 10c capture a stereo image of the same scene, thereby capturing an attention target object such as an obstacle.
  • the detection range can be expanded.
  • the possibility that a plurality of camera malfunctions occur at the same time is generally low, and the railway vehicle imaging system 100 is configured to be able to acquire a stereo image by at least two cameras. It is possible to avoid a situation where the vehicle RV cannot be automatically driven.
  • the first imaging unit 18 includes two HD cameras (the first camera 10a and the second camera 10b), and the second imaging unit 20 includes one 4K camera.
  • the example constituted by the camera (the third camera 10c) is shown.
  • the imaging unit 10 may be configured by four or more cameras, and similar effects can be obtained.
  • the railway vehicle imaging system 100 may be applied to a railway vehicle on which a driver can board (regardless of whether or not the driver drives), and similar effects can be obtained.
  • the output unit 16 displays an image as shown in FIG. 4 on a display device installed in a driver's seat or the like, displays a detection result of an obstacle or the like, and outputs a voice message or a warning corresponding to the result. A sound may be output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Electromagnetism (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Train Traffic Observation, Control, And Security (AREA)
  • Image Analysis (AREA)
  • Measurement Of Optical Distance (AREA)
  • Cameras In General (AREA)
  • Stereoscopic And Panoramic Photography (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)

Abstract

L'invention concerne un système d'imagerie pour un véhicule ferroviaire qui comprend : une première unité d'imagerie; et une seconde unité d'imagerie. La première unité d'imagerie est disposée sur l'extrémité avant d'un véhicule ferroviaire avant, et peut acquérir une image stéréo dans une première région dans la direction d'avance du véhicule ferroviaire. La seconde unité d'imagerie est disposée sur l'extrémité avant, peut imager une seconde région comprenant au moins une région plus éloignée qu'au moins la première région, et peut acquérir une image à haute résolution plus haute en résolution que celle acquise par la première unité d'imagerie.
PCT/JP2019/025429 2018-07-24 2019-06-26 Système d'imagerie pour véhicule ferroviaire Ceased WO2020021949A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
SG11202012917RA SG11202012917RA (en) 2018-07-24 2019-06-26 Imaging system for railway vehicle
EP19840166.3A EP3829155B1 (fr) 2018-07-24 2019-06-26 Système d'imagerie pour véhicule ferroviaire
CN201980048961.0A CN112470456B (zh) 2018-07-24 2019-06-26 铁路车辆用摄像系统

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-138596 2018-07-24
JP2018138596A JP7150508B2 (ja) 2018-07-24 2018-07-24 鉄道車両用撮像システム

Publications (1)

Publication Number Publication Date
WO2020021949A1 true WO2020021949A1 (fr) 2020-01-30

Family

ID=69180463

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/025429 Ceased WO2020021949A1 (fr) 2018-07-24 2019-06-26 Système d'imagerie pour véhicule ferroviaire

Country Status (5)

Country Link
EP (1) EP3829155B1 (fr)
JP (1) JP7150508B2 (fr)
CN (1) CN112470456B (fr)
SG (1) SG11202012917RA (fr)
WO (1) WO2020021949A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113486783A (zh) * 2021-07-02 2021-10-08 浙江省交通投资集团有限公司智慧交通研究分公司 轨道交通车辆的障碍物检测方法及系统
JP2025019899A (ja) * 2023-07-28 2025-02-07 株式会社東芝 情報処理装置、情報処理システム、および情報処理方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001242521A (ja) 2000-02-28 2001-09-07 Fuji Heavy Ind Ltd カメラの組み立て構造、カメラの調整方法、および調整用治具
JP2010165142A (ja) * 2009-01-15 2010-07-29 Nissan Motor Co Ltd 車両用道路標示認識装置および車両用道路標示認識方法
JP2016192109A (ja) * 2015-03-31 2016-11-10 公益財団法人鉄道総合技術研究所 障害物検知方法およびその装置
WO2017056266A1 (fr) * 2015-09-30 2017-04-06 株式会社小松製作所 Dispositif d'imagerie

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10318805A (ja) * 1997-05-15 1998-12-04 Toshiba Fa Syst Eng Kk 移動点検システム
JP2008196891A (ja) * 2007-02-09 2008-08-28 Mitsubishi Precision Co Ltd 多眼カメラ
JP5030983B2 (ja) * 2009-02-10 2012-09-19 株式会社日立製作所 列車の停止検知システム並びに列車の移動速度及び位置検出システム
JP2010271143A (ja) * 2009-05-20 2010-12-02 Ricoh Co Ltd 撮像装置、距離測定装置、及び車外監視装置、並びに撮像方法
JP5407898B2 (ja) * 2010-01-25 2014-02-05 株式会社豊田中央研究所 対象物検出装置及びプログラム
US9109891B2 (en) * 2010-02-02 2015-08-18 Konica Minolta Holdings, Inc. Stereo camera
US9720089B2 (en) * 2012-01-23 2017-08-01 Microsoft Technology Licensing, Llc 3D zoom imager
DK3027482T3 (da) * 2013-07-31 2021-12-20 Rail Vision Ltd System og fremgangsmåde til identificering og undvigelse af forhindring
DE102013110615B3 (de) * 2013-09-26 2014-11-27 Sick Ag 3D-Kamera nach dem Stereoskopieprinzip und Verfahren zum Erfassen von Tiefenkarten
WO2015045501A1 (fr) * 2013-09-27 2015-04-02 日立オートモティブシステムズ株式会社 Dispositif de reconnaissance d'environnement externe
US20170359565A1 (en) * 2015-02-20 2017-12-14 Sony Corporation Image pickup device and image pickup method
JP2016197795A (ja) * 2015-04-03 2016-11-24 日立オートモティブシステムズ株式会社 撮像装置
JP6660751B2 (ja) * 2016-02-04 2020-03-11 日立オートモティブシステムズ株式会社 撮像装置
US10007854B2 (en) * 2016-07-07 2018-06-26 Ants Technology (Hk) Limited Computer vision based driver assistance devices, systems, methods and associated computer executable code

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001242521A (ja) 2000-02-28 2001-09-07 Fuji Heavy Ind Ltd カメラの組み立て構造、カメラの調整方法、および調整用治具
JP2010165142A (ja) * 2009-01-15 2010-07-29 Nissan Motor Co Ltd 車両用道路標示認識装置および車両用道路標示認識方法
JP2016192109A (ja) * 2015-03-31 2016-11-10 公益財団法人鉄道総合技術研究所 障害物検知方法およびその装置
WO2017056266A1 (fr) * 2015-09-30 2017-04-06 株式会社小松製作所 Dispositif d'imagerie

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3829155A4

Also Published As

Publication number Publication date
EP3829155B1 (fr) 2025-04-23
CN112470456A (zh) 2021-03-09
EP3829155A1 (fr) 2021-06-02
CN112470456B (zh) 2022-04-26
JP2020017824A (ja) 2020-01-30
EP3829155A4 (fr) 2022-04-20
SG11202012917RA (en) 2021-02-25
EP3829155C0 (fr) 2025-04-23
JP7150508B2 (ja) 2022-10-11

Similar Documents

Publication Publication Date Title
JP6060282B2 (ja) 車載カメラ画像処理装置
TWI530409B (zh) 自動追蹤防撞警示系統及其方法
JP2014222429A (ja) 画像処理装置、距離測定装置、移動体機器制御システム、移動体及び画像処理用プログラム
JP2001216519A (ja) 交通監視装置
JP6221464B2 (ja) ステレオカメラ装置、移動体制御システム及び移動体、並びにプログラム
JP2009239664A (ja) 車両用環境認識装置および先行車追従制御システム
JP2009169776A (ja) 検出装置
US10974719B2 (en) Mobile object control system, mobile object control method, and program
JP6447289B2 (ja) 撮像装置、撮像方法、プログラム、車両制御システム、および車両
CN107431747B (zh) 摄像装置
US10247551B2 (en) Vehicle image processing device for environment recognition
JP7278846B2 (ja) 物体位置検出装置、走行制御システム、および走行制御方法
JP5261752B2 (ja) ドライブレコーダ
JP5027710B2 (ja) 車両用環境認識装置および先行車追従制御システム
CN112470456B (zh) 铁路车辆用摄像系统
JP6899673B2 (ja) 物体距離検出装置
JP6891082B2 (ja) 物体距離検出装置
US20200111227A1 (en) Orientation detection apparatus for vehicle, image processing system, vehicle, and orientation detection method for vehicle
CN104002730A (zh) 立体物检测装置、驾驶支援装置以及立体物检测方法
JP2024152819A (ja) 撮像情報記憶装置、撮像情報記憶方法及びプログラム
JP5580062B2 (ja) 障害物検知警報装置
JP2019184342A (ja) 線路検査装置
JP2008021069A (ja) 周辺監視システム
WO2023089834A1 (fr) Système d'affichage d'image, procédé d'affichage d'image et dispositif d'affichage d'image
JP4797441B2 (ja) 車両用画像処理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19840166

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019840166

Country of ref document: EP

Effective date: 20210224

WWG Wipo information: grant in national office

Ref document number: 2019840166

Country of ref document: EP