EP3028006A2 - Système permettant l'acquisition d'une image d'un front d'attaque - Google Patents

Système permettant l'acquisition d'une image d'un front d'attaque

Info

Publication number
EP3028006A2
EP3028006A2 EP14744032.5A EP14744032A EP3028006A2 EP 3028006 A2 EP3028006 A2 EP 3028006A2 EP 14744032 A EP14744032 A EP 14744032A EP 3028006 A2 EP3028006 A2 EP 3028006A2
Authority
EP
European Patent Office
Prior art keywords
image
working face
image data
drill head
arrangement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP14744032.5A
Other languages
German (de)
English (en)
Other versions
EP3028006B1 (fr
Inventor
Robert Galler
Robert WENIGHOFER
Martin Entacher
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Montanuniversitaet Leoben
Original Assignee
Montanuniversitaet Leoben
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=52432495&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=EP3028006(A2) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Montanuniversitaet Leoben filed Critical Montanuniversitaet Leoben
Publication of EP3028006A2 publication Critical patent/EP3028006A2/fr
Application granted granted Critical
Publication of EP3028006B1 publication Critical patent/EP3028006B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • E—FIXED CONSTRUCTIONS
    • E21—EARTH OR ROCK DRILLING; MINING
    • E21B—EARTH OR ROCK DRILLING; OBTAINING OIL, GAS, WATER, SOLUBLE OR MELTABLE MATERIALS OR A SLURRY OF MINERALS FROM WELLS
    • E21B47/00—Survey of boreholes or wells
    • E21B47/002—Survey of boreholes or wells by visual inspection
    • E—FIXED CONSTRUCTIONS
    • E21—EARTH OR ROCK DRILLING; MINING
    • E21B—EARTH OR ROCK DRILLING; OBTAINING OIL, GAS, WATER, SOLUBLE OR MELTABLE MATERIALS OR A SLURRY OF MINERALS FROM WELLS
    • E21B44/00—Automatic control systems specially adapted for drilling operations, i.e. self-operating systems which function to carry out or modify a drilling operation without intervention of a human operator, e.g. computer-controlled drilling systems; Systems specially adapted for monitoring a plurality of drilling variables or conditions
    • E—FIXED CONSTRUCTIONS
    • E21—EARTH OR ROCK DRILLING; MINING
    • E21C—MINING OR QUARRYING
    • E21C39/00—Devices for testing in situ the hardness or other properties of minerals, e.g. for giving information as to the selection of suitable mining tools
    • E—FIXED CONSTRUCTIONS
    • E21—EARTH OR ROCK DRILLING; MINING
    • E21D—SHAFTS; TUNNELS; GALLERIES; LARGE UNDERGROUND CHAMBERS
    • E21D9/00—Tunnels or galleries, with or without linings; Methods or apparatus for making thereof; Layout of tunnels or galleries

Definitions

  • the invention relates to an arrangement for determining an image of at least a region of a tunnel boring machine worked by a tunnel of a mountain, a tunnel boring machine and a method for determining an image of at least a portion of one of
  • a tunnel boring machine is a machine used to build tunnels.
  • Components of a tunnel boring machine are a decal with feed and bracing devices, facilities for the installation of support and expansion measures, facilities for material removal, a
  • a front boring head of a tunnel boring machine is provided with mining tools for loosening a mountain.
  • geological documentation of the working face is essential in tunneling.
  • mechanical drives as in the case of a tunnel boring machine, such documentation is only possible to a very limited extent because the boring head of the tunnel boring machine is mechanically in the way and thus visual assessments are possible only to a very limited extent.
  • geologists and geotechnicians look through small openings on the working face and produce photos and / or sketches. Occasionally, the drill head is pulled back a bit so that you can photograph larger parts of the face, but from extremely unfavorable angles. With the retraction of the drill head also a lot of effort is connected.
  • an arrangement for obtaining an image of at least a portion of a tunnel boring machine ie, the boundary surface of a mountain facing the tunneling direction of a tunnel boring machine therewith and at which tunneling effected by the tunnel boring machine takes place or takes place has created) of a mountain
  • a tunnel boring machine for excavating rock material of the working face, an image pickup device mounted for picking up image data of at least a portion of the working face on the boring head, and a
  • Image processing device which processes (in particular during rotation of the drill head) by means of the image recording device recorded image data of several sections of the working face and from a
  • a tunnel boring machine for excavating rock material from a face of a mountain, the tunnel boring machine having an arrangement having the features described above.
  • the invention provides a method of determining an image of at least a portion of a tunnel boring machined face of a rock, the method comprising rotating a boring head of a tunnel boring machine for excavating rock material of the face, image data of a respective portion of the face by means of a boring head Image pickup device are recorded (in particular while the drill head rotates, the drill head preferably during recording not is advanced, ie the recording is not performed during the drilling operation), and the (in particular during the rotation of the
  • Drill head recorded by the image pickup device image data of multiple sections of the face are processed, whereby an image is at least reconstructed or generated at least the region of the working face.
  • TBM tunneling For this purpose, for example, complete images or circular rings of the working face are produced according to the invention.
  • a camera or other image pickup device is mounted on a drill head, for example placed in openings of the drill head, and image data (for example a video or images in rapid succession) is taken, preferably while the drill head is rotating.
  • image data for example a video or images in rapid succession
  • the individual image data are combined by methods of digital image processing in particular into an overall image.
  • the quality of the images produced has proven to be very good, so that they can meet the high quality requirements for geological documentation. This makes it possible to generate image data with a very short time requirement (for example within approximately 10 minutes), from which subsequently a complete, undistorted face breast image can be derived.
  • the entire working face or large or desired parts of the working face can be fully documented. The process is very fast and
  • a camera can be placed manually as an image recording device in maintenance layers in the drill head.
  • a permanent or fixed implementation of the image recording device (for example, cameras) in a drill head is possible and very efficient.
  • the acquisition of the bi-data may be performed in maintenance shifts or during brief breaks in the drilling operation. However, long interruptions of the drilling operation do not occur thereby.
  • Imaging units of the image pickup device are recorded, and that at a certain angular position or at several different angular positions of the drill head.
  • the drill head rests during the acquisition of the image data and, for example, several image acquisition units at different positions of the drill head while the drill head is at rest receive image data from different sections of the working face.
  • the drill head can rotate, so that one and the same image acquisition unit can be rotated at different times during the rotation and thus at different times
  • Angular positions of the drill head absorbs image data of different local sections of the working face.
  • the drill head at least one opening, in particular a through hole (between
  • the image pickup device can be mounted on the drill head for receiving the image data through the through opening.
  • the image pickup device Permanently or temporarily attaching the image recording device or even in its interior, the image pickup device in the orifices protected from the harsh mechanical conditions in one or boring tunnel.
  • the integration of the image recording device in openings of the drill head means that the drill head mechanically does not interfere with the recording of image data of the working face.
  • the at least one opening may be selected from a group consisting of a manhole, one
  • An image pickup unit can be mounted in such a manhole (for example, detachable) and used to pick up the image pickup unit when the manhole is not needed otherwise.
  • An image pickup unit can be mounted in such a manhole (for example, detachable) and used to pick up the image pickup unit when the manhole is not needed otherwise.
  • mine tools rotatably mounted discs can be used with cutting edges, which are set upon rotation of the drill head itself in rotation and then
  • a discharge system of a tunnel boring machine for conveying away excavated material contains arranged in the drill head
  • Scraper slots through which, using scrapers, the released cuttings from the working face passes behind the drill head and then transported away, for example via conveyor belts. Such scraper slots can also be used for at least partially accommodating the image recording device.
  • Image capture device not restrict at all, can also special provided for separate openings (for example, recesses or passages) are formed in the drill head.
  • the at least one recess may be designed for at least partially receiving the image recording device. If the image pickup device is wholly or partly in the
  • Receiving opening is housed, it can be damage-proof before
  • the image pickup device may be selected from a group consisting of at least one
  • a video camera for capturing video data from at least a portion of the face, and at least one still camera for capturing photos of at least a portion of the face.
  • a continuous video recording of portions of the working face during rotation of the drill head can be recorded. This is a wide
  • the image pickup device may include a
  • the individual image capture units can video and / or
  • Photographic cameras which may be mounted at different radii of the drill head, so that during rotation of the drill head, each of the
  • Image recording units can record, for example, an annular portion of the working face.
  • the image recording device can be set up image data of the working face during a predetermined time
  • the image capture may be temporarily terminated and a corresponding data set of image data may be terminated
  • Image processing unit for the reconstruction of the interesting area of the working face are transmitted.
  • the image recording device may comprise a plurality of image recording units attached to the drill head,
  • video cameras and / or cameras have, for example,
  • Imaging units can be used, even at small distances between a front of the drill head and the working face a basically unlimited range of the working face in the form of an image can be displayed.
  • the image recording device sets together with the
  • Image processing device the individual sub-images corresponding to the recorded from the individual image recording units sections of the face to a total image corresponding to the area of the working face of interest together.
  • Imaging units are mounted on the drill head such that the different portions of the working face imaged by these image pickup units overlap at least partially with each other.
  • the overlapping areas allow the image processing device an error-free composition of the individual images to an overall image. Also allows the provision of EP2014 / 065652
  • Overlapping areas the compensation of brightness differences between individual partial images, as they can be caused for example by a different degrees of illumination of different sections of the face.
  • Color bar artifacts such as discontinuous color changes at an interface of two image sections on the composite image can thereby be avoided or at least suppressed.
  • the image processing device can be set up as the region of the working face a circular ring or a
  • a contiguous annulus may be visually displayed as a mapped area of the face when the drill head rotates.
  • Circular ring can already be sufficiently meaningful for the geological documentation of the working face.
  • several photo and / or video cameras can record individual, preferably overlapping, circular rings, which can then be assembled by the image processing device into a complete circular area.
  • the arrangement may have a light source or light source mounted on the drill head for illuminating the portion of the working face imaged by the image recording device. Since it is dark in a borehole, the partial illumination of
  • a photo camera with a flash light source can be used.
  • the assembly may comprise a mounting bracket adapted to receive the image pickup device, and in particular the light source, and for mounting to the drill head.
  • a mounting bracket can clearly a mechanical adapter between an arbitrarily shaped image pickup unit or
  • Imaging device on the one hand and an already available or specially trained recess can be created as a receiving space in the drill head.
  • the arrangement may include a
  • Incline sensor which for detecting a current inclination angle (in particular the image pickup device or one or more
  • certain image pickup units thereof is arranged and may be attached, for example, to the mounting bracket or alternatively directly to the image pickup device or a respective image pickup unit.
  • Inclination sensor can thus be a positioning of the mounting bracket (and thus also the image recording unit recorded therefrom) or directly
  • Capture image capture device itself and continue to report to the image processing device.
  • Such inclination information provides additional knowledge about the way in which an image of a region of the working face is to be interpreted or evaluated or processed.
  • Image processing means for reconstructing the image processes the image data associated with the sections in consideration of the tilt angle detected by the tilt sensor. This avoids artifacts that result from mis-evaluation of image data when the
  • the mounting bracket for Imaging device from an incorrect current orientation of the image pickup device or the image pickup units relative to the working face emanates.
  • Mounting bracket on the drill head but can be mounted protected.
  • the processing may preferably be performed by assistance of an image processing device on the
  • Tunnel boring machine is physically removed.
  • the post-processing be carried out.
  • This has the advantage that a geologist can already draw conclusions from the face pattern on site and can make the control of the tunnel boring machine based on the determined image of the working face.
  • the evaluation of the images can be carried out at a post-processing station, for example in an office.
  • the processing of the image data may be performed such that as the image, a three-dimensional relief image of at least the region of the face is reconstructed. Accordingly, in the arrangement, the image processing device may be such
  • the image recording device can be set up to record sufficient image information for reconstructing relief images.
  • the image evaluation device can be set up accordingly to derive the relief image from such image data. From the spatial information of a relief, a geologist can derive further technical information.
  • Image capture unit (s) configured as a stereo system
  • Image recording device for example, several image recording units for particular simultaneous recording of multiple images or a single image pickup unit, which rotates with the rotation of the drill head and while temporally successively records each other to be linked images) take place.
  • the image processing device may be configured to process image data of different sections of the face using contrast analysis (ie determination of contrast information for image data of the sections and comparison of the contrast information determined for different image data) to corresponding regions of the face but to identify, for example, partially overlapping sections) corresponding sections of the working face on the different ones
  • contrast analysis ie determination of contrast information for image data of the sections and comparison of the contrast information determined for different image data
  • Sections are determined and assigned to each other. In other words, a sequence of light and dark areas in image data associated with each section can be examined and areas of the same or the same
  • the image recording device may comprise a plurality of image recording units (for example several cameras).
  • Times are each at least overlapping, in particular identical, sections of the face are formed.
  • the image processing device can be set up to process the image data of the sections at least overlapping at any time such that the image is reconstructed as a three-dimensional relief image of at least a part of the overlapping sections. According to such an embodiment, several take
  • Imaging units for example, multiple cameras at different positions of the drill head
  • Imaging units in particular simultaneously associated with their respective positions and orientations images of different sections of the working face.
  • the image pickup units are arranged so that the space areas that you capture include a common subvolume
  • the individual images that correspond to different view directions may be assembled to obtain three-dimensional information.
  • the overlapping areas picked up from different directions contain complementary steric information.
  • the image recording device may have one, in particular only one, image recording unit, which during the
  • the image processing device may be configured to process the image data of the at least overlapping sections such that a three-dimensional relief image of at least a part of the overlapping sections is reconstructed as the image.
  • a single image pickup unit may be sufficient to provide a compact arrangement (also, according to the described embodiment, several image pickup units are alternatively possible).
  • Such an image pickup unit may be due to rotate with their fixed mounting on the drill head with the drill head so that it scans image technically continuously different surface areas of the working face. If the images are taken in sufficiently short temporal (and consequently spatial) distances, then they can be immediate
  • successive captured images contain an overlapping region of the face, which in turn results in the acquisition of three-dimensional
  • Image information and a correct composition of the individual images allows. By assigning face chest areas to images created by one and the same image capture unit, any artifacts that may occur, unlike images from different image capture units (with slightly different physical properties), can be avoided.
  • the image recording device may have one, in particular only one, image recording unit, which during the
  • the arrangement may further comprise pattern generating means for generating a respective pattern of electromagnetic radiation (eg, visible light, ultraviolet, infrared, etc.) on at least a part of the portions during recording of the image data.
  • the image processing means may be arranged to process the image data of the portions using the pattern or the patterns as assignment information so as to reconstruct, as the image, a three-dimensional relief image of at least a part of the portions.
  • the patterns of electromagnetic radiation impressed on the individual images can therefore enable or facilitate the assignment of the individual images to one another. To distinguish visible patterns of face breast image information, one shot can be taken with and one shot without
  • the pattern generating means for generating the pattern in a wavelength range outside of visible light (for example, in the infrared range, in the ultraviolet range, etc.) or in 14 065652
  • the arrangement may further comprise sensor means for detecting the pattern in the wavelength range outside visible light.
  • the pattern on the images of the image pickup device which can be recorded, for example, in the optically visible range, not seen and thus does not interfere with the artifact-free generation of image information.
  • Wavelength range outside the visible light sensitive sensor device for example, an infrared camera when the pattern in the IR region are generated
  • profile scanners is possible. If the relative orientation between the sensor device, pattern generation device and image recording device is known, the patterns associated with the individual images can be used to correctly assemble the images to derive three-dimensional image information.
  • Pattern generating means for generating a means of the
  • Image recording device detectable pattern, in particular a
  • Laser illumination be set up and be positioned relative to the image pickup device in a previously known manner.
  • the image processing device may be configured to process the image data of the sections using the patterns and the prior art positioning as assignment information such that the image is reconstructed into a three-dimensional relief image of at least a portion of the sections.
  • the generated patterns it is thus possible for the generated patterns to lie in the visible range or in a wavelength range detectable by the image recording device, in particular by means of a laser.
  • the pattern information for generating a three-dimensional relief of the face can be used to connect the individual images with each other and / or to gain 3D information.
  • a (in particular rectilinear) laser line may be used as the pattern, which then passes from a non-planar surface of the working face into a laser line that is perceptibly (in particular non-rectilinear) visible on the images
  • the arrangement may comprise a transit time measuring device for electromagnetic radiation.
  • Image processing means may be arranged to process the image data of the portions using the runtime information so as to reconstruct, as the image, a three-dimensional relief image of at least a part of the portions.
  • the radiation serves to determine the distance or distance differences of the source from the considered part of the working face and as a result of obtaining SD information.
  • the arrangement may also have a
  • Have device for acoustic transit time measurement for example, an echo sounder.
  • Figure 1 shows essential parts of a tunnel boring machine with a
  • Figure 2 shows a, adjacent to a working face drill head of a
  • Tunnel boring machine according to an exemplary embodiment of the invention.
  • Figure 3 shows a small portion of a working face, through a
  • FIG. 4 shows a face breast photograph taken after retraction of a drill head.
  • Figure 5 shows a plan view of an arrangement according to a
  • FIG. 6 shows an arrangement of four cameras for collectively approximately complete recording of a working face according to an exemplary embodiment
  • Figure 7 shows an arrangement of a camera, a light source and a magnet holder for mounting on a drill head of a
  • Tunnel boring machine according to an exemplary embodiment of the invention.
  • FIG. 8 shows a mounting bracket for a camera, light sources and inclination sensors for an arrangement for producing an image of a working face according to an exemplary embodiment of the invention.
  • FIG. 9 shows images or video frames which overlap, and according to an exemplary embodiment of the invention by means of digital
  • FIG. 10 shows a section of a circular ring as an image of a region of a working face, which has been assembled from a plurality of individual video frames according to an exemplary embodiment.
  • FIG. 11 shows, according to an exemplary embodiment, ascertained height-layer lines over a 3D-reconstructed part of a working face.
  • FIG. 12 shows two overlapping images obtained in succession according to an exemplary embodiment, containing SIFT interest points marked with symbols (ring / cross) and their assignment over both images.
  • FIG. 13 shows a boring head of a tunnel boring machine adjoining a working face according to an exemplary embodiment of the invention Invention in a side view, in which by means of several
  • Imaging units three-dimensional images of the working face are generated.
  • FIG. 14 shows a boring head of a tunnel boring machine adjoining a working face in accordance with an exemplary embodiment of the invention in a plan view in which a boring-rotating boring head is used
  • Image recording unit for recording data overlapping portions of the working face three-dimensional images of the working face are generated.
  • FIG. 15 shows a boring head of a tunnel boring machine adjoining a working face in accordance with an exemplary embodiment of the invention in a plan view in which a boring-rotating boring head is used
  • Imaging unit and a pattern generating device three-dimensional images of the working face are generated.
  • Figure 1 shows essential parts of a tunnel boring machine 180 having an arrangement 120 for determining an image of a region of one of
  • Tunnel boring machine 180 machined working face 122 of a mountain 102 according to an exemplary embodiment of the invention.
  • the tunnel boring machine 180 for dismantling the rock 102 is in a position in which a borehole 182 has already been introduced into the rock 102.
  • the drilling takes place in such a way that the borehole 182 according to FIG. 1 is widened successively to the right.
  • the tunnel boring machine 180 moves along a direction indicated by reference numeral 190 in FIG.
  • the person skilled in the art is aware that the
  • a drill head 150 having a multiplicity of herein
  • the drill head 150 has a drill body 152 that is rotatable and translationally movable with respect to the mountain 102, to whose front side or the face 122 facing the front side, the plurality of removal tools 100 is mounted. These are about the circular
  • Each of the mining tools 100 may have a rotatable disk mounted in a holder with a circumferentially sharp cutting edge. Upon rotation of the drill body 152 rotate the discs of the mining tools 100 and bear with their sharp cutting edge material of the mountains 102 at the face of 122.
  • disassembly tools 100 may be used instead of discs, for example paring knives or gravers.
  • a drive motor 157 of the tunnel boring machine 180 drives the drill head 150 via a schematically illustrated drive shaft 159 so that it rotates about an axis of the drive shaft 159 (see reference numeral 161) and also along the direction 190 deeper and deeper in the direction of the mountain 102 while removing it inside moves.
  • Tunnel boring machine 180 machined face 122 described in more detail.
  • the rotatably mounted drill head 150 is part of the arrangement 120 and serves for the removal of material at the working face 122.
  • recesses 128 which are formed as passage openings in the embodiment shown
  • a plurality of cameras are mounted as image recording units which together form an image recording device 124 form. Each of the cameras takes during the rotation of the 150 boring head, which also to
  • an image (in particular a 360 ° panorama) corresponding to an annular portion of the end face face 122 on.
  • the cameras themselves are held stationary relative to the drill head 150, ie do not need a separate drive.
  • the image data taken by the cameras during a complete 360 ° rotation of the drill head 150 is supplied to an image processing device 126 (for example a processor, in particular a microprocessor or a CPU, central processing unit).
  • the image processing device 126 processes the image data of the different cameras of the image recording device 100 together, ie sets the individual images of the individual cameras or the recorded images
  • methods of image processing can be used.
  • the constructed image of the working face 122 is then displayed to a display device 155, for example an LCD display of a
  • the image is thus at the place of
  • Tunnel boring machine 180 reconstructed.
  • Imaging device 124 can be implemented integrated into the drill head 150. In the embodiment shown is also the
  • Image processing device 126 integrated in the drill head 150 and transmits the reconstructed image data in the illustrated embodiment wirelessly to a display device 155, not shown in Figure 2 for displaying the image of
  • Working face 122 The geological and geotechnical documentation of the working face 122 in the case of drives with tunnel boring machines 180 is an essential basis for the reliable and safe operation of the tunnel boring machine 180. For example, this takes place once a day in maintenance shifts. Arrangements 120 according to the invention are significantly more advantageous than conventional approaches for determining geological data of a working face 122, as will be described in more detail below:
  • Manholes, reamer slots or side of mining tools 100 Manholes, reamer slots or side of mining tools 100. However, it is only possible to view small parts of the working face 122, as shown for example in FIG. In this photo, a circular area with a diameter of about 60 cm is visible, while the entire working face 122 a
  • bit 150 is withdrawn a bit to allow for better documentation, the chance of taking pictures covering a larger portion of the face does arise, but only extremely
  • geological face chest documentation is included
  • a method is provided according to the invention, which allows to photograph a large part or even the entire working face 122 or film and subsequently to a to process the entire picture or photo.
  • Such a method thus makes it possible to graphically represent the working face 122 in tunnel boring machine drives.
  • the difficulty overcome with the image pickup device 124 integrated into and / or rotating the boring head 150 is that the working face 122 for operators is almost completely detached from the operator
  • Drill 150 of the tunnel boring machine 180 is concealed, can be viewed conventionally only through small openings, but by the inventive provision of the image pickup device 124 as well as in terms of
  • Creating an image of a larger area of the working face 122 can be used.
  • FIG. 5 shows a top view of components of an arrangement 120 according to an exemplary embodiment of the invention, in which cameras as the image recording device 124 on the drill head 150 of FIG
  • a complete photograph of the working face 122 or circular rings of the working face 122 can be created.
  • a photo or video camera (or more photo or video cameras) as an image pickup device 124 in one or more openings in
  • Drilling head 150 placed, for example,1%,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg,kg, etc.
  • the boring head 150 is slowly rotated either clockwise or counterclockwise (see reference numeral 161), and simultaneously photographs are taken at frequent intervals, or video is simultaneously taken, for example, until the drill head 150 has once fully rotated.
  • the number of cameras needed to get one A complete image of the entire face 122 (or region of interest of the face 122) is dependent upon the angle of the lens and whether and, if so, how far, the bit 150 is retracted (for example, fully adjacent the face 122 or a predetermined distance withdrawn from, for example, 1.5m).
  • FIG. 6 shows an arrangement of four cameras as
  • Image recording device 124 for approximately complete coverage of a working face 122 according to an exemplary embodiment of the invention.
  • the camera arrangement shown in FIG. 6 permits the recording of four annular images 600 including small overlaps 602.
  • the regions of intersections 602 can be used, in particular, to assemble the individual images without artifacts.
  • Figure 7 shows an arrangement of a camera as
  • Imaging device 124 a light source 700 (formed here in the form of an LED array) and a magnet holder 702 for placement on a drill head 150 of a tunnel boring machine 180 according to an exemplary embodiment of the invention.
  • the attachment of the image recording device 124 can be done in different ways.
  • a simple and user-friendly variant is the provision of the camera including light source 700 with the magnet holder 702 for joint placement on the drill head 150.
  • Figure 8 shows a mounting bracket 702 for a camera as
  • Imaging device 124 for an arrangement 120 for generating an image of a face breast 122 according to an exemplary embodiment of the invention.
  • FIG. 8 shows such a mounting bracket 702 which can be mounted in maintenance layers in a manhole.
  • strong light sources 700 not shown
  • sensors such as Inclinometer / inclinometer 800
  • a preferred embodiment is to integrate the image pickup device 124 into the drill head 150 right from the start when it is designed.
  • FIG. 9 shows images or video frames 900 which overlap and, according to an exemplary embodiment of the invention, are combined to form an overall image by means of digital image processing.
  • FIG. 9 schematically shows the overlapping images, which are subsequently assembled.
  • FIG. 10 shows a detail 1000 of a circular ring as an image of a region of a working face 122, which is shown in FIG
  • Embodiment has been composed of several individual video frames.
  • composition into an overall picture can be carried out by means of various methods (for example pixel or texture-oriented). in the
  • FIG. 10 shows the detail 1000 of a circular ring, which is the result of several
  • FIG. 11 shows height layer lines 1102 via a 3D reconstructed part of a working face 122 (determined with an arrangement 120 according to FIG. 13).
  • FIG. 12 shows two successively acquired recordings 1202, 1204 with overlap, the SIFT interest points marked with symbols (ring / cross), ie scale-invariant feature transforms, ie scale invariant
  • FIG. 13 shows a boring head 150 of a tunnel boring machine 180 adjoining a working face 122 according to an exemplary embodiment
  • FIG. 13 shows a side view of the drill head 150 in a simplified representation.
  • the image pickup device 124 has two image pickup units that are adapted to receive image data from
  • each overlapping portions 600 of the face 122 are formed.
  • An overlapping area of the overlapping portions 600 is indicated by reference numeral 601.
  • Image processing means 126 is arranged to process the image data of the at least overlapping portions 600 at any time so as to reconstruct, as the image, a three-dimensional relief image of at least a part of the overlapping portions 600 (see FIG. 11).
  • Stereo systems operate, which synchronously pictures by means of the two
  • Image pickup units of the (at least partially) same area of the working face 122 are recorded during a rotation of the tunnel boring machine 180.
  • the image recording device 124 is fixedly connected to the rotating drill head 150 during the recording.
  • the images may be created under uniform illumination with artificial light generated by light sources (not shown) from openings of the drill head 150 to the face 122.
  • rectification is carried out, for example, using an internal and external camera orientation known from a calibration Pictures determined. Both contain specific details for the image recording units, such as their focal length, position of the main image point, translation vector and rotation between the image recording units.
  • Both contain specific details for the image recording units, such as their focal length, position of the main image point, translation vector and rotation between the image recording units.
  • the position of corresponding image contents can be determined by means of stereo masters, the lines of the left image with the same lines of the right image from the stereo system to similar image content compare. Similarities can be calculated, for example, with computational methods such as SAD (Sum of Absolute Difference), where minima of differences between pixel values result in image mapping.
  • SAD Sud of Absolute Difference
  • minima of differences between pixel values result in image mapping.
  • a dense three-dimensional color point cloud of at least portions of the working face 122 can be generated. This allows the differentiation of different, in the
  • FIG. 14 shows a boring head 150 of a tunnel boring machine 180 adjoining a working face 122 (see FIG. 1) according to an exemplary embodiment of the invention.
  • the drill head 150 can by means of a co-rotating image pickup unit as an image pickup device 124 for
  • Reference numeral 600 denotes a portion which is picked up when the co-rotating image pickup unit is at a first position. Likewise, reference numeral 600 denotes another portion, which is recorded when the co-rotating image pickup unit is at a later time at a second position. At this second position, the image pickup device is indicated by reference numeral 124 '. An overlapping region of the overlapping sections 600, 600 'is again identified by reference numeral 601.
  • FIG. 14 shows a top view of the drill head 150 in a simplified representation (for example, the removal tools 100 attached to the drill head 150 are not shown).
  • the image pickup device 124, 124 ⁇ has a single image pickup unit, which is formed during the rotation of the drill head 150 for temporally successive recording of image data of sections 600, 600 'of the face breast 122. At a first time, the image pickup unit is shown in solid lines. At a later second time, the image capture unit is shown in dashed lines.
  • the image processing device 126 (not shown, see FIG. 1) is adapted to process the image data of the overlapping sections 600, 600 ⁇ recorded at different times such that the image is a three-dimensional relief image of at least a part of the overlapping one
  • Sections 600, 600 ⁇ is reconstructed.
  • Imaging device 124, 124 ⁇ for 3D reconstruction provided a camera unit, wherein temporally successively obtained images are used for SD reconstruction.
  • Tunnel boring machine 180 effects translation and rotation of the image acquisition unit. This causes a partial overlap of the
  • FIG. 15 shows a drill head 150 of a tunnel boring machine 180 adjoining a working face 122 (see FIG. 1) (see FIG. 1) according to an exemplary embodiment of the invention.
  • the drill head 150 can by means of a co-rotating image pickup unit and a
  • Pattern generator (shown at a first position with reference numeral 1500 and at a second position with reference numeral 1500 ")
  • FIG. 15 again shows a top view of the drill head 150 in a simplified representation.
  • the image pickup device (again indicated at a first position with reference numeral 124 and at a second position with reference numeral 124) comprises an image pickup unit which, during rotation of the drill head 150, captures image data from sections 600, 600 (see FIG Figure 14) of the working face 122 is formed.
  • the assembly 120 further includes the pattern generating means 1500, 1500 ⁇ for generating a respective pattern of electromagnetic radiation on at least a portion of the portions 600, 600 during acquisition of the image data. At a first time are the
  • Pattern generator 1500, 1500 ' shown in solid lines. At a later second time, the image pickup unit and the pattern generation means 1500, 1500 ⁇ with dashed lines are shown.
  • the image processing device 126 (see FIG. 1) is for processing the image data of the sections 600, 600 ⁇ using the
  • Image capturing device 124 for 3D reconstruction can for the purpose of Independence from the visual contrast of the face 122 of a strip projector.
  • An associated pattern generator 1500, 1500 ⁇ is formed of a fringe projector having a predefined regular or randomly generated areal pattern (in the same or different electromagnetic frequency ranges as the visible light, one separate for uster detection and not shown in the figure
  • Sensor device can be provided) throws. These patterns can be captured with a calibrated stereo system. These images can be edited for 3D reconstruction using image mapping techniques.
  • the pattern generating means 1500 be 1500 a, adapted to generate by means of the image pickup device 124, 124 ⁇ detectable pattern, in particular a laser illumination and relative (not shown) to the image pickup device 124, 124 x positioned in previously known way.
  • the image processing device 126 may then be configured to process the image data of the sections 600, 600 using the patterns and the prior art positioning as mapping information such that a three-dimensional relief image of at least a portion of the sections 600, 600 ⁇ is reconstructed as the image.
  • the image recording device 124, 124 ⁇ can thus with a
  • Light section method for 3D reconstruction are equipped, which uses a laser line as a signal.
  • the image recording device 124, 124 ⁇ is then composed, for example, of a laser illumination and a camera. Both are aligned at a certain angle to each other and in the recording of the camera, the laser line can be detected as a height profile.
  • the frequency of profiling may be in the form of rotation of the
  • Tunnel boring machine 180 be tuned so that a 3D reconstruction by means of a dense cloud point is possible.
  • the combination with the visual recording of the working face 122 takes place via a calibration step, which brings together visual information and the results of the height profile recording.
  • a similar image pickup 124, 124 ⁇ according to yet another Embodiment of the invention may be a profile scanner, which replaces laser illumination and the associated camera.
  • the image recording device 124, 124 ⁇ can be equipped with sensors (not shown) for detecting wavelengths outside the visible light. Techniques such as spectroscopy using handheld devices can be used analytically to achieve an assignment by means of quartz or mafic indices based on the different emissivities, for example in the LWIR (long wavelength infrared).

Landscapes

  • Engineering & Computer Science (AREA)
  • Mining & Mineral Resources (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geology (AREA)
  • Physics & Mathematics (AREA)
  • General Life Sciences & Earth Sciences (AREA)
  • Geochemistry & Mineralogy (AREA)
  • Environmental & Geological Engineering (AREA)
  • Fluid Mechanics (AREA)
  • Geophysics (AREA)
  • Earth Drilling (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un agencement (120) permettant l'acquisition d'une image d'au moins une zone d'un front d'attaque (122) d'une montagne (102) traitée par une machine de creusement de tunnels (180), l'agencement (120) comprenant une tête foreuse rotative (150) de la machine de creusement de tunnels (180) permettant l'excavation du matériau rocheux du front d'attaque (122), un dispositif de prise de vues (124) servant à enregistrer des données d'image d'au moins une partie (600) du front d'attaque (122) sur laquelle est appliquée la tête foreuse (150), et un dispositif de traitement d'images (126) qui est conçu pour traiter les données d'image de plusieurs parties (600) du front d'attaque (122) qui ont été enregistrées par le dispositif de prise de vues (124), en particulier pendant la rotation de la tête foreuse (150), et pour reconstruire l'image d'au moins la zone du front d'attaque (122) à partir des données d'image des multiples parties (600) du front d'attaque (122). Une analyse optique continue ou périodique du front d'attaque peut en outre fournir des informations complémentaires utiles pour une amélioration ou une optimisation de la commande de la machine de creusement de tunnels.
EP14744032.5A 2013-07-30 2014-07-21 Système permettant l'acquisition d'une image d'un front d'attaque Active EP3028006B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ATGM50107/2013U AT14252U1 (de) 2013-07-30 2013-07-30 System zum Ermitteln eines Ortsbrustbilds
PCT/EP2014/065652 WO2015014655A2 (fr) 2013-07-30 2014-07-21 Système permettant l'acquisition d'une image d'un front d'attaque

Publications (2)

Publication Number Publication Date
EP3028006A2 true EP3028006A2 (fr) 2016-06-08
EP3028006B1 EP3028006B1 (fr) 2019-12-18

Family

ID=52432495

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14744032.5A Active EP3028006B1 (fr) 2013-07-30 2014-07-21 Système permettant l'acquisition d'une image d'un front d'attaque

Country Status (3)

Country Link
EP (1) EP3028006B1 (fr)
AT (1) AT14252U1 (fr)
WO (1) WO2015014655A2 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108729919B (zh) * 2018-06-07 2019-07-05 深圳市市政设计研究院有限公司 一种填石地层弧形变截面隧道浅埋暗挖施工方法
CN109139023B (zh) * 2018-09-25 2019-12-17 清华大学 基于tbm管片拼装机的围岩观测系统
CN111272753B (zh) * 2019-09-06 2021-04-13 山东大学 基于图像识别与分析的隧道内围岩石英含量预测系统与方法
CN114607464A (zh) * 2022-03-14 2022-06-10 新疆额尔齐斯河流域开发工程建设管理局 隧道掌子面岩体质量评价系统和方法
DE202022106442U1 (de) 2022-11-17 2024-02-20 Herrenknecht Aktiengesellschaft Tunnelbohrmaschine

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4119180A1 (de) * 1991-06-11 1992-12-17 Merkel Peter Dr Verfahren und vorrichtung zur vermessung und dokumentation der geometrie, struktur- und materialbeschaffenheit von dreidimensionalen objekten, wie fassaden und raeume sowie deren waende, einrichtungen und installationen
US5657073A (en) * 1995-06-01 1997-08-12 Panoramic Viewing Systems, Inc. Seamless multi-camera panoramic imaging with distortion correction and selectable field of view
US6009359A (en) * 1996-09-18 1999-12-28 National Research Council Of Canada Mobile system for indoor 3-D mapping and creating virtual environments
JP3031542B2 (ja) * 1997-07-17 2000-04-10 鹿島建設株式会社 Ｔｂｍ切羽画像撮影処理システム
AU2003226047A1 (en) * 2002-04-10 2003-10-27 Pan-X Imaging, Inc. A digital imaging system
DE202012102496U1 (de) * 2012-07-05 2012-09-13 Montanuniversität Leoben Fehlerrobuste Sensorik zum Ermitteln einer räumlichen Beziehung zwischen Anlagenkomponenten unter rauen Bedingungen

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2015014655A3 *

Also Published As

Publication number Publication date
WO2015014655A2 (fr) 2015-02-05
WO2015014655A3 (fr) 2015-10-01
EP3028006B1 (fr) 2019-12-18
AT14252U1 (de) 2015-07-15

Similar Documents

Publication Publication Date Title
EP3028006B1 (fr) Système permettant l'acquisition d'une image d'un front d'attaque
EP2690459B1 (fr) Dispositif et procédé d'identification et de documentation d'au moins un objet traversant un champ de rayonnement
EP2850388B1 (fr) Procédé et dispositif d'enregistrement pour un système de détection de position
DE202017007481U1 (de) System zur Überwachung von Nutzpflanzen
CN106225770A (zh) 隧道掌子面地质多维数字化记录识别方法及系统
DE102004026702B3 (de) Vorrichtung zur Untersuchung von Ankerbohrlöchern
DE3516939A1 (de) Verfahren zum erfassen der bilder geologischer proben zu ihrer optischen analyse und vorrichtung zur durchfuehrung des verfahrens
KR101966732B1 (ko) 석회석 광산 갱내의 디지털 사진 자료를 이용한 암상 분석 방법
DE102008054453A1 (de) Messsystem zur Vermessung von Räumen und/oder von Objekten
DE69104688T2 (de) Strassenbeläge-Untersuchungsfahrzeug.
EP3486606A1 (fr) Caméra stéréo et procédé stéréophotogrammétrique
DE102007047499A1 (de) Verfahren und Vorrichtung zur Erfassung von Informationen eines Werkzeugs
DE102006055652A1 (de) Verfahren zur Aufarbeitung dreidimensionaler Daten und Vorrichtung zur Aufarbeitung dreidimensionaler Daten
Gaich et al. Digital rock mass characterization 2017–Where are we now? What comes next?
EP3236440B1 (fr) Dispositif, systeme et procede de surveillance de pente et/ou de batiment sans marquage
Gaich et al. 3D images for digital tunnel face documentation at TBM headings–Application at Koralmtunnel lot KAT2/3D‐Bilder zur digitalen Ortsbrustdokumentation bei TBM‐Vortrieben–Anwendung beim Koralmtunnel Baulos KAT2
DE102007038785A1 (de) Verfahren und Vorrichtung zum Bestimmen von Geometriedaten eines Messobjekts
DE10248416B4 (de) Vorrichtung und Verfahren zur Bestimmung einer Relativbewegung eines Fahrzeugs
DE102004027769B3 (de) Verfahren und Vorrichtung zur Untersuchung von Bohrkern-Proben
Gaich et al. 3D images for data collection in tunnelling–applications and latest developments/3D‐Bilder für die Datenerfassung im Tunnelbau–Anwendung und aktuelle Entwicklungen
EP4410685B1 (fr) Système d'inspection de canal
CN110043252B (zh) 一种岩屑图像特征随钻监测装置
EP2831839A1 (fr) Procédé d'exploitation automatique d'un système de surveillance
DE102010029627B4 (de) Vorrichtung und Verfahren zur Bestimmung der Struktur einer spiegelnden Oberfläche eines Objekts
DE102008009643A1 (de) Bildverarbeitungsverfahren, Bildanzeigeverfahren, Bildverarbeitungsprogramm, Speichermedium, Bildverarbeitungsvorrichtung und Röntgenbildvorrichtung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20160129

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20170609

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190607

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAL Information related to payment of fee for publishing/printing deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR3

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAR Information related to intention to grant a patent recorded

Free format text: ORIGINAL CODE: EPIDOSNIGR71

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

INTC Intention to grant announced (deleted)
AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

INTG Intention to grant announced

Effective date: 20191111

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502014013288

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1215089

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200115

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200318

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200319

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

Ref country code: NO

Ref legal event code: T2

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200513

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200418

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

REG Reference to a national code

Ref country code: DE

Ref legal event code: R026

Ref document number: 502014013288

Country of ref document: DE

PLBI Opposition filed

Free format text: ORIGINAL CODE: 0009260

PLAX Notice of opposition and request to file observation + time limit sent

Free format text: ORIGINAL CODE: EPIDOSNOBS2

26 Opposition filed

Opponent name: 3GSM GMBH

Effective date: 20200910

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PLBB Reply of patent proprietor to notice(s) of opposition received

Free format text: ORIGINAL CODE: EPIDOSNOBS3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200721

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200721

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200721

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200721

REG Reference to a national code

Ref country code: DE

Ref legal event code: R100

Ref document number: 502014013288

Country of ref document: DE

PLCK Communication despatched that opposition was rejected

Free format text: ORIGINAL CODE: EPIDOSNREJ1

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

PLBN Opposition rejected

Free format text: ORIGINAL CODE: 0009273

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: OPPOSITION REJECTED

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191218

27O Opposition rejected

Effective date: 20220215

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230517

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240703

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NO

Payment date: 20250731

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20250729

Year of fee payment: 12

Ref country code: AT

Payment date: 20250723

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20250806

Year of fee payment: 12

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502014013288

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20260203