EP3408210A2 - Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention - Google Patents

Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention

Info

Publication number
EP3408210A2
EP3408210A2 EP17721522.5A EP17721522A EP3408210A2 EP 3408210 A2 EP3408210 A2 EP 3408210A2 EP 17721522 A EP17721522 A EP 17721522A EP 3408210 A2 EP3408210 A2 EP 3408210A2
Authority
EP
European Patent Office
Prior art keywords
machine
control device
crane
remote control
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP17721522.5A
Other languages
German (de)
English (en)
Other versions
EP3408210B1 (fr
Inventor
Michael PALBERG
Jürgen Resch
Oliver Fenker
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Liebherr Werk Biberach GmbH
Original Assignee
Liebherr Components Biberach GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from DE202016002295.5U external-priority patent/DE202016002295U1/de
Application filed by Liebherr Components Biberach GmbH filed Critical Liebherr Components Biberach GmbH
Priority to EP21197655.0A priority Critical patent/EP3950558A1/fr
Publication of EP3408210A2 publication Critical patent/EP3408210A2/fr
Application granted granted Critical
Publication of EP3408210B1 publication Critical patent/EP3408210B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
    • B66C13/00—Other constructional features or details
    • B66C13/18—Control systems or devices
    • B66C13/40—Applications of devices for transmitting control pulses; Applications of remote control devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
    • B66C13/00—Other constructional features or details
    • B66C13/52—Details of compartments for driving engines or motors or of operator's stands or cabins
    • B66C13/54—Operator's stands or cabins
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/16—Cabins, platforms, or the like, for drivers
    • E02F9/166—Cabins, platforms, or the like, for drivers movable, tiltable or pivoting, e.g. movable seats, dampening arrangements of cabins
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/20—Drives; Control devices
    • E02F9/2025—Particular purposes of control systems not otherwise provided for
    • E02F9/205—Remotely operated machines, e.g. unmanned vehicles

Definitions

  • Remote control device for crane, construction machine and / or industrial truck
  • the present invention relates to a remote control device for a crane, a construction machine and / or a truck, with a control station, the at least one input means for inputting control commands and a signal transmission device for transmitting the input control commands to the control device of the crane, the construction machine or Truck and a display device for displaying a representation of the machine environment and / or a working tool such as jib or hooks, wherein a movement determination module for determining movements and / or deformations of the machine components is provided in response to the input control commands.
  • Cranes and similar large equipment such as Bohrrammello, Surface Miner or Crawler Excavators are very complex in operation and control. This is not only the variety of control functions and their interaction and the associated, in their entirety quite complex input means such as joysticks, foot pedals and control switches a problem, but also the often unfamiliar, machine-specific reactions of the machine structure to movements of the actuators.
  • Cranes such as tower cranes or Telescopic luffing cranes, but also harbor or Maritimkrane have long, slim structural components such as boom or tower structures that twist and are relatively soft, so that associated with acceleration or deceleration operations of the actuators structural deformations and pendulum movements that for experienced crane operators as a kind Feedback for the control operation serve.
  • tower cranes can deform the tower structure and boom system when a load is picked up, or it can pendulum the load when turning about the upright axle and the boom will vibrate accordingly. The same can occur in cable excavators or Bohrrammellon, so that a crane operator or operator is unsafe if he does not learn the usual, corresponding crane reactions to control operations.
  • an approximately realistic control station which, for example, corresponds to the crane operator cab of a respective crane type and can have control commands via input means such as joysticks, pedals, control switches or touchscreens provided there.
  • a real or virtual representation of the crane environment as well as the crane components visible from the control station, such as, for example, outriggers and load hooks is represented on a display device, which is known to comprise a plurality of screens arranged in the field of view of the control station, wherein a "real" representation of cameras on the remote-controlled machine and the virtual representation of the crane environment and the crane components can be calculated by a graphical simulation module as a function of the input control commands.
  • a remote control system for a crane is known for example from the document DE 10 2012 216 489 A1, this known remote control for machines of different types should be used.
  • the remote control is provided with a selection mechanism, which allows to reduce the number of control and selection menus of the display of the remote control, depending on the configuration in which the connected machine is located.
  • this remote control in the manner described above, there is a lack of sufficient realism, which is hardly improved by the adaptation of the operating menus to the respective working machine.
  • a crane simulator that works with virtual representations of the crane environment is known, for example, from document DE 10 2013 011 818 A1.
  • a crane cab is provided as a control station with corresponding input means, wherein the viewing window or the glazing of the simulated crane cab are replaced by screens on which the virtual representation of the crane environment is displayed.
  • a technical simulation module and the dynamic behavior of the control and drive components is to be simulated and taken into account in the screen display, in which case especially occurring during certain crane movements positioning movements of crane components such as the hoist are presented.
  • the present invention has the object to provide an improved remote control device of the type mentioned, which avoids the disadvantages of the prior art and further develops the latter in an advantageous manner.
  • a more realistic simulation of crane or machine operation is to be achieved, which better communicates the actual crane or machine behavior and makes the remote controllability safer.
  • the object is achieved by a remote control device according to claim 1.
  • Preferred embodiments of the invention are the subject of the dependent claims.
  • control station which may for example comprise an operator's chair, is no longer statically rigidly mounted in the room or on the floor, but can be moved by a drive device in the room.
  • the control station of the remote control device is movably mounted and can be moved by a drive device as a function of the movements and / or deformation of the machine components determined by the movement determination module.
  • the propulsion device is driven by a propulsion control device to simulate the movement of the crane operator's cab and move the control station accordingly. For example, if a command to rotate the crane about an upright axis is entered at the helm, the helm is rotated by the drive device according to the upright axis. If, for example, the control command for lifting a heavy load is entered, which in reality can lead to a slight nodding of the crane structure with slight twisting of the tower, the control station is moved slightly forward by the drive device and / or slightly tilted forwards.
  • the drive device can be designed to be multi-axially movable and / or to perform both rotational and translatory movements.
  • the control station can be multi-axially movably mounted and the drive device comprise at least one upright rotation axis and at least one horizontal rocking axis and / or two lying lying translational axes.
  • the drive device can have three rotation or tilt axes or three axes rotatory working trained and three axes translational working, so that the control station rotated by all three spatial axes or tilted and translational in all three spatial directions can be moved.
  • simpler designs of the drive device with fewer axes of motion may also be considered.
  • the remote control device can be connected to various devices, in particular different types of devices and set up a control communication with the selected device.
  • This can be, in particular, various equipment used on a construction site, such as cranes, excavators, caterpillars or similar machines, but also devices used at various locations.
  • a control communication connection can be set up from the remote control device and / or from the respective device, for example by selecting a respective IP address if the connection is made via a network, or in another way .
  • a high-bandwidth communication link is advantageously selected to avoid real-time problems or skews between control commands and machine responses.
  • the remote control device in a further development of the invention Have configuration module, by means of which the remote control device can be optionally preconfigured for each remote controlled machine type.
  • the preconfiguration means mentioned can adapt the assignment of the control levers and control keys as well as the selection of the operating menus and screen representations to the respective device in a manner known per se.
  • preconfiguration means can also be provided for the motion control and / or imaging of the machine reactions, by means of which the drive devices can be reconfigured to move the control station and adapted to the machine reactions of the respective machine type, since, for example, a tower tower crane rotating above reacts differently than a tower crane turning down or in turn a telescopic luffing crane reacts differently than a tower crane, or an excavator reacts differently than a crane.
  • working and / or boundary parameters of the movement determination module can also be reconfigured by means of the configuration means mentioned, so that the movement determination module determines the movements and / or deformations of machine components suitable for the respective machine type, depending on the selected machine type.
  • the configuration module may also include preconfiguration means for adapting the display device and the illustration of the machine environment and / or the working tool reproduced thereon, wherein said preconfiguration means may preferably adapt the display and / or the display device to the cameras present or usable on the respective remote-controlled machine type and / / or, if the representation is calculated virtually, the parameters for the calculation of the virtual representation can be adapted to the respective machine type.
  • the said movement determination module can basically be designed differently.
  • the motion determination module may be genuine Detecting reactions of the remote-controlled machine by means of a suitable sensor, with corresponding sensor signals can be transmitted to the remote control to then actuate the control station accordingly or to control.
  • the detection of such real machine reactions of the remote-controlled machine may include, for example, the detection of movements, such as the travel of a trolley, the playing and playing a hoist of a crane, etc.
  • deformations of the structure of the remote-controlled machine can be detected, such as pitching movements of the crane cab as a result of tower deformations, for example by means of corresponding inclination and / or acceleration sensors.
  • the movement determination module can also simulate the responses to input control commands in the form of movements and / or deformations of machine parts and for this purpose have a corresponding simulation module, which depends on the entered Control commands corresponding reaction movements and / or deformations calculated and / or with the help of control and / or movement components that belong to the simulation module, can map and measure.
  • a simulation can avoid time offset problems between the remote control station and the remote-controlled machine, especially with slower communication links or other distance links.
  • the configuration module can adapt the calculation parameters and / or algorithms for the simulation of the movements and / or deformations to the respectively selected machine type.
  • dynamic packets or data and algorithm sets for various machines to be remotely controlled can be read from a database into the simulation module in order to suit the respectively selected one Machine to simulate the movements and deformations.
  • the reading of the said dynamic data packets can be done online from a database.
  • the simulation module itself may be provided with a memory device in which various data and algorithmic records are stored in the manner of a library.
  • hybrid forms are also possible in which part of the reactions on the real, remote-controlled machine are detected and another part of the reactions is determined by the simulation module.
  • the movement determination module is designed such that the crane or machine structure is not considered as a rigid, so to speak infinitely stiff structure, but is assumed as elastically deformable and / or resilient and / or relatively soft structure, which - in addition to the actuating movement axes of the machine such as the cantilever rocker axis or the tower axis of rotation - allows movements and / or position changes by deformations of the structural components.
  • the consideration of the mobility of the machine structure as a result of structural deformations under load or dynamic loads is particularly important for elongated, slender and deliberately static and dynamic boundary conditions - taking into account the necessary collateral - structures as in cranes of importance, since here noticeable movement shares for example the crane operator cab, but also the load hook position are added by the deformations of the structural components.
  • the movement determination module takes into account such deformations of the machine structure under static or dynamic loads.
  • the determination device for determining such structural deformations may comprise a calculation unit which determines these structural deformations on the basis of a stored calculation model in FIG Dependence of the control commands entered at the control station calculated.
  • a model can be constructed similar to a finite element model or be a finite element model, but advantageously a model that is significantly simplified compared to a finite element model is used, for example empirically by detecting structural deformations under certain control commands and / or load conditions on the real crane or the real machine can be determined.
  • Such a calculation model can, for example, work with tables in which specific deformations are assigned to specific control commands, wherein intermediate values of the control commands can be converted into corresponding deformations by means of an interpolation device.
  • the structural part deformations considered by the motion determination module can, on the one hand, be taken into account in the control of the drive device for moving the control station, so that the control station imitates the control station movements occurring due to the structural part deformations.
  • certain or detected structural part deformations can also be taken into account in the representation of the machine environment and / or the machine components visible therein, for example in that the deflection of the boom is represented in the illustration or the horizon of the crane environment is moved upwards a bit to simulate a slight forward nodding of the crane cab by, for example, a tower deformation.
  • the representation of the machine environment and / or the working tool of the remote-controlled machine displayed on the control station can comprise a real representation provided by means of at least one camera provided in the area of the remote-controlled machine, and / or also comprise a virtual representation which is displayed by means of a graphic simulation module Dependence of the input control commands is calculated.
  • Such a real, camera-generated representation of the machine environment and / or the working tool can be generated in particular in the form of a live image or television-like video image, wherein transmitted from the at least one camera on the remote-controlled machine, a corresponding video signal to the remote control station of the remote control device and from the there provided display unit is played.
  • imaging sensors for example an infrared sensor and / or a radar sensor and / or a photo mixing detector and / or a light transit time sensor in the sense of a so-called time-of-flight detector
  • the measuring object is illuminated by light pulses and the signal transit time is measured, on the basis of which the distance between the camera and the object can be calculated and a three-dimensional image can be generated.
  • a plurality of imaging sensors of different types such as a camera can be used in conjunction with an infrared sensor, their images can be superimposed and displayed in a common image, for example, such that in the camera image, the warm parts of an infrared image, for example, indicate people can be displayed.
  • a video and / or television camera is advantageous in that it produces a realistic image that gives the operator a feeling of seeing with his own eyes.
  • Such a camera or imaging sensor for representing the machine environment or the working tool of the remote-controlled machine may be mounted fixedly or pivotably on the machine itself, for example on the crane operator cab of a crane and / or on the trolley of a tower crane or at another suitable location, the camera advantageously being oriented in such a way that the working tool, for example the load hook of a crane, can be observed.
  • the working tool for example the load hook of a crane
  • several cameras can be provided in this case in order to view the working tool from different perspectives.
  • a movable and / or movable to different locations camera can be used, which allows to represent the working tool and / or the machine environment from different perspectives.
  • a camera can be used, which is mounted on a flying drone. It is therefore proposed to use a remote-controlled, equipped with a camera flight drone, by means of which the desired camera image of the working tool and / or the tool environment can be provided from different directions.
  • a perspective camera image of the working tool and its surroundings can be provided from oblique viewing axes, which can be obtained from a point spaced from the machine and the driver's cab and above the ground, so that the machine tool and the working environment a visual line can be visualized, as they would see an external viewer.
  • camera images of the load hook and its surroundings can be provided, which can be seen from the side obliquely or perpendicular to the vertical crane center plane through the boom. Also, from drone positions lying in this vertical crane center plane passing through the boom, images of the load hook can be provided. By such perspective camera images from different visual axes, the relative positions between Work tool and its environment or a target point are much better visualized.
  • the aircraft drone can be controlled depending on a machine position and / or working tool position in such a way that the drone machine, especially working tool movements automatically follows and a desired position relative to the machine and / or their Work tool also at machine movements, especially working tool movements at least approximately holds or tries to keep trying and nacha.
  • the aircraft drone may automatically follow the hook of the crane when the automatic follow mode is activated.
  • the aircraft drone can automatically reduce or increase its altitude when the load hook is lowered or raised, and / or in parallel fly forward or backward to the vertical crane midplane as the trolley of the crane moves and / or laterally crosses left or right as the crane is twisted.
  • the flying drone can also be remotely controlled autonomously such that various desired positions relative to the machine and / or its working tool can be flown freely from the drone. This can be done, for example, by entering a desired position for the aircraft drone relative to the load hook or the working tool, for example, such that in a position control module, which may be provided in the control station of the remote control device, a position with respect to the load hook is entered, for example in shape "2 m above the right side of the load hook.”
  • the flying drone can also be completely cleared away from the machine and its working tool, for example with the aid of a joystick, for as long as possible fly away until the camera position and its angle on the work tool and its environment appeals to the crane operator or machine operator.
  • the images provided by the camera and / or the graphic simulation module of the surroundings of the remote-controlled machine with live images from the control station for example movements of the remote control device user can be superimposed.
  • the real or virtual representations of the machine environment and / or the machine components visible therein and, on the other hand, live images of a live camera recorded on the control station can be displayed simultaneously and superimposed on the display device.
  • Such a superposition of images from the simulation world and live images gives the remote control device user a particularly strong sense of reality.
  • a display device wearable on the head in particular spectacle-like display device, for example in the form of a virtual reality glasses and advantageously also on the head portable, for example, designed as a helmet camera or integrated into said virtual reality glasses camera, the provides the said live images, which are displayed together with the artificially generated, virtual representation on the display device, in particular the virtual reality glasses.
  • Said camera for providing the live images from the control station can advantageously be a stereoscopic camera which preferably provides stereoscopic images in at least approximately matching the viewing direction of the user's eye pair camera viewing direction, which are displayed at the appropriate location of the display device, in particular the virtual reality glasses can.
  • This allows a particularly realistic user experience can be achieved.
  • Such a 360 ° camera also simplifies the construction of the imaging system, in particular, the camera image of the head movement of the wearer of the virtual reality glasses can follow relatively easily, whereby a complex remote control of the camera can be avoided.
  • the superimposition device for superimposing the live images of the control station camera with the representation of the environment of the remote-controlled machine can advantageously be designed according to the so-called green-screen technique, wherein the superimposition device recognizes color surfaces of a predetermined color in the live image and these image areas are then recognized by the virtual representation replaced from the simulation module.
  • the control station include a driver's cab wall, are colored in the window areas - for example, according to the viewing windows of a real crane cab - in a key color, which are different from the remaining colors of the other lying in the camera field components such as the color of the window framing, the input means and the Operator clothing as well as - skin color as clearly as possible, so that recorded in the control station live image said colored surfaces in a specific Color rendering while all other image areas are shown in different colors.
  • Device whose components and in the field of view of the live camera body limbs of the user really shows as a live image and on the other hand in the captured by the live camera window areas of the cab wall shows the camera-generated real or virtual representation of the machine environment and the machine components visible therein.
  • the aforementioned virtual representation of the machine environment can advantageously be changed by the graphical simulation module and be adapted to different scenarios depending on different data records which can be imported via an interface into the simulation module.
  • planning data such as CAD data of a building to be manufactured and / or construction site actual data, which reflect the actual state of a building or structure depending on the construction progress, be imported into the simulation module via a corresponding data interface and used by the simulation module, the virtual representation the machine environment according to the recorded data set, in particular depending on the recorded planning data and / or construction site actual data to generate or adapt to it.
  • the construction site or building information mentioned can be CAD data or other geometric data of the building or the construction site in the aforementioned manner, it also being possible to use digital image data which reflects the actual building and its construction progress.
  • image data can be used as machine environment data via the said CAD interface or a suitable image data interface in the graphic Simulation module can be imported, which then adapts the virtual representation of these acquired CAD and / or image data.
  • the modeling of a planned or already existing or partly executed construction site and the corresponding generation of the virtual representation of the machine environment by the graphical simulation module is in particular a valuable tool to ensure the logistics on a construction site and to be able to simulate and practice critical processes before the start of construction ,
  • the remote control device can advantageously not have the motion parameters required for the motion simulation calculated by a simulation computer, but at least partially by means of data emulation using actually moving hardware components may form part of the remote control facility.
  • a data emulation module of the remote control device can in particular comprise actuator components and / or power electronic components by means of which actual positioning movements are carried out, which simulate the real crane or machine movements and provide data characterizing these movements, for example in the form of sensor signals representing the positioning movements of said Play drive components.
  • said motion determination module can be designed as a hybrid device or hybrid module, on the one hand a computer for the simulation of motion and / or position parameters and on the other hand the real crane or machine actuators at least similar hardware components such as drive units, encoders or frequency converter comprises, simulated by means of which positioning movements and movement and / or position parameters are determined.
  • "genuine" hardware components are used, which are also installed in the crane to be simulated or the machine to be simulated as actuator and / or control device components.
  • the movement determination module may comprise the control cabinet or at least a part of the control cabinet and its components, which is also used in the machine to be simulated and forms part of the machine control there.
  • the power electronics and / or at least part of the power electronics such as a frequency converter can be used to simulate the actuating movements that are triggered by control command inputs on the control station.
  • actuator units for example in the form of servomotors
  • a drive unit for example in the form of a servo drive unit is used here for a respective actuator axis, which - in particular via the aforementioned frequency converter - is driven according to a control command
  • a further drive unit for example in the form of a servo drive unit, can be coupled by means of which a counter-torque and / or a counter-load can be exercised to simulate actual loads, resistances or inertias.
  • a load can be simulated by means of said second drive unit, which counteracts a hoist, or it can be simulated a wind torque, which counteracts a slewing drive.
  • the adjusting movement of the first-mentioned drive unit can be detected by a suitable detection device, whereby a corresponding detection signal reproduces the actually obtained actuating movement and can be used as a sensor signal in the further simulation, in particular in the aforementioned To determine movements and / or positions and / or deformations of the structural parts and / or to simulate the virtual representation of the machine environment and / or the machine components visible therein.
  • a plurality of such drive units or a plurality of such drive unit pairs comprising drive and counter-load drive and a respectively associated detection device are used to determine the various actuating movement axes and the related executed positioning movements of the machine operation to be simulated.
  • the sensor values of the drive units of the actuating movement axes which are actuated and moved as a function of the control commands entered at the control station, are therefore not simulated or calculated by means of a computing model but based on hardware components as close as possible to the real actuator components of the machine to be simulated come, emulated or readjusted and output directly as actual sensor values.
  • Such a data emulation system allows the motion determination module to determine movements and / or positions of the machine components much faster and with less processing power, so that the virtual representation of the machine environment and / or of the machine components and the associated positioning movements of the machine state are much faster and more realistic can be achieved.
  • the generated sensor signals can be displayed on the control station and / or used for further monitoring measures such as load monitoring or workspace limitations that can be displayed and / or simulated on the control station.
  • the aforementioned drive unit pairs for carrying out the appropriate positioning movements and providing the corresponding counter-torque or counter-load in particular the tower lathe - or in a top-slewing the boom turning -, the hoist and the Cat trolley correspond.
  • FIG. 1 shows a schematic representation of a remote-controlled crane in the form of a tower crane whose load hook maneuvers a load in the non-visible area behind a building, wherein a flying drone with a camera observes the load hook to transmit a corresponding camera image to the control station of the remote control device, and
  • Fig. 2 a schematic representation of the components of a remote control device with a control station for the operator according to an advantageous embodiment of the invention.
  • the remote control device 1 can be designed as a crane remote control, which comprises a control station 2 in the form of a crane operator cabin, which is essentially a "real" crane operator cabin, as on a crane, for example a tower crane, a port crane or a maritime crane or mobile telescopic crane can be used.
  • the crane 200 may in this case be designed as a tower crane whose tower 202 carries a boom 203, on which a trolley 204 is movably mounted.
  • the boom 203 can be rotated together with the tower 202 or even without the tower 202 - depending on the design of the crane as a top or bottom rotator - about an upright axis, including a slewing drive is provided.
  • the extension arm 203 could also be designed so that it can be tilted up and down about a horizontal transverse axis, wherein a suitable rocker drive could be provided, for example, in cooperation with the boom bracing.
  • Said trolley 204 can be moved by means of a trolley or other Katzfahrantriebs.
  • Said control station 2 can in this case comprise, in a manner known per se, an operator seat 21, for example in the form of an operator's chair 20, around which various input means 18 for inputting control commands are arranged.
  • the aforementioned input means 18 may comprise, for example, a joystick, a touch screen, control levers, input keys and switches, rotary controls, sliders and the like.
  • the operator station is in this case surrounded by a cab wall 22, which may correspond to a cabin housing and windows 23, which are glazed in real crane cabs, but in the present case colored in a particular color, for example, coated with a green film to Green by means of green Screen technique to show a real, camera-generated or virtual machine environment, as will be explained.
  • the control station 2 is mounted on a movement platform 7, by means of which the control station 2 is multi-axially movable.
  • the movement platform 7 is advantageously multi-axially movable, in particular about all three spatial axes x, y and z verkipp- or rotated and along these axes translationally displaceable.
  • Actuators of a drive device 8 are associated with the axes of movement x, y and z of the movement platform 7, for example in the form of electric motors and / or hydraulic cylinders and / or hydraulic motors in order to be able to move the control station 2 about or along the axes.
  • the drive device 8 is driven by a motion control device 24, which can be realized for example by an industrial PC.
  • the said movement control device 24 may in particular be part of a movement determination module 10, by means of which crane movements and / or positions and / or orientations of crane components such as the boom or the tower and also twists of structural components such as the boom or tower depending on the respective control station 2 input control commands can be determined.
  • the said movement determination module 10 determines, as it were, the effects of the inputted control commands on the crane to be controlled, i. which movements, positions, orientations and twists of the crane components would result as a result of input control commands on the crane to be controlled and outputs corresponding, the said variables characterizing motion signals.
  • the said movement determination module 10 does not determine the said movement variables hereby or not completely by calculation on the basis of a calculation model, but resorts to actual hardware components in the form of drive and control components, the actual ones Perform movements and the corresponding hardware components are modeled on a real crane.
  • the movement determination module 10 comprises at least the essential components of a crane control 25, as can be realized in the control cabinet of a crane, for example.
  • said crane control 25 comprises the frequency converters 15 of various crane drives, for example the slewing gear, the trolley and the hoist.
  • Said crane control 25 may optionally comprise further control and / or power electronics components, in particular
  • the crane control 25 is communicatively connected to the control station 2 and its input means 18, so that the crane control 25 can process the input control commands, in particular the frequency converter 15 depending on the input control commands drive units 12, for example in the form of servo drives.
  • the control commands entered at the control station 2 are thus converted into real movements or drive torques and forces of the drive units 12.
  • the drive units 12 mentioned here can be coupled with mating drive units 14 via which motion resistances can be applied to the drive units 12 in order to be able to simulate real resistances such as lifting loads, wind forces, inertias or dynamic loads.
  • the mating drive units 14 can be controlled by the aforementioned industrial PC, which also implements the motion control device 24.
  • the control of the mating drive units 14 can be carried out on the basis of various specifications or programs, for example, by predeterminable lifting loads, predeterminable wind programs or based on predetermined functions or tables such as dynamic reactions when braking the trolley or the rotary motion.
  • a memory module of the control device to drive the mating drive units 14 corresponding models, tables or functions stored.
  • the drive units 12 are assigned detection devices 13, for example in the form of encoders or other position and / or motion sensors, by means of which movement or position signals are provided which characterize the actuating movements of the drive units 12.
  • the motion determination module 10 thus provides real sensor signals as motion parameters that can be displayed on the one hand at the control station 2 and on the other hand can also be used for further simulation functions.
  • structural distortions such as tower bends, cantilever bends and similar deformations can be determined using a computer model and the drive device 8 of the movement platform 7 can be controlled to move the control station 2 and the virtual representation of the crane environment can be generated, respectively as a function of the said, actually generated sensor signals.
  • the movement determination module 10 may comprise a computer unit 11, which in turn may be realized by the aforementioned industrial PC, by means of which computer unit 11 in response to the inputted at the control station 2 control commands and / or the emulated data generated by the data emulation device 19 and the drive units 12 associated detection device 13 generated sensor signals are generated, structural distortions are determined, in particular bends and torsion in the crane tower and in the crane boom, the computer unit 11 for this purpose uses a structural stiffness considering calculation model, as explained above.
  • the movement determination can also detect "real" movements and / or deformations of the remote-controlled crane by means of a suitable sensor system and by Remote data transmission transmitted to the control station 2, and then to control certain functions of the control station 2 of the remote control device based on these real crane reactions.
  • the motion control device 24 controls the drive device 8 of the movement platform 7 to move the control station 2 and emulate real crane operator movements that are true upon input of appropriate control commands Crane would occur.
  • the aforementioned movement data and possibly also the deformation data mentioned are used to take into account crane reactions in a virtual representation which is generated by a graphic simulation module 9 and displayed on a display device 3.
  • the mentioned virtual representation shows in particular the crane environment and visible crane components such as the crane boom and the load hook and can essentially correspond to the image that a crane operator would see from the crane operator cabin.
  • the said virtual representation can hereby correspond in the form of a photo- or film-like digital image, for example a pixel representation in several colors.
  • a simplified graphical representation can also be provided, although the most realistic, photo- or film-like display image is preferred.
  • a true, camera-generated representation of the crane environment and / or the load hook can also be used at the control station 2.
  • at least one camera may be mounted on the crane 200, the live images of which are transmitted to the control station 2.
  • Such a camera 220 may for example be mounted on the crane cab 206 of the remote-controlled crane 200 and advantageously at least approximately have a visual axis, which is the visual axis of a crane operator in corresponds to the crane cab 206 and / or goes from the crane cab 206 to the load hook.
  • a flying drone 209 can be used, which is equipped with at least one camera and can be moved remotely relative to the crane 200.
  • the flying drone 209 is provided, on which at least one camera 210 is mounted, by means of which a camera image is provided by the load hook 208 and / or the load hook environment can.
  • Said camera image is advantageously a live or real-time image in the form of a television or video image and is transmitted wirelessly from the camera 210 of the aerial drone 209 to a display unit 211 and / or the control device 205 of the crane 201, said display unit 211 being, for example, a machine operator display may be in the nature of a tablet or a screen or a monitor that may be mounted in the crane cab 206. If a remote control station or a mobile operating unit is used to control the crane 201 in the aforementioned manner, said display unit 211 may be provided in the remote control station or on the mobile operating unit.
  • the airborne drone 209 is provided with a remote control device 212 which allows the airborne drone 209 to be remotely controlled, in particular the flight control units such as rotor blades, to remotely control the flight position of the aerial drone 209 and / or remotely control the camera 210, in particular with respect to the swivel angle of the camera 210 relative to the body of the flying drone 209 and / or the focal length of the camera 210.
  • a remote control device 212 which allows the airborne drone 209 to be remotely controlled, in particular the flight control units such as rotor blades, to remotely control the flight position of the aerial drone 209 and / or remotely control the camera 210, in particular with respect to the swivel angle of the camera 210 relative to the body of the flying drone 209 and / or the focal length of the camera 210.
  • a corresponding remote control module may be provided in the crane cab 206 and / or the remote control station or the mobile control unit, for example, be equipped with appropriate joysticks.
  • voice control and / or menu control may also be provided for the aerial drone 209, for example to select a desired relative position relative to the crane from a plurality of predetermined relative positions of the aircraft drone 209. This can be done, for example, by inputting "drone position 1" via voice control and / or menu control, which can be preprogrammed or predetermined in the position control device 213.
  • said real or virtual representation of the crane environment and the cranial components visible therein is superimposed by a live image showing real components from the helm 2, in particular from the head of the remote helper in its line of sight visible components such as the input means 18, the hands and the forearm of the user and other components in view.
  • a camera 16 is advantageously provided, which may be designed as a head camera of the user portable head and may have appropriate fastening and / or holding means for attachment to the head, for example in the form of a helmet camera. If the display device 3 is advantageously designed in the form of a virtual reality glasses 4 that the user wears, the camera 16 can be integrated in these VR glasses.
  • the camera 16 is embodied as a stereoscopic camera in order to be able to provide stereoscopic images corresponding to the visual axes of the two eyes of the user.
  • the overlay device 17 for superimposing the representation of the crane environment and the live image of the camera 16 from the control station 2 can in this case in particular comprise a color-based image processing module 26, which can operate according to the so-called green-screen technique.
  • said color-based image processing module 26 can recognize image regions which have a specific color deviating from the remaining image sub-areas and then replace these image regions with the representation from the simulation module 9.
  • control station 2 include a driver's cab wall 22, are colored in the window areas 23 - for example, according to the viewing windows of a real crane cab - in a key color, which differs from the remaining colors of the other lying in the camera field components such as the color of the window frames, the Input means 18 and the operator clothing and skin color as clearly as possible, so that the recorded in the control station 2 live image shows the said colored areas in a particular color reproduction, while all other image areas are shown in different colors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Mining & Mineral Resources (AREA)
  • Civil Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Structural Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Control And Safety Of Cranes (AREA)

Abstract

La présente invention concerne un dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention, muni d'un poste de commande qui présente au moins un moyen d'entrée d'instructions de commande, un dispositif de transmission de signaux servant à transmettre au dispositif de commande de la grue, de l'engin de chantier ou du chariot de manutention les instructions de commande entrées, un dispositif d'affichage affichant une représentation de l'environnement de l'engin et/ou d'un outil de travail, par exemple une flèche ou un crochet de charge, ainsi qu'un module de détermination de mouvements servant à déterminer les mouvements et/ou les déformations des composants de l'engin en fonction des instructions de commande entrées. Selon l'invention, les réactions de la grue ou de l'engin aux instructions de commande entrées au poste de commande, par exemple sous la forme de mouvements et/ou de déformation de la grue, ne sont pas uniquement représentées sous la forme d'une représentation sur le dispositif d'affichage, mais sont converties en un mouvement effectif du poste de commande de la commande à distance accompagnant la réaction de la grue ou de l'engin, pour que l'utilisateur de la commande à distance puisse recevoir et percevoir les réactions dynamiques de l'engin au plus proche de la réalité.
EP17721522.5A 2016-01-14 2017-04-07 Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention Active EP3408210B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP21197655.0A EP3950558A1 (fr) 2016-01-14 2017-04-07 Dispositif de commande à distance pour grues, machines de construction et/ou chariots de manutention

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE202016002295.5U DE202016002295U1 (de) 2016-01-14 2016-04-08 Fernsteuer-Einrichtung für Kran, Baumaschine und/oder Flurförderzeug
PCT/EP2017/000460 WO2017174205A2 (fr) 2016-01-14 2017-04-07 Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention

Related Child Applications (1)

Application Number Title Priority Date Filing Date
EP21197655.0A Division EP3950558A1 (fr) 2016-01-14 2017-04-07 Dispositif de commande à distance pour grues, machines de construction et/ou chariots de manutention

Publications (2)

Publication Number Publication Date
EP3408210A2 true EP3408210A2 (fr) 2018-12-05
EP3408210B1 EP3408210B1 (fr) 2021-10-20

Family

ID=58671562

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17721522.5A Active EP3408210B1 (fr) 2016-01-14 2017-04-07 Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention

Country Status (1)

Country Link
EP (1) EP3408210B1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4053071A1 (fr) * 2021-03-04 2022-09-07 The Raymond Corporation Systèmes et procédés d'assistance pour véhicule de manutention de matériaux
WO2022238228A1 (fr) * 2021-05-10 2022-11-17 Schwing Gmbh Système de télécommande pour engin de chantier et procédé de commande d'un engin de chantier
CN116946876A (zh) * 2023-07-27 2023-10-27 河钢工业技术服务有限公司 一种用于大型冶金铸造起重机的实时监测系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4053071A1 (fr) * 2021-03-04 2022-09-07 The Raymond Corporation Systèmes et procédés d'assistance pour véhicule de manutention de matériaux
EP4517719A3 (fr) * 2021-03-04 2025-03-26 The Raymond Corporation Systèmes et procédés d'assistance pour un véhicule de manipulation de matériau
WO2022238228A1 (fr) * 2021-05-10 2022-11-17 Schwing Gmbh Système de télécommande pour engin de chantier et procédé de commande d'un engin de chantier
CN116946876A (zh) * 2023-07-27 2023-10-27 河钢工业技术服务有限公司 一种用于大型冶金铸造起重机的实时监测系统

Also Published As

Publication number Publication date
EP3408210B1 (fr) 2021-10-20

Similar Documents

Publication Publication Date Title
WO2017174205A2 (fr) Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention
EP3437036B1 (fr) Procédé et dispositif pour planifier et/ou commander et/ou simuler le fonctionnement d'un engin de chantier
EP3504148B1 (fr) Dispositif de télécommande pour grue, engin de chantier et/ou chariot élévateur, et système comprenant ledit dispositif de télécommande et une grue, un engin de chantier et/ou un chariot élévateur
EP3403253B1 (fr) Simulateur de grues
EP3411534A1 (fr) Procédé et dispositif de commande d'une grue, d'une pelle mécanique, d'un engin à chenilles ou d'un engin de construction similaire
CN112884710B (zh) 作业机械的辅助影像生成方法、远程操控方法及其装置
EP3408210B1 (fr) Dispositif de commande à distance d'une grue, d'un engin de chantier et/ou d'un chariot de manutention
JPH0665949A (ja) 建設ロボットにおける遠隔臨場制御方法
CN115148063A (zh) 一种基于unity3D的虚拟挖掘机系统
EP3048597A1 (fr) Agencement de simulation
DE102012110372A1 (de) System zur Steuerung eines Roboters zum Tragen einer Kamera

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180830

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20200903

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210518

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502017011774

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1439795

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211115

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2904634

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20220405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220120

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220220

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220221

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220120

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220121

RAP2 Party data changed (patent owner data changed or rights of a patent transferred)

Owner name: LIEBHERR-WERK BIBERACH GMBH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502017011774

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

REG Reference to a national code

Ref country code: NL

Ref legal event code: PD

Owner name: LIEBHERR-WERK BIBERACH GMBH; DE

Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), ASSIGNMENT; FORMER OWNER NAME: LIEBHERR-COMPONENTS BIBERACH GMBH

Effective date: 20220718

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20220721

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 502017011774

Country of ref document: DE

Owner name: LIEBHERR-WERK BIBERACH GMBH, DE

Free format text: FORMER OWNER: LIEBHERR-COMPONENTS BIBERACH GMBH, 88400 BIBERACH, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

REG Reference to a national code

Ref country code: AT

Ref legal event code: PC

Ref document number: 1439795

Country of ref document: AT

Kind code of ref document: T

Owner name: LIEBHERR-WERK BIBERACH GMBH, DE

Effective date: 20221018

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220407

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20220430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220407

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220407

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220407

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230704

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20170407

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20250426

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250430

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20250505

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20250428

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20250426

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20250501

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20250424

Year of fee payment: 9