EP3877896A2 - Procédé et dispositif pour usiner au moins une zone de travail au moyen d'un outil d'usinage - Google Patents

Procédé et dispositif pour usiner au moins une zone de travail au moyen d'un outil d'usinage

Info

Publication number
EP3877896A2
EP3877896A2 EP19809714.9A EP19809714A EP3877896A2 EP 3877896 A2 EP3877896 A2 EP 3877896A2 EP 19809714 A EP19809714 A EP 19809714A EP 3877896 A2 EP3877896 A2 EP 3877896A2
Authority
EP
European Patent Office
Prior art keywords
image
work area
sensor data
processing tool
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP19809714.9A
Other languages
German (de)
English (en)
Inventor
Robert BÜCHER
Nick JAGIELLA
Frank SCHRÖDER
Konrad ZÜRL
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Advanced Realtime Tracking & Co Kg GmbH
Original Assignee
Advanced Realtime Tracking & Co Kg GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Advanced Realtime Tracking & Co Kg GmbH filed Critical Advanced Realtime Tracking & Co Kg GmbH
Publication of EP3877896A2 publication Critical patent/EP3877896A2/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/06—Recognition of objects for industrial automation

Definitions

  • the invention relates to a device with a processing tool for processing at least one working area according to the preamble of claim 1 and a method for processing at least one working area with a processing tool according to claim 40.
  • a device for logging screw connections is known from the prior art from DE 10 2012 219 871 A1.
  • an image is automatically recorded by a digital camera.
  • the screw and its immediate surroundings are included in the recorded image.
  • human errors can occur. It may therefore be necessary for the device to include systems for mechanical and / or automatic quality assurance in order to prevent, avoid and / or correct human errors.
  • the invention is therefore based on the problem of avoiding errors in the processing of the at least one work area and of increasing the quality of the processing.
  • this problem is solved by a device and a method for processing at least one work area.
  • a device for processing at least one working area of an object comprises a processing tool.
  • processing can be, for example, industrial assembly and in particular screwing and riveting on an object such as a vehicle, but is not limited to this.
  • the machining tool can be formed by an assembly tool, such as an industrial screwdriver, a flange mounting device, a gun screwdriver, a rivet gun, a stapler, a drilling device, an axial screwdriver or an angle screwdriver.
  • An image recording device for example a camera module, for generating at least one image of the at least one working area and at least one sensor, for example an inertial sensor, for generating sensor data during processing of the at least one working area are arranged on the processing tool.
  • the at least one image and the sensor data can be used for the dynamic identification of work areas or work points at which the processing tool acts on the work area.
  • the at least one image can show the working area, a section of the working area with the working point and, optionally, a section, for example a tool tip, of the Show editing tool.
  • the at least one image can be, for example, a thermal image or a 2-D or 3-D image or representation.
  • the at least one image can be a topography of the surface of the at least one work area.
  • the at least one image can also be generated by scanning the surface. In one embodiment, the scanning can take place using lighting technology. Ultrasound or laser scanning can be used for scanning, for example. In particular, a laser scanner or a time-of-flight camera can be used for scanning.
  • the at least one image and / or the sensor data can be generated before and / or after the processing of the at least one work area.
  • the at least one image and / or the sensor data can be generated when the processing tool approaches the at least one work area.
  • the at least one image can be generated as a function of the sensor data.
  • the generation of the at least one image can begin or end by the generation of specific sensor data or a specific pattern of sensor data.
  • sensor data can also be generated as a function of the at least one image.
  • a specific image or a specific pattern on an image can start or stop the generation of sensor data by the at least one sensor.
  • a sensor designed as a screwdriver control can measure whether a correct angle of rotation has been reached. Faulty angles of rotation can thus be measurable, for example. When machining a large number of work areas, the angle of rotation can be measured by the screw control for each work area.
  • the at least one sensor can be an inertial measuring unit (IMU), a position sensor, an acoustic sensor, an ultrasonic sensor, an acceleration sensor, a vibration sensor, which in particular plays a tool component, in particular when the processing tool starts processing the object , can detect a distance sensor, in particular a time-of-flight (TOF) camera, a QR code scanner, a barcode scanner or a magnetic field sensor can be formed.
  • IMU inertial measuring unit
  • a position sensor an acoustic sensor
  • an ultrasonic sensor an acceleration sensor
  • a vibration sensor which in particular plays a tool component, in particular when the processing tool starts processing the object
  • a distance sensor in particular a time-of-flight (TOF) camera, a QR code scanner, a barcode scanner or a magnetic field sensor can be formed.
  • TOF time-of-flight
  • QR code scanner QR code scanner
  • barcode scanner a barcode scanner
  • magnetic field sensor can be formed.
  • the magnetic field sensor can take advantage of the fact that different working areas modify the geomagnetic field locally differently, that is to say they have a signature in the geomagnetic field.
  • the sensor data can include location bearing data, for example.
  • the location bearing data can be generated in particular by means of marker-based tracking or using an existing radio infrastructure, which in particular includes wireless technologies such as WLAN, Bluetooth, ultra-wideband or ZigBee.
  • the generation of the at least one image can thus begin or end by measuring the distance.
  • the recording of the at least one image can be started by prior laser distance measurement.
  • the device further comprises an evaluation unit, for example a processing device such as an industrial PC.
  • the evaluation unit can include software that includes modules for the acquisition, processing and evaluation of images.
  • the evaluation unit has a computing model that can be used to evaluate the at least one image and the sensor data.
  • the at least one work area and / or a situation during the processing of the at least one work area can be identified with the computing model.
  • the work area being worked on or to be worked on can thus be determined by means of the evaluation by the computing model.
  • the evaluation unit can therefore also be used to systematically and / or statistically analyze the environment on the basis of the at least one image and / or the sensor data.
  • an operating point to be edited or edited in the at least one working area can be determinable by the computing model.
  • preconditions for processing the working point can be determined using the computing model.
  • a precondition is, for example, the use of a tool component, a component to be assembled and / or a location or position of the processing tool.
  • the calculation model can also be used with the at least one image and / or the sensor data a relative position of the processing tool to the working point or to the at least one working area can be determined.
  • the computing model was trained using machine learning using images of work areas and sensor data.
  • the training can take place before the object is processed for the first time by the processing tool.
  • an image is recorded in a first step as a reference image of each work area and / or work point.
  • different situations and combinations of images and sensor data can be recorded while processing the at least one work area.
  • (representative) positions, positions and / or angles of the machining tool, in particular tool positions can be recorded relative to the working area and / or working point.
  • machine learning comprises at least one neural network, at least one folding neural network (convolutional neural network), at least one bag-of-words and / or at least one uncorrelated decision tree (random forest).
  • neural network at least one exemplary embodiment
  • folding neural network convolutional neural network
  • bag-of-words at least one bag-of-words
  • uncorrelated decision tree random forest
  • the computing model can generate at least one classifier from at least one image and / or the sensor data.
  • the at least one classifier can represent at least one essential feature of the at least one image and / or the sensor data.
  • An essential feature can be image information and / or sensor information that is more relevant than other image information and / or sensor information.
  • An essential feature can be, for example, a section of the at least one image and / or the sensor data that differs more from an adjacent section than other mutually adjacent sections of the at least one image and / or the sensor data differ from one another.
  • an essential feature can be used to recognize a specific work area from different angles, positions and / or positions of the machining tool.
  • an essential feature can also be used for a movement of the machining tool relative to the machined or machined work area by the evaluation unit to recognize that the work area is the same as long as the essential feature is present (feature tracking).
  • a movement of the machining tool relative to the machined or machined work area can be, for example, a change in the position or position, in particular the angle and / or the distance to the work area to be machined.
  • a dark circle on a light surface is an essential feature of the at least one image.
  • a sequence of images can contain an essential feature that can be represented by the at least one classifier.
  • combinations of the at least one image and the sensor data can contain an essential feature that can be represented by the at least one classifier.
  • the computing model can differentiate optically similar work areas in which the images are identical on the basis of the sensor data. For example, in the case of optically identical screw points (due to the symmetry of the object), it is possible to differentiate the screw points based on an angle of the machining tool relative to the object.
  • the computing model can identify the at least one work area using at least one image and / or sensor data by means of the at least one classifier.
  • a degree of probability for identification can be determined by the computing model.
  • the degree of probability can indicate the probability with which the at least one work area is the identified work area (confidence). For example, the degree of probability for a correct identification can lie between 0.00 (low degree of probability) and 1.00 (high degree of probability).
  • an alternative degree of probability can additionally be determined that the at least one work area is a different work area than the identified one. From the difference between the probability level for the identified work area and the alternative A degree of probability can be used to estimate the reliability of the identification.
  • the reliability of the identification can increase with the size of the difference. So the certainty of a prediction can be quantified by score differences between first best to second best prediction that the work area is an identified or an alternative work area. If the difference is large, the prediction is more certain. If the difference is marginal, the risk of confusion increases and with it the uncertainty.
  • the computational model can use the at least one classifier to determine whether a prerequisite has been met on the basis of the at least one image and / or the sensor data.
  • the at least one classifier can perform a pixel-by-pixel evaluation, e.g. in the form of a heat map.
  • the evaluation can indicate which parts of the current image have particularly high correspondences and / or differences to already known images in the training data record.
  • the computing model can thus be designed to visually superimpose evaluation information on the at least one image.
  • the evaluation information can be based on a similarity comparison of at least a section of the at least one image with the at least one classifier and / or images of work areas with which the computing model was trained.
  • the evaluation information can indicate how similar the at least one section with the at least one classifier and / or images of work areas with which the computing model was trained is.
  • a user can be able to display the at least one image with the visually superimposed evaluation information.
  • the evaluation information can be used to identify anomalies in the at least one image.
  • An anomaly can be, for example, an (unwanted, accidental) masking of a section of the at least one image.
  • a lens of a camera of the image recording device can be covered (accidentally, unintentionally) by an object or a body part of the user.
  • Such objects can be, for example, a cable, a line and / or a rope.
  • the evaluation unit is designed to recognize static areas on the at least one image which, despite the movement of the processing tool relative to the object, are unchanged in a sequence of several images.
  • Static areas of the at least one image can be identified via all training images and / or sensor data during the training process, that is to say during the training of the computing model.
  • a static area can, for example, be a section of a tool component, in particular a machining tool.
  • the static area can be used to check that the image recording device is functioning correctly. For example, if the static range changes with respect to imaging parameters such as brightness or sharpness, there could be a camera defect, camera contamination, a camera misalignment and / or a camera misalignment.
  • the computing model extracts features from the at least one image, then weights and compresses the features to find essential features, and generates the at least one classifier from the at least one essential feature.
  • the generation of the at least one classifier can, for example, result in the computing model identifying the at least one work area on the basis of the shape and / or arrangement of holes in the at least one work area when the at least one work area is identified on the basis of the at least one image.
  • Training the computing model can also include identifying an already trained work area with the computing model and using the result (right or wrong) to improve the computing model (supervised learning; back propagation). This can improve the weighting of the characteristics.
  • the computing model can be trained as follows: in a first step, at least one image is generated from at least one working point with the image recording device. In a second step, it is checked for the at least one image whether the computing model determines the at least one working point on the basis of the at least one Can identify the image. In the event of incorrect identification, in a third step the at least one image is added to a training data record or a data pool for the at least one classifier, with which the at least one classifier is re-trained. The steps are repeated until the at least one operating point is identified correctly by the computing model or with a desired degree of probability, for example above 0.75. In one variant, the at least one working point is designed as a screw point.
  • the at least one image is used to correct the sensor data by the evaluation unit.
  • a measurement of the acceleration, the angle, the position and / or the position, in particular relative to the at least one working area can be corrected by the at least one sensor, in particular an inertial sensor, by means of the at least one image.
  • a recognition of (essential) features, such as holes or edges of the object, in particular of the at least one work area, on the at least one image can be used.
  • a correction can include, for example, that a measured acceleration is corrected on the basis of the movement of an essential feature on at least three images.
  • the measurement of a direction of rotation of the processing tool by the at least one sensor can be correctable by means of the at least one image.
  • a tool component such as an extension, a nut and / or a bit
  • the evaluation unit can, for example, recognize from the at least one image and / or the sensor data which size or which type of bit is arranged on the processing tool.
  • the tool component can be determinable on the basis of a characteristic pattern, a label and / or a marking on the tool component.
  • the marking can be optical.
  • other, in particular magnetic, barcode or QR code markings are also conceivable and possible.
  • the tool component can be worn by the evaluation unit, in particular by the image recording device and / or the at least one Sensor to be recognizable.
  • wear of the tool component can be identified on the basis of a play in the tool component, in particular a screwdriver / bit system.
  • the evaluation unit can be designed, for example, on the basis of the play of the tool component or on the basis of wear, to determine a period of use of the tool component which indicates how long the tool component has already been used.
  • a position or position of the processing tool such as, for example, an angle relative to the at least one working area, or a position relative to the object, can be determined by the evaluation unit.
  • a component such as a screw
  • the evaluation unit can be designed to recognize further deviations in the at least one working area, such as color deviations.
  • the evaluation unit is designed to identify an area of the object that is not to be processed and / or to distinguish it from the at least one work area.
  • an information device which is designed to indicate to a user of the processing tool that the at least one tool component of the processing tool needs to be replaced.
  • the notification device can indicate to the user that the tool component must be replaced when a predetermined degree of wear of the tool component is exceeded.
  • the notification device can indicate that a different tool component than the one currently arranged on the processing tool is required for processing the work area.
  • the information device can monitor the exchange of the at least one tool component by evaluating the at least one image and / or the sensor data.
  • the information device can inform the user based on the at least one image and / or monitor the sensor data when replacing the tool component and, if necessary, indicate errors during the exchange.
  • the computing model is designed to identify an assembly step in an assembly process on the basis of a sequence of images and / or sensor data.
  • a sequence of images can be formed, for example, by a plurality of images that were generated at a time interval from one another.
  • a sequence of sensor data can be formed, for example, by a plurality of sections of sensor data that were generated at a time interval from one another.
  • An assembly step can include, for example, a change in a position and / or a position of the processing tool and / or an action or a sequence of actions on the at least one work area by the processing tool.
  • An assembly process can be a sequence of assembly steps.
  • an assembly step comprises screwing at a working point such as a screwing point.
  • the evaluation unit is designed to select a key area of the at least one image for training the computing model based on a user input.
  • a key area can be an image area (region of interest) that is to be used to generate the at least one classifier by the computing model.
  • the image area can in particular contain at least one essential feature.
  • the selection of a key area can be used to ignore a background of the at least one image for training the computing model.
  • the background can be an area that has fewer features than an area that is closer (along an axis from the processing tool to the object) than the background at the working point.
  • the selection of the key area can save time for training the computing model.
  • the background can also be used to identify the at least one work area by the computing model, for example by means of the different depth of field of the background for different work areas.
  • the background in particular a far range, can be masked automatically by means of the evaluation unit, for example on the basis of a threshold value of the depth of field. A masked area is not considered for evaluation.
  • a distance between the machining tool and the object can be measurable by means of a distance measuring module of the machining tool.
  • the distance measuring module comprises a laser beam aligned with the working point in combination with a camera.
  • An installation depth to be achieved can be determined by means of the distance measuring module.
  • a further quality statement can thus be added, with the aid of which successful processing of the at least one work area can be identified. For example, it may happen that the pre-set parameters have been reached in the case of incorrect screw connections and that an OK. -The screw connection is displayed by the evaluation unit.
  • the measurement of the mounting depth can serve to determine that a head, for example a screw head, of the component to be assembled, for example a screw, is not yet in contact and thus there is an incorrect assembly, for example an incorrect screw connection.
  • the distance measuring module can comprise, for example, a TOF camera with which a time-of-flight (TOF) measurement can be carried out.
  • TOF time-of-flight
  • the distance of the background in different work areas can be determined with the distance measuring module.
  • the removal of the background can also be used to identify the at least one work area by the computing model.
  • the evaluation unit is designed to select a key area of the at least one image for identifying the at least one work area based on a user input.
  • the user can select the key area, for example, while the work area is being processed by the processing tool.
  • the image recording device is designed to take a large number of images before, during and / or after the action of the To generate machining tool on the at least one work area.
  • the image recording device can generate images at any time.
  • the image recording device generates at least one image when the processing tool acts on the at least one work area for processing.
  • the image recording device can be designed to automatically start and / or end the generation of images based on sensor data.
  • the image recording device can end the generation of images when the processing tool is put down by the user.
  • the placement of the processing tool can be measurable, for example, by means of an (inertial) sensor.
  • the image recording device can start the generation of images, for example, when a user approaches the processing tool to the at least one work area.
  • a number of the images generated per unit of time can be varied by the image recording device.
  • the image capturing device can thus vary a rate of image capturing.
  • a (wireless) transmission rate, ie a frequency of transmission, of images and / or sensor data from the image recording device and / or the at least one sensor to the evaluation unit can be varied.
  • the transmission rate can be low if the processing tool is stored and high if the at least one work area is processed by the processing tool.
  • times and distances in the cyclical filing of the processing tool can be estimated using the sensor data.
  • a transmission with a high frequency can be started at the beginning of the processing of the at least one working area or when approaching the at least one working area.
  • images and / or sensor data can be transmitted at a low frequency from the image recording device and / or the at least one sensor to the evaluation unit at any time during operation of the device.
  • the image recording device comprises at least one camera.
  • the at least one camera can be used to record the at least one image.
  • the at least one camera can comprise a zoom lens.
  • the at least one camera can then be in zoom in on at least one work area.
  • the at least one camera can use the zoom lens to record a reduced image.
  • the at least one camera can then zoom out of the at least one work area.
  • the image recording device in particular the at least one camera, can be arranged on the processing tool by means of an adapter.
  • the adapter can be designed specifically for the machining tool.
  • the image recording device, in particular the at least one camera can be arranged on processing tools of various types via the adapter. Different types of processing tools can differ in terms of different shapes, for example round or angular in cross section, so that an adapter is required for mounting the image recording device.
  • the at least one camera is adjustable relative to the processing tool.
  • the at least one camera can be movably and / or rotatably mounted on the processing tool relative to the processing tool.
  • the at least one camera can, for example, be rotatable about a tool axis and / or be displaceable along the tool axis.
  • the tool axis can essentially extend along the processing tool from a handle for gripping by the user to a tool tip for processing the at least one work area.
  • the at least one camera is designed as a barcode reader or can be used for reading barcodes.
  • the image recording device comprises at least two cameras for the simultaneous generation of at least two images of the at least one work area.
  • the images generated by the at least two cameras can show overlapping, identical or different areas of the at least one work area.
  • the at least two images can also be stitched together to form a panorama, can be fused or fused with one another (in terms of content) and / or can be designed as a stereo image.
  • a Stereo image can be used in particular to calculate depth information.
  • the at least two cameras can therefore generate a 3D view.
  • the 3D view can be used to distinguish near and far areas, for example the at least one work area and a background.
  • the image recording device comprises at least one lens.
  • the at least one lens can include the end of at least one glass fiber.
  • the image recording device records the at least one image through the at least one lens.
  • At least one optical axis of the at least one objective can intersect a working point in which the processing tool acts on the at least one working area.
  • at least one detection axis of the at least one sensor can intersect the working point.
  • the at least one detection axis of the at least one sensor can be, for example, a direction in which the at least one sensor is oriented.
  • the at least one optical axis of the at least one objective of the image recording device and / or the at least one detection axis of the at least one sensor extends parallel to an action axis along which the processing tool acts on the at least one work area.
  • the parallel alignment of the at least one optical axis and the axis of action can enable a viewing direction parallel to the axis.
  • a large number of objectives of the image recording device and / or a large number of sensors are arranged in a ring around the axis of action.
  • a parallel alignment of the action axis and the at least one optical axis and / or the at least one detection axis can be made possible by integrating at least one camera and / or at least one sensor in a ring component, such as an adapter ring.
  • the ring component can be arranged on the machining tool.
  • the axis of action can extend through the ring-shaped opening of the ring component.
  • the ring component can comprise at least one objective.
  • a connecting straight line intersects between two objectives, in particular between two ends of two glass fibers, and / or between two sensors the action axis. So the two lenses and / or the two sensors can be opposite each other with respect to the axis of action.
  • the image recording device comprises at least one lighting device.
  • the at least one work area can be illuminated by means of the lighting device. In one variant, only a section of the at least one work area can be illuminated with the lighting device.
  • the lighting device can also illuminate the machining tool or at least a part thereof. In particular, the lighting device can illuminate a tool component of the machining tool with which the machining tool acts on the at least one work area.
  • the lighting device can comprise, for example, at least one light source.
  • the spectrum of the at least one light source can in principle comprise any wavelength or a spectrum of wavelengths. In particular, the at least one light source can emit a full spectrum or infrared light. As a result, the device can be independent of external lighting, for example the lighting of a workshop.
  • the at least one light source can emit light synchronized with the generation of the at least one image, in particular with the recording of the at least one image by the at least one camera.
  • the depth information of the image recording device can also be used to control the at least one light source. For example, a far area can be illuminated less than a near area. The more distant areas can therefore be illuminated less.
  • the at least one light source can of course emit light at any time and with any length of time and / or in any time interval.
  • the at least one light source can emit light for marking a position, in particular a working point to be processed, of the at least one working area.
  • the light source can be designed, for example, as a projector.
  • the light source can then be able to project at least one pattern onto the at least one work area.
  • the at least one pattern can be a designation, for example a section of the at least one work area or include directional instructions.
  • the at least one pattern can be projectable in any direction from the processing tool onto the at least one work area. The risk of the machining tool being incorrectly positioned by the user can thus be reduced.
  • a training period for a new employee can be reduced.
  • the lighting device comprises a projection unit with which the working point to be machined can be recognized and marked as soon as the machining tool approaches the working area.
  • the device comprises a simulation system with which the at least one work area for generating the at least one image, the sensor data and / or the at least one classifier can be simulated.
  • a work area for training can be simulated for the evaluation unit using the simulation system.
  • the simulation system can provide at least one image and / or sensor data of the evaluation unit.
  • the evaluation unit can then generate at least one classifier from the at least one image and / or the sensor data.
  • the simulation system derives the at least one classifier from the at least one simulated image and / or the sensor data.
  • the at least one classifier can then be made available to identify the at least one work area.
  • the at least one classifier can be made available to the evaluation unit and / or transferred to a classifier memory (classificator repository).
  • the simulation system can simulate a 3D model of the at least one work area and make it available to the evaluation unit for training.
  • the simulation system can include a system for computer-aided design (CAD system).
  • CAD system computer-aided design
  • the 3D model of the at least one work area can depend on the computing model.
  • the 3D model of the at least one work area can be selected depending on a method of machine learning.
  • the simulation system can simulate different perspectives of the image recording device. Different perspectives of the image recording device can be, for example, different viewing angles of the at least one camera of the image recording device. Perspectives that deviate from the training can arise, for example, due to a change in the tool component, for example using an extension.
  • the simulation system can be used to generate the at least one classifier for a different perspective.
  • the at least one work area can be simulated with the simulation system during the action of the machining tool on the at least one work area, that is to say in real time.
  • the simulation in real time enables an adaptation of the calculation model, in particular the generation of additional classifiers, during the processing of the work area.
  • the at least one classifier can be called up by the simulation system, in particular for selected work areas and / or work points, in particular important screwing points (A and B screw connections).
  • the at least one classifier can be called up using an app of the evaluation unit.
  • the image recording device for generating images of the at least one work area comprises a mobile terminal, in particular a mobile device, a cell phone or a smartphone.
  • the mobile terminal can be arranged on the processing tool.
  • the image recording device comprises at least one further camera for generating images of the at least one working area.
  • the mobile terminal can comprise at least one sensor.
  • the at least one sensor of the mobile terminal can, for example, determine a position, position and / or a location of the processing tool for generating sensor data.
  • the mobile terminal can be connected to the evaluation unit for the transmission of the images and / or the sensor data.
  • the mobile terminal device generates sensor data that represent a state of the machining tool, in particular screwdriver and screwing data.
  • a state of the machining tool can include, for example, a power used, switch-on and switch-off data, a torque, a screwing time, a screwing curve, a speed, an angle relative to the object and / or other settings of the machining tool.
  • the screwing curve can be, for example, a ratio of a torque of the machining tool on a screw to a rotated angle of the screw or to a screwing time.
  • the mobile terminal can be connected to the processing tool in order to generate the sensor data, for example wirelessly, in particular by means of WLAN.
  • the mobile terminal can receive classifiers from the classifier memory and use them to identify the at least one work area on the basis of the images and sensor data generated.
  • the images and sensor data generated can be stored in connection with an identification of the processed object and / or processed work area.
  • the object or the work area can be clearly identified on the basis of the marking. This means that the images and sensor data can be used for quality evaluation in the long term. In particular, a statistical analysis of service branches and service personnel can be made possible by means of the quality evaluation.
  • the evaluation unit is designed in a sequence of a plurality of known images and / or known sensor data, in which at least one unknown image and / or at least a section of unknown sensor data is contained, the at least one unknown image and / or the at least one detect an unknown section of sensor data.
  • a sequence of known images can be, for example, an image sequence during the processing of the object, within which the evaluation unit recognizes all of the images. Recognition can mean that the probability of identification is above a certain threshold. The threshold can be, for example, 0.75 or 0.95.
  • the evaluation unit can recognize the images in particular if the images have been trained in training beforehand were.
  • the at least one unknown image can be, for example, an image or an image sequence that was not previously trained in the training. The same applies to a sequence of known sensor data.
  • the duration of a section of unknown sensor data can be shorter than the duration of the previous and subsequent known sensor data.
  • the evaluation unit can furthermore be designed to be trained by means of machine learning to identify the at least one known working area on the basis of the at least one unknown image and / or the at least one section of unknown sensor data.
  • the at least one unknown image and / or the at least one unknown section of sensor data can be used to generate at least one classifier. This can be achieved by taking the at least one unknown image and / or the at least one unknown section of sensor data into the data pool for the next classifier run. At least one classifier can be generated or improved from the data pool in one classifier run. This can increase the likelihood of the work area being identified.
  • the at least one classifier can be continuously improved during the operation of the device.
  • the device can collect images for retraining during operation.
  • the computing model is designed to be trained by itself. This happens through unsupervised learning, i.e. without an expert.
  • the at least one image can be used for retraining.
  • the prerequisite for this is that the at least one work area has been identified on the basis of the at least one image by the computing model with a high degree of probability, for example a degree of probability above 0.75.
  • any image in which the classifier can make a prediction with a high degree of certainty can be used for retraining. It is assumed that the prediction was correct.
  • the overall degree of probability can be more correct Identification of a variety of work areas can be increased. So the overall certainty of predictions should continue to increase over time.
  • the computing model is designed to be trained by a user. This is done through supervised learning, i.e. with the help of an expert.
  • the at least one image can be used for retraining.
  • the prerequisite for this is that the at least one work area has been identified on the basis of the at least one image by the computing model with a low degree of probability, for example a degree of probability below 0.45.
  • images are collected in which the classifier has a high degree of uncertainty in the prediction.
  • a user can train the computing model to recognize the at least one work area on the basis of the at least one image.
  • the at least one picture is therefore labeled by an expert, i.e. the correct recognition is assigned and can then be used for retraining.
  • sensor data can be used for retraining instead of or in addition to the at least one image.
  • the evaluation unit can also be designed to recognize at least one unknown combination of at least one image and at least a section of sensor data. It is conceivable and possible that the at least one image and the at least one section of sensor data are known in each case.
  • the evaluation unit can then also be designed to be trained by means of machine learning to identify the at least one known work area on the basis of the at least one unknown combination.
  • a control unit which is designed to document a sequence of work areas which have been processed by the processing tool.
  • the control unit can therefore document which work areas have been processed and in which order the work areas have been processed.
  • the control unit can be a used tool component, a used component, such as one screwed screw, document the at least one image and / or the sensor data.
  • the use of a component can be documented by the control unit, for example for planning the logistics of components or as proof of use. In addition to the documentation of the component used, the position and / or the surroundings of the component can be documented.
  • control unit is designed to specify a sequence of work areas and / or work points for processing by the processing tool.
  • the control unit can in particular be designed to group at least two work areas in a sequence of work areas for processing by the processing tool.
  • control unit can be used to group work areas or work points for a work area which are to be processed with a tool component and / or identical states of the processing tool, such as angles relative to the object or torque.
  • control unit can assign incorrectly processed work areas while processing a work area or after processing the plurality of work areas or a group of the plurality of work areas. If necessary, the control unit can specify incorrectly processed work areas for reprocessing. The control unit can also identify an incorrect sequence when processing work areas.
  • control unit can be connected to the processing tool, the image recording device, the at least one sensor and / or the evaluation unit for exchanging data, for example by radio or cable.
  • a control unit is provided which is connected to the processing tool, the control unit and / or the evaluation unit and with which the processing tool can be controlled.
  • States of the machining tool such as a rotational speed or a torque, can be adaptable by means of the control unit.
  • states of the machining tool can be automatically determined by means of the device based on the identified one Workspace be customizable.
  • the status of the machining tool can be adjusted in real time with dynamic detection of the working point currently being machined.
  • work points and / or work areas with the same requirements for the state of the machining tool can also be grouped by means of the control unit.
  • a signal device for outputting a signal to a user of the processing tool.
  • the signal can be used to signal the user that at least one work area was not recognized, at least one work area was processed incorrectly and / or at least one work area was not processed in a sequence of work areas.
  • the signaling device can optionally signal the need for reworking at least one work area to a user of the processing tool.
  • the signaling device can also signal to the user that the evaluation unit cannot identify a work area.
  • the signaling device signals the user a result of a determination by the evaluation unit and / or control unit.
  • the signal device can therefore basically serve to output signals from the control unit and / or the evaluation unit to the user.
  • the signaling device can, for example, output a signal for exchanging the tool component.
  • the signal can be output, for example, on the processing tool.
  • the signal can be a sound, a light, or a vibration.
  • any signals are conceivable and possible.
  • the device includes a test work area for calibrating the device.
  • a user can, for example, calibrate the image recording device and in particular the at least one camera by means of the test work area.
  • the evaluation unit can be designed to automatically recognize that calibration is required.
  • the test work area can also be designed to detect damage and / or soiling on the processing tool.
  • a standard processing routine can be carried out on the test work area for damage and / or soiling. Deviations in the at least one image and / or the sensor data from the expected at least one image and / or the expected sensor data can be an indication of damage and / or contamination.
  • a possible process can include one or the following steps: 1. Approach the test work area. 2. Check the function of the processing tool. 3. Calibrate the machining tool. 4. Focus the imaging device on the test work area. 5. Detect dirt and / or damage. 6. Correct dirt and / or damage.
  • the device comprises at least one interface which is provided for connection to another system. Information can be exchanged between the device and the other system via the interface.
  • the interface can be designed as an application programming interface (API).
  • API application programming interface
  • the device can be accessible remotely by means of the interface, for example from a server or a mobile device.
  • the device can be managed or controlled centrally by means of the interface, for example by a server.
  • the processing tool comprises a display for displaying context information, for example the at least one image, the sensor data and / or parameters, settings or states of the processing tool for the user, in particular after the at least one working area has been identified.
  • use of the device can include the following optional steps:
  • a person visits the contract workshop with the object, for example a motor vehicle.
  • the person needs a repair on the object (guarantee or critical component).
  • the authorized workshop uses the device to report the repair to a server, for example a server of an OEM network.
  • the authorized workshop may receive a required component, for example a spare part for the object.
  • the authorized workshop receives approval for repairs.
  • the device receives at least one classifier via the interface from the server, which is optionally loaded into the classifier memory. In principle, the device can receive data for monitoring and / or ensuring proper processing.
  • the processing of the object begins - for example, assembly or repair.
  • the processing tool is intended for this.
  • the machining tool is known to the OEM, connected to the server via the interface and / or has received an OEM approval.
  • the at least one classifier is loaded.
  • the at least one classifier is made available to the evaluation unit.
  • the at least one classifier can be specific to the person's motor vehicle.
  • a user of the device for example a workshop MA, arranges the image recording device on the processing tool.
  • the image recording device can be, for example, a mobile camera or a flandy.
  • the user carries out the processing of the object with the processing tool with predefined values, for example a predefined state and / or predefined situations.
  • the processing is optionally documented by means of the processing tool, in particular the image recording device and / or the at least one sensor (recognizable in the environment by a classifier), via the control unit.
  • the data for monitoring and / or ensuring proper processing is compared with the at least one image and / or the sensor data by the evaluation unit and / or control unit. Matching the data and images provides proof of successful processing.
  • the at least one classifier is provided to carry out a diagnosis on the object.
  • a diagnosis on the object By means of the at least one classifier, for example, an incorrect location on the object can be identified.
  • the at least one classifier can be optimized by current diagnostic values. For example, if a incorrect or unskilled position in at least one work area of the object, the at least one work area can be relearned.
  • the OEM can provide at least one currently valid classifier based on a unique identification number of the object, in particular the motor vehicle, via the interface, for example via an intranet connection.
  • the system can also be used for general diagnosis.
  • a trained work area with at least one structure for example on a motor vehicle, can be used to check problems such as missing, displaced and / or defective components.
  • the image recording device can be aimed at an area that may have a problem. The area can be checked visually under driving conditions during the test drive by means of the image recording device.
  • the device can also detect components that are deformed under wind load and / or acceleration and other external influences, in particular during the test drive. Inadmissible deformations e.g. due to damage can be detectable by means of the device. The deformations can still be registered and documented.
  • a method according to the invention can of course be used using a device proposed according to the invention, so that the advantages and features of embodiment variants of such a device mentioned above and below also apply to embodiment variants of a proposed method and vice versa.
  • Fig. 4 device with a control unit
  • Fig. 5 device with a control unit, a signal device and a
  • Fig. 1 1 sectional view of a device with an image recording device with two lenses;
  • Fig. 12 perspective view of a processing tool in a
  • FIG. 14 perspective view of a processing tool in a
  • 15B is a perspective view of a processing tool in a
  • the 1 shows a device with a processing tool 1 for processing a work area A of an object B.
  • the work area A extends over a surface of the object B.
  • the work area A is rectangular. In principle, any shape of the work area A is conceivable and possible.
  • a working point P is arranged within the working area A, with which the machining tool 1 interacts.
  • the machining tool 1 can be, for example, an assembly screwdriver, a torque wrench or another tool.
  • Object B can be processed, for example, for assembly or repair.
  • An image recording device 12 for generating at least one image 121 of the working area A of the object B is arranged on the processing tool 1.
  • a sensor 13 for generating sensor data 131 is arranged on the machining tool 1.
  • the device also includes an evaluation unit 2 with a computing model 21.
  • the computing model 21 is trained by means of machine learning using images 120 of work areas A and sensor data 130, as shown in FIG. 2A.
  • the work area A can be identified with the computing model 21 during the processing of the work area A.
  • the work area A can be identified with the computing model 21 before the machining, for example when the machining tool 1 approaches the work area A.
  • Working area A can also be identified on the basis of sensor data 131.
  • the work area A can be identified in space by a position, an acceleration or a position of the machining tool 1.
  • the sensor 13 can be designed, for example, as a position sensor, an acceleration sensor or a position sensor.
  • the work area A can also be identifiable via an acoustic profile, that is to say the course of a sound signal over time.
  • the working point P can be identified on the basis of the sound which arises when the machining tool 1 is placed on the working point P.
  • the sensor 13 can be designed as an acoustic sensor.
  • the work area A can also be identified using a combination of at least one image 121 and sensor data 131.
  • a situation during the processing of the at least one work area A can also be identified with the computing model 21.
  • a situation can be characterized individually by features of the processing tool 1, such as a state, in particular a torque, sensor data 131 and images 121 of the working area A, in combination or in a summary during the processing of the at least one working area A.
  • the computing model 21 forms classifiers 221, 222, 231, 232 based on sensor data 130, images 120 and / or features such as a state, the processing tool 1.
  • 2B shows the formation of a classifier 221 based on an image 120.
  • the image 120 shows a work area A which is processed by a processing tool 1, a tool component 11 interacting with the work area A.
  • the computing model 21 extracts features and forms a feature assignment.
  • the features are weighted and compressed in a second step S2.
  • the computing model 21 forms a feature vector from the result of the weighting and compression.
  • the feature vector is used to generate the classifier 221.
  • the computing model 21 uses the images 120 and sensor data 130 trained during the training, in particular the classifiers 221, in order to identify the work area A and / or a situation on the basis of the images 121 and sensor data 131 .
  • the computing model 21 also takes combinations of images 121 and sensor data 131 into account.
  • the identification of a work area A using the at least one image 121 can then depend on the sensor data 131.
  • a position or location of the machining tool 1 determined during training may be required in order to identify the at least one work area A on the basis of the at least one image 121.
  • missing or incorrect positions and / or positions of the machining tool 1 can thereby be ascertainable.
  • the evaluation unit 2 can predefine allowed positions and / or positions of the machining tool 1 to a user, for example via a signal device 5.
  • the computing model 21 can learn the missing positions and / or positions, for example in response to an input from the user.
  • the computing model 21 can also use classifiers that are formed or imported in some other way, for example on another evaluation unit, in order to identify the work area A and / or a situation on the basis of images 121 and sensor data 131.
  • the computing model 21 can be designed to relearn a work area A or a situation from a context from unknown images 12T and / or unknown sensor data 13T.
  • a context can be, for example, a sequence of known images 121 and / or known sensor data 131, in which unknown images 121 and / or sensor data 131 are included. After an unknown image 121 'and / or unknown sensor data 131' from a work area A or a situation has been learned by the computing model 21, the work area A or the situation can be identified by the computing model 21.
  • 2C shows an assignment of an angle of the machining tool 1 between 0 ° and 360 ° relative to the work area A to a degree of probability which indicates the confidence with which the work area A is identified by the computing model 21 on the basis of at least one image 121.
  • the angle is representative of any parameters and in particular any state and / or situation.
  • sensor data 131 it is also conceivable and possible to carry out such an assignment for sensor data 131. The following explanations therefore also apply to sensor data 131 and to combinations of sensor data 131 and images 121.
  • An inner area 251 indicates the degree of probability that images recorded so far have reached.
  • a middle area 252 indicates which degree of probability would potentially be achievable for the at least one image 121.
  • An outer area 253 indicates the maximum degree of probability that would theoretically be attainable.
  • the at least one image 121 is taken at a certain angle. If previously acquired images that achieve a higher degree of probability than the at least one image 121 at an angle of the processing tool 1 that is adjacent to the determined angle, the at least one image 121 is used to train the computing model 21. In particular, the at least one image 121 can be transferred to a data pool for the next classifier run (pool of reference images).
  • a difference threshold can be provided, which indicates how much higher the degree of probability of the previously acquired images must be in comparison to the at least one image 121, in order to use the at least one image 121 for training the computing model 21.
  • the difference threshold can be 0.25.
  • the degree of probability of the at least one image 121 in comparison with the degree of probability that can potentially or theoretically be attained as a maximum can be used to determine whether the at least one image 121 is to be used for training the computing model 21. The difference threshold is then relative to the potentially or theoretically maximum attainable degree of probability.
  • the image 121 produced shows the working area A during processing by the processing tool 1.
  • a circular component T can be seen on this.
  • the image 121 can show any other section of the object B.
  • the image 121 can show a section of the object B that is likely to be processed by the processing tool 1 or that has been processed by the processing tool 1.
  • a sensor 13 for generating sensor data 131 is arranged on the machining tool 1.
  • the sensor data 131 show a superposition of two waveforms with different frequencies.
  • the sensor 13 can generate the sensor data 131 at any time.
  • the sensor 13 can generate sensor data 131 before the processing of the working area A by the processing tool 1 or after the processing of the working area A by the processing tool 1.
  • the device further comprises an evaluation unit 2.
  • the evaluation unit 2 receives images 121 and sensor data 131 each from the image recording device 12 and the sensor 13 for evaluation.
  • the evaluation unit 2 is connected to the image recording device 12 and the sensor 13.
  • the evaluation unit 2 can receive the images and sensor data 131 by radio or via another transmission path.
  • the evaluation unit 2 is arranged on the machining tool 1.
  • the evaluation unit 2 has a computing model 21 which is designed to generate a classifier 221, 231 from the images 121 and / or the sensor data 131.
  • the classifiers 221, 231 each represent at least one essential feature of the images 121 and / or the sensor data 131.
  • the evaluation unit 2 generates a pane as a classifier 221 for the image 121.
  • the evaluation unit 2 in FIG. 3A selects the waveform with the lower frequency than Classifier 231.
  • the generation of the classifier depends on the computing model 21. The examples shown are for illustration only.
  • the classifiers 221, 231 generated by the evaluation unit 2 can be stored in a classifier memory 24.
  • the classifiers 221, 231 can be called up by the evaluation unit 2 and / or the processing tool 1 from the classifier memory 24.
  • the classifier memory 24 can be connected wirelessly to the processing tool 1, for example.
  • a simulation system 6 is connected to the evaluation unit 2.
  • the simulation system 6 simulates the work area A. It can be used to generate classifiers 222, 232.
  • the simulation system 6 can also be used to make simulated images 120 ′′ or sensor data 130 ′′ available to the evaluation unit 2.
  • the simulated images 120 'or sensor data 120' can be used to generate classifiers 222, 232 by the computing model 21.
  • the simulation system 6 can in particular be releasably connected to the evaluation unit 2. In one variant, the simulation system 6 can be connected to the evaluation unit 2 for training and can be dispensed with during use.
  • the simulation system 6 in FIG. 3B also generates a classifier 222, which can be used to identify a work area A with a rectangular component T.
  • Another classifier 232 generated by the simulation system 6 can be used to identify sensor data with an angular waveform.
  • the classifiers 222, 232 generated by the simulation system 6 are in the
  • Classifier memory 24 can be stored.
  • the simulation system 6 can generate classifiers 222, 232 for any work areas A and / or situations.
  • the classifiers 222, 232 can be generated in real time, that is to say for example during the processing of the work area A by the processing tool 1 with the simulation system 6.
  • the simulation system 6 can ensure, for example, that variants of work areas A or objects B can be identified by the evaluation unit 2, that were not trained using images 120 of work areas A and sensor data 130.
  • the device comprises a control unit 4 for controlling the machining tool 1.
  • the control unit 4 is connected to the machining tool 1 by means of a cable 123.
  • the control unit 4 can be connected to the machining tool 1 wirelessly, integrally or in some other way.
  • the control unit 4 can also be connected to the processing tool 1 via the evaluation unit 2.
  • the control unit 4 can also be connected to the machining tool 1 and the evaluation unit 2.
  • the device in FIG. 5 comprises a control unit 3 which is designed to document and / or log which work areas A have been processed by the processing tool 1.
  • the control unit 3 can save a sequence of work areas A that have been processed by the processing tool 1.
  • the control unit 3 is connected to the evaluation unit 2.
  • the evaluation unit 2 thus forwards information about the processed work areas A to the control unit 3.
  • the control unit 4 can also be connected to the control unit 3. The control unit 4 can therefore forward information about the processed work areas A to the control unit 3.
  • a signal device 5 is connected to the evaluation unit 2.
  • the signal device 5 can be arranged on the machining tool 1, for example.
  • the signal device 5 can also be arranged on another component of the device or can be designed separately as an independent unit.
  • the signal device 5 can be integrated into a component of the device, for example the evaluation unit 2.
  • a signal can be output to a user of the machining tool 1 by means of the signal device 5.
  • the signal is used, for example, to signal the user that the evaluation unit 2 has detected an error.
  • An error can be, for example, the processing of an incorrect work area A, the incorrect processing of a work area A and / or the processing of work areas A in an incorrect sequence.
  • a test work area 7 is used to calibrate the device.
  • a calibration of the device may be necessary, for example, in order to increase the probability of identifying a work area A.
  • the test work area 7 for example, it can be checked whether the image recording device 12 is functioning properly.
  • the image recording device 12 can be calibrated by means of the test work area 7.
  • the image recording device 12 can generate images by means of at least one camera.
  • the image recording device 12 can be arranged on a side of the processing tool 1 facing the working area A when the processing area A is being processed by the processing tool 1, as shown in FIG. 6A.
  • the image recording device 12 can be arranged at any point on the processing tool 1.
  • the image recording device 12 can comprise a multiplicity of cameras, which are arranged at different locations of the processing tool 1. Using the plurality of cameras, images of an environment of the processing tool 1, one or more work areas A and / or different perspectives of a work area A and / or an environment of the work area A can be generated by the image recording device 12.
  • the image recording device 12 comprises an objective 122, the optical axis O of which intersects a working point P, at which the processing tool 1 processes the working area A.
  • the working point P can thereby be arranged in the center of the image generated by the image recording device 12.
  • the image recording device 12 can be adjustable relative to the processing tool 1.
  • the image recording device 12 is designed to zoom into and out of a working area A.
  • the image recording device 12 can comprise at least one camera with a zoom lens.
  • the optical axis O can intersect an action axis E, along which the machining tool 1 acts on the work area A, as shown in FIG. 7.
  • the optical axis O extends parallel to the action axis E.
  • An illumination device 14 is arranged on the image recording device 12, which illuminates the work area A and a part of the processing tool 1.
  • the illumination device 14 emits light essentially along the optical axis of the image recording device 12 in the direction of the working area A.
  • the light from the lighting device 14 illuminates in particular the working point P.
  • the lighting device 14 can emit light of any wavelength in any direction. It is also conceivable and possible for the lighting device 14 to be arranged at another location on the machining tool 1, as shown in FIG. 6B, and in particular on the at least one sensor 13.
  • the lighting device 14 can comprise a multiplicity of light sources, which are arranged at different locations of the machining tool 1 and emit light in any direction.
  • the lighting device 14 can continuously emit light, emit light when processing the work area A by the processing tool 1 or emit light synchronously with the recording of at least one image by the image recording device 12 and / or the generation of sensor data by the at least one sensor 13.
  • the sensor 13 is arranged on a tip of the machining tool 1 facing away from the user, in particular the tip of the machining tool 1.
  • the sensor 13 can be arranged, that is integrated, at any desired point on the machining tool 1 and in particular within the machining tool 1.
  • a detection axis D of the sensor 13 can intersect the working point P on the working area A, as shown in FIG. 7.
  • the detection axis D can in principle extend along a direction in which the sensor 13 generates the sensor data 131 with the greatest accuracy.
  • the detection axis D can extend perpendicular to an opening of the microphone.
  • the detection axis D can in particular also extend parallel to the action axis E, as shown in FIG. 8.
  • the detection axis D can extend in any direction.
  • FIG. 9 shows a work area A with a nut T.
  • a component T can be determined by the evaluation unit 2.
  • the position of the component T can be determined by the evaluation unit 2.
  • any component T can be determinable by the evaluation unit 2.
  • the absence that is, the absence of a component T or a defective component T can be recognized by the evaluation unit 2.
  • FIG. 10 shows a processing tool 1 with an extension.
  • a tool component 11 can be depicted on the at least one image 121.
  • the evaluation unit 2 can then determine the presence of the tool component 11 on the basis of the image. Likewise, the evaluation unit 2 can recognize a wrong tool component 11, which is not suitable for processing the work area A.
  • the tool component 11 can for example comprise a marking.
  • the tool component 11 can also be recognized by the at least one sensor 13.
  • Fig. 1 1 shows an image recording device 12, the lenses 122a, 122b are arranged in a ring around the axis of action E.
  • a connecting straight line between two lenses 122a, 122b intersects the axis of action E.
  • a plurality of lenses can, for example, be arranged in a star shape around the axis of action E.
  • a variety of images that can be captured by the variety of lenses can provide a variety of perspectives of the work area A.
  • Other arrangements of objectives for example distributed over a surface of the machining tool 1 or in clusters, are of course conceivable and possible.
  • the processing tool 1 comprises an extension as tool component 1 1.
  • the image recording device 12 comprises a camera with a lens 122 and is screwed to the processing tool 1 by the tool component 11 using an adapter 15.
  • the image recording device 12 generates the at least one image 121 from the at least one working area A in an area within an image recording cone 124.
  • the image recording cone 124 cuts the tool component 11.
  • the tool component 11 is therefore also shown on the at least one image 121.
  • Cut 124 other or no section of the machining tool can be adjustable by means of an adjustable image recording device 12.
  • FIG. 14 An image 121, on which the tool component 11 is shown, is shown in FIG. 14.
  • the working point P is arranged in the center of the image 121.
  • the tool component 1 formed by an extension and a bit, extends from the edge of the image 121 into the center to the working point P.
  • Features of the work area A on the basis of which the evaluation unit 2 identifies the work area A, optionally in combination with sensor data, are represented by closed shapes and lines.
  • Object B is formed by a vehicle.
  • FIG. 15A Another embodiment is shown in Fig. 15A.
  • the tool component 1 1 is formed by a bit.
  • 15B a component T, formed by a screw, is screwed to an object B formed by a vehicle seat at a working point P in FIG. 15B.
  • the work area A is formed by one end of a seat rail.
  • the image recording device 12 generates at least one image 121 of the seat rail, the bit and the screw for identifying the work area.
  • 16A-16C each show an image 121 of a work area A on a motor vehicle door.
  • the angle of the machining tool 1 in FIG. 16A is defined as 0 ° in an image plane which is arranged essentially parallel to the working area A.
  • Several essential features 121 a, 121 b, 121 c, 121 d, 121f, 121g are visible on the work area A.
  • 121 c, 121 d, 121f, 121g are formed by holes, screws and a cover.
  • the processing tool 1 is rotated in FIG. 16B by 3 ° relative to the processing tool 1 in FIG. 16A in the image plane.
  • the rotation reveals a larger section of the essential feature 121g, which is identified by a rectangle.
  • the processing tool 1 in FIG. 16C is rotated by a further 5 ° to 8 ° relative to FIG. 16C in the image plane.
  • the essential features 121a, 121b, 121c, 121d, 121f, 121g have shifted further within the image 121.
  • Another essential feature 121 e which was previously hidden by the tool component 11, is visible.
  • the images shown in Figures 16A-C form a successive series of images. Basically, the images in a series of images have no relation to one another.
  • the images 121 can be linked via the essential features 121 a, 121 b, 121 c, 121 d, 121e, 121 f, 121g contained therein in order to establish a relationship between the images of a series of images (feature recognition) .
  • a degree of probability of identifying the processed work area can be increased by linking images.
  • 17A shows a working area A of a motor vehicle, in which a component T is missing.
  • the absence of the component can be detected by means of the device.
  • the evaluation unit 2 determines the missing component T.
  • the evaluation unit 2 thus recognizes the lack, ie the absence of the component T, of the motor vehicle.
  • 17B shows the working area A of the motor vehicle on which the screw T is mounted.
  • a work area A work area

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)
  • Machine Tool Sensing Apparatuses (AREA)

Abstract

L'invention concerne un dispositif comprenant un outil d'usinage pour usiner au moins une zone de travail d'un objet. Un dispositif d'acquisition d'image conçu pour générer au moins une image de la ou des zones de travail et au moins un capteur conçu pour générer des données de détection pendant l'usinage de la ou des zones de travail sont disposés sur l'outil d'usinage. Le dispositif comprend une unité d'évaluation comportant un modèle de calcul entraîné par apprentissage machine au moyen d'images de zones de travail et de données de détection et permettant d'identifier la ou les zones de travail et/ou une situation pendant l'usinage de la ou des zones de travail. Cette invention concerne en outre un procédé pour usiner au moins une zone de travail d'un objet.
EP19809714.9A 2018-11-05 2019-11-04 Procédé et dispositif pour usiner au moins une zone de travail au moyen d'un outil d'usinage Pending EP3877896A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102018127518.8A DE102018127518A1 (de) 2018-11-05 2018-11-05 Vorrichtung und Verfahren zur Bearbeitung mindestens eines Arbeitsbereiches mit einem Bearbeitungswerkzeug
PCT/EP2019/080080 WO2020094558A2 (fr) 2018-11-05 2019-11-04 Procédé et dispositif pour usiner au moins une zone de travail au moyen d'un outil d'usinage

Publications (1)

Publication Number Publication Date
EP3877896A2 true EP3877896A2 (fr) 2021-09-15

Family

ID=68699381

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19809714.9A Pending EP3877896A2 (fr) 2018-11-05 2019-11-04 Procédé et dispositif pour usiner au moins une zone de travail au moyen d'un outil d'usinage

Country Status (5)

Country Link
US (1) US12361706B2 (fr)
EP (1) EP3877896A2 (fr)
CN (1) CN113272817B (fr)
DE (1) DE102018127518A1 (fr)
WO (1) WO2020094558A2 (fr)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2584439B (en) * 2019-06-03 2023-02-22 Inspecvision Ltd Projector assembly system and method
TR201922399A1 (tr) * 2019-12-30 2021-07-26 Tusas Tuerk Havacilik Ve Uzay Sanayii Anonim Sirketi Bir işleme takımı ömür tespit sistemi.
US11354796B2 (en) 2020-01-28 2022-06-07 GM Global Technology Operations LLC Image identification and retrieval for component fault analysis
DE102021104661A1 (de) 2021-02-26 2022-09-01 Robert Bosch Gesellschaft mit beschränkter Haftung Bildklassifikator mit Indikatorschicht für Nachvollziehbarkeit der Entscheidungen
DE102021132023A1 (de) 2021-12-06 2023-06-07 Workmation GmbH Verfahren und Vorrichtung zur Erkennung von Objekten an einem industriellen Arbeitsplatz
CN114662621B (zh) * 2022-05-24 2022-09-06 灵枭科技(武汉)有限公司 基于机器学习的农机作业面积计算方法及系统
CN114863750B (zh) * 2022-06-17 2024-05-31 中国人民解放军32128部队 一种螺栓拆装训练装置及训练方法
DE102022208827A1 (de) * 2022-08-25 2024-03-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein Sicherheitseinrichtung und Verfahren zu deren Betrieb
DE102023118299A1 (de) * 2023-07-11 2025-01-16 Soft2Tec Gmbh Verfahren und Vorrichtung zur Prozesssteuerung von Arbeitsaufgaben mittels eines Werkzeugs an einem Objekt

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19843162C2 (de) 1998-09-21 2001-02-22 Alfing Montagetechnik Gmbh Bearbeitungsvorrichtung mit einem Bearbeitungswerkzeug zur Bearbeitung eines Werkstücks
DE102006014345B3 (de) * 2006-03-28 2007-08-23 Siemens Ag Sichtprüfgerät und Verfahren zu dessen Konfiguration
EP2047617B1 (fr) 2006-07-12 2024-02-14 Imprenditore Pty Limited Appareil et système de contrôle
DE102007041097A1 (de) * 2006-09-04 2008-03-06 Robert Bosch Gmbh Werkzeugmaschinenüberwachungsvorrichtung
US20090192644A1 (en) 2008-01-30 2009-07-30 Meyer Thomas J Method and system for manufacturing an article using portable hand-held tools
EP2275990B1 (fr) 2009-07-06 2012-09-26 Sick Ag Capteur 3D
WO2011106797A1 (fr) 2010-02-28 2011-09-01 Osterhout Group, Inc. Déclenchement de projection par un repère externe dans des lunettes intégrales
WO2012000648A1 (fr) * 2010-06-28 2012-01-05 Precitec Kg Procédé de commande en boucle fermée d'une opération de traitement laser et tête de traitement de matériau au laser utilisant ce procédé
US9511274B2 (en) 2012-09-28 2016-12-06 Bally Gaming Inc. Methods for automatically generating a card deck library and master images for a deck of cards, and a related card processing apparatus
DE102012219871A1 (de) 2012-10-30 2014-04-30 Marco Systemanalyse Und Entwicklung Gmbh Verfahren und Vorrichtung zur Protokollierung von Verschraubungen
US9233470B1 (en) * 2013-03-15 2016-01-12 Industrial Perception, Inc. Determining a virtual representation of an environment by projecting texture patterns
EP2916189B1 (fr) 2014-03-06 2019-05-08 Hexagon Technology Center GmbH Fabrication de qualité contrôlée
DE102014204969A1 (de) * 2014-03-18 2015-09-24 Daimler Ag Verfahren zur Überwachung von Fertigungsvorgängen, sowie zugehörige Überwachungsvorrichtung
CN104933436B (zh) * 2014-03-19 2018-09-14 通用汽车环球科技运作有限责任公司 具有动态完整性评分的基于视觉的多摄像头工厂监测
GB201413991D0 (en) 2014-08-07 2014-09-24 Ubisense Ltd Tool tracking
WO2016157593A1 (fr) 2015-03-27 2016-10-06 富士フイルム株式会社 Appareil d'acquisition d'image de distance et procédé d'acquisition d'image de distance
DE102016118617B4 (de) 2016-09-30 2019-02-28 Carl Zeiss Industrielle Messtechnik Gmbh Messsystem
JP6548690B2 (ja) 2016-10-06 2019-07-24 株式会社アドバンスド・データ・コントロールズ シミュレーションシステム、シミュレーションプログラム及びシミュレーション方法
WO2018158601A1 (fr) * 2017-03-01 2018-09-07 Omron Corporation Dispositifs de surveillance, systèmes de commande surveillés et procédés de programmation desdits dispositifs et systèmes
DE102017121098A1 (de) * 2017-09-12 2019-03-14 Trumpf Werkzeugmaschinen Gmbh & Co. Kg Objektverfolgung-basierte steuerung von fertigungsprozessen in der metallverarbeitenden industrie
GB201800534D0 (en) 2018-01-12 2018-02-28 Ubisense Ltd Tool tracking
JP6697501B2 (ja) * 2018-03-26 2020-05-20 ファナック株式会社 工作システム

Also Published As

Publication number Publication date
US20210358110A1 (en) 2021-11-18
US12361706B2 (en) 2025-07-15
DE102018127518A1 (de) 2020-05-07
WO2020094558A2 (fr) 2020-05-14
CN113272817A (zh) 2021-08-17
WO2020094558A3 (fr) 2020-08-27
CN113272817B (zh) 2025-03-11

Similar Documents

Publication Publication Date Title
EP3877896A2 (fr) Procédé et dispositif pour usiner au moins une zone de travail au moyen d'un outil d'usinage
DE102012219187B4 (de) System zur binären Klassifikation von interessierenden Elementen in einem reproduzierbaren Prozess
DE102009029478B4 (de) System und Verfahren für einen schnellen ungefähren Fokus
DE102012219190A1 (de) Qualitätsstatusanzeige für einen Vibrationsschweissprozess
CN109840900A (zh) 一种应用于智能制造车间的故障在线检测系统及检测方法
DE102012206079A1 (de) Inspizieren von potenziell störenden Eigenschaften in einem maschinellen, visuellen System
DE102018108042B4 (de) Optisches Messsystem
EP0999433B1 (fr) Dispositif et procédé d'acquistion des signaux des vibrations et/ou de son structurel
EP3608701A1 (fr) Procédé de fourniture d'au moins un procédé d'évaluation pour échantillons
WO2015074763A1 (fr) Système équipé d'une vanne et procédé de fonctionnement d'un tel système
DE102016109030A1 (de) Verfahren zum Bewerten eines Scheinwerfers
WO2019020791A1 (fr) Procédé et dispositif mis en œuvre par un ordinateur pour générer automatiquement des données d'image caractérisées et dispositif d'analyse pour contrôler un composant
EP3726425B1 (fr) Procédé de détection physique, en particulier optique, d'au moins un objet utile
EP3872447B1 (fr) Détection optique d'un dispositif de réception pour composants interchangeables d'un appareil de mesure de coordonnées
DE102017128032A1 (de) Codelesevorrichtung und Verfahren zur Online-Verifikation eines Codes
EP3098033A1 (fr) Determination d'un angle des axes d'un robot et selection d'un robot a l'aide d'une camera
DE102008005034A1 (de) Überwachungsvorrichtung
WO2020074722A1 (fr) Procédé et système robotique pour entrer dans une zone de travail
EP3781999B1 (fr) Installation de guidage d'opérateur, installation de traitement, procédé, programme informatique, support d'enregistrement lisible par machine et unité de commande électronique
DE102020006160A1 (de) Verfahren zur Lageerkennung eines Objekts mittels eines Lageerfassungssystems, Verfahren zum Bearbeiten eines Objekts sowie Lageerfassungssystem
DE102014221753A1 (de) Verfahren zum Führen eines Werkers während einer Montage
EP4101602A1 (fr) Procédé d'analyse d'un outil et machine-outil mobile
DE102014212141A1 (de) Vorrichtung und Verfahren zum Identifizieren und Integrieren von elektrischen Bauteilen in Anlagen
DE102021119132A1 (de) Systeme und verfahren zur optimierung einer leistung eines bildverarbeitungssystems
DE102016109027A1 (de) Verfahren zur Lageüberprüfung charakteristischer Punkte in Lichtverteilungen

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20210520

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230504