EP4136567A1 - Procédé et agencement pour la capture numérique d'espaces dans un bâtiment - Google Patents

Procédé et agencement pour la capture numérique d'espaces dans un bâtiment

Info

Publication number
EP4136567A1
EP4136567A1 EP21721851.0A EP21721851A EP4136567A1 EP 4136567 A1 EP4136567 A1 EP 4136567A1 EP 21721851 A EP21721851 A EP 21721851A EP 4136567 A1 EP4136567 A1 EP 4136567A1
Authority
EP
European Patent Office
Prior art keywords
scanning device
building
obi
digital
mgi
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP21721851.0A
Other languages
German (de)
English (en)
Inventor
Christian Frey
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens Schweiz AG
Original Assignee
Siemens Schweiz AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Schweiz AG filed Critical Siemens Schweiz AG
Publication of EP4136567A1 publication Critical patent/EP4136567A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10—Recognition assisted with metadata
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition

Definitions

  • the invention relates to a method and an arrangement for digitally recording rooms in a building, a corresponding room in the building being scanned by a scanning device and recorded in a digital point cloud and / or an image.
  • a digital building model enables an overarching exchange of information between the roles (stakeholders) involved in the creation and operation (i.e. management) of a building.
  • the components and elements of a building are usually mapped in IFC (Industry Foundation Classes) notation in the digital building model.
  • Scanning devices or scanning devices are predominantly used for digital recording of the rooms / area of a building Create a point cloud and use it to create an orientation plan and even a floor plan.
  • the scanners mostly work on the basis of laser, infrared or acoustic sensors.
  • the scanning equipment can be used statically or mobile. The latter are then portable or mobile to move through the building.
  • a so-called indoor viewer increases the The importance of 3D laser scanning as it makes scan data visible and usable for everyone involved in the building.
  • An indoor viewer can be used as a collaborative platform for the display and exchange of building information or it can be integrated into existing software platforms to enable a new dimension of spatial understanding. Users can explore scanned rooms as if they were on site by moving around in a highly detailed, realistic digital twin and changing floors.
  • Point clouds are usually limited to professionals who work with floor plans and digital building models (BIM models; BIM stands for Building Information Model).
  • BIM models BIM stands for Building Information Model.
  • An indoor viewer combines point clouds with highly detailed, urgent 360 ° images to create a digital twin with which any interested building observer can explore and interact with the scanned rooms just as he would in real life.
  • An indoor viewer usually has an easy-to-use, scalable content management system for adding geotagged information (i.e. information is assigned location information) and media for 3D scans. Users can right-click anywhere on the screen to add and display this information as points of interest or points of interest (POI).
  • a corresponding search function in the indoor viewer makes it easier for a user to find the desired information in the 2D plan or in the 3D scan.
  • Finding and annotating objects that have been digitally captured by a scanning device has so far only been a laborious process with manual visual identification. It is therefore very time-consuming and error-prone, since digitally captured objects are in the Point cloud or can easily be overlooked in a 3D scan.
  • the object is achieved by a method for digital recording of rooms (e.g. rooms, corridors) of a building, whereby a corresponding room in the building is scanned (measured) and in a digital point cloud and by a scanning device (e.g. NavVis scanner) / or by means of an image recording (the image recording can take place, for example, by a digital camera), with object recognition (advantageously also an object identification) taking place with means of artificial intelligence based on the digital point cloud and / or the image recording Object recognition the digital point cloud and / or the image recording is mapped in a digital building model, whereby when recording defined objects (e.g. point of interest, POI, sights) in the building, the respective defined object (e.g.
  • the respective defined object e.g.
  • a fire alarm, an actuator, or a sensor is detected in a dedicated manner by the scanning device (for example by a camera and / or point cloud), and un d whereby attributes (eg location information, installation information, type information, product information) are assigned to the respective defined object by a voice input of an operator of the scanning device.
  • attributes eg location information, installation information, type information, product information
  • Objects are thus enriched with additional meta information (eg tags, attributes) through audio input by an operator during the building scan.
  • the image content is annotated by the scan person using suitable speech recognition.
  • the speech recognition is advantageously a trained speech recognition based on mechanisms of machine learning.
  • the dedicated detection of a defined or previously specified object e.g.
  • a fire alarm, an actuator, or a sensor by the scanning device can be done, for example, by a recording device set up for this purpose (e.g. by zooming the object through a camera attached to the scanning device).
  • the object recognition advantageously also includes object identification.
  • the image recording can take place, for example, in the form of a digital image recording, using appropriately suitable digital cameras. This can be done by individual recordings (or a sequence of individual recordings) and / or recording of one or more video sequences.
  • the image recording advantageously includes image recognition for the recognition or identification of objects on the image recording. Algorithms for pattern recognition and / or pattern classification and / or pattern analysis are advantageously used for image recognition. Artificial intelligence methods are also advantageously used for image recognition.
  • Verbal or linguistic annotation is particularly advantageous when capturing or recording round objects.
  • a defined object eg point of interest, POI
  • POI point of interest
  • the user has access to the inventory catalog and / or the inventory plan.
  • the inventory catalog and / or the inventory plan is advantageously displayed on a display of the scanning device.
  • a first advantageous embodiment of the invention is that the voice input takes place at the scan location of the respective defined object in the building.
  • the position of the scanning device can automatically be used as annotation for the defined object.
  • a further advantageous embodiment of the invention is that the voice input at the scan location of the respective defi ned object is made by a user, whereby a user receives the respective attributes for a defined object on an output device (e.g. display of a mobile communication terminal or display on the scanning device ) to be provided.
  • an output device e.g. display of a mobile communication terminal or display on the scanning device
  • a user can add the annotations to the corresponding object by reading (at a suitable volume) the information from the display.
  • the information on the display is advantageously provided by an alternative source (e.g. an existing building plan (e.g. as a PDF document).
  • a further advantageous embodiment of the invention is that the respective attributes for a defined object are provided on the output device (eg display) as a function of the location of the scanning device.
  • the operator of the scanning device must the Do not look for corresponding information on an object in an already existing building plan, because the information to be annotated is provided as a function of the position of the scanning device.
  • An object that has been annotated is advantageously acknowledged by the operator. This ensures that no object is overlooked.
  • a further advantageous embodiment of the invention is that for object recognition (and advantageously also for object identification) of defined objects, the attributes assigned to the respective defined object by the voice input are used. This enables an increase in the confidence of the object recognition also for the object identification.
  • the confidence can e.g. be determined by using precision recall diagrams (PR diagrams).
  • a further advantageous embodiment of the invention is that a speaker-independent speech recognition (Siri, etc .; optimization via deep learning) is used for the recognition of the speech input. Speaker-independent speech recognition does not require a previous training phase.
  • a further advantageous embodiment of the invention lies in the fact that the scanning device sends a message (optically and / or acoustically) to the user of the Scanning device outputs. This prevents objects from being forgotten or overlooked.
  • Another advantageous embodiment of the invention resides in a scanning device (scanning device) for carrying out the inventive method.
  • An already existing scanning device or scanning device can easily be upgraded in order to enable the method according to the invention.
  • an arrangement for the digital recording of rooms (rooms, corridors) in a building comprising: a scanning device (e.g. NavVis scanner) for scanning and / or recording a room (room, corridor) in the building, wherein the scanning device is set up to generate a digital point cloud and / or a digital image recording based on the data obtained by the scanning and / or recording; a processing device which is set up to undertake an object recognition (advantageously also an object identification) based on the digital point cloud and / or the image acquisition with means of artificial intelligence, wherein the processing device is further set up to map the digital point cloud and / or the image recording in a digital building model (BIM, Building Information Model, digital twin); wherein the scanning device comprises a speech recognition device (e.g.
  • a speech recognition device e.g.
  • the microphone for capturing speech input to defined objects in the room, whereby attributes (e.g. location information, installation information, manufacturer information, product information) are assigned to a respective defined object through a speech input the generation of the point cloud and / or the image acquisition can be used.
  • attributes e.g. location information, installation information, manufacturer information, product information
  • the point cloud and / or the digital image recording is stored in a suitable storage medium.
  • the storage medium can be present in the scanning device itself.
  • the storage medium can, however, also be implemented in a cloud infrastructure with a suitable data connection (eg radio connection, IP connection) to the scanning device.
  • a suitable data connection eg radio connection, IP connection
  • the scanning advantageously also includes a measurement of the spatiality.
  • the image recording can take place, for example, in the form of a digital image recording by means of correspondingly suitable digital cameras.
  • the image recording advantageously includes image recognition for the recognition or identification of objects on the image recording.
  • Algorithms for pattern recognition and / or pattern classification and / or pattern analysis are advantageously used for image recognition.
  • Artificial intelligence methods are also advantageously used for image recognition.
  • a further advantageous embodiment of the invention is that when defining objects in the building are detected, the respective defined object is detected in a dedicated manner by the scanning device (e.g. by a camera), and dedicated attributes can be assigned to the respective defined object through voice input.
  • the dedicated detection of a defined or previously determined object (e.g. a fire alarm, an actuator, or a sensor) by the scanning device can, for example, be done by a recording device set up for this purpose (e.g. by zooming the object by a camera attached to the scanning device).
  • a further advantageous embodiment of the invention is that the arrangement comprises a position determination system (e.g. IPS, indoor GPS) for recognizing the location of the scanning device in the building, with the respective attributes for a defined object on an output device (e.g. display of a mobile Communication terminal or display on the scanning device) depending on the location of the scanning device can be provided.
  • a position determination system e.g. IPS, indoor GPS
  • an output device e.g. display of a mobile Communication terminal or display on the scanning device
  • the operator of the Scanning device do not search for the corresponding information on an object in an existing building plan, because the information to be annotated is provided as a function of the position of the scanning device.
  • An object that has been annotated is advantageously acknowledged by the operator. This ensures that no object is overlooked.
  • a further advantageous embodiment of the invention is that the voice input takes place at the location of the respective defi ned object by a user, the user being given the respective attributes for a defined object on an output device (e.g. display of a mobile communication terminal or display on the scanning device ) are available.
  • an output device e.g. display of a mobile communication terminal or display on the scanning device
  • a user can add the annotations to the corresponding object by reading (at a suitable volume) the information from the display.
  • an alternative source for the information on the display e.g. an existing building plan (e.g. as a PDF document)
  • the voice input is advantageously carried out by the user at the scan location of the respective defined object
  • a further advantageous embodiment of the invention is that the scanning device outputs a message (optically and / or acoustically) for the user of the scanning device upon detection of a defined object in the vicinity. This prevents objects from being forgotten or overlooked.
  • the processing device is integrated in the scanning device.
  • the processing device is a processor or computer set up for this purpose with corresponding input / output means, memory, and communication means.
  • the processing device is integrated in a cloud infrastructure.
  • the scanning device is connected to the processing device (e.g. computer) by suitable means of communication (e.g. radio, WLAN).
  • suitable means of communication e.g. radio, WLAN.
  • the scanning device and the processing device advantageously comprise suitable storage means (e.g. database, flash memory).
  • FIG. 1 shows a first exemplary arrangement for the digital acquisition of rooms in a building
  • FIG. 3 shows an exemplary arrangement for a speech recognition device
  • FIG. 4 shows an exemplary flow chart for a method for digitally recording rooms in a building.
  • Points of interest are points defined in the 3D coordinate system of the indoor viewer instance with additional information and have a WGS 84 coordinate (GPS) for positioning. All POIs have one Type, a type group and a position. The content of a POI description can range from simple text to embedded iFrames. Adding custom data to POIs is useful for connecting to applications based on indoor viewers.
  • the POIs are mostly assigned to objects in the 3D scan and which can be subsequently identified by hand by a person in the indoor viewer and assigned using an editor.
  • Object recognition is a computer vision technique used to identify objects in images or videos. Object recognition is an important output of deep learning and machine learning algorithms. When people look at a photo or video, we can easily see people, objects, scenes and visual details. The aim is to teach a computer to do what is natural to humans: to understand what an image contains.
  • the methods for object identification include 3D models, component identification, edge detection and analysis of appearances from different angles.
  • Object recognition takes place at the convergence points of robotics, machine vision, neural networks and AI (artificial intelligence).
  • Deep learning techniques have become a popular method for object recognition.
  • Deep learning models like Conventional Neural Networks (CNNs) are used to automatically learn the inherent characteristics of an object in order to identify that object.
  • CNNs Conventional Neural Networks
  • a CNN can learn to tell differences between cats and dogs by analyzing thousands of training images and the characteristics learned that make cats and dogs different.
  • Training a model from scratch To train a deep network from scratch, collect a very large, labeled data set and design a network architecture that learns the functions and creates the model. The results can be impressive, but this approach requires a large amount of training data and you need to set up the shifts and weights on CNN.
  • Deep learning offers a high level of accuracy, but requires a large amount of data to make accurate predictions. Deep Learning for Image-Based Localization
  • Machine learning techniques are also popular for object recognition and offer different approaches than deep learning. Common examples of machine learning techniques are:
  • a feature extraction algorithm can extract edge or corner features that can be used to distinguish classes in your data.
  • MATLAB automates the provision of the models on company systems, clusters, clouds and embedded devices.
  • FIG. 1 shows a first exemplary arrangement for digitally recording rooms of a building or part of a building RI.
  • the exemplary arrangement according to FIG. 1 for the digital recording of rooms RI (e.g. room, corridor) of a building comprises: a mobile scanning device MGI (e.g. NavVis scanner) for scanning and / or recording (e.g.
  • MGI mobile scanning device
  • MGI e.g. NavVis scanner
  • the scanning device MGI is set up to generate a digital point cloud PW1 and / or a digital image based on the data obtained by the scanning and / or the recording; a processing device S, which is set up to carry out an object recognition with means of artificial intelligence based on the digital point cloud PW1 and / or the digi tal image, wherein the processing device S is further set up to map the digital point cloud PW1 and / or the digital image into a digital building model BIM (BIM, Building Information Model, digital twin); characterized in that the scanning device MGI comprises a speech recognition device SPEV1 (e.g.
  • the point cloud PW1 is, for example, a 3D point cloud.
  • OBI attributes e.g. location information, installation information, manufacturer information, Product information
  • the point cloud PW1 is, for example, a 3D point cloud.
  • the scanning device MGI comprises a suitable recording device AVI (e.g. camera, lidar (light detection and running), ladar (laser detection and ranging), laser scanning, etc.) for scanning the space RI.
  • AVI e.g. camera, lidar (light detection and running), ladar (laser detection and ranging), laser scanning, etc.
  • the scanning advantageously also includes a measurement of the spatiality RI.
  • an object identification takes place together with the object identification or in addition to the object identification.
  • a suitable file format or graphic format is used for the digital image, e.g. for raster (e.g. .ami, .apx,
  • Graphic formats can be, for example, JPG; Exif, IPTC, or XMP. The graphic formats are advantageously compressed accordingly.
  • the MGI scanning device e.g. NavVis scanner
  • scans (measures) a corresponding room in the building records it in a digital point cloud and / or a digital image and advantageously processes it further (e.g. mapping in the BIM).
  • a defined object is, for example, an object that is already known in the room.
  • an inventory object located in the room i.e. the defined object is an inventory object in the room.
  • the previously known object or the inventory object are advantageously listed in an inventory catalog or an inventory plan for the space.
  • the user has a look at the inventory catalog and / or the inventory plan.
  • the inventory catalog and / or the inventory plan is advantageously displayed on a display of the scanning device.
  • the point cloud is stored in a suitable storage medium DB (e.g. database, flash memory).
  • the storage medium can be present in a data processing unit (e.g. processor, computer) of the scanning device MGI itself.
  • the storage medium can, however, also be implemented in a cloud infrastructure C, with a suitable data connection KV1 (e.g. radio connection, IP connection) to the scanning device MGI.
  • KV1 e.g. radio connection, IP connection
  • the scanning device can also be a suitably set up mobile communication terminal MG2 (for example smartphone).
  • the mobile communication terminal MG2 is equipped with a suitable recording device AV2 (e.g. camera) fitted.
  • the point cloud PW2 generated by the recording device AV2 can be forwarded from the scanning device MG2 (e.g. smartphone, tablet computer) to the processing device S (appropriately configured server) via a suitable communication connection KV2 (e.g. radio connection, IP connection) , for mapping the digital point cloud PW2 in a digital building model BIM (BIM, Building Information Model, digital twin).
  • a suitable communication connection KV2 e.g. radio connection, IP connection
  • the server S and the BIM database DB are advantageously implemented in a cloud infrastructure C.
  • the scanning device MGI it is advantageous to have a mobile, drivable device operated by a user PI.
  • the scanning device MG2 is a correspondingly directed mobile, portable device (e.g. smartphone) which is operated by a user PI.
  • the voice annotations for an object OBI are carried out by the operator PI of the corresponding scanning devices MGI, MG2 or by another person.
  • the scanning device can also be a drone set up accordingly.
  • the respective defined object OBI is advantageously detected in a dedicated manner by the scanning device MGI, MG2.
  • a voice input SPEV1 by the operator PI assigns dedicated attributes (e.g. type properties, installation properties, relationships to the building infrastructure) to the respective defined object OBI.
  • the exemplary arrangement according to FIG. 1 comprises a position determination system IPS (e.g. indoor positioning system; I beacons) for recognizing the location of the scanning device MGI, MG2 in the building RI, where the respective attributes for a defined object OBI can be provided on an output device (eg display of the scanning device MGI, MG2) as a function of the location of the scanning device MGI, MG2.
  • IPS e.g. indoor positioning system
  • I beacons for recognizing the location of the scanning device MGI, MG2 in the building RI, where the respective attributes for a defined object OBI can be provided on an output device (eg display of the scanning device MGI, MG2) as a function of the location of the scanning device MGI, MG2.
  • the voice input is advantageously carried out at the location of the respective defined object OBI by a user PI, with the user PI being able to provide the respective attributes for a defined object OBI on an output device of the scanning device MGI, MG2. This ensures, among other things, that all known attributes are assigned to the OBI object.
  • the attributes are advantageously provided by an alternative or further source.
  • the scanning device MGI, MG2 advantageously outputs a message (optically and / or acoustically) for the user PI of the scanning device MGI, MG2 when it detects a defined object OBI in the vicinity. This ensures that no defined object (pole) OBI is forgotten when assigning attributes
  • the processing device is advantageously integrated in the scanning device.
  • the processing device S can, however, also be integrated in a cloud infrastructure C.
  • the processing device S can be integrated in a cloud infrastructure C, for example, as a BIM server with access to a BIM database DB.
  • the point cloud PW1, PW2 generated by the scanning device (scanning device) MGI, MG2 is transmitted from the scanning device MGI, MG2 to the BIM server S via suitable communication links KV1, KV2. With the communication connections KV1, KV2 it is, for example, radio connections, WLAN, IP network connection).
  • FIG. 2 shows a second exemplary arrangement for digita len detection of rooms of a building or building part R2.
  • the exemplary arrangement according to FIG. 2 for the digital detection of rooms R2 (eg room, corridor) of a building comprises: a mobile scanning device MG3 (eg NavVis scanner) for scanning a room R2 in the building, the scanning device MG3 being set up a digital point cloud PW3 is to be generated based on the data obtained by the scanning; a processing device S, which is set up, based on the digital point cloud PW3, to carry out an object recognition with means of artificial intelligence, where the processing device S is further set up to map the digital point cloud PW3 into a digital building model BIM (BIM, Building Information Model) , digital twin); characterized in that the scanning device MG3 comprises a speech recognition device SPEV2 (e.g.
  • SPEV2 e.g.
  • the point cloud PW3 is, for example, a 3D point cloud.
  • the point cloud PW3 is transmitted from the scanning device MG3 to the processing device S via a suitable communication link KV3 (for example radio). The transmission is advantageously carried out in real-time.
  • the point cloud PW3 can be accessed via the appropriate Communication link KV3 (for example radio) from the scanning device MG3 to the processing device S, but also transmitted by a batch run, for example triggered by the operator P2, or daily at a specific point in time.
  • the processing device S eg BIM server
  • the scanning device MG3 comprises a suitable recording device AV3 (e.g. camera, lidar (light detection and running), ladar (laser detection and ranging), laser scanning, etc.) for scanning the space R2.
  • AV3 e.g. camera, lidar (light detection and running), ladar (laser detection and ranging), laser scanning, etc.
  • the scanning advantageously also includes a measurement of the spatiality R2.
  • an object identification takes place together with the object identification or in addition to the object identification.
  • the defined object OB2 (e.g. point of interest, pole) is assigned the attributes A by operator P2 through a voice input SPRE.
  • the defined object OB2 is a fire detector.
  • the fire detector OB2 is recorded in a dedicated manner by the recording device AV3 of the scanning device MG3 (e.g. by a camera that is located directly or almost directly below the fire detector OB2).
  • the camera AV3 is advantageously located in a plumb line from the object OB2 to the floor or in a range of 2 meters, in particular 1 meter, around the plumb axis when the object OB2 is recorded (or fixed).
  • Operator P2 assigns the following exemplary attributes A to object OB2 (fire alarm) by means of a voice input SPRE when scanning room R2: Smoke alarm, Sinteso,
  • the fire detector is to be recorded by camera 0, for example.
  • these attributes are assigned to the fire detector OB2 and These attributes are also assigned to the object OB2 in the BIM digital building model.
  • the attribute A is assigned to the object OB2 as additional meta information when the building is scanned.
  • a further advantageous embodiment of the invention is that a speaker-independent speech recognition (Siri, etc .; advantageously with optimization via deep learning and / or machine learning algorithms) is used for the recognition of the voice input SPRE. Speaker-independent speech recognition does not require a previous training phase.
  • deep learning methods e.g. neural networks
  • speech recognition lie in the implicit learning of representations of the input data that lead to an optimal result (based on the number of available examples).
  • deep learning methods e.g. neural networks
  • deep learning has contributed greatly to a significant number of machine learning applications, but regardless of its innovative strength and obvious practical applications, this type of machine learning is still a considerable effort.
  • the object to be identified e.g. smoke detector
  • the object to be recognized is approached and the description is made via the voice input at the location: e.g. "Camera 0, Sinteso FDOOT241-9 smoke detector". "Camera 0" is the camera in the scanner pointing upwards.
  • a microphone release function push-to-talk or voice activation triggers the voice recording for the scan.
  • the object to be recognized is advantageously approached as close as possible, with the possibility at a distance at which the object can be recorded essentially selectively (advantageously alone in a recording). If the object to be recorded is on the ceiling (as is common with fire alarms, for example), the recording is advantageously carried out essentially in a plumb line from the object downwards.
  • the object recognition can thus “concentrate” on the object described by speech recognition and assigns the description from the speech recognition to the object recognized in camera 0.
  • a “speaker-independent” speech recognition is advantageously used.
  • a characteristic of the "speaker-independent" speech recognition is the property that the user can start speech recognition immediately without a previous training phase.
  • the vocabulary is limited to a few thousand words. But that is completely sufficient for the objects in the building.
  • a filtered assignment of the object selection can also be made.
  • fire alarms in office buildings are usually mounted on the ceiling, so they can best be seen in camera 0.
  • a light switch for example, the area on the wall, e.g. at a height of 1 m to 1.4 m next to a passage, is analyzed, which is best captured by the cameras 1 or 3 on the side.
  • the WGS 84 coordinate (GPS) for determining the position can also be determined, whereby the object can be annotated in the 2D / 3D plan. A corresponding symbol can also be placed in a 2D plan.
  • placement rules can reduce the error rate or report incorrect placements.
  • a Fire detectors have a minimum distance from the room delimitation (wall, window).
  • FIG. 3 shows an exemplary arrangement for a speech recognition device SPEV3.
  • Analog language AS of a user is recorded by a preprocessing unit W E and transformed into corresponding reference vectors RV.
  • the reference vectors RV are forwarded to a decoder D of the speech recognition unit SPEE.
  • Decoder D creates a "list of words", i.e. a word list WL, based on an acoustic model AM, a dictionary WB, and a language model SM.
  • Speech recognition is a method of speech analysis in which a computer-based system with automatic speech recognition analyzes, classifies and saves the entered speech information.
  • the time-consuming learning of interesting objects contained in the scan (training data) can be reduced since the automatic speech recognition contains the exact description.
  • the system can thus train and optimize itself by including software for recognizing and processing naturally spoken language.
  • FIG. 4 shows an exemplary flow chart for a method for digitally recording rooms (e.g. rooms, corridors) in a building
  • VS1 whereby a corresponding room in the building is scanned (and / or measured) by a scanning device (e.g. NavVis scanner) and recorded in a digital point cloud,
  • a scanning device e.g. NavVis scanner
  • VS3 where after the object has been recognized, the digital point cloud is mapped into a digital building model, (VS4) where, when recording defined objects (e.g. point of interest, POI, sights) in the building, the respective defined object (e.g. a fire alarm) Ak tor, or a sensor) is dedicated by the scanning device (e.g. by camera and / or point cloud), and attributes (e.g. location information, installation information, type information, product information) to the respective defined object through a voice input of an operator of the scanning device ) be assigned.
  • defined objects e.g. point of interest, POI, sights
  • the respective defined object e.g. a fire alarm
  • attributes e.g. location information, installation information, type information, product information
  • Objects are thus enriched with additional meta information (eg tags, attributes) through audio input by an operator during the building scan.
  • the image content is advantageously annotated by the scan person using suitable speech recognition.
  • the speech recognition is advantageously a trained speech recognition based on mechanisms of machine learning.
  • the dedicated detection of a defined or previously specified object (e.g. a fire alarm, an actuator, or a sensor) by the scanning device can for example be done by a recording device set up for this purpose (e.g. by zooming the object by a camera attached to the scanning device).
  • Object recognition also includes object identification.
  • a defined object is, for example, an object that is already known in the room.
  • an inventory object located in the room i.e. the defined object is an inventory object in the room.
  • the previously known object or the inventory object are advantageously listed in an inventory catalog or an inventory plan for the space.
  • the user has a look at the inventory catalog and / or the inventory plan.
  • the inventory catalog and / or the inventory plan is advantageously displayed on a display of the scanning device.
  • the scanning device eg NavVis scanner
  • scans measures (measures) a corresponding room in the building and records it in a digital point cloud and / or through image recognition (image recognition can be carried out, for example, by a digital camera) further processed with advantage (e.g. mapping in the BIM).
  • the voice input is advantageously carried out at the scan location of the respective defined object in the building.
  • the voice input at the scan location of the respective defined object is advantageously carried out by a user, the respective attributes for a defined object being made available to a user on an output device.
  • the respective attributes for a defined object are advantageously provided on the output device (eg display) as a function of the location of the scanning device.
  • the attributes assigned to the respective defined object by the voice input are advantageously used for object recognition of defined objects.
  • Speaker-independent speech recognition is advantageously used for recognizing the speech input.
  • the scanning device advantageously outputs a message (optically and / or acoustically) for the user of the scanning device.
  • the message is output by appropriate output means on the scanning device, e.g. loudspeaker, display).
  • the method according to the invention for digitally recording rooms (e.g. rooms, corridors) in a building can be implemented using a correspondingly set up scanning device.
  • a defined object If a defined object is recognized in the scan, it can automatically be annotated at the recognized position in the indoor viewer.
  • the position of the object can be referenced in the digital building model (BIM, digital twin).
  • BIM digital building model
  • the WGS84 position or the room position (distance from wall, ceiling, floor) can be used for this.
  • the scan provides very precise measurement results here.
  • the 360 ° images are not just digital images.
  • the pixels in these images are enriched by laser scans (cloud of points), which makes it possible to interact with the scanned areas as if you were there, including precise point-to-point measurements.
  • Point clouds have proven to be a very useful representation of an indoor scene for solving basic computer vision problems. It takes advantage of the color image that Provides information about the appearance of an object, but also the depth image, which is immune to fluctuations in color, lighting, angle of rotation and scaling.
  • the automatic object recognition is very advanced nowadays, which promises a reliable use.
  • Stored rules can increase the quality of the object recognition.
  • a message can be output if, according to a positioning rule, an object (e.g. fire alarm) should be present, but it is not recognized in the scan area.
  • an object e.g. fire alarm
  • the objects clearly identified by the automatic speech recognition are advantageously stored as training data in a database, which are advantageously used successively by the corresponding deep learning processes.
  • the method according to the invention offers efficient acquisition of data for the provision of so-called “digital twins” (digital building models). Training data for machine learning or deep learning are generated “on the scan job”. This means cost savings, quality improvement, and also a time / scan optimization.
  • Method and arrangement for the digital acquisition of rooms in a building with a scanning device corresponding space in the building is scanned and recorded in a digital point cloud, based on the digital point cloud an object recognition takes place with means of artificial intelligence, whereby after the object recognition, the digital point cloud is mapped in a digital building model, with the acquisition of defined objects in the building, the respective defined object is recorded in a dedicated manner by the scanning device, and attributes are assigned to the respective defined object by means of a voice input and / or a voice message.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention porte sur un procédé et sur un agencement pour la capture numérique d'espaces dans un bâtiment. Un espace correspondant dans le bâtiment est numérisé par un dispositif de numérisation et capturé dans un diagramme de dispersion numérique et/ou par une capture d'images. Un objet est détecté par intelligence artificielle sur la base du diagramme de dispersion numérique et/ou de la capture d'images. Une fois que l'objet est détecté, le diagramme de dispersion numérique et/ou la capture d'images sont mappés dans un modèle numérique de bâtiment. Pour la capture d'objets spécifiés dans le bâtiment, l'objet spécifié particulier est capturé par le dispositif de numérisation lors d'une opération dédiée, et des attributs sont assignés à l'objet spécifié particulier par entrée vocale.
EP21721851.0A 2020-04-17 2021-04-09 Procédé et agencement pour la capture numérique d'espaces dans un bâtiment Pending EP4136567A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102020204921.1A DE102020204921A1 (de) 2020-04-17 2020-04-17 Verfahren und Anordnung zur digitalen Erfassung von Räumlichkeiten eines Gebäudes
PCT/EP2021/059318 WO2021209341A1 (fr) 2020-04-17 2021-04-09 Procédé et agencement pour la capture numérique d'espaces dans un bâtiment

Publications (1)

Publication Number Publication Date
EP4136567A1 true EP4136567A1 (fr) 2023-02-22

Family

ID=75690248

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21721851.0A Pending EP4136567A1 (fr) 2020-04-17 2021-04-09 Procédé et agencement pour la capture numérique d'espaces dans un bâtiment

Country Status (5)

Country Link
US (1) US12361730B2 (fr)
EP (1) EP4136567A1 (fr)
CN (1) CN115362480A (fr)
DE (1) DE102020204921A1 (fr)
WO (1) WO2021209341A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20250148750A1 (en) * 2023-11-08 2025-05-08 International Business Machines Corporation Multimodal artificial intelligence-based communication systems
CN118551434B (zh) * 2024-04-29 2025-04-25 安徽水利开发有限公司 一种基于数字孪生的钢箱提篮拱桥施工模拟方法及系统
CN118840427B (zh) * 2024-09-24 2024-12-10 上海建工四建集团有限公司 基于bim的模型标注和实景点三维定位方法及设备

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3586553B1 (fr) * 2017-02-22 2021-06-09 Middle Chart, LLC Modèle de construction amélioré à capture virtuelle de caractéristiques telles que construites et suivi de performance d'objectif

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202798910U (zh) * 2012-09-20 2013-03-13 艾塔斯科技（镇江）有限公司 可语音控制的扫描仪
CN103152505A (zh) * 2012-10-23 2013-06-12 艾塔斯科技（镇江）有限公司 智慧型扫描仪及操作方法
US10282722B2 (en) * 2015-05-04 2019-05-07 Yi Sun Huang Machine learning system, method, and program product for point of sale systems
US10762251B2 (en) * 2017-02-22 2020-09-01 Middle Chart, LLC System for conducting a service call with orienteering
CN106709481A (zh) * 2017-03-03 2017-05-24 深圳市唯特视科技有限公司 一种基于二维‑三维语义数据集的室内场景理解方法
DE102017208174A1 (de) 2017-05-15 2018-11-15 Siemens Schweiz Ag Verfahren und Anordnung zur Berechnung von Navigationspfaden für Objekte in Gebäuden oder auf einem Campus
EP3534240A1 (fr) 2018-03-01 2019-09-04 CMORE Automotive GmbH Procédé et dispositif d'annotation de données
CN108629849A (zh) * 2018-05-16 2018-10-09 浙江大学 一种基于bim和点云技术的构件质检系统
US11301683B2 (en) * 2018-10-10 2022-04-12 Autodesk, Inc. Architecture, engineering and construction (AEC) construction safety risk analysis system and method for interactive visualization and capture

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3586553B1 (fr) * 2017-02-22 2021-06-09 Middle Chart, LLC Modèle de construction amélioré à capture virtuelle de caractéristiques telles que construites et suivi de performance d'objectif

Also Published As

Publication number Publication date
DE102020204921A1 (de) 2021-10-21
US12361730B2 (en) 2025-07-15
WO2021209341A1 (fr) 2021-10-21
US20230154214A1 (en) 2023-05-18
CN115362480A (zh) 2022-11-18

Similar Documents

Publication Publication Date Title
CN110488975B (zh) 一种基于人工智能的数据处理方法及相关装置
WO2021209341A1 (fr) Procédé et agencement pour la capture numérique d'espaces dans un bâtiment
CN113781519A (zh) 目标跟踪方法和目标跟踪装置
CN116518973A (zh) 一种适用于现实室内环境的机器人视觉语言导航方法
CN113408329B (zh) 基于人工智能的视频处理方法、装置、设备及存储介质
US20220237224A1 (en) Methods and system for coordinating uncoordinated content based on multi-modal metadata through data filtration and synchronization in order to generate composite media assets
CN114677644A (zh) 一种基于教室监控视频的学生入座分布识别方法及系统
CN113705510A (zh) 目标识别跟踪的方法、装置、设备及存储介质
CN119809925B (zh) 一种多模态模型视觉感知能力增强方法、设备及介质
CN112087590A (zh) 图像处理方法、装置、系统及计算机存储介质
CN117177005A (zh) 基于多模态与动态视角调整的花絮视频生成方法
CN116528015A (zh) 数字人视频的生成方法、装置、电子设备和存储介质
CN112242002A (zh) 基于深度学习的物体识别和全景漫游方法
CN118741037A (zh) 一种适用于施工现场的多功能会议管理系统
CN113673478A (zh) 基于深度全景拼接的港口大型设备检测与识别方法
CN117827047A (zh) 基于视觉图像的对话方法、电子设备及存储介质
DE102018217032A1 (de) Verfahren und Vorrichtung zur Bereitstellung von Anmerkungen in der erweiterten Realität
US20200311901A1 (en) Classifying panoramic images
CN112925470B (zh) 交互式电子白板的触摸控制方法、系统和可读介质
DE102025123445A1 (de) Steuern von dialog unter verwendung kontextueller informationen für streaming-systeme und -anwendungen
CN116996707B (zh) 一种虚拟角色的渲染方法、电子设备及存储介质
DE102004001595A1 (de) Verfahren zur informativen Beschreibung von Bildobjekten
DE202018100752U1 (de) Bayesische Methodik für georäumliche Objekt-/Merkmalserkennung
Srilatha et al. 3D Smartlearning Using Machine Learning Technique
Pavlovskis et al. Application of multi-criteria decision making for the selection of sensing tools for historical gravestones

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220826

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20250109