WO2026020242A1 - Système d'aide à la coupe d'arbre en temps réel pour équipement forestier - Google Patents

Système d'aide à la coupe d'arbre en temps réel pour équipement forestier

Info

Publication number
WO2026020242A1
WO2026020242A1 PCT/CA2025/051001 CA2025051001W WO2026020242A1 WO 2026020242 A1 WO2026020242 A1 WO 2026020242A1 CA 2025051001 W CA2025051001 W CA 2025051001W WO 2026020242 A1 WO2026020242 A1 WO 2026020242A1
Authority
WO
WIPO (PCT)
Prior art keywords
tree
cutting
real
trees
point cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CA2025/051001
Other languages
English (en)
Inventor
Maxime VAIDIS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Groupe Carvi Inc
Original Assignee
Groupe Carvi Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Groupe Carvi Inc filed Critical Groupe Carvi Inc
Publication of WO2026020242A1 publication Critical patent/WO2026020242A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G23/00—Forestry
    • A01G23/02—Transplanting, uprooting, felling or delimbing trees
    • A01G23/099—Auxiliary devices, e.g. felling wedges
    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G23/00—Forestry
    • A01G23/02—Transplanting, uprooting, felling or delimbing trees
    • A01G23/08—Felling trees
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/02—Agriculture; Fishing; Forestry; Mining
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/87—Combinations of systems using electromagnetic waves other than radio waves

Definitions

  • the invention relates to the field of forestry, and more specifically to systems and methods for assisting efficient timber harvesting.
  • a real-time tree-cutting assistance system for forestry equipment is provided.
  • the tree-cutting assistance or support system is installable in the cabin of the forestry equipment and is configured to detect trees, determine their characteristics and provide cutting instructions to the operator in real time.
  • the real-time tree-cutting assistance system for forestry equipment may comprise a computer, at least one camera, and at least one 3D scanner.
  • the computer can include program instructions configured to use images and point clouds from the at least one camera and from the 3D scanner to detect objects, including trees and the woodprocessing head of the forestry equipment, prior to the generation of a 3D map of the environment of the forestry equipment.
  • the camera(s) and the 3D scanner(s) may be mounted on an in-cabin bracket.
  • the camera(s) and the 3D scanner(s) may be calibrated prior to installation of the system in the cabin.
  • the real-time tree-cutting assistance system may comprise one or more of: a sensor module configured to manage sensors, including information from the at least one camera and 3D scanner and to synchronize the images captured and the point clouds generated; a detection module configured to detect trees and the harvester head from the synchronized images and point clouds; a mapper module configured to generate, based on the detection of the trees by the detection module, a 3D map comprising trees and their characteristics; and at least one tree-cutting support module configured to provide instructions to the operator based on the computed characteristics of the trees.
  • a sensor module configured to manage sensors, including information from the at least one camera and 3D scanner and to synchronize the images captured and the point clouds generated
  • a detection module configured to detect trees and the harvester head from the synchronized images and point clouds
  • a mapper module configured to generate, based on the detection of the trees by the detection module, a 3D map comprising trees and their characteristics
  • at least one tree-cutting support module configured to provide instructions to the operator based on the
  • the system comprises a computer comprising at least one memory and at least one processor, the memory having stored thereon instructions executable by the at least one processor to perform one or more of the following steps: process and synchronize the captured images and the point cloud; detect and classify objects in the environment of the forestry equipment by analyzing the synchronized images and the point cloud, wherein at least some of the objects classified are trees; position the classified trees relative to each other and to the forestry equipment and generate a 3D map of the environment comprising the detected trees, the detection and classification of objects being carried out prior to the generation of the 3D map; determine tree characteristics associated with the classified trees and provide a cutting recommendation as to whether or not a tree should be cut.
  • the system comprises an electronic display configured to present a user interface to an operator of the forestry equipment, the user interface showing the classified trees in real-time and the cutting recommendation, as the forestry equipment is being operated.
  • the at least one image sensor for capturing images comprises a color camera and the at least one depth sensor for generating the point cloud comprises a LIDAR sensor.
  • the at least one depth sensor for generating the point cloud comprises a first high-density LIDAR sensor generating a first point cloud; and a second lower- density LIDAR sensor generating a second point cloud.
  • the point cloud results from merging of the first and the second point clouds generated by the first high-density LIDAR and the second lower-density sensor.
  • the at least one image sensor for capturing images and the at least one depth sensor for generating the point cloud operate at a frequency of at least 10 Hz.
  • the system comprises an in-cabin bracket installable inside the cabin, with the at least one image sensor and the at least one depth sensor mounted on the in-cabin bracket.
  • the in-cabin bracket is configured to maintain a fixed and know spatial relationship between the at least one image sensor and the at least one depth sensor in fixed, enabling pre-calibration of the system prior to installation and providing protection for the at least one image sensor and the at least one depth sensor from environmental factors during harvesting operations.
  • the at least one sensor for capturing images and at least one sensor for generating the point cloud have overlapping fields of view, facilitating merging of the image data and point cloud data.
  • the system comprises a geo-positioning system to provide a global position of the forestry equipment, the global position being used to position the classified trees.
  • the instructions are further executable by the at least one processor to synchronize the captured images and the point cloud based on a clock signal derived from the geo-positioning system.
  • the geo-positioning system comprises a Global Navigation Satellite System (GNSS), or a GPS.
  • GNSS Global Navigation Satellite System
  • GPS Global Navigation Satellite System
  • the system comprises one or more microphones, for capturing the sound of the environment of the forestry machine, to detect and time cutting or delimbing operations.
  • the computer is located in the cabin.
  • the computer comprises CAN bus port for interfacing with a CAN bus of the forestry equipment.
  • the system may comprise an electronic board configured to read analog machine signals from the forestry equipment.
  • synchronization of the captured images and point cloud, and detection and classification of the objects is performed at a refresh rate of at least 10Hz.
  • the instructions are further executable by the at least one processor to remove dynamic objects from the point cloud, including the wood-processing head of the forestry equipment, prior to passing the point cloud for 3D mapping, thus reducing processing time for 3D mapping.
  • the system comprises different modules.
  • the instructions to process and to synchronize the captured images and the point cloud are part of a capture module, the instructions to detect and to classify the objects in the environment of the forestry equipment are part of a detection module; the instructions to position the classified trees are part of a 3D mapping module; and the instructions to determine tree characteristics and to provide the cutting recommendation is part of at least one tree-cutting aid module.
  • a set of tree-cutting aid modules configured to analyze the classified trees and generate the cutting recommendation based on the tree characteristics and operational parameters.
  • the operational parameters comprise at least one or more of: current cutting mode selected by the operator, forestry equipment type, harvester head configuration, machine position and orientation, harvesting objectives, tree species-specific requirements, predefined cutting rules, and market value data associated with different log types or lengths.
  • the set of tree-cutting aid modules comprises one or more of: a normal cut value estimation module, configured to extract information from the point cloud to monitor and assess forestry operations; a wood-processing head and tree cut detection module, configured to determine the position of the wood-processing head in its environment and detects when a tree has been cut in real time; a long log volume estimation module, configured to estimate the volume of long timber; an optimal cut prediction module, configured for predicting an optimum cut of a short wood tree; a partial cut support module, configured to assist operators for partial cutting operations; and a data transfer module, configured to export data generated by the system, including maps, classified trees, cutting recommendations, to external devices or remote servers.
  • a normal cut value estimation module configured to extract information from the point cloud to monitor and assess forestry operations
  • a wood-processing head and tree cut detection module configured to determine the position of the wood-processing head in its environment and detects when a tree has been cut in real time
  • a long log volume estimation module configured to estimate the volume of long timber
  • the cutting recommendation further comprises at least one or more of: an indication of the preferred cutting location along the tree trunk; and a classification of the tree as suitable for long log or short wood processing.
  • the instructions are further executable by the at least one processor to distinguish, based on the tree positions from the 3D map, logs on the ground from standing trees.
  • the instructions are further executable by the at least one processor to monitor forestry operations by the forestry equipment, by linking, over time, the standing trees to the logs on the ground.
  • the instructions are further executable by the at least one processor to estimate, using one or more Al models and based on the logs on the ground, the volume of wood cut by species.
  • the at least one cutting aid module is configured to determine, using one or more Al models, a diameter, a tree species, a grade and/or wounds of the standing trees.
  • the at least one cutting aid module is adapted to measure, based on the combined the images and the point cloud, tree characteristics with a precision between 1 and 3 cm within 20 meters from the sensors.
  • the instructions are further executable by the at least one processor to measure distances between standing trees and the wood-processing head, the cutting recommendation being function of the distances measured.
  • the at least one cutting aid module is configured to predict optimal cutting locations of a standing tree based on the tree silhouette graphs, tree species, distance from the tree harvester head and commercial value for wood products, the display means presenting the cutting locations predicted to the operator.
  • the instructions are further executable by the at least one processor to identify trees to be cut to perform partial tree cutting, based on the diameters and species of the trees and partial tree cutting specifications.
  • partial tree cutting specifications include the number of trees to be removed according to their diameter and species, the track width to be respected by the forestry equipment, and the wound ratio on the trees.
  • the system comprises a data transfer module, configured to transmit high-level data to a mobile application via short-distance communication links, and to transmit more detailed or heavy data to a cloud-based application when internet communication links are available.
  • the system is provided as a kit which can be installed in the cabin of a forestry equipment.
  • a forestry equipment is provided with the system described above.
  • the forestry equipment is configured to process and/or cut trees based on the recommendations provided by the cutting aid module.
  • a method for real-time tree-cutting assistance for forestry equipment comprises one or more of the following steps : generating images and a point cloud of the environment of the forestry equipment; processing and synchronizing the captured images and the point cloud; detecting and classifying objects in the environment of the forestry equipment by analyzing the synchronized images and the point cloud, at least some of the objects of interest being identified as trees; positioning the classified trees relative to each other and to the forestry equipment and generating a 3D map of the environment comprising the detected trees, the detection and classification of objects being carried out prior to the generation of the 3D map, determining tree characteristics associated with the identified trees; providing a recommendation as to whether or not a tree should be cut; and present a user interface on an electronic display in a cabin of the forestry equipment to show the identified trees in real-time and the cutting recommendation as the forestry equipment is operated.
  • FIG. 1 is a front view of a forestry equipment provided with a tree-cutting assistance system, according to a possible implementation.
  • FIG. 2 is a side view of a forestry equipment provided with a tree-cutting assistance system, according to another possible implementation.
  • FIG. 3A is a front view of a harvester head, which can be provided on the crane of a forestry equipment, according to a possible implementation.
  • FIG. 3B is a side view of a delimber head, provided on a forestry equipment, according to a possible implementation.
  • FIG. 4 is a schematic view of possible components of a tree-cutting assistance system, according to a possible implementation.
  • FIG. 5 shows an operator's view of the interior of a cabin of a forestry equipment, provided with a tree-cutting assistance system, according to a possible implementation.
  • FIG. 6 shows another operator's view of the interior of the cabin of a forestry equipment.
  • FIG. 7 is an image of a 2D map of trees, according to a possible implementation.
  • FIG. 8 is a block diagram of tree-cutting assistance system, according to a possible implementation.
  • FIG. 9 is a block diagram of a sensor module of the tree-cutting assistance system, according to a possible implementation.
  • FIG. 10 is a block diagram of a detection module of the tree-cutting assistance system, according to a possible implementation.
  • FIG. 11 is a block diagram of a mapper module of the tree-cutting assistance system, according to a possible implementation.
  • FIG. 12 is a block diagram of a process within the mapper module of FIG. 11 , according to a possible implementation.
  • FIG. 13 is a block diagram of a tree-cutting support module, for estimating cutting values, according to a possible implementation.
  • FIG. 14A is a schematic view of a tree with the trunk and branches delineated by the tree-cutting assistance system, according to a possible implementation.
  • FIG. 14B is a schematic view of a tree trunk, segmented by the tree cutting aid system, to determine whether the tree qualifies as a long log, according to a possible implementation.
  • FIG. 14C is a schematic view of a tree trunk, segmented by the tree cutting aid system, to determine whether the tree qualifies as short log, according to a possible implementation.
  • FIG. 15 is a block diagram of a tree-cutting support module of the tree-cutting assistance system, for detecting trees which have been cut, according to a possible implementation.
  • FIG. 16 is a block diagram of a tree-cutting support module of the tree-cutting assistance system, for estimating the volume of long logs that have been cut, according to a possible implementation.
  • FIG. 17 is a block diagram of a tree-cutting support module of the tree-cutting assistance system, for determining an optimal cut for a given tree within the environment of the forestry equipment, according to a possible implementation.
  • FIG. 18 is a block diagram of a tree-cutting support module of the tree-cutting assistance system, for determining which trees need to be cut to carry out a partial cut of woodland, according to a possible implementation.
  • FIG. 19 is an exemplary graphical user interface of the partial cut support module for the operator, according to a possible implementation.
  • FIG. 20 is a block diagram of data transfer module of the tree-cutting assistance system, according to a possible implementation.
  • forestry equipment refers to machines specifically adapted for wood-processing operations, including but not limited to logging, harvesting, felling, delimbing, cutting, mulching and transporting trees.
  • Forestry equipment typically comprises a vehicle chassis supporting an operator’s cabin, means of movement such as wheels or tracks, a boom or crane assembly, and a wood-processing head supported by the crane or boom, such as a harvester head, mulching head, or delimber head.
  • These machines are configured to operate in forested environments and may be equipped with hydraulic or electromechanical systems for precise control of tree-handling operations.
  • the system is designed to assist forestry equipment operators by detecting trees in the equipment’s environment, determining relevant tree characteristics, and providing real-time cutting recommendations to optimize harvesting operations.
  • the tree characteristics refer to measurable and classifiable attributes of a tree, including but not limited to: species, diameter, total height, trunk curvature or bending, branching structure and volume estimations.
  • the real-time cutting recommendations refer to operational recommendations automatically generated by the system and presented to the operator in real-time during active machine operation. These instructions may include one more of:
  • the system is installable in the cabin of forestry equipment and is suitable for use with various types of wood-processing heads, such as harvester heads or delimber heads.
  • the system can be part of forestry equipment and integrated into it during manufacture. The system can thus be used to retrofit existing forestry equipment or can be integrated into equipment during manufacture.
  • the real-time tree-cutting assistance system comprises at least one image sensor configured to capture images, such as a digital or color camera, and at least one depth sensor configured to generate a point cloud of the environment, such as a three-dimensional (3D) scanner or a Light Detection and Ranging (LIDAR) device.
  • the system further comprises a computer including at least one memory and at least one processor.
  • the memory stores program instructions, which are executable by the processor and configured to process both the captured images and the point cloud data. These instructions enable the realtime detection and classification of objects in the environment of the forestry equipment, including trees and the harvester head.
  • microphones may be used to capture sound generated by the forestry equipment, such as the activation of saws or delimbing operations.
  • Specific sound signatures acoustic signals converted into electrical signals
  • the computer may include program instructions configured to analyze the captured sound to detect and coordinate cutting or delimbing operations. This information can then be used to update the system’s operation logs and cutting and/or wood-processing recommendations in real time.
  • the camera(s) and 3D scanner(s) of the real-time tree-cutting assistance system are mounted on a customized in-cabin bracket.
  • This bracket is adapted to the layout of the specific forestry equipment cabin, enabling optimal sensor placement while maintaining operator visibility.
  • the term “sensor” refers to detection components, such as cameras and 3D scanners in this embodiment, which are configured to capture visual and spatial data from the environment surrounding the forestry equipment. Mounting the sensors inside the cabin provides protection from external environmental factors, such as weather, wood debris, and impact damage, which may be common during forestry operations. While a bracket designed to be placed inside the cabin is preferred, other arrangements are possible, where some of the components are placed outside the cabin.
  • the bracket is designed to hold the sensors in fixed, known positions relative to one another, allowing calibration to be performed prior to installation. This pre-calibration ensures that the spatial relationship between sensors is preserved during operation and reduces the setup time required in the field.
  • the combination of at least an image camera and at least a 3D scanner, such as a LIDAR, in a unified in-cabin sensing system allows a wide range of novel capabilities.
  • forestry machinery either lacks sensing capabilities altogether or relies on a single type of sensor.
  • single-sensor configurations are often limited in spatial accuracy, object differentiation, and real-time performance, particularly under challenging environmental conditions like poor lighting or dense foliage.
  • the present system leverages the complementary strengths of both sensing modalities: the camera captures high-resolution visual information, useful for identifying colors, textures, and species-specific visual markers, while the LIDAR provides dense, three-dimensional point clouds that enable accurate spatial localization.
  • This sensor fusion enables a wide range of novel capabilities, some of which will be described in detail in the following sections. Furthermore, in certain embodiments, applying object detection to data from both modes (images and point clouds) before generating a 3D map allows for a more efficient use of processing resources, reducing processing time, which in turn enables information and recommendations to be displayed to the operator in real time.
  • a forestry equipment 100 in a tree-cutting site 10 is shown.
  • the forestry equipment 100 typically travels on the ground 20, on a forest road 30, to cut trees 50.
  • the forestry equipment 100 is in this case a vehicle 120 comprising a cabin 122, a cabin window 124, a boom or crane 130 and a forestry head 140, in this case a harvester head.
  • Harvester heads are sometimes referred to as feller-buncher head, as they can gather trees and cut them at the same time.
  • the forestry equipment travels near a forest or woodland, where standing trees 52 are candidates for felling.
  • the cut trees are typically placed along the forest road 30, as logs or cut trees 54, where other operations can be carried out, such as delimbing to remove the branches and/or cutting the trees into smaller tree segments.
  • the realtime tree-cutting assistance system is hardly visible, since it is installed in the operator’s cabin.
  • FIG.2 shows another forestry equipment 100, also provided with a harvester head 140, but this time shown from the side and where the cut trees 54 stacked along the forestry road 30 are more visible.
  • the harvesting head 140 can be seen at work, where three logs 54 have been picked up and gathered by the arms of the head and cut by the circular saw at the bottom of the head.
  • FIG. 3A shows a front view of a harvester head 140, with its arms 142 and circular saw 144.
  • the exemplary implementations described in the following paragraphs apply to forestry equipment provided with a harvester head, but the head could be of another type, and the system may be adapted accordingly.
  • the forestry equipment 100 may be fitted with a delimber head 160, and the system as described below could still work and be adapted accordingly.
  • the assistance system and method described herein can be used on forestry equipment equipped with different types of wood-processing heads or tools, such as harvesters, delimbers, mulchers or mulching heads, as examples only.
  • the system 200 comprises various detection and input components, including an image sensor 210 for generating images (e.g., a camera), one or more microphones 212 configured to pick up acoustic signals from the forestry machine, particularly noise generated by the harvester head (such as activating or stopping motors, opening or closing clamps) or other wood-processing heads, and at least one depth sensor 222, 224 for generating point clouds of the environment.
  • an image sensor 210 for generating images (e.g., a camera)
  • microphones 212 configured to pick up acoustic signals from the forestry machine, particularly noise generated by the harvester head (such as activating or stopping motors, opening or closing clamps) or other wood-processing heads
  • at least one depth sensor 222, 224 for generating point clouds of the environment.
  • the image-generating sensor may be a color camera, while the one or more depth sensors may include 3D sensors like LIDAR units, such as a low-density LIDAR 222 and a high-density LIDAR 224.
  • the system may include a first LIDAR sensor 224 with a high point density configured to generate a first point cloud, and a second LIDAR sensor 222 with a lower point density and wider field of view configured to generate a second point cloud.
  • the point cloud used by the system is formed by merging the initial first and second point-clouds produced respectively by the high-density and low-density LIDAR sensors, thereby enhancing the spatial coverage and resolution of the sensed environment. More details will be provided in the following description.
  • the system 200 also preferably includes a positioning system, such as a Global Navigation Satellite System (GNSS) or Global Positioning System (GPS) 226, and an Inertial Measurement Unit (IMU) for motion tracking.
  • the system may further include an electronic display 246, such as a display screen or a projector for presenting real-time information to the operator, for example on the interior surface of the cabin window 124.
  • the system 200 may also include a communications unit or subsystem and sensor data ports.
  • the system may also include a CAN bus port 230, which can interface directly with the vehicle’s CAN bus (as shown at reference 150 in FIG. 1) in one embodiment.
  • the system may be connected to an electronic or electric board capable of reading digital and/or analog signals from the machine, such as the activation or deactivation of control buttons and switches, or other operating indicators.
  • the communication ports may also enable wireless communications (via Wi-Fi, Bluetooth or other equivalent protocol) with devices 282 located in the vicinity of the equipment, such as a tablet or laptop accessible via a local network and/or communication ports for communicating with external networks or servers 280, via the Internet, when this type of connection is available.
  • the system also includes a computer 240, which serves as the central processing unit.
  • the computer comprises at least one processor 242, volatile and non-volatile memory 244, and one or more input/output interfaces 248 for communication with sensors, local devices, or remote systems.
  • the processor may be a central processing unit (CPU) or a graphics processing unit (GPU), depending on the computational needs.
  • the computer memory which may be partly remote, includes program code to manage the information received from the various sensors, and to detect the trees and their characteristics and issue cutting recommendations, in real time, to the operator. In possible embodiments, depending on the availability of a high-speed wireless connection, part of the processing can be carried out remotely.
  • FIGs. 5 and 6 showing an operator's view of the interior of a cabin of a forestry equipment, a possible arrangement of image sensor 210 and depth-sensors 222, 224 (in this case higher and lower resolution LIDAR sensors), mounted on an in-cabin bracket or support, is visible.
  • the computer 240 and display screen 246 are also visible.
  • the bracket 202 secures the camera and LIDARS in predetermined positions, enabling rapid installation of the system 200, and allowing at least part of the calibration to be carried out in advance, reducing the time required to install the system in the vehicle cabin.
  • the physical distances between each sensor being known, a pre-calibration can be performed based on these distances.
  • the IMU and GPS can also be secured to the bracket, such that all components are at fixed and predetermined positions for a given cabin model.
  • the real-time tree-cutting assistance system can be adapted and customized for different models of forestry equipment.
  • the system can be installed rapidly, typically in less than half an hour. This easy and rapid installation may be possible thanks to the configuration of the in-cabin bracket, which may comprise only a few attachment points on the cabin frame or wall 126, such as only five attachment points.
  • the incabin sensor bracket is configured to have the camera and LIDAR sensors positioned near the cabin window 124, without obstructing the operator's view, while being protected from the weather and debris outside the cabin.
  • the orientation of the sensors on the in-cabin bracket can be changed to better target the cutting area of the tree harvester head 140.
  • the incabin bracket includes sensor connectors for the camera and LIDARs, enabling their respective field of views to be adjusted according to the configuration of the operator's cabin.
  • the computer 240 comprising the ports and software application, may also be located in forestry equipment cabin and may be rigidly fixed therein to prevent any abrupt movement during operation of the forestry vehicle.
  • the computer 240 may be housed in a protective case. Connectors are also provided to connect the sensors to the computer for data acquisition.
  • the protective case, computer and/or sensors may also be connected to the forestry equipment for power supply.
  • the system when the operator starts the forestry equipment, the system can be configured such that it automatically starts up in the mode chosen by the operator, for the activity to be performed.
  • Data processed by the computer may be displayed on a screen or window in the cabin, for example, as in FIG. 7, to show the operator information in real-time, and/or to assist him with tasks, such as partial cutting.
  • partial cutting While clearcutting involves cutting down the majority of trees in a mature forest in a single operation, temporarily leaving the land free of tall vegetation, partial cutting consists in preserving a percentage of the forest cover, where only selected trees having specific characteristics are cut.
  • the proposed tree cutting system requires little or no maintenance by the operator, as well as limited knowledge of harvesting methods, since the system identifies the trees to be collected and may also indicate the cutting locations along the trunk.
  • the real-time tree cutting assistance system may be fully automated once the initial parameters have been set. In the event of breakage or if maintenance is required, the components of the system may be removed and replaced with another set of sensors or functional computer. As shown in Figure 7, the different tree species detected by the system can be identified with colours and/or a distinct pattern, allowing the operator to better distinguish the tree species in front of them.
  • metadata associated with the trees 52 can also be displayed, including, for example, tree diameter, curvature, whether the tree is a good candidate for felling, the order in which the trees should be felled, etc.
  • the software application may also include a 3D mapping module 3000 for positioning the trees relative to each other and to the forestry vehicle.
  • the detection and classification of objects can be carried out prior to the 3D mapping. Performing object detection on images and/or point clouds before generating the 3D map saves significant processing time and CPU/GPU resources, as objects and information that are not relevant for generating cutting recommendations are discarded before creating the 3D map, an operation which is an extremely resource intensive.
  • the software application also includes one or more tree-cutting aid modules 258, configured to determine tree characteristics associated with the identified trees and to provide a cutting recommendation as to whether or not a tree should be cut.
  • the system 200 may include a camera 210, microphones 212, 3D scanners 220, including for example a low point density LIDAR 222 and a high point density LIDAR 224, a positioning sensor (GPS) 226 and an inertial sensor (IMU) 228.
  • a camera 210 may include a camera 210, microphones 212, 3D scanners 220, including for example a low point density LIDAR 222 and a high point density LIDAR 224, a positioning sensor (GPS) 226 and an inertial sensor (IMU) 228.
  • GPS positioning sensor
  • IMU inertial sensor
  • the different sensors 202 are in communications with the computer 242, which comprises a software application 250 including the above-mentioned modules.
  • the software application includes a sensor module 1000, referred to as Module 1, a detection module 2000, referred to as Module 2 and a 3D mapping module 3000, referred to as Module 3.
  • the software application can also include a series of tree-cutting aid modules 258, which support real-time operational decisions. These may include:
  • Optimal cut prediction module 7000 Optimal cut prediction module 7000.
  • Module 8 Partial cut support module 8000.
  • the sensor, detection and mapper modules 1000, 2000, 3000 can perform common or core operations of the real-time tree-cutting aid system 200, which are needed for one or more of the tree-cutting aid modules.
  • the cuttingaid or any additional modules can be used independently or in combination, depending on the task in hand.
  • the tree-cutting support module(s) 258 are mainly configured to provide instructions to the operator based on the computed characteristics of the trees, and based on additional parameters, such as a quota to be met for a given tree species, or prices for certain tree species or log lengths.
  • the various tree-cutting aid modules 258 can be used in combination with each other, or independently, depending on the operator's needs.
  • cutting long timber may require the outputs of modules 5, 6 and the data transfer module 9000; cutting short wood may require the outputs of modules 4 and the data transfer module 9000; optimizing short wood cutting may require the use of modules 4, 5, 7 and the data transfer module 9000; and partial cutting assistance may require the output of modules 4, 5, 8 and the data transfer module 9000.
  • several processes can be derived from the use of the basic modules, via the real-time tree-cutting aid system 200.
  • data from these modules can be collected and transferred, using the data transfer module 9000, to a local computer, other than the one in the cabin, via short-distance wireless communication, or to a remote server, when an Internet connection is available.
  • the sensor/capture module 1000 is configured to communicate with and/or manage the sensors, including receiving information from the camera and 3D scanners.
  • the sensor module 1000 may also be configured to synchronize the images captured 1016 and the point clouds generated 1018 and 1020, based on a clock 1022 signal, which can be derived from the geo-positioning system 226.
  • a clock 1022 signal which can be derived from the geo-positioning system 226.
  • Using both a camera and LIDAR(s) improves the detection performance of objects of interest.
  • the combination of two types of sensors (image generating sensor and point cloud generating sensor) enables the system to simultaneously obtain the classification of objects of interest, as well as their 3D geometry, at around 10Hz.
  • Detecting objects prior to 3D mapping also enables pre-processing of the point cloud as input to 3D mapping, in particular to remove dynamic objects such as the head of the tree harvester, so as not to include them during 3D mapping.
  • Sensor module 1000 will be described in greater detail with reference to FIG. 9.
  • the sensor/capture module 1000 may comprise the data acquisition part, with the various sensors used, as well as data pre-processing.
  • the system 200 may comprise the following sensors:
  • At least one image camera 210 such as an RGB camera, preferably taking images 1016 at a minimum frequency of 10Hz;
  • At least one depth sensor such as LIDAR (Light Detection and Ranging) or laser sensor 224 with a field of view (FOV), such as 120 degrees by 25 degrees or higher, and dense point cloud scanning (such as 400,000 points per second or higher) at a frequency of at least 10 Hz.
  • the data range is preferably at least thirty meters;
  • At least one other depth sensor such as LIDAR sensor 222 with a wide field of view (such as 360 degrees by 180 degrees or higher) and a sparse point cloud scan (such as 200,000 points per second or higher) at a frequency of at least 10 Hz.
  • the data range is also preferably at least thirty meters;
  • GNSS Global Navigation Satellite System
  • IMU Inertial Measurement Unit
  • sensors such as an accelerometer, gyroscope and magnetometer 1024, preferably at a minimum frequency of 40 Hz
  • access to the CAN bus of the forestry equipment/machine used to obtain machine data such as bit representing tree saw use, as an example only.
  • access to digital and/or analog control signals from the forestry equipment can be used.
  • a microphone sensor recording sound signals 1014 of the environment of the forestry machine and of the working area, can be used to detect start and end of long wood cutting, start and end of delimbing, or start and end of short wood cutting.
  • the images 1016 obtained from the camera 210 may be in color.
  • the images 1016 may have an HD resolution (resolution of 1280 x 720 pixels) or higher.
  • the GNSS device 226 may be used as a reference for the computer clock. Data from the cameras and LIDARs may be synchronized 1026 so that they are acquired at the same time, preferably at a minimum of 10Hz.
  • the synchronized images 1028 may then be recalibrated according to an intrinsic calibration 1038 performed prior to starting up the system.
  • GNSS and inertial unit data may be pre-processed to extract the approximate position and orientation odometry of the forestry machine.
  • a filter such as a Kalman filter, may be applied to merge the forestry equipment’s position and movement data to obtain a six-degree-of-freedom ( 6 DOF) pose of the forestry equipment.
  • the pose 1052 corresponds to the position and orientation of the equipment, in three-dimension, and may be stored as a matrix.
  • the pose 1052 may be used to deskew 1040 and 1042 (or “straighten”) the point clouds.
  • the LIDAR 3D point clouds may be distorted.
  • the unskweded point clouds 1048 and 1050 may then be merged into a single point cloud 1054 for the rest of the process.
  • the sensor/capture module 1000 outputs calibrated images 1046, a deskewed or unskweded point cloud 1054 and pose odometry in six degrees of freedom, preferably at the same frequency such as 10Hz.
  • the camera, LIDAR and IMU sensors may all be positioned on the same support (or in-cabin bracket), facing the front of the forestry equipment (such as a harvester). Extrinsic calibration between the sensors may be carried out prior to operating the system 200.
  • the computer 240 has installed thereon drivers to communicate with the different sensors, including a microphone driver 1002, a camera driver 1004, a first LIDAR driver 1006, a second LIDAR driver 1008, a GPS driver 1010 and a IMU driver 1012.
  • the images 1016 for the camera, and the high and low-density point clouds 1018 and 1020 from the LIDARs are synchronized 1026, using the clock 1022 signal from the GPS device 226.
  • the synchronized images 1028 can be calibrated and the synchronized point clouds 1030 and 1032 can be deskewed 1040, 1042, and combined as one unskewed point cloud 1054.
  • the odometry data from the GPS and IMU may be filtered 1034, 1036 and may be merged 1044.
  • the filtered 1034, 1036 and merged odometry data 1044 from the GPS and the IMU is used to continuously calculate and update the pose of the forestry equipment.
  • low and high density point cloud it is meant that one of the LIDAR sensors can generate a higher density point cloud than the other.
  • the second lower-density LIDAR sensor may thus have a larger field of view than the first high density LIDAR sensor.
  • the sensor for generating images and the sensor(s)for generating the point cloud have capture frequencies of at least 10 Hz.
  • a compromise between field-of-view (FOV) and data density may be needed.
  • FOV field-of-view
  • a first LIDAR 222 with a high point density and a second LIDAR 224 with a lower density but larger field of view a greater portion of the environment in front of the tree harvester can be captured.
  • the LIDARs having a FOV closer to that of the camera the linking or association of the 3D data from the LIDARs and of the pixels from the camera(s) is simplified.
  • the sensors chosen are also small, so as not to obstruct the operator's field of view during these activities. Installing the camera and LIDAR scanners on the in-cabin bracket reduces the risk of extrinsic miscalibration between the sensors.
  • the system 200 may also comprise instructions, part of a detection module 2000, configured to detect objects, such as trees and the harvester head, from the synchronized images and point clouds.
  • object detection upstream of 3D map generation speeds up the processing of relevant information, allowing providing the operator with real-time instructions. More specifically, the detection of objects of interest prior to 3D-mapping, speeds up the extraction process concerning the position and classification of these objects of interest. As the number of points in a reconstructed 3D map is much greater, it would take several hours or days to correctly segment a 3D tree in such a high- density point cloud, if conducted in post-processing.
  • the detection module 2000 (or “Module 2”) concerns the use of Al models to detect objects of interest in the images 2002 and in the unskewed point clouds 2004.
  • a first Al (Artificial Intelligence) model 2006 may be used on the images 2002 to detect at least standing trees, cut logs on the ground, the harvester head, and/or the delimber’s clamps.
  • the images 2002 may be segmented, i.e. detection may be performed on each pixel.
  • the output of the first Al model may include the segmentation mask of the detected objects, together with their classification and percentage of confidence in the classification.
  • another Al model 2008 may be used on the unskewed point cloud 2004, this time to detect standing trees and the harvester head, as trees on the ground may be too difficult to segment.
  • the output of the second Al model may also include the segmentation mask of the objects detected on the point cloud, together with their classification and percentage of confidence in the classification.
  • a function linking pixels to points in the point cloud may be used to match them. This function can be calculated beforehand of the use of the system according to the extrinsic calibration of the different sensors used. Some points of the point cloud are filtered out depending on whether they are in the field of view of both the cameras and lidars used.
  • each point in the remaining point cloud is assigned to a pixel.
  • the matched pixel and the point will both have a label value based on the previous results given by the Al models. If the label values are identical, the label is set on the point cloud (as identified by the step “merge detection 2010” in FIG. 10.) If one label is only available coming from the pixel or the point, the label is set on the point either way. If there are no labels or the labels mismatched, no label is set on the point. This merging of the detected objects is done, and some of the detections may be later rejected if the confidence rate is too low. Then, each detected object retained may be tagged in the unskewed point cloud 2004.
  • each point having a correspondence with an object may be referenced and outliers may then be filtered 2012.
  • a final sorting may be performed between the points detected as belonging to the harvester head and the remainder of the environment 2014. This sorting may be performed to remove 'dynamic' points, in order to facilitate the 3D mapping of the environment carried out in the mapper module (“Module 3”).
  • the output of the detection module may therefore consist of a point cloud labeled with objects of interest other than the harvester head, and a harvester head point cloud 2016.
  • the system 200 also includes instructions, which may be part of a 3D mapping module, configured to generate, based on the detection of the trees by the detection module, a 3D map comprising trees and their characteristics.
  • a 3D mapping module configured to generate, based on the detection of the trees by the detection module, a 3D map comprising trees and their characteristics.
  • possible steps performed by the mapper module 3000 (“’module 3”) are presented.
  • the mapper module corresponds to the module used to map the 3D environment in real time and obtain the forestry equipment’s location.
  • the mapper module may include instructions to take as input the labeled point cloud from the detection module 2000, as well as the estimated pose of the forestry machine 3002 from the sensor module 1000.
  • the mapping module may comprise three parts: a pre-processing submodule, an Iterative Closest Point (ICP) submodule, and a post-processing submodule.
  • ICP Iterative Closest Point
  • the mapper module 3000 is configured to generate a 3D map of the forestry environment and to maintain an updated estimation of the pose of the forestry equipment in six degrees of freedom (6 DOF).
  • the process begins with two inputs: a pose 3008 and an unskewed labeled point cloud 3010 comprising points corresponding to detected trees. These inputs are used in a 3D-mapping step 3012, in which a preliminary 3D map of the forestry environment is generated. The resulting 3D map is then processed and refined of 6 DOF to adjust over time 3004, based on accumulated deviations or corrections.
  • the output of step 3004 may also be used to update the 6 DOF pose 3006.
  • the pose corrections from step 3004 are merged with odometry data from Module 1 3002.
  • the results of the 3D mapping step 3012 and the updated 6 DOF pose 3006 are used to generate a 3D map with tree features 3018.
  • This map includes spatial information and characteristics of the detected trees and is provided to one or more downstream modules, such as Module 4 (normal cutting aid), Module 5 (cut tree detection), or Module 6 (partial cutting aid) 3020.
  • the corrected 6 DOF pose output from step 3006 is used to update internal mapping data and associated information 3014.
  • the outputs of the 3D mapping process and the updated maps and pose data are then stored in ROM 3016 for persistent access.
  • the mapper module 3000 enables the system to construct and maintain a dynamic and accurate 3D map of the operating environment, while ensuring precise localization of the forestry equipment, thereby supporting real-time decision-making and guidance in subsequent processing modules.
  • the pre-processing part 1210 is performed on the input point cloud data 1204. Density may be reduced by applying uniform probability, and normal vectors are calculated. Further pre-processing can be carried out as required.
  • the ICP process may be iterative, and performed in two phases: a first phase where point matching 1220 is performed (by a matcher submodule) between the reference (reference cloud) and the reading (source cloud).
  • points in the reading that are close to those in the reference are matched.
  • Outliers may be filtered 1230, for example a distance threshold may be applied to remove pairs that are too far apart.
  • the distance between all pairs may be iteratively minimized 1240, using for example the least-squares method. The method seeks to minimize the distance between the reading point and the plane or normal vector of the reference point.
  • the result of the minimization is a rigid pose transformation matrix in six degrees of freedom. At each iteration, the match is recalculated by applying the resulting transformation to the reading (the source reference) and the minimization is recalculated.
  • the output of ICP is a rigid transformation of the point cloud registration 1246 and the reading (source cloud) corrected with the transformation.
  • a final step in the process may be post-processing, typically reading point cloud filtered in global frame 1248.
  • the resulting reading may be merged 1250 with the reference point cloud, and filters can be applied to homogenize the density of the final point cloud 1260.
  • the 3D mapping gives the forestry equipment location, but this location may deviate over time due to noise in the measurements.
  • the resulting localization may then be processed by another sub-module to correct the localization if it has deviated too much over time.
  • the comparison may be made with the prior of the sensor module, based on the global GNSS position. If the deviation is too great, a correction can be applied to the forestry equipment pose, and this correction may be propagated to the 3D mapping results 3018 to correct the resulting point clouds.
  • the correction can be made using a rigid transformation represented as one matrix including the position and the orientation.
  • the correction may also be propagated to the 2D map with features in the ROM to correct the position of the features prior to the calculation of the correction and generated after the last applied corrections.
  • the results are then saved on the computer's hard disk (ROM 3016).
  • the reference map is also partially saved in ROM 3016 during calculations.
  • the 3D map may be generated within a time window. This means that if the forestry equipment passes over the same spot twice, one hour apart, for example, the previously calculated map will not be taken into account.
  • the result of the mapper module is a temporary 3D point cloud of the forestry machine's environment and its pose in six degrees of freedom.
  • FIG. 13 a block diagram of the real-time normal cut estimation module 4000 or “Module 4” is shown.
  • This module comprises instructions to extract information from the point cloud to monitor forestry operations.
  • the estimation module takes as input the camera images from the detection module, the forestry machine pose and the 3D map 4014 from the mapper module.
  • the input data may be frozen, i.e. they may not be processed at 10Hz like the output data from the mapper module, since the processing time is slower.
  • the data is updated and subsequently reprocessed.
  • the point cloud may be segmented into two parts: one point cloud concerning the detection of logs on the ground (log point cloud) 4006 and another point cloud concerning standing trees (standing tree point cloud) 4022.
  • the corresponding point clouds are segmented from the main point cloud.
  • the log positions are then averaged 4006, and stored in a 2D map 4008 in global coordinates, to make it easier to track operations and manage the memory size of the information.
  • the main point cloud is also segmented, providing the position of each tree detected.
  • a tree-feature or tree-characteristic extraction algorithm may be implemented 4024.
  • the tree features or characteristics may include the diameter, specie, grade and number of wounds of a tree.
  • the diameter at breast height may be estimated directly from the point cloud, thanks to pre-processing that removes noise from the measurements and the calculation of the ground normal.
  • Camera images 4030 may be used by a specific specie-detection Al model 4032 to detect tree species, trunk grade (quality, e.g. whether there are any defects) or wounds I harvester head injuries may have been made during harvesting as shown at step 4026.
  • All these tree-features or characteristics may be recorded on a 2D map 4018 using the forestry equipment’s position to track collected/harvested trees.
  • wood products 4028 may be analyzed and merged with log detection information at step 4020.
  • This merging step allows associating detected logs with their corresponding processed output, and the resulting information is recorded as features on a 2D map 4012.
  • These merged features are then combined with other features previously extracted, including those derived from log position data 4008 and tree features 4018 and are used to update the 2D map with new information in global coordinates at step 4010.
  • the updated 2D map is then used to generate a consolidated 2D map with features, as shown at step 4002.
  • This consolidated 2D map reflects both harvested and standing tree data, and serves as a centralized record of processed forestry operations.
  • the final 2D map 4002 is stored in ROM 4004, allowing persistent access to enriched tree feature data and enabling downstream modules to retrieve updated cutting and mapping information as needed.
  • FIGs. 13, 14A and 15 further analysis of tree structure may be carried out in parallel on the individual point cloud of each tree in line.
  • priority may be given to trees close to the harvester head using a proximity value defined in by the harvester head and the cut tree detection module (“Module 5”).
  • In-depth analysis of the tree's shape may be performed on the tree's point cloud.
  • the trunk 56 of a tree 52 can be segmented or divided in segments, such as cylinders, where adjacent points may be assigned to a segment (such as a cylinder) with a predetermined diameter and spacing.
  • a pre-processing algorithm may be used to remove noise from the measurements.
  • an algorithm may be used to increase the number of points belonging to the tree that would not have been detected by the Al models of the detection module, by detecting the nearest neighbors to the points already labeled.
  • an algorithm may traverse the tree from bottom to top, following the main trunk to determine the tree's overall silhouette. The following paragraphs will provide a more detailed explanation of this process.
  • the initial pre-processing may include ground filtering, eliminating points located close to the terrain, and a density reduction by voxelization, in order to preserve the structure of the trunk while reducing computational load. From the filtered point cloud, normal vectors may be computed for each point and stored in memory.
  • a local cylindrical primitive detection algorithm may be applied.
  • This algorithm can use a 3D Hough transform across spatial blocks of the horizontally structured point cloud.
  • the detection may be performed stochastically, by randomly resampling different regions of the point cloud, allowing for the identification of trunks that are partially visible or heavily occluded.
  • Each detected cylinder may be parameterized by its axis, radius, and height.
  • Detected cylindrical segments may then be grouped into coherent tree structures using clustering techniques based on their spatial proximity, alignment of axes, diameter regularity, and vertical continuity.
  • a graph-based filtering step may be used to discard inconsistent or overly short segments.
  • a subsequent validation step may be performed to further refine the retained segments based on geometric criteria, such as vertically, local point density, and conicity coherence.
  • Validated trunks are ultimately modeled as a sequence of fitted cylinders, forming a skeletal representation of the trees within the scene.
  • three key attributes may be recorded: the section origin (e.g., the center of the tree trunk at a given height, T o ), the diameter at the origin (e.g., d 0 ), and the orientation vector of the cross- sectional disk (e.g., a 0 ).
  • the analysis proceeds upward with a predefined spacing between sections until the summit of the tree is reached.
  • the section origin i.e. the center of the tree trunk at a given height, for instance TO
  • the trunk diameter for the origin for instance dO
  • the trunk disk vector of the origin level for instance aO.
  • the algorithm continues in this way, with a predefined spacing between the various sections, until it reaches the summit of the tree.
  • the positions of the origin of the secondary branches 58 are stored in memory.
  • the secondary branches are analyzed. The same method may be used as for the main trunk, and the same information may be recorded, except that the analysis is performed only on the branch origin and not on the entire branch, to reduce calculation time.
  • each node represents the information on the main trunk and the branches or forks on the main stem.
  • information on the biomass present in the tree and the size of the tree can be found using allometric equations. All this information may be recorded on the same 2D map as before.
  • a sub-module can be used to link the standing tree that has been detected to the logs cut. This information may then be updated via the detection of cut logs, enabling their volume and species to be estimated.
  • the global 2D map may be updated and saved to ROM. This map is the output of Module 4. It contains all the information needed to monitor felling operations.
  • the harvester head and tree cut detection module 5000 or “Module 5” are shown. Instructions in this module can be configured to determine the position of the harvester head (or any other wood-processing head) in its environment and detect when a tree has been cut. This module may operate in parallel with Modules 1, 2 and 3, with their data continuously updated.
  • the harvester head and tree cut detection module 5000 may take as input the forestry equipment location determined by the mapper module, the given harvester head point cloud 5002 and pre-processed camera images given by the detection module, and optionally the machine's CAN bus data 230 or the electric board data and a microphone 212.
  • the ICP method may be applied to determine the position (in 6DOF) of the harvester head, based on a point cloud of the head taken upstream of the current operations, the "reading” or “source” point cloud corresponding to the point cloud from the detection module, and the forestry equipment location and the six-degree-of-freedom pose previously calculated as "prior”.
  • a tree proximity sub-module evaluates the possibility of a cut being made 5006. This calculation may be based on the distance from the harvester head to the tree. If the distance is below a certain threshold, proximity is established.
  • an Al model may use camera, machine data and sound to detect whether or not a cut has been made, as shown at step 5008.
  • an Al model may be trained to associate noises from the wood-processing head provided by the microphone’s driver 5014 with particular types of operation (long cut, short cut, delimbing) and to detect the start and end of these operations.
  • an algorithm uses the images to calculate the number of trunks cut and sends the information to the normal cut estimation module (4000) to update the cutting information on the 2D map 5012.
  • Module 5 may output information on the position of the harvester head, the indication of proximity to the harvester head to a given tree and the number of timbers cut.
  • FIG. 16 a block diagram of a module comprising instructions for estimating the volume of long logs 6000 is illustrated.
  • the Module 6 is configured to estimate the volume of long timber.
  • Long timber normally trees complete with branches, lying on the ground near a forestry road
  • the volume estimation module may work with Modules 1 , 2, 3 and 5, taking as input their continuously updated data.
  • Module 6 may include as input the 3D map of the forestry equipment environment 6006, camera images 6008, delimbing head position 6002, the overall delimber position 6026 and the CAN bus on electronic signal of the equipment 6010.
  • the head position and camera images it is possible to detect the catching of a long tree by the forestry equipment with an Al model as shown at step 6004.
  • the pickup phase of the logs may be followed by the detection of the start of delimbing process 6012.
  • this start-up can be detected based on the noise generated by the machine's clamps closing, and by the sound of the tree(s) being pushed into position, before delimbing begins.
  • the detection of the delimbering process is based on the signal coming from the electric board/CAN bus 6010 of the delimber. The signal will indicate the use of buttons or switches by the operator and the beginning and end of the delimbering process.
  • the delimbing step performed on the trunk(s) using the same Al model as that used for detecting the gathering phase.
  • the end of the delimbing operation can be detected 6014 by the noise generated by the opening of the clamps, via the image feed captured by the camera or by the electric board/CAN bus signal.
  • the delimbed trunk may be detected in the images, enabling its segmentation with an Al model of the same type as the Al model of the detection module.
  • different Al models may be used.
  • the trunk is tracked using the 3D point cloud provided by the mapper module, as shown at step 6016. This ensures that trunks are not mistaken with other trunks in the environment.
  • Trunk tracking is used to monitor the position of the trunk to be delimbed, as shown at step 6018. While the trunk is being delimbed and placed on a pile of delimbed trunks, the point cloud provides information on the volume and length of the trunk 6020. The camera data provides information on the grade of wood and the species of trunk. All this information may then be added to a global 2D map, as shown at step 6022, and this 2D map with all the features 6024 may them be stored in the ROM 6028, for transmission or display to the operator or other users.
  • FIG. 17 a block diagram of a tree cutting prediction module 7000, or Module 7, is illustrated.
  • This module comprises instructions for predicting where to perform cuts of a short wood tree, as shown at step 7006.
  • This module may require outputs from Modules 1, 2, 3, 4 and 5 to operate.
  • the tree cutting prediction module may take as input information on the tree (tree silhouette graph, species), the proximity value of the tree to the harvester head 7004 and the list of commercial values of the wood products, such as a price list or table. Other information indicating limits or special cases to be taken into account when selecting an expected value of the tree can be considered. If a tree is close to the harvester head and the tree silhouette calculation has been completed, the cutting (or bucking) optimization process may be initiated.
  • FIG. 17 Several tree-features or characteristics may be checked. Still referring to FIG. 17, but also to FIGs. 14B and 14C, the following tree features may be verified by the module: the presence of pronounced bends on the main trunk, for long wood (ref. 60 on FIG. 14B) or short wood (ref. 70 on FIG. 14C), the presence of forks, root shrinkage on the trunk, a tolerance value for long bends.
  • the segmentation of the main trunk into cylinders can be used to estimate the curvature of the main trunk at different tree-heights.
  • An optimization algorithm may start by estimating a “short angle” or "short curvature" of the tree between the different cylinders of the main trunk.
  • the trunk is segmented in two, starting from the point of greatest curvature.
  • the end and start diameters of the impacted levels are estimated by interpolation.
  • the distance chosen to cut the trunk may be calculated so that the angle of curvature (ref. 74 on FIG. 14C) for the cut segments returns below an acceptable threshold.
  • a minimum required such as dictated by a price list supplied by sawmills and which may be stored and provided by the ROM 7002. If the log is too small, it is no longer taken into account in the calculations. Otherwise, the optimization process continues by looking at the value of the "long bend angle".
  • the optimum cutting points are saved in memory (ROM 7002) and presented to the operator 7008.
  • the tree-cutting aid module is adapted to measure, using the point cloud and the position of the trees, distances between standing trees and the tree harvester head.
  • the cutting recommendations are function of the distances measured.
  • Cutting locations can be predicted for a standing tree based on the tree silhouette graph, tree species, distance from the tree harvester head and commercial value for wood products. These cutting locations can be displayed to the operator 7008 on the display screen 246, in the operator’s cabin.
  • a block diagram of a partial cut support module 8000 is shown, also referred to as “Module 8”.
  • This module is designed to assist operators with partial cutting operations.
  • the module 8000 may take as input the 2D map of the "tree features" determined by module 4, e.g. tree species and diameter, the pose of the forestry equipment, as well as the rules or regulations to be followed for the partial cut, which may all be extracted from the ROM 8010. These rules may be broken down. For example, they may include: the number of trees to be removed according to their diameter and species, the track width to be respected in the forest by the forestry equipment or machinery, and the injury (or wound) rate on the trees.
  • the partial cut assistance process may be divided into three parts.
  • a module may calculate an inter-tree distance 8002 graph from the 2D map, which includes a real-time inventory of standing trees.
  • the inter-tree distance graph may be completed by adding tree diameter and species values 8004 for each standing tree.
  • an algorithm may determine which trees need to be cut in order to comply with the partial cut regulations, as shown at step 8006.
  • the algorithm may be executed iteratively, taking as input the inter-tree distance graph, as well as the position (or pose) of the forestry equipment. As the forestry equipment progresses through the forest to cut the trees, the 2D map is updated at step 8008 using the information collected by Module 5, such as standing tree positions and tree characteristics.
  • Trees in the graph in the vicinity of the forestry equipment may be taken into account according to the maximum distance of the harvester head for cutting according to a threshold (e.g. 10 m).
  • the 2D map may be transformed into a graph which is continuously updated as the forestry equipment moves forward in the forest. In this graph, only the values of trees present below a threshold distance from the forestry equipment are taken into account.
  • For each node i.e. for each group of trees evaluated in relation to a given tree, the distance to the nearest neighbor tree is calculated.
  • the diameter value of the given tree is taken as a node weight.
  • the algorithm may then search for the minimum diameters of trees within a radius defined in advance by the partial cutting regulations or rules.
  • all trees may be labeled with an “unlocked” state.
  • the algorithm starts from one of the existing nodes and then searches for a local minimum from its nearest neighbors within the maximum radius. If no neighboring tree has a smaller diameter, the selected tree is marked as to-cut and all others within the maximum radius are labeled with a “locked” state. The “locked trees” are no longer taken into account for the remainder of the process, as they are not to be cut. The process then starts again from another unlocked node. If another tree with a smaller diameter is detected within the maximum radius, calculations start again from this one. The algorithm continues in this way until all trees have been cut or are in the “locked” state.
  • the algorithm detects the position of the forestry equipment as well as the position of the harvester head, and informs the operator via the display screen, as to whether the tree is to be cut or not. Colors or other distinctive markings may be used when displaying trees to the operators on the display screen to distinguish trees to be cut from other trees.
  • the system may evaluate spatial constraints and health-related metrics for visualization.
  • road space estimation is performed to assess whether the available space between trees satisfies the required width for safe machinery movement, taking into account the forestry equipment and its environment.
  • wound rate estimation is carried out, where the likelihood of damaging surrounding trees during harvesting operations is calculated, based on observed wound patterns or proximity data.
  • the estimated road width and the wound rate values are respectively transmitted to display modules 8016 and 8020, which present this information to the operator in real-time, and are also used to update the features on the 2D map 8008.
  • FIG. 19 an example of a graphical user interface 269 is shown.
  • the forestry machine may be displayed with an icon 100, such as an arrow or triangle, to show its position and orientation compared to the trees 50 around it. Circles of different diameters, such as 5m and 10m in this case, are represented around the forestry machine to display the possible range of the boom for the operator.
  • the history of the harvester's trajectory may be represented, in this case by a dark solid line, and the future trajectory to be followed to comply with the regulations is represented, in this case by a dotted line.
  • Trees are represented by distinctive icons, such as colored dots. Black dots represent trees which were cut.
  • white dots represent trees which are in the ‘locked’ state and shouldn’t be cut.
  • Striped dots represent trees which should be cut by the operator, and grid-patterned dots represent undetermined trees for which the algorithm didn’t calculated a solution yet.
  • Tracking indicators may be displayed in real time on the screen for the operator as shown in FIG 19. These indicators may include the width of the tracks or path in the forest, as well as their curvature. These values may be compared with track parameters defined by the partial cut rules or regulations.
  • the curvature of the forestry equipment’s trajectory may be given by the trajectory analysis provided by Module 3.
  • the width of the path may be calculated in relation to boundary trees that have not been cut.
  • the rate of injury (or wounds) to the trees may also be displayed to the operator using data provided by Module 4. Once the trees have been cut, the 2D map can be updated with the new information.
  • the inter-tree distance and tree diameter graphs may be reset to zero.
  • the process may be restarted for each track or path traversed by the forestry equipment.
  • the information is made available to the end users, to monitor the operation and validate that the indicators (or parameters) of the partial cut rules have been respected.
  • Information determined by the system in connection with the progression of forestry equipment along cutting paths can be saved in memory (ROM) for future reference.
  • FIG. 20 a block diagram of a tree-cutting data transfer module 9000 is shown.
  • different types of information can be transferred to online servers for monitoring logging operations.
  • data can be transferred in two stages using a mobile of software application 250.
  • a mobile application may automatically connect, via BluetoothTM or Wi-FiTM connection, to the computer’s system, to collect high-level tree cutting data (2D maps and information on trees detected during operations, information on cuts made, machine productivity data), which does not require a broadband connection.
  • the complete tree cutting data may be transmitted to online servers, as shown at step 9002. If a connection can be made in the forest, the complete tree-cutting data may be sent to the online servers 280.
  • the mobile application can be used by several people at the same time to monitor tree-cutting operations.
  • the tree-cutting aid module may thus be configured and adapted to measure, based on the combined the images and the point cloud, tree characteristics with precision between 1 and 3 cm, within 20 meters from the sensors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Environmental Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Business, Economics & Management (AREA)
  • Ecology (AREA)
  • Forests & Forestry (AREA)
  • Agronomy & Crop Science (AREA)
  • Human Resources & Organizations (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Animal Husbandry (AREA)
  • Marine Sciences & Fisheries (AREA)
  • Mining & Mineral Resources (AREA)
  • Evolutionary Computation (AREA)
  • Economics (AREA)
  • Vascular Medicine (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé et un système d'aide à la coupe d'arbre en temps réel pour un équipement forestier. Des images et des nuages de points de l'environnement de l'équipement forestier sont générés. Les images et les nuages de points sont traités et synchronisés. Des objets, tels que des arbres et la tête de traitement du bois, sont détectés et classés par analyse des images et du nuage de points. La détection et la classification d'objets sont effectuées avant la cartographie 3D de l'environnement de l'équipement forestier. Des caractéristiques d'arbre sont déterminées pour les arbres identifiés et des recommandations de coupe sont fournies pour savoir si un arbre doit être coupé ou non, et comment. Les recommandations de coupe sont présentées à l'opérateur de l'équipement forestier en temps réel, lorsque l'équipement forestier est actionné.
PCT/CA2025/051001 2024-07-24 2025-07-23 Système d'aide à la coupe d'arbre en temps réel pour équipement forestier Pending WO2026020242A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202463674865P 2024-07-24 2024-07-24
US63/674,865 2024-07-24

Publications (1)

Publication Number Publication Date
WO2026020242A1 true WO2026020242A1 (fr) 2026-01-29

Family

ID=98564638

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2025/051001 Pending WO2026020242A1 (fr) 2024-07-24 2025-07-23 Système d'aide à la coupe d'arbre en temps réel pour équipement forestier

Country Status (1)

Country Link
WO (1) WO2026020242A1 (fr)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090095375A1 (en) * 2007-10-15 2009-04-16 Elliot Little Tree counter for a saw head in a tree feller
US20130235205A1 (en) * 2010-11-24 2013-09-12 Minna Lappalainen Method for monitoring wood harvesting, and a system
US20140238545A1 (en) * 2013-02-28 2014-08-28 Waratah Nz Limited System, device, and method for processing a length of material
US20170089797A1 (en) * 2015-09-30 2017-03-30 Deere & Company Stability warning and control intervention system for a forestry vehicle
CN115376096A (zh) * 2022-07-04 2022-11-22 北京拓疆者智能科技有限公司 一种工程机械的行驶区域生成方法、装置、存储介质及终端
US20230126631A1 (en) * 2015-03-25 2023-04-27 Waymo Llc Vehicle with Multiple Light Detection and Ranging devices (LIDARs)
US20230205231A1 (en) * 2020-05-27 2023-06-29 Airforestry Ab Method and system for remote or autonomous ligno harvesting and/or transportation
WO2024134033A1 (fr) * 2022-12-23 2024-06-27 Ponsse Oyj Procédé dans une machine forestière et système

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090095375A1 (en) * 2007-10-15 2009-04-16 Elliot Little Tree counter for a saw head in a tree feller
US20130235205A1 (en) * 2010-11-24 2013-09-12 Minna Lappalainen Method for monitoring wood harvesting, and a system
US20140238545A1 (en) * 2013-02-28 2014-08-28 Waratah Nz Limited System, device, and method for processing a length of material
US20230126631A1 (en) * 2015-03-25 2023-04-27 Waymo Llc Vehicle with Multiple Light Detection and Ranging devices (LIDARs)
US20170089797A1 (en) * 2015-09-30 2017-03-30 Deere & Company Stability warning and control intervention system for a forestry vehicle
US20230205231A1 (en) * 2020-05-27 2023-06-29 Airforestry Ab Method and system for remote or autonomous ligno harvesting and/or transportation
CN115376096A (zh) * 2022-07-04 2022-11-22 北京拓疆者智能科技有限公司 一种工程机械的行驶区域生成方法、装置、存储介质及终端
WO2024134033A1 (fr) * 2022-12-23 2024-06-27 Ponsse Oyj Procédé dans une machine forestière et système

Similar Documents

Publication Publication Date Title
US10278333B2 (en) Pruning robot system
KR101948728B1 (ko) 데이터 수집 방법 및 시스템
EP3158409B1 (fr) Visualisation et mappage de jardin par l'intermédiaire de véhicule robotique
US8275547B2 (en) Method and system for locating a stem of a target tree
US20090185741A1 (en) Apparatus and method for automatic airborne LiDAR data processing and mapping using data obtained thereby
WO2016103067A1 (fr) Cartographie et planification d'un jardin par l'intermédiaire d'un véhicule robotisé
KR20140009737A (ko) 하이브리드 맵 기반 로봇의 위치인식방법
EP3761136B1 (fr) Dispositif de commande, corps mobile et programme
WO2022121460A1 (fr) Chariot élévateur intelligent agv, et procédé et appareil pour détecter l'état de plateforme de zones d'inventaire d'empilements au sol
CN115599119A (zh) 一种无人机的避障系统
Rossmann et al. Realization of a highly accurate mobile robot system for multi purpose precision forestry applications
JP2020188735A (ja) 玉葱収穫用作業機及び玉葱収穫作業方法
US12592075B2 (en) Remote sensing for intelligent vegetation trim prediction
WO2026020242A1 (fr) Système d'aide à la coupe d'arbre en temps réel pour équipement forestier
JP2004012233A (ja) 農作物検出装置および農作物検出方法
JP4102885B2 (ja) 駐車車両検知方法及び駐車車両検知システム
CN121300167A (zh) 一种玉米收获机自动对行系统及方法
CN117608285A (zh) 割草机器人的脱困方法及装置、设备、存储介质
CN109709554A (zh) 作业设备及其控制方法和装置
Grote et al. Road extraction in suburban areas by region-based road subgraph extraction and evaluation
CN121325888B (zh) 一种辣椒智能收获机的自动避障方法、系统、设备及介质
CN112297011A (zh) 农林机器人避障方法、装置、计算机设备和存储介质
CN120558235B (zh) 果园行内导航方法和系统
US20260068805A1 (en) Methods of detecting anomalies in agricultural fields, and related agricultural vehicles
CN113932851B (zh) 基于视觉的扶梯识别处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25843142

Country of ref document: EP

Kind code of ref document: A1