WO2019019819A1 - Dispositif électronique mobile et procédé de traitement de tâches dans une région de tâche - Google Patents

Dispositif électronique mobile et procédé de traitement de tâches dans une région de tâche Download PDF

Info

Publication number
WO2019019819A1
WO2019019819A1 PCT/CN2018/090579 CN2018090579W WO2019019819A1 WO 2019019819 A1 WO2019019819 A1 WO 2019019819A1 CN 2018090579 W CN2018090579 W CN 2018090579W WO 2019019819 A1 WO2019019819 A1 WO 2019019819A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
mobile electronic
wireless signal
camera
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/090579
Other languages
English (en)
Chinese (zh)
Inventor
潘景良
陈灼
李腾
陈嘉宏
高鲁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vestorch Technology Ltd
Original Assignee
Vestorch Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vestorch Technology Ltd filed Critical Vestorch Technology Ltd
Publication of WO2019019819A1 publication Critical patent/WO2019019819A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0022—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement characterised by the communication link
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means

Definitions

  • the present invention relates to the field of electronic devices.
  • the invention relates to the field of intelligent robot systems.
  • the traditional sweeping robot randomly moves according to the scanned map autonomously positioned and moved or collided, and sweeps the ground at the same time. Therefore, the traditional sweeping robot cannot fully judge the complex situation of the ground during the work process because of the immature or inaccurate drawing and positioning technology, and it is easy to lose the position and direction.
  • some models can only change direction by the physical principle of collision rebound because they do not have the positioning ability, and even cause damage to the household goods or the robot itself or even personal injury, causing interference to the user.
  • the present invention proposes a technique in which a user can use a mobile handset terminal APP to delineate a target operation area (cleaning) and send an instruction to the robot to automatically complete the delineated area automatic operation (cleaning).
  • a target operation area cleaning
  • an instruction to the robot to automatically complete the delineated area automatic operation (cleaning).
  • three ways to establish an indoor environment map are proposed.
  • a path planning algorithm that accurately reaches the circled area of the user's mobile phone terminal APP and effectively covers the circled area to complete cleaning is also implemented.
  • One embodiment of the present invention discloses a mobile electronic device for processing a task of a task area, including a first wireless signal transceiver, an image processor, a positioning module, a path planning module, and a motion module, wherein: the first The wireless signal transceiver is communicably coupled to the second mobile electronic device, configured to acquire a photo taken by the user of the second mobile electronic device to the mission site and a selected area on the photo; the image processor Communicatingly coupled to the first wireless signal transceiver, configured to extract feature information of a photo containing the selected area, and determine by comparing the extracted feature information with the stored feature information of the image map including the location information An actual coordinate range corresponding to the selected area in the photo; the positioning module is communicably coupled to the image processor, configured to record a current location of the mobile electronic device and to select a range of distances between corresponding actual coordinate ranges of the fixed area; the path planning module is communicably coupled to the map An image processor, configured to generate a path planning scheme according to an actual coordinate range
  • Another embodiment of the present invention discloses a method for processing a task of a task area in a mobile electronic device, the mobile electronic device including a first wireless signal transceiver, an image processor, a positioning module, a path planning module, and a motion module, the method comprising: obtaining, by the first wireless signal transceiver communicably connected to the second mobile electronic device, a photo taken by a user of the second mobile electronic device to a mission site and in the a selected area on the photo; extracting feature information of the photo containing the selected area by the image processor communicably coupled to the first wireless signal transceiver, and comparing the extracted feature information and the stored Characteristic information of an image map containing location information, determining an actual coordinate range corresponding to the selected region in the photo; recording the movement by the positioning module communicably coupled to the image processor a range of distances between the current location of the electronic device and the actual coordinate range corresponding to the selected region; Communicatingly coupled to the path planning module of the image processor, generating a path planning scheme based
  • FIG. 1 shows a schematic diagram of a system in which a mobile electronic device is located, in accordance with one embodiment of the present invention.
  • FIGS. 2A and 2B respectively illustrate a task area photographed by a second camera of the second mobile electronic device and a delineation of the task area by the second mobile electronic device, in accordance with one embodiment of the present invention.
  • FIG. 3 shows a schematic diagram of a system in which a mobile electronic device and a second mobile electronic device are located, in accordance with one embodiment of the present invention.
  • FIG. 4 shows a flow chart of a method in a mobile electronic device in accordance with one embodiment of the present invention.
  • FIG. 5 is a diagram showing a checkerboard diagram of a black and white rectangle displayed on a display screen on the mobile electronic device 100.
  • FIG. 1 shows a schematic diagram of a system in which a mobile electronic device is located, in accordance with one embodiment of the present invention.
  • the mobile electronic device 100 includes, but is not limited to, a cleaning robot, an industrial automation robot, a service robot, a disaster relief robot, an underwater robot, a space robot, a drone, and the like. It can be understood that the mobile electronic device 100 can also be referred to as the first mobile electronic device 100 in order to distinguish it from the following second mobile electronic device 140.
  • the second mobile electronic device 140 includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a remote controller, and the like.
  • the mobile electronic device optionally includes an operator interface.
  • the second mobile electronic device is a mobile phone, and the operation interface is a mobile phone APP.
  • the signal transmission manner between the mobile electronic device 100 and the charging station 160 includes but is not limited to: Bluetooth, WIFI, ZigBee, infrared, ultrasonic, ultra-wide bandwidth (UWB), etc., in this embodiment, signal transmission It is WIFI as an example for description.
  • the mission area represents the venue where the mobile electronic device 100 performs the task. For example, when the task of the mobile electronic device 100 is a cleaning robot, the task area indicates an area that the cleaning robot needs to clean. For another example, when the task of the mobile electronic device 100 is to vent a disaster relief robot, the task area indicates the area where the disaster relief robot needs to be rescued.
  • a mission site represents a venue that contains the entire mission area.
  • the mobile electronic device for processing tasks of the task area includes a first wireless signal transceiver 102, an image processor 104, a positioning module 106, a path planning module 108, and a motion module 110.
  • the first wireless signal transceiver 102 can be communicatively coupled to the second mobile electronic device 140, configured to acquire a photo taken by the user of the second mobile electronic device 140 to the mission location and a selected area on the photo.
  • FIGS. 2A and 2B respectively illustrate a task area captured by a second camera 144 of the second mobile electronic device 140, and a user of the second mobile electronic device 140 delineating the selected area, in accordance with an embodiment of the present invention. .
  • the second mobile electronic device 140 is used as a mobile phone, and the task area is a cleaning area as an example.
  • the user of the second mobile electronic device 140 uses the second camera 144 on the second mobile electronic device 140 to take a photo of the position to be cleaned by using the mobile phone APP (as shown in FIG. 2A). Show) and circle the target cleaning area in the photo (as shown in Figure 2B).
  • the photo (including the delineated target cleaning area) is transmitted to the mobile electronic device 100 via a local wireless communication network (WIFI, etc.) and stored in the memory 116.
  • WIFI local wireless communication network
  • the image processor 104 is communicably coupled to the first wireless signal transceiver 102, configured to extract feature information of a photo containing the selected region, and by comparing the extracted feature information with the stored feature information of the image map including the location information, Determine the actual coordinate range that corresponds to the selected area in the photo.
  • the location information refers to the location information of the image feature points in the image map during the process of establishing the map, that is, the actual coordinate position.
  • the location information includes, for example, the location of the charging post 180 and/or the location of the mobile electronic device 100 itself.
  • the image processor 104 can use the position of the charging post 180 as a coordinate origin.
  • the memory 116 of the mobile electronic device 100 stores an image map established during the first use of the indoor environment map, such as indoor image map information, including image feature points and their location information.
  • the image processor 104 extracts feature information and position information in the captured photos, and further utilizes an image feature point matching algorithm (such as SIFT, SURF, etc.) and an indoor image map (including position information) in the memory 116 to perform fast. Comparison analysis.
  • the image processor 104 in the mobile electronic device 100 determines the photo in the photo by comparing the image feature points in the indoor image map. The coordinate range of the indoor actual area corresponding to the user selected area of the mobile electronic device 140.
  • the indoor actual area range corresponding to the user selected area is determined as follows: For example, the image processor 104 may appropriately increase the corresponding area on the basis of the user's original selected area, such as the circled area indicated by the finger pattern in FIG. 2B.
  • the percentage range for example, is increased by a 10% range to ensure that the selected area indicated by the finger pattern is within the actual cleaning range to determine the actual coordinate range.
  • the image processor 104 may offset the original area outward by a certain distance to determine the actual coordinate range.
  • image processor 104 may blur the construction of a standard graphic that includes the actual coordinate range.
  • the selected range indicated by the finger pattern in the figure is an irregular approximate rectangular shape
  • the image processor 104 can convert the approximate rectangle into an actual coordinate range corresponding to the rectangle, thereby facilitating the cleaning of the mobile electronic device and completing the cleaning. task.
  • the image feature points may be identified by a Scale Invariant Feature Transform (SIFT) algorithm or a Speeded Up Robust Features (SURF) algorithm.
  • SIFT Scale Invariant Feature Transform
  • SURF Speeded Up Robust Features
  • the image processor 1040 first identifies the key points of the object of the reference image stored in the memory 110, extracts the SIFT features, and then compares the SIFT features of the respective key points in the memory 110 with the SIFT features of the newly acquired image, and then based on the K nearest neighbor
  • the matching feature of the algorithm K-Nearest Neighbor KNN
  • the SURF algorithm is based on an approximate 2D Haar wavelet response and uses an integral image for image convolution using a Hessian matrix-based measure for the detector. And use a distribution-based descriptor.
  • determining the coordinate range of the indoor real area corresponding to the user selected area of the second mobile electronic device 140 in the photo may determine the actual coordinate range of the task area by coordinate mapping conversion.
  • the feature points in the image in the second mobile electronic device 140 will match the image feature points in the image map, ie the actual coordinate position of the feature points in the image in the second mobile electronic device 140 can be determined.
  • the coordinate system conversion relationship of the camera coordinate system where the image captured by the user camera is located with respect to the actual world coordinate system where the charging pile is located can be calculated.
  • the bounding area boundary line in the image can be discretized into a boundary line composed of points.
  • the positional information of the discretized points on the boundary line in the image relative to the image feature points, the actual coordinate position of the image feature points, and the coordinate system conversion relationship can be used to calculate the discretization points on the boundary line in the actual world coordinate system (ie, the charging pile)
  • the actual coordinate position in the coordinate system that is, the coordinate range of the actual indoor area corresponding to the boundary line.
  • the location module 106 is communicably coupled to the image processor 104 and is configured to record a range of distances between the current location of the mobile electronic device 100 and the actual coordinate range corresponding to the selected region. For example, the positioning module 106 sets the location of the charging post 180 as the coordinate origin, and each point in the image corresponds to a coordinate value (X, Y). The positioning module 106 and the encoder cause the mobile electronic device 100 to know its current location.
  • the positioning module 106 is a module that calculates the position of the first electronic device 100 in the room. The first electronic device 100 always needs to know its indoor location at all times during operation, and is implemented by the positioning module 106.
  • the path planning module 108 is communicably coupled to the image processor 104 and configured to generate a path planning scheme based on actual coordinate ranges corresponding to the selected regions.
  • the path planning module 108 is further configured to perform path planning on the selected area by using a grid-based spanning tree path planning algorithm.
  • the path planning module 108 optimizes the cleaning path for the coordinate range according to the generated corresponding region coordinate range (the user-defined target cleaning region).
  • Grid-based Spanning Tree Path Planning is used to implement cleaning path planning for selected target cleaning areas. The method uses gridding processing for the corresponding coordinate region, establishes a tree node for the mesh and generates a tree, and then uses a Hamiltonian path surrounding the spanning tree as an optimized cleaning path for cleaning the region.
  • the mobile electronic device 100 is located at the smart charging station 180.
  • the path planning module 108 will read the path that the mobile electronic device 100 follows to reach the region when first used (if the mobile electronic device 100 adopts the following mode) Or adopting the walking path in the user mapping process of the second mobile electronic device 140 as the path to the area (if the first time the mobile electronic device 100 does not follow the user), and optimizing the path and the selected area
  • the sweep path synthesizes the sweep task path.
  • the synthesis can connect the two paths in a simple sequence, the first path realizes reaching the target cleaning area, and the second path realizes optimal coverage of the circled cleaning area to complete the cleaning task.
  • the motion module 110 can be communicatively coupled to the path planning module 108, configured to perform motion in accordance with a path planning scheme.
  • the mobile electronic device 100 (for example, a robot) includes a camera, and the user of the second mobile electronic device 140 wears a positioning receiver.
  • the mobile electronic device 100 further includes a first camera 112, wherein the second mobile electronic device 140 further includes a second wireless signal transceiver 142 that is configured to operate in a map-building mode.
  • First wireless signal transceiver 102 and second wireless signal transceiver 142 are communicably coupled to a plurality of reference wireless signal sources, respectively, configured to determine mobile electronic device 100 and based on signal strengths obtained from a plurality of reference wireless signal sources The location of the mobile electronic device 140.
  • the signals received from the reference wireless signal source can be converted to distance information by any method known in the art including, but not limited to, Time of Flight (ToF), Angle of Arrival (Angle) Of Arrival, AoA), Time Difference of Arrival (TDOA) and Received Signal Strengh (RSS).
  • TOF Time of Flight
  • Angle Angle of Arrival
  • AoA Angle of Arrival
  • TDOA Time Difference of Arrival
  • RSS Received Signal Strengh
  • the motion module 110 is configured to follow the motion of the second mobile electronic device 140 in accordance with the location of the mobile electronic device 100 and the second mobile electronic device 140.
  • mobile electronic device 100 includes a monocular camera 112, a user of second mobile electronic device 140 wears a wireless positioning receiver wristband, or a user handheld mobile phone equipped with a wireless positioning receiver peripheral.
  • the use of the monocular camera 112 can reduce hardware cost and computational cost, and the use of a monocular camera achieves the same effect as using a depth camera. Image depth information may not be needed.
  • the distance depth information is sensed by the ultrasonic sensor and the laser sensor.
  • a monocular camera is taken as an example for description.
  • the mobile electronic device 100 follows the user through its own wireless location receiver. For example, for the first time, the user of the second mobile electronic device 140 realizes the interaction with the mobile electronic device 100 through the mobile phone APP to complete the indoor establishment of the map.
  • the wireless signal transmitting group in a fixed position placed indoors as a reference point, for example, UWB, the mobile APP of the second mobile electronic device 140 and the wireless signal module in the mobile electronic device 100 read the signal strength (RSS) for each signal source.
  • RSS signal strength
  • the location of the user of the second mobile electronic device 140 and the mobile electronic device 100 within the room is determined.
  • the motion module 110 of the mobile electronic device 100 completes user tracking according to real-time location information (mobile phone and robot location) transmitted by the smart charging station.
  • the first camera 112 is configured to capture a plurality of images when the motion module 110 is in motion, the plurality of images including feature information and corresponding photographing position information.
  • the follow-up process is completed by the robot's monocular camera.
  • the mobile electronic device 100 captures the entire indoor layout by using the first camera 112, such as a monocular camera, and takes the captured image with a large number of features and corresponding shooting position information and the mobile electronic device 100.
  • the memory 116 is transmitted to the memory 116 in real time via a local wireless communication network (WIFI, Bluetooth, ZigBee, etc.).
  • WIFI local wireless communication network
  • Bluetooth Bluetooth
  • ZigBee ZigBee
  • FIG. 1 memory 116 is shown as being included in mobile electronic device 100.
  • the memory 116 may also be included in the smart charging station 180, ie, the cloud.
  • the image processing module 104 is communicably coupled to the first camera 112 and configured to generate feature maps by extracting the plurality of images, extracting feature information and shooting location point information of the plurality of images, and generating an image map.
  • the image processing module 104 performs map stitching creation on a large number of images captured by the first camera 112 via the image processor 104 according to the height and internal and external parameters of the first camera 112 of the mobile electronic device 100, and feature selection extraction (eg, SIFT, The SURF algorithm or the like adds feature point position information to generate indoor image map information (including a large number of image feature points), and stores the processed image map information in the memory 116.
  • feature selection extraction eg, SIFT, The SURF algorithm or the like adds feature point position information to generate indoor image map information (including a large number of image feature points), and stores the processed image map information in the memory 116.
  • the internal parameters of the camera refer to parameters related to the camera's own characteristics, such as the camera's lens focal length, pixel size, etc.; the camera's external parameters are parameters in the world coordinate system (the actual coordinate system in the charging pile room), such as the camera's Position, direction of rotation, angle, etc.
  • the photos taken by the camera have their own camera coordinate system, so the internal and external parameters of the camera are required to realize the conversion of the coordinate system.
  • the mobile electronic device 100 (robot) includes a camera and the displayable camera corrects the black and white checkerboard, and the user of the second mobile electronic device 140 does not need to wear the positioning receiver.
  • the mobile electronic device 100 further includes a display screen 118, the mobile electronic device 100 is configured to operate in a map-building mode, and the second mobile electronic device 140 includes a second camera 144, the first wireless signal
  • the transceiver 142 is communicably coupled to a plurality of reference wireless signal sources configured to determine a location of the mobile electronic device 100 based on signal strengths obtained from the plurality of reference wireless signal sources.
  • the first camera 112 is configured to detect the location of the second mobile electronic device 140.
  • the mobile electronic device 100 further includes an ultrasonic sensor and a laser sensor, and the distance between the mobile electronic device 100 and the second mobile electronic device 140 can be detected.
  • the motion module 110 is configured to follow the motion of the second mobile electronic device 140 in accordance with the location of the mobile electronic device 100 and the second mobile electronic device 140.
  • the user of the second mobile electronic device 140 implements user interaction with the mobile electronic device 100 through the mobile phone APP to complete the indoor establishment of the map.
  • the first wireless signal transceiver 102 in the mobile electronic device 100 reads the signal strength (RSS) for each signal source to determine the mobile electronic device by using a wireless signal transmitting group (UWB or the like) of a fixed position placed indoors as a reference point. 100 indoors location.
  • RSS signal strength
  • Target positioning and following of the user of the second mobile electronic device 100 is achieved by the first camera 112 of the mobile electronic device 100, such as a monocular camera, an ultrasonic sensor, and a laser sensor 114.
  • the user of the second mobile electronic device 140 can set the following distance through the mobile phone APP, so that the mobile electronic device 100 adjusts and the second mobile electronic according to the following distance and the angle between the second mobile electronic device 140 measured in real time. The distance and angle between the devices 140.
  • the mobile electronic device 100 transmits the following path coordinates to the smart charging post 180 in real time.
  • display 118 of mobile electronic device 100 is configured to display, for example, a black and white checkerboard.
  • the image processor 104 is communicably coupled to the second camera 144 and is configured to receive a plurality of images taken from the second camera 144 as the motion module 110 moves.
  • image processor 104 may receive a plurality of images taken from second camera 144 via first wireless signal transceiver 102 and second wireless signal transceiver 142.
  • the plurality of images includes an image of the display 118 of the mobile electronic device 100 that is displayed as a black and white checkerboard.
  • the image processor 104 is further configured to generate an image map by splicing a plurality of images, extracting feature information in the plurality of images, and capturing position point information.
  • the calibration picture is a checkerboard composed of black and white rectangles, as shown in Figure 5.
  • the mobile electronic device 100 ie, the robot, includes a first camera 112, such as a monocular camera, and a display 118 that can display a black and white camera to correct the board.
  • the user does not need to wear the wireless positioning receiver bracelet, and the user does not need to hold the mobile phone equipped with the wireless positioning receiver peripheral.
  • the mobile electronic device 100 follows the user visually, and the user of the second mobile electronic device 140 uses the mobile phone APP to complete the drawing. For example, each time a room is reached, the user of the second mobile electronic device 140 launches the room building application via the mobile phone APP, at which time the liquid crystal display 118 of the mobile electronic device 100 displays a classic black and white checkerboard for correcting the camera.
  • the mobile electronic device 100 simultaneously transmits its own coordinate and direction information to the positioning module 106.
  • the user of the second mobile electronic device 140 photographs the room environment using the mobile phone APP, and the photograph taken needs to include a black and white checkerboard in the liquid crystal display of the mobile electronic device 100.
  • the user of the second mobile electronic device 140 takes a plurality of photos according to the layout of the room (the photos all need to take a black and white checkerboard in the robot LCD screen), and the room environment and the mobile electronic device 100 that are photographed by the mobile phone APP, for example
  • the image of the robot 100 is transferred to the memory 116 via a local wireless communication network (WIFI, Bluetooth, ZigBee, etc.).
  • WIFI local wireless communication network
  • the image processor 104 performs map stitching creation on a large number of images taken by the user of the second mobile electronic device 140, feature selection extraction, The feature point position information is added, the indoor image feature point map information is generated, and the processed image map information is stored in the memory 116.
  • Mode 3 The mobile electronic device 100 (robot) does not include a camera, and the user of the second mobile electronic device 140 wears a positioning receiver.
  • the second mobile electronic device 140 further includes a second wireless signal transceiver 142 and a second camera 144.
  • a second wireless signal transceiver 142 is communicably coupled to the plurality of reference wireless signal sources, configured to determine a location of the second mobile electronic device 140 based on signal strengths obtained from the plurality of reference wireless signal sources.
  • the second camera 144 is configured to capture a plurality of images of the mission location.
  • the image processor 104 is communicably coupled to the second camera 140, and is configured to generate an image map by splicing a plurality of images, extracting feature information of the plurality of images, and capturing position point information.
  • the mobile electronic device 100 such as a robot, does not include a monocular camera and the robot does not follow the user of the second mobile electronic device 140.
  • the user of the second mobile electronic device 140 wears a wireless positioning receiver wristband, or the user holds a mobile phone equipped with a wireless positioning receiver peripheral, and uses the mobile phone APP to complete the indoor drawing.
  • the user of the second mobile electronic device 140 realizes the indoor establishment of the map through the mobile phone APP or the wireless positioning receiver wristband worn by the user or the wireless positioning receiver peripheral of the mobile phone equipment.
  • the wireless signal transceiver 142 in the second mobile electronic device 140 reads the received signal strength (RSS) for each reference wireless signal source by using a fixed position reference wireless signal source (UWB or the like) placed indoors as a reference point. The location of the user of the second mobile electronic device 140 indoors is determined. Each time a room is reached, the user of the second mobile electronic device 140 initiates a room building program via the mobile APP. The user of the second mobile electronic device 140 photographs the room environment using the mobile phone APP, for example, multiple photos can be taken according to the layout of the room.
  • RSS received signal strength
  • UWB fixed position reference wireless signal source
  • the mobile APP of the second mobile electronic device 140 will record the pose information of the second camera 144 for each shot and the second mobile electronic device 140 recorded by the second wireless signal transceiver 142, such as the height information of the mobile phone relative to the ground and its
  • the location information of the room is transmitted to the memory 116 via a local wireless communication network (WIFI, Bluetooth, ZigBee, etc.).
  • WIFI local wireless communication network
  • a large number of images captured by the image processor 104 are created by map stitching, feature selection extraction, feature point position information is added, and an indoor is generated.
  • the image feature point map information is stored in the memory 116.
  • the mobile electronic device 100 for example, the robot 100 further includes an encoder and an inertial measurement module (IMU) to assist the first camera 112 in acquiring the position and attitude of the mobile electronic device 100, such as a robot.
  • an encoder and an inertial measurement module IMU
  • both the encoder and the IMU can provide the position and attitude of the robot.
  • the encoder can be used as an odometer to record the trajectory of the robot by recording the rotation information of the robot wheel.
  • the mobile electronic device 100 may further include a sensor 114 that transmits obstacle information around the mobile electronic device 100 to the motion module 110.
  • the motion mode 110 is also configured to adjust the motion orientation of the mobile electronic device 100 to avoid obstacles. It can be understood that, because the height of the installation is different, the height of the first camera 112 mounted on the mobile electronic device 100 is different from the height of the sensor 114 mounted on the mobile electronic device 100, so the obstacle information and the sensor device captured by the first camera 112 are The obstacles taken may be different because there may be obscuration.
  • the first camera 112 can change the visual direction by means of rotation, pitch, etc. to obtain a wider visual range.
  • the senor 114 can be mounted at a relatively low horizontal position, which may be a blind spot of the first camera 112. Objects do not appear in the viewing angle of the first camera 112, so these conventional sensors 112 are relied upon to avoid obstacles.
  • camera 112 may acquire obstacle information in conjunction with ultrasound and laser sensor 114 information. The image obtained by the monocular camera 112 is used for object recognition, and the ultrasonic and laser sensors 114 are ranging.
  • the sensor 114 includes an ultrasonic sensor and/or a laser sensor.
  • the first camera 112 and the sensor 114 can assist each other. For example, if there is shielding, the mobile electronic device 100 needs to rely on its own laser sensor, ultrasonic sensor 114, etc. to avoid obstacles in the shaded portion.
  • the laser sensor and the ultrasonic sensor mounted on the mobile electronic device 100 detect static and dynamic environments around the mobile electronic device 100, and assist in avoiding static and dynamic obstacles and adjusting the optimal path.
  • the mobile electronic device 300 further includes a charging post 380, which may include an image processor 386, a path planning module 388, a memory 384, such as a memory data module, a first wireless transmitter 381 (eg, UWB) And the second wireless signal receiver 382, for example implemented by WIFI.
  • a charging post 380 may include an image processor 386, a path planning module 388, a memory 384, such as a memory data module, a first wireless transmitter 381 (eg, UWB) And the second wireless signal receiver 382, for example implemented by WIFI.
  • the body of mobile electronic device 300 may include a first wireless signal receiver 302, such as implemented by UWB, first camera 310, map positioning module 304, obstacle avoidance module 306, motion module 308, and sensors 314 and 316, and The encoder 318 and the second wireless signal receiver 320 are implemented, for example, by WIFI.
  • the second mobile electronic device 340 such as a mobile phone, also includes a mobile phone APP and a second camera.
  • at least one of the image processor 386, the path planning module 388, and the memory 384 may also be included in the body of the mobile electronic device 300. As shown in FIG.
  • the first wireless transmitter 381 in the smart charging post 380 is communicably coupled to the first wireless signal receiver 302 in the cleaning robot 300, and the second wireless signal transmitter 320 and the intelligent in the cleaning robot 300.
  • the fifth wireless signal receiver 382 in the charging post 380 is communicably coupled.
  • the path planning module 388 in the smart charging station 380 is communicatively coupled to the motion module 308 in the cleaning robot 300.
  • the handset 340 of the second electronic mobile device is communicably coupled to the second wireless signal receiver 382 in the smart charging station 380.
  • Path planning module 388 sends the generated path to motion module 308 for execution.
  • the first wireless signal receiver 302 is communicably coupled to the map location module 304.
  • the map location module 304 communication is in turn communicatively coupled to the second wireless signal transmitter 320 and the motion module 308.
  • the first camera 310 is communicably coupled to the second wireless signal transmitter 320.
  • Ultrasonic sensor 314, laser sensor 316, and encoder 318 are communicatively coupled to obstacle avoidance module 306.
  • the obstacle avoidance module 306 is communicatively coupled to the motion module 308. There is also information interaction between the positioning module 304 and the motion module 308.
  • the motion module 308 needs to locate the location information input by the module 304 when performing the planning path.
  • a second wireless signal receiver 382 is communicably coupled to the memory 384 for communication.
  • Memory 384 is communicably coupled to image processor 386.
  • Image processor 386 is communicatively coupled to path planning module 388.
  • FIG. 4 shows a flow diagram of a method 400 in a mobile electronic device in accordance with one embodiment of the present invention.
  • the mobile electronic device includes a first wireless signal transceiver, an image processor, a positioning module, a path planning module, and a motion module.
  • the method 400 includes, in block 410, acquiring a photo taken by a user of the second mobile electronic device on a mission location and a selected area on the photo by communicably connecting to the second mobile electronic device first wireless signal transceiver;
  • feature information of the photo containing the selected region is extracted by an image processor communicably coupled to the first wireless signal transceiver, and by comparing the extracted feature information with the stored image map containing the location information Feature information determining an actual coordinate range corresponding to a selected area in the photo; in block 430, recording the current location of the mobile electronic device and the selected area by a positioning module communicably coupled to the image processor a range of distances between corresponding actual coordinate ranges; in block 440, a path planning scheme is generated by a path planning module communicatively coupled to the image processor, based on an actual coordinate range corresponding to the selected region; The motion is performed according to the path planning scheme by a motion module communicably connected to the path planning module.
  • the mobile electronic device further comprises a first camera, the second mobile electronic device further comprising a second wireless signal transceiver, the mobile electronic device being configured to operate in a map mode
  • the method 400 further comprising Not showing) the mobile electronic device and the first wireless signal transceiver and the second wireless signal transceiver communicably connected to the plurality of reference wireless signal sources, respectively, based on signal strengths obtained from the plurality of reference wireless signal sources Positioning of the second mobile electronic device; following the movement of the second mobile electronic device according to the position of the mobile electronic device and the second mobile electronic device; and capturing, by the first camera, multiple images during the movement of the motion module, The image includes feature information and corresponding shooting position information; and the image processing module communicably connected to the first camera, by splicing the plurality of images, extracting feature information and shooting position information in the plurality of images, and generating Image map.
  • the mobile electronic device further includes a display screen
  • the mobile electronic device is configured to operate in a map mode
  • the second mobile electronic device includes a second camera
  • the method 400 further includes (not shown) a first wireless signal transceiver communicably coupled to the plurality of reference wireless signal sources, determining a location of the mobile electronic device based on signal strengths obtained from the plurality of reference wireless signal sources; through the motion module, according to the mobile electronic device and the second Positioning of the mobile electronic device, following the movement of the second mobile electronic device; displaying a black and white chessboard through the display screen of the mobile electronic device; receiving the image from the second camera in the motion module through an image processor communicably connected to the second camera a plurality of images captured during exercise, wherein the plurality of images comprise images of a display screen of the mobile electronic device displayed as a black and white checkerboard, and the image information is extracted by splicing the plurality of images by the image processor to extract feature information of the plurality of images And shooting location information to generate an image
  • the method 400 further comprising (not shown) communicatively coupled to the plurality of reference wireless signal sources a second wireless signal transceiver, determining a position of the second mobile electronic device according to the signal strength acquired from the plurality of reference wireless signal sources; and capturing a plurality of images of the mission site through the second camera, communicably connecting to the first
  • the image processor of the two cameras generates image maps by splicing a plurality of images, extracting feature information and shooting position point information in the plurality of images.
  • the method 400 further includes (not shown) routing the selected area by using a path planning module and using a grid-based spanning tree path planning algorithm.
  • the method 400 further includes (not shown) further comprising obtaining the location of the mobile electronic device by assisting the first camera by an encoder and an inertial measurement module communicably coupled to the image processor attitude.
  • the mobile electronic device further includes a charging post, wherein the charging post includes an image processor, a path planning module, and a positioning module.
  • the senor comprises an ultrasonic sensor and/or a laser sensor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Electromagnetism (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

Dispositif électronique mobile (100), comprenant un premier émetteur-récepteur de signal sans fil (102), un processeur d'image (104), un module de positionnement (106), un module de planification d'itinéraire (108) et un module de mouvement (110). Le premier émetteur-récepteur de signal sans fil (102) acquiert une image photographiée par un utilisateur d'un second dispositif électronique mobile (140) pour un lieu de tâche et une région sélectionnée sur l'image ; le processeur d'image (104) extrait des informations de caractéristique contenant l'image de la région sélectionnée, et détermine une plage de coordonnées réelle de la région sélectionnée dans l'image par comparaison des informations de caractéristique extraites et des informations de caractéristique stockées, concernant une carte d'image, contenant des informations d'emplacement ; le module de positionnement (106) enregistre une plage de distance entre l'emplacement actuel du dispositif électronique mobile (100) et une plage de coordonnées réelle d'une région de tâche ; le module de planification d'itinéraire (108) génère un schéma de planification d'itinéraire selon la plage de coordonnées réelle de la région sélectionnée ; et le module de mouvement (110) se déplace selon le schéma de planification d'itinéraire.
PCT/CN2018/090579 2017-07-26 2018-06-11 Dispositif électronique mobile et procédé de traitement de tâches dans une région de tâche Ceased WO2019019819A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710620792.6 2017-07-26
CN201710620792.6A CN108459597B (zh) 2017-07-26 2017-07-26 一种用于处理任务区域的任务的移动电子设备以及方法

Publications (1)

Publication Number Publication Date
WO2019019819A1 true WO2019019819A1 (fr) 2019-01-31

Family

ID=63220256

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/090579 Ceased WO2019019819A1 (fr) 2017-07-26 2018-06-11 Dispositif électronique mobile et procédé de traitement de tâches dans une région de tâche

Country Status (2)

Country Link
CN (1) CN108459597B (fr)
WO (1) WO2019019819A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115060262A (zh) * 2019-05-09 2022-09-16 深圳阿科伯特机器人有限公司 在地图上定位设备的方法、服务端及移动机器人
CN115705349A (zh) * 2021-08-05 2023-02-17 华为技术有限公司 地图构建的方法及其装置

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109376262B (zh) * 2018-11-12 2022-02-11 万瞳(南京)科技有限公司 一种基于大数据处理的景点离线图像识别方法及其装置
CN109460040A (zh) * 2018-12-28 2019-03-12 珠海凯浩电子有限公司 一种通过手机拍摄照片识别地板建立地图系统及方法
CN110285799B (zh) * 2019-01-17 2021-07-30 杭州志远科技有限公司 一种带有三维可视化技术的导航系统
CN111481111A (zh) * 2019-01-29 2020-08-04 北京奇虎科技有限公司 扫地机内存的使用方法及装置
CN111667531B (zh) * 2019-03-06 2023-11-24 西安远智电子科技有限公司 定位方法及装置
CN113296495B (zh) * 2020-02-19 2023-10-20 苏州宝时得电动工具有限公司 自移动设备的路径形成方法、装置和自动工作系统
CN111784797A (zh) * 2020-06-29 2020-10-16 济南浪潮高新科技投资发展有限公司 一种基于ar的机器人物联网交互方法、装置及介质
CN112256345A (zh) * 2020-10-10 2021-01-22 深圳供电局有限公司 一种基于最先适应算法和遗传算法的计算任务卸载方法
CN116965745A (zh) * 2022-04-22 2023-10-31 追觅创新科技(苏州)有限公司 坐标重定位方法、系统及清洁机器人

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6667592B2 (en) * 2001-08-13 2003-12-23 Intellibot, L.L.C. Mapped robot system
CN101655369A (zh) * 2008-08-22 2010-02-24 环达电脑(上海)有限公司 利用图像识别技术实现定位导航的系统及方法
CN104750008A (zh) * 2015-04-14 2015-07-01 西北农林科技大学 一种ZigBee网络中的农业机器人无线遥控系统
CN105259898A (zh) * 2015-10-13 2016-01-20 江苏拓新天机器人科技有限公司 一种智能手机控制的扫地机器人
CN106725119A (zh) * 2016-12-02 2017-05-31 西安丰登农业科技有限公司 一种基于三维模型定位的扫地机器人导航系统
CN207115193U (zh) * 2017-07-26 2018-03-16 炬大科技有限公司 一种用于处理任务区域的任务的移动电子设备

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101333496B1 (ko) * 2010-11-30 2013-11-28 주식회사 유진로봇 과거 지도 데이터 기반의 이동 로봇 제어 장치 및 방법
CN102866706B (zh) * 2012-09-13 2015-03-25 深圳市银星智能科技股份有限公司 一种采用智能手机导航的清扫机器人及其导航清扫方法
US20140309925A1 (en) * 2013-04-14 2014-10-16 Pablo Garcia MORATO Visual positioning system
CN103439973B (zh) * 2013-08-12 2016-06-29 桂林电子科技大学 自建地图家用清洁机器人及清洁方法
CN205068153U (zh) * 2015-08-07 2016-03-02 浙江海洋学院 一种基于行走机器人的分布式视觉定位系统
CN105352508A (zh) * 2015-10-22 2016-02-24 深圳创想未来机器人有限公司 机器人定位导航方法及装置
KR20170058612A (ko) * 2015-11-19 2017-05-29 (주)예사싱크 영상 기반 실내측위 방법 및 그의 시스템
CN106292697B (zh) * 2016-07-26 2019-06-14 北京工业大学 一种移动设备的室内路径规划与导航方法
CN106382930B (zh) * 2016-08-18 2019-03-29 广东工业大学 一种室内agv无线导航方法及装置
CN106444750A (zh) * 2016-09-13 2017-02-22 哈尔滨工业大学深圳研究生院 一种基于二维码定位的智能仓储移动机器人系统
CN106647766A (zh) * 2017-01-13 2017-05-10 广东工业大学 一种基于复杂环境的uwb‑视觉交互的机器人巡航方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6667592B2 (en) * 2001-08-13 2003-12-23 Intellibot, L.L.C. Mapped robot system
CN101655369A (zh) * 2008-08-22 2010-02-24 环达电脑(上海)有限公司 利用图像识别技术实现定位导航的系统及方法
CN104750008A (zh) * 2015-04-14 2015-07-01 西北农林科技大学 一种ZigBee网络中的农业机器人无线遥控系统
CN105259898A (zh) * 2015-10-13 2016-01-20 江苏拓新天机器人科技有限公司 一种智能手机控制的扫地机器人
CN106725119A (zh) * 2016-12-02 2017-05-31 西安丰登农业科技有限公司 一种基于三维模型定位的扫地机器人导航系统
CN207115193U (zh) * 2017-07-26 2018-03-16 炬大科技有限公司 一种用于处理任务区域的任务的移动电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115060262A (zh) * 2019-05-09 2022-09-16 深圳阿科伯特机器人有限公司 在地图上定位设备的方法、服务端及移动机器人
CN115705349A (zh) * 2021-08-05 2023-02-17 华为技术有限公司 地图构建的方法及其装置

Also Published As

Publication number Publication date
CN108459597B (zh) 2024-02-23
CN108459597A (zh) 2018-08-28

Similar Documents

Publication Publication Date Title
WO2019019819A1 (fr) Dispositif électronique mobile et procédé de traitement de tâches dans une région de tâche
JP7236565B2 (ja) 位置姿勢決定方法、装置、電子機器、記憶媒体及びコンピュータプログラム
US9710919B2 (en) Image-based surface tracking
CN207115193U (zh) 一种用于处理任务区域的任务的移动电子设备
WO2019001237A1 (fr) Dispositif électronique mobile, et procédé dans un dispositif électronique mobile
CN112567201A (zh) 距离测量方法以及设备
WO2022077296A1 (fr) Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur
US20160253814A1 (en) Photogrammetric methods and devices related thereto
US20180190014A1 (en) Collaborative multi sensor system for site exploitation
WO2018140107A1 (fr) Système de filtrage d'images en 3d
CN207488823U (zh) 一种移动电子设备
Ye et al. 6-DOF pose estimation of a robotic navigation aid by tracking visual and geometric features
WO2018228258A1 (fr) Dispositif électronique mobile et procédé associé
JP2016085602A (ja) センサ情報統合方法、及びその装置
CN113701750A (zh) 一种井下多传感器的融合定位系统
KR102190743B1 (ko) 로봇과 인터랙션하는 증강현실 서비스 제공 장치 및 방법
CN207067803U (zh) 一种用于处理任务区域的任务的移动电子设备
CN117257170A (zh) 清洁方法、清洁展示方法、清洁设备和存储介质
KR20120108256A (ko) 로봇 물고기 위치 인식 시스템 및 로봇 물고기 위치 인식 방법
JP7437930B2 (ja) 移動体及び撮像システム
CN112672134B (zh) 基于移动终端三维信息采集控制设备及方法
WO2019037517A1 (fr) Dispositif électronique mobile et procédé de traitement de tâche dans une zone de tâche
CN206833252U (zh) 一种移动电子设备
JP7317684B2 (ja) 移動体、情報処理装置、及び撮像システム
CN112304250B (zh) 一种移动物体之间的三维匹配设备及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18838419

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18838419

Country of ref document: EP

Kind code of ref document: A1