WO2022012471A1 - 自移动设备的控制方法、装置、存储介质及自移动设备 - Google Patents

自移动设备的控制方法、装置、存储介质及自移动设备 Download PDF

Info

Publication number
WO2022012471A1
WO2022012471A1 PCT/CN2021/105792 CN2021105792W WO2022012471A1 WO 2022012471 A1 WO2022012471 A1 WO 2022012471A1 CN 2021105792 W CN2021105792 W CN 2021105792W WO 2022012471 A1 WO2022012471 A1 WO 2022012471A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
scene
self
model
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/105792
Other languages
English (en)
French (fr)
Inventor
郁顺昌
王朕
汤盛浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dreame Innovation Technology Suzhou Co Ltd
Original Assignee
Dreame Innovation Technology Suzhou Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202010666134.2A external-priority patent/CN111539398B/zh
Priority claimed from CN202010666140.8A external-priority patent/CN111539400A/zh
Priority claimed from CN202010666135.7A external-priority patent/CN111539399B/zh
Priority to KR1020237004202A priority Critical patent/KR20230035610A/ko
Priority to EP21842796.1A priority patent/EP4163819A4/en
Priority to JP2023501666A priority patent/JP2023534932A/ja
Priority to AU2021308246A priority patent/AU2021308246A1/en
Application filed by Dreame Innovation Technology Suzhou Co Ltd filed Critical Dreame Innovation Technology Suzhou Co Ltd
Priority to CA3185243A priority patent/CA3185243A1/en
Priority to US18/015,719 priority patent/US12478241B2/en
Publication of WO2022012471A1 publication Critical patent/WO2022012471A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4011—Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4061—Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • A47L9/2805—Parameters or conditions being sensed
    • A47L9/2826—Parameters or conditions being sensed the condition of the floor
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • A47L9/2836—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means characterised by the parts which are controlled
    • A47L9/2852—Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • A47L9/2868—Arrangements for power supply of vacuum cleaners or the accessories thereof
    • A47L9/2873—Docking units or charging stations
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/955—Hardware or software architectures specially adapted for image or video understanding using specific electronic processors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/02—Docking stations; Docking operations
    • A47L2201/022—Recharging of batteries
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/04—Automatic control of the travelling movement; Automatic obstacle detection
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/06—Control of the cleaning action for autonomous devices; Automatic detection of the surface condition before, during or after cleaning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects

Definitions

  • the present application relates to a control method, device, storage medium and self-moving device for self-moving equipment, and belongs to the technical field of computers.
  • Common sweeping robots collect environmental pictures through a camera component fixed above the fuselage, and use image recognition algorithms to identify items in the collected pictures.
  • the image recognition algorithm is usually obtained by training based on a neural network model or the like.
  • the existing image recognition algorithm requires a combination of a graphics processor (Graphics Processing Unit, GPU) and a neural network processor (Neural Processing Unit, NPU), which requires high hardware requirements for the sweeping robot.
  • a graphics processor Graphics Processing Unit, GPU
  • a neural network processor Neural Processing Unit, NPU
  • the present application provides a control method, device and storage medium for a self-moving device, which can solve the problem that the existing image recognition algorithm has high hardware requirements for the cleaning robot, resulting in the limited application scope of the object recognition function of the cleaning robot.
  • This application provides the following technical solutions:
  • a method for controlling a self-moving device wherein an image acquisition component is installed on the self-moving device, and the method includes:
  • the computing resources occupied when the image recognition model is running is lower than the maximum computing resources provided by the self-mobile device;
  • the image recognition model is obtained by training a small network detection model.
  • the method before the acquiring the image recognition model, the method further includes:
  • the training data includes training images of each object in the working area of the self-mobile device and a recognition result of each training image
  • the small network detection model is trained based on the difference between the model result and the recognition result corresponding to the training image to obtain the image recognition model.
  • the method further includes:
  • Model compression is performed on the image recognition model to obtain an image recognition model for recognizing objects.
  • the small network detection model is: a miniature YOLO model; or, a MobileNet model.
  • the method further includes:
  • the self-mobile device is controlled to move to complete the corresponding task.
  • a liquid cleaning component is installed on the self-moving device, and the self-moving device is controlled to move to complete a corresponding task based on the object recognition result, including:
  • the liquid sweeping assembly is used to sweep liquid from the area to be cleaned.
  • a power supply component is installed in the self-moving device, the power supply component is charged by a charging component, and the self-moving device is controlled to move to complete a corresponding task based on the object recognition result, including:
  • the actual position of the charging assembly is determined according to the image position of the charging assembly.
  • a positioning sensor is also installed on the self-moving device, and the positioning sensor is used to locate the position of the charging interface on the charging assembly; after the self-moving device is controlled to move to the charging assembly, the positioning sensor is further include:
  • the self-moving device is controlled to move according to the positioning result, so as to realize the docking of the self-moving device with the charging interface.
  • a control device for a self-moving device is provided, an image acquisition component is installed on the self-moving device, and the device includes:
  • an image acquisition module configured to acquire an environmental image acquired by the image acquisition component during the movement of the self-mobile device
  • a model acquisition module configured to acquire an image recognition model, the computing resources occupied when the image recognition model is running is lower than the maximum computing resources provided by the self-mobile device;
  • a device control module configured to control the environment image to be input into the image recognition model to obtain an object recognition result, where the object recognition result is used to indicate the category of the target object.
  • a method for controlling a self-moving device wherein an image acquisition component is installed on the self-moving device, and the method includes:
  • the self-mobile device during the movement of the self-mobile device, acquiring a scene image collected by the image acquisition component, where the scene image is an image of the current scene where the self-mobile device is located;
  • the scene recognition model is obtained by training based on the sample object information and the sample scene type corresponding to the sample object information;
  • the object information is controlled to be input into the scene recognition model to obtain the scene type of the current scene.
  • the scene recognition model is obtained by training a probability model.
  • the method before the acquiring the scene recognition model, the method further includes:
  • training data includes sample attribute information of each object and a sample scene type corresponding to each sample attribute information
  • the probability model is trained based on the difference between the model result and the sample scene type to obtain the scene recognition model.
  • the object information obtained by performing image recognition on the scene image further includes:
  • the computing resources occupied when the image recognition model is running is lower than the maximum computing resources provided by the self-mobile device;
  • the image recognition model is obtained by training using sample scene images and sample object results based on a small network model.
  • the image recognition model is obtained after model compression processing.
  • the scene recognition model also outputs the confidence level of each scene type; the controlling the object information to be input into the scene recognition model to obtain the scene type of the current scene, further comprising:
  • the method further includes:
  • the self-moving device is controlled to perform cleaning work according to the cleaning strategy.
  • a scene recognition device for a self-moving device is provided, an image acquisition component is installed on the self-moving device, and the device includes:
  • an image acquisition module configured to acquire a scene image collected by the image acquisition component during the movement of the self-mobile device, where the scene image is an image of the current scene where the self-mobile device is located;
  • an image recognition module configured to perform image recognition on the scene image to obtain object information, where the object information refers to attribute information of an object located in the current scene;
  • a model acquisition module configured to acquire a scene recognition model, the scene recognition model is obtained by training based on the sample object information and the sample scene type corresponding to the sample object information;
  • a device control module configured to control the object information to be input into the scene recognition model to obtain the scene type of the current scene.
  • a method for controlling a self-mobile device comprising:
  • the method further includes:
  • the scene type of the independent area is determined according to the scene prediction result of the independent area.
  • dividing the independent area in the working area based on the access door information and the edge information including:
  • Each closed area formed by the combined boundary information is divided into corresponding independent areas.
  • the obtaining access door information based on the environment image includes:
  • position information of the access door in the work area is acquired.
  • the access door includes a door frame in the work area.
  • the determining the scene type of the independent area according to the scene prediction result of the independent area includes:
  • the pose information includes the position information and orientation information of the corresponding independent area in the working area;
  • the probability distribution strategy is used for each target scene type to determine from each independent area that the scene type is the independent area with the highest probability of the target scene type.
  • the determining the scene prediction result corresponding to the independent area based on the environment image includes:
  • the computing resources occupied when the image recognition model is running is lower than the maximum computing resources provided by the self-mobile device;
  • the scene recognition model is obtained by using the sample attribute information of the object and the sample scene type training;
  • the image recognition model is obtained by training using sample environment images and sample object results based on a small network model.
  • the scene recognition model is obtained by using the sample attribute information of the object and the sample scene training based on the probability model.
  • a control device for a self-moving device comprising:
  • a first information acquisition module configured to acquire edge information of the working area where the self-mobile device is located
  • the environmental image acquisition module is used to acquire the environmental image collected while moving;
  • a second information obtaining module configured to obtain access door information based on the environment image
  • an area division control module configured to divide independent areas in the working area based on the access door information and the edge information
  • a control apparatus for a self-mobile device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to realize the first aspect and the third The control method of the self-mobile device according to the aspect or the fifth aspect.
  • a computer-readable storage medium is provided, and a program is stored in the storage medium, and the program is loaded and executed by the processor to realize the self-service described in the first aspect, the third aspect or the fifth aspect. Control method of mobile device.
  • a self-moving device including:
  • a moving assembly for driving the self-moving device to move
  • a mobile drive assembly for driving the movement of the mobile assembly
  • an image capture component installed on the self-moving device for capturing images of the environment in the direction of travel
  • the control method of the self-mobile device according to the aspect, the third aspect or the fifth aspect.
  • the beneficial effects of the present application are: by acquiring the environmental image collected by the image acquisition component in the process of moving from the mobile device; by acquiring the image recognition model, the computing resources occupied by the image recognition model when running are lower than the maximum computing resources provided by the self-mobile device; Control the environmental image and input the image recognition model to obtain the object recognition result, and the object recognition result is used to indicate the category of the target object in the environmental image; it can solve the high requirement of the existing image recognition algorithm on the hardware of the cleaning robot, which leads to the application of the object recognition function of the cleaning robot.
  • the problem of limited scope by using an image recognition model that consumes less computing resources to recognize the target object in the environmental image, the hardware requirements of the object recognition method for self-mobile devices can be reduced, and the application scope of the object recognition method can be expanded.
  • FIG. 1 is a schematic structural diagram of a self-mobile device provided by an embodiment of the present application.
  • FIG. 2 is a flowchart of a control method for a self-mobile device provided by an embodiment of the present application
  • FIG. 3 is a flowchart of an execution work strategy provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an execution work strategy provided by an embodiment of the present application.
  • FIG. 5 is a flowchart of an execution work strategy provided by another embodiment of the present application.
  • FIG. 6 is a schematic diagram of an execution work strategy provided by another embodiment of the present application.
  • FIG. 7 is a block diagram of a control apparatus for a self-mobile device provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a self-mobile device provided by an embodiment of the present application.
  • FIG. 9 is a flowchart of a control method from a mobile device provided by an embodiment of the present application.
  • FIG. 10 is a flowchart of a control method for a self-mobile device provided by another embodiment of the present application.
  • FIG. 11 is a block diagram of a control apparatus for a self-mobile device provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a self-mobile device provided by an embodiment of the present application.
  • FIG. 13 is a flowchart of a control method from a mobile device provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of control from a mobile device provided by an embodiment of the present application.
  • 16 is a block diagram of a control apparatus for a self-mobile device provided by an embodiment of the present application.
  • FIG. 17 is a block diagram of a control apparatus for a self-mobile device provided by an embodiment of the present application.
  • Model compression refers to the method of reducing the parameter redundancy in the trained network model, thereby reducing the storage occupation, communication bandwidth and computational complexity of the network model.
  • Model compression includes, but is not limited to: model clipping, model quantization, and/or low-yield decomposition.
  • Model tailoring refers to the search process of the optimal network structure.
  • the model pruning process includes: 1. training the network model; 2. pruning unimportant weights or channels; 3. fine-tuning or retraining the pruned network.
  • the second step usually relies on iterative layer-by-layer clipping, fast fine-tuning or weight reconstruction to maintain accuracy.
  • Quantization model is a general term for a model acceleration method. It is a process of representing a limited range (such as 32-bit) floating-point data with a data type with fewer digits, so as to reduce the size of the model, reduce model memory consumption and Goals such as speeding up model inference.
  • Low-rank decomposition The weight matrix of the network model is decomposed into multiple small matrices.
  • the calculation amount of the small matrix is smaller than that of the original matrix, so as to reduce the amount of model operation and the memory occupied by the model.
  • YOLO model One of the basic network models is a neural network model that can achieve target location and recognition through the Convolutional Neural Networks (CNN) network.
  • YOLO models include YOLO, YOLO v2, and YOLO v3.
  • YOLO v3 is another target detection algorithm of the YOLO series after YOLO and YOLO v2, which is an improvement based on YOLO v2.
  • YOLO v3-tiny is a simplified version of YOLO v3. On the basis of YOLO v3, some feature layers are removed, so as to achieve the effect of reducing the amount of model operation and making the operation faster.
  • MobileNet model It is a network model whose basic unit is depthwise separable convolution. Among them, the depth-level separable convolution can be decomposed into depthwise separable convolution (Depthwise, DW) and pointwise convolution (Pointwise, PW).
  • DW is different from standard convolution. For standard convolution, its convolution kernel is used on all input channels, while DW uses different convolution kernels for each input channel, that is, one convolution kernel corresponds to one input channel.
  • PW is an ordinary convolution, but it uses a 1x1 convolution kernel. For depth-level separable convolution, it first uses DW to convolve different input channels separately, and then uses PW to combine the above outputs. In fact, the overall calculation result is approximately the same as that of a standard convolution process. But it will greatly reduce the amount of calculation and model parameters.
  • FIG. 1 is a schematic structural diagram of a self-mobile device provided by an embodiment of the present application. As shown in FIG. 1 , the system at least includes: a control component 110 , and an image acquisition component 120 communicatively connected to the control component 110 .
  • the image acquisition component 120 is used for acquiring the environmental image 130 during the movement of the mobile device; and sending the environmental image 130 to the control component 110 .
  • the image acquisition component 120 may be implemented as a camera, a video camera, or the like, and the implementation manner of the image acquisition component 120 is not limited in this embodiment.
  • the field of view angle of the image acquisition component 120 is 120° in the horizontal direction and 60° in the vertical direction; of course, the field of view angle can also be other values, and this embodiment does not affect the field of view of the image acquisition component 120 The value of the angle is limited.
  • the field of view of the image capture assembly 120 can ensure that the environment image 130 in the travel direction can be captured from the mobile device.
  • the number of image acquisition components 120 may be one or more, and this embodiment does not limit the number of image acquisition components 120 .
  • the control component 110 is used to control the self-mobile device. For example: control the start and stop of the self-mobile device; control the start and stop of each component in the self-mobile device (eg, the image capture component 120 ).
  • control component 110 is communicatively connected to a memory; the memory stores a program, and the program is loaded by the control component 110 and executes at least the following steps: in the process of moving from the mobile device, acquire the 120 pieces of images collected by the image collection group. environment image 130 ; obtain an image recognition model; control the environment image 130 to input the image recognition model to obtain an object recognition result 140 , the object recognition result 140 is used to indicate the category of the target object in the environment image 130 .
  • the program is loaded and executed by the control component 110 to implement the control method from the mobile device provided by the present application.
  • the object recognition result 140 when the target object is included in the environment image, the object recognition result 140 is the type of the target object; when the target object is included in the environment image, the object recognition result 140 is empty. Or, when the target object is included in the environment image, the object recognition result 140 is an indication that the target object is included (for example, "1" indicates that the target object is included) and the type of the target object; when the target object is not included in the environment image, The object recognition result 140 is an indication that the target object is not included (eg, a "0" indicates that the target object is not included).
  • the computing resources occupied by the image recognition model when running are lower than the maximum computing resources provided by the mobile device.
  • the object recognition result 140 may further include, but is not limited to, information such as the position and size of the image of the target object in the environment image 130 .
  • the target object is an object located in the work area of the mobile device.
  • the target objects can be objects such as beds, tables, chairs, and people in the room;
  • the target objects can be boxes, people in the warehouse etc., this embodiment does not limit the type of the target object.
  • the image recognition model is a network model in which the number of model layers is less than the first value; and/or the number of nodes in each layer is less than the second value.
  • the first numerical value and the second numerical value are both small integers, so as to ensure that the image recognition model consumes less computing resources when running.
  • the self-moving device may further include other components, such as: a moving component (such as a wheel) for driving the self-moving device to move, a mobile driving component (such as a wheel) for driving the moving component to move : motor) etc., wherein, the mobile drive component is connected to the control component 110 in communication, and under the control of the control component 110, the mobile drive component operates and drives the mobile component to move, thereby realizing the overall motion of the self-moving device.
  • a moving component such as a wheel
  • a mobile driving component such as a wheel
  • the mobile drive component operates and drives the mobile component to move, thereby realizing the overall motion of the self-moving device.
  • the components included in the mobile device are not listed one by one.
  • the self-moving device may be a sweeping robot, an automatic lawn mower, or other devices with an automatic driving function, and the present application does not limit the device type of the self-moving device.
  • the hardware requirements of the object recognition method on self-mobile devices can be reduced, and the application scope of the object recognition method can be expanded.
  • control method of the self-mobile device is used in the self-mobile device shown in FIG. 1 as an example, and the execution subject of each step is the control component 110 for illustration.
  • the method at least includes the following steps :
  • Step 201 during the process of moving from the mobile device, acquire the environment image collected by the image collection component.
  • the image acquisition component is used to collect video data, and at this time, the environmental image can be a frame of image data in the video data; or, the image acquisition component is used to collect a single image data, and at this time, the environmental image is Image data of a single sheet sent by the image acquisition component.
  • step 202 an image recognition model is acquired, and the computing resources occupied by the image recognition model when running are lower than the maximum computing resources provided by the mobile device.
  • the hardware requirements of the image recognition model on the self-mobile device can be reduced, and the application scope of the object recognition method can be expanded.
  • a pre-trained image recognition model is read from a mobile device.
  • the image recognition model is obtained by training the small network detection model.
  • Training the small network detection model includes: obtaining the small network detection model; obtaining training data; inputting the training image into the small network detection model to obtain the model result; based on the difference between the model result and the recognition result corresponding to the training image, the small network The detection model is trained to obtain an image recognition model.
  • the training data includes training images of various objects in the working area of the mobile device and the recognition result of each training image.
  • the small network model refers to a network model in which the number of model layers is less than the first value; and/or the number of nodes in each layer is less than the second value.
  • the first numerical value and the second numerical value are both small integers.
  • the small network detection model is: the miniature YOLO model; or, the MobileNet model.
  • the small network detection model may also be other models, which are not listed one by one in this embodiment.
  • the self-mobile device can also perform model compression processing on the image recognition model to obtain the image recognition model used for recognition.
  • Image recognition models for objects can also perform model compression processing on the image recognition model to obtain the image recognition model used for recognition.
  • model compression process includes but is not limited to: model clipping, model quantization, and/or low-yield decomposition, etc.
  • the self-mobile device can use the training data to train the compressed image recognition model again, so as to improve the recognition accuracy of the image recognition model.
  • Step 203 Control the environment image to input the image recognition model to obtain the object recognition result, where the object recognition result is used to indicate the category of the target object.
  • the object recognition result further includes, but is not limited to, information such as the position and/or size of the image of the target object in the environment image.
  • the control method provided by this embodiment from a mobile device obtains the environmental image collected by the image acquisition component during the movement of the mobile device, and obtains the image recognition model, and the computing resources occupied when the image recognition model runs are low. It is based on the maximum computing resources provided by the mobile device; the object recognition result is obtained by controlling the input image recognition model of the environmental image, and the object recognition result is used to indicate the category of the target object in the environmental image; it can solve the hardware requirements of the existing image recognition algorithm for the cleaning robot high, which leads to the problem that the application scope of the object recognition function of the sweeping robot is limited; by using an image recognition model that consumes less computing resources to recognize the target object in the environmental image, the hardware requirements of the object recognition method for self-mobile devices can be reduced, and the expansion of The scope of application of object recognition methods.
  • the image recognition model is obtained by training and learning with a small network model, and the object recognition process can be realized without the combination of a graphics processor (Graphics Processing Unit, GPU) and an embedded neural network processor (Neural-network Processing Units, NPU). Therefore, the requirements on the device hardware of the object recognition method can be reduced.
  • a graphics processor Graphics Processing Unit, GPU
  • an embedded neural network processor Neuro-network Processing Units, NPU
  • the image recognition model is subjected to model compression processing to obtain an image recognition model for recognizing objects; the computing resources occupied by the image recognition model during operation can be further reduced, the recognition speed can be improved, and the application scope of the object recognition method can be expanded.
  • the self-mobile device is also controlled to move based on the object recognition result to complete the corresponding task.
  • the tasks include but are not limited to: the task of avoiding obstacles for certain objects, such as chairs, pet feces, etc.; the task of locating certain objects, such as the task of locating doors, windows, charging components, etc.; Tasks for monitoring and following people; tasks for cleaning specific items, such as cleaning liquids; and/or, automatic refill tasks. Below, the tasks performed corresponding to different object recognition results are introduced.
  • a liquid cleaning assembly is installed on the self-moving device.
  • controlling the self-mobile device to move based on the object recognition result to complete the corresponding task including: when the object recognition result indicates that the environmental image contains a liquid image, controlling the self-mobile device to move to the area to be cleaned corresponding to the liquid image ;Use the Liquid Sweep Kit to sweep liquid from the area to be cleaned.
  • the liquid sweeping assembly includes an absorbent mop mounted on the periphery of a wheel body from the mobile device.
  • the self-moving device is controlled to move to the area to be cleaned corresponding to the liquid image, so that the wheel body of the self-moving device passes through the area to be cleaned, so that the absorbent mop absorbs the liquid on the ground.
  • the self-moving equipment is also provided with a cleaning pool and a reservoir; the cleaning pool is located under the wheel body; the water pump sucks the water in the reservoir, sprays it from the nozzle to the wheel body through the pipe, and washes the dirt on the water-absorbing mop until it is cleaned. pool.
  • liquid cleaning assembly is only illustrative, and in actual implementation, the liquid cleaning assembly can also be realized in other ways, which are not listed one by one in this embodiment.
  • the self-moving device may be a cleaning robot, and in this case, the self-moving device has the function of uniformly removing dry and wet garbage.
  • the liquid cleaning component by activating the liquid cleaning component when there is a liquid image in the environmental image, the problem that the self-moving device can bypass the liquid and cause the cleaning task to fail, and the cleaning effect of the self-moving device can be improved.
  • the liquid can be prevented from entering the interior of the self-moving device, causing circuit damage, and the risk of damage to the self-moving device can be reduced.
  • a power supply component is installed in the mobile device. Controlling the self-mobile device to move based on the object recognition result to complete the corresponding task, including: when the remaining power of the power supply component is less than or equal to the power threshold, and the environmental image includes the image of the charging component, the self-mobile device determines the charging according to the image position of the charging component. Actual position of the component; controls movement from the mobile device to the charging component.
  • the self-mobile device After the self-mobile device captures the image of the charging component, the orientation of the charging component relative to the self-mobile device can be determined according to the position of the image in the environment image. Therefore, the self-mobile device can charge according to the roughly determined direction. Components move.
  • a positioning sensor is further installed on the self-moving device, and the positioning sensor is used to locate the position of the charging interface on the charging assembly.
  • the self-moving device controls the positioning sensor to locate the position of the charging component to obtain the positioning result; the self-moving device is controlled to move according to the positioning result, so as to realize the docking between the self-moving device and the charging interface .
  • the positioning sensor is a laser sensor.
  • the charging interface of the charging assembly emits laser signals at different angles, and the positioning sensor determines the position of the charging interface based on the angle difference of the received laser signals.
  • the positioning sensor may be other types of sensors, and the type of the positioning sensor is not limited in this embodiment.
  • FIG. 5 and FIG. 6 use the image recognition model to obtain the object recognition result of the environment image; when the object recognition result is that the current environment includes the charging component 51, use the positioning sensor 52 to locate the position of the charging interface 53 on the charging component 51; move to the charging interface 53 to The self-moving device is electrically connected to the charging component 51 through the charging interface to realize charging.
  • the charging component is identified by the image recognition model and moved to the vicinity of the charging component; the self-mobile device can automatically return to the charging component for charging, and the intelligence of the self-mobile device can be improved.
  • the accuracy of the automatic return of the self-mobile device to the charging component can be improved, and the automatic charging efficiency can be improved.
  • FIG. 7 is a block diagram of an apparatus for controlling a self-moving device provided by an embodiment of the present application. This embodiment is described by taking the device applied to the self-moving device shown in FIG. 1 as an example.
  • the apparatus includes at least the following modules: an image acquisition module 710 , a model acquisition module 720 and a device control module 730 .
  • An image acquisition module 710 configured to acquire an environment image acquired by the image acquisition component during the moving process from the mobile device;
  • Model acquisition module 720 for acquiring an image recognition model, the computing resources occupied when the image recognition model is running is lower than the maximum computing resources provided by the self-mobile device;
  • the device control module 730 is configured to control the environment image to be input into the image recognition model to obtain an object recognition result, where the object recognition result is used to indicate the category of the target object.
  • control device of the self-mobile device when the control device of the self-mobile device provided in the above-mentioned embodiment controls the self-mobile device, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions may be allocated as required. It is completed by different functional modules, that is, the internal structure of the control device of the self-mobile device is divided into different functional modules, so as to complete all or part of the functions described above.
  • the control apparatus for a self-mobile device provided by the above-mentioned embodiment and the control method for a self-mobile device belong to the same concept, and the specific implementation process thereof is detailed in the method embodiment, which will not be repeated here.
  • FIG. 8 is a schematic structural diagram of a self-mobile device provided by an embodiment of the present application. As shown in FIG. 8 , the system at least includes: a control component 810 and an image acquisition component 820 that is communicatively connected to the control component 810 .
  • the image acquisition component 820 is used for acquiring scene images; and sending the scene images to the control component 810 .
  • the image acquisition component 820 may be implemented as a camera, a video camera, or the like, and the implementation manner of the image acquisition component 820 is not limited in this embodiment.
  • the field of view angle of the image acquisition component 820 is 20° in the horizontal direction and 60° in the vertical direction; of course, the field of view angle can also be other values, and this embodiment does not apply to the field of view of the image acquisition component 120 The value of the angle is limited.
  • the field of view of the image capturing component 820 can ensure that the scene image in the traveling direction can be captured from the mobile device.
  • the number of the image acquisition components 820 may be one or more, and this embodiment does not limit the number of the image acquisition components 820 .
  • the control component 810 is used to control the self-mobile device. For example: control the start and stop of the self-mobile device; control the start and stop of each component in the self-mobile device (eg, the image capture component 820 ).
  • control component 810 is communicatively connected to a memory; the memory stores a program, and the program is loaded by the control component 810 and executed to implement at least the following steps: in the process of moving from the mobile device, acquire the scene collected by the image acquisition component 820 Image 830; perform image recognition on the scene image 830 to obtain object information 840, the object information 840 refers to the attribute information of the object located in the current scene; obtain the scene recognition model; control the object information 840 to input the scene recognition model to obtain the scene of the current scene Type 850.
  • the program is loaded and executed by the control component 810 to implement the control method from the mobile device provided by the present application.
  • the scene recognition model is obtained by training based on the sample object information 840 and the sample scene type 850 corresponding to the sample object information 840 .
  • the attribute information of the object includes but is not limited to: the type of the object and the position of the image of the object in the scene image 830 .
  • the scene type 850 is used to indicate the type of the working environment where the self-mobile device is currently located, and the division method of the scene type 850 is set according to the working environment of the self-mobile device. For example, if the working environment of the mobile device is a room, the scene type 850 includes: bedroom type, kitchen type, study room type, bathroom type, etc. This embodiment does not limit the division method of the scene type 850 .
  • the self-moving device may further include other components, such as: a moving component (such as a wheel) for driving the self-moving device to move, a mobile driving component (such as a wheel) for driving the moving component to move : motor), etc., wherein, the mobile drive component is connected to the control component 810 in communication, and under the control of the control component 810, the mobile drive component operates and drives the mobile component to move, thereby realizing the overall movement of the self-moving device.
  • a moving component such as a wheel
  • a mobile driving component such as a wheel
  • the mobile drive component operates and drives the mobile component to move, thereby realizing the overall movement of the self-moving device.
  • the components included in the mobile device are not listed one by one.
  • the self-moving device may be a sweeping robot, an automatic lawn mower, or other devices with an automatic driving function, and the present application does not limit the device type of the self-moving device.
  • the self-mobile device can identify the scene type of the current working environment and provide more information for the user.
  • FIG. 9 is a flowchart of a method for controlling a self-mobile device provided by an embodiment of the present application.
  • the method is applied to the self-mobile device shown in FIG. 8 , and the execution subject of each step is the control in the system.
  • the component 810 is described as an example.
  • the method includes at least the following steps:
  • Step 901 in the process of moving from the mobile device, acquire a scene image collected by the image acquisition component, where the scene image is an image of the current scene where the self-mobile device is located.
  • the image acquisition component is used to collect video data, in this case, the scene image can be a frame of image data in the video data; or, the image acquisition component is used to collect a single image data, in this case, the scene image is Image data of a single sheet sent by the image acquisition component.
  • Step 902 Perform image recognition on the scene image to obtain object information, where the object information refers to attribute information of an object located in the current scene.
  • the image recognition model is stored in the mobile device.
  • an image recognition model is obtained from a mobile device; the scene image is input into the image recognition model to obtain object information.
  • the computing resources occupied by the image recognition model when running are lower than the maximum computing resources provided by the self-mobile device.
  • the image recognition model is trained using sample scene images and sample object results based on a small network model.
  • the small network model refers to a network model in which the number of model layers is less than the first value; and/or the number of nodes in each layer is less than the second value.
  • the first numerical value and the second numerical value are both small integers.
  • the small network model is a miniature YOLO model; or, the MobileNet model.
  • the small network model may also be other models, which are not listed one by one in this embodiment.
  • the self-mobile device can also perform model compression processing on the image recognition model to obtain the image recognition model used for recognizing objects.
  • the model compression processing includes, but is not limited to, model clipping, model quantization, and/or low-yield decomposition.
  • the image recognition model may also be established based on a deep network neural model, for example, established based on a convolutional neural network.
  • the object is an object in the work area of the mobile device.
  • the attribute information of the object includes type information of the object.
  • the type information of the object is divided based on the working environment of the mobile device. For example, when the working environment of the mobile device is a room, the type information of the object includes: table type, chair type, sofa type, liquid type, charging station type, etc. , this embodiment does not limit the manner of classifying the types of objects.
  • the attribute information of the object may also include other information, such as: position information of the object in the scene image, size of the object, etc. This embodiment does not limit the specific content included in the attribute information of the object.
  • the self-mobile device can use the training data to train the compressed image recognition model again, so as to improve the recognition accuracy of the image recognition model.
  • Step 903 Obtain a scene recognition model, where the scene recognition model is obtained by training based on the sample object information and the sample scene type corresponding to the sample object information.
  • a pre-trained scene recognition model is read from a mobile device, where the scene recognition model is obtained by training a probability model.
  • the probability model needs to be trained. Training the probability model includes: acquiring the probability model; and acquiring training data, where the training data includes sample attribute information of each object and sample scene types corresponding to each sample attribute information ; Input the sample attribute information into the probability model to obtain the model result; train the probability model based on the difference between the model result and the sample scene type to obtain the scene recognition model.
  • the self-mobile device can be provided with the function of recognizing the scene type, and the intelligence of the self-mobile device can be improved.
  • Step 904 the control object information is input into the scene recognition model to obtain the scene type of the current scene.
  • the scene type is pre-divided based on the working environment of the mobile device.
  • the working environment of the self-mobile device is a room
  • the scene types include but are not limited to: kitchen type, living room type, bedroom type, bathroom type, storage room type, and/or study type.
  • the scene type is one or more of the pre-divided scene types.
  • the scene recognition method is illustrated below with an example.
  • the scene image is collected from the mobile device using the image collection component 820
  • the scene image is input into the image recognition model to obtain object information 32
  • the scene type 33 of the current scene is obtained after the object information is output to the scene recognition model.
  • the scene recognition model also outputs a confidence level of each scene type, and the confidence level is used to indicate the output accuracy of each scene type.
  • the mobile device sorts the scene types in descending order of confidence; and outputs the scene types ranked in the top N.
  • N is an integer greater than 1.
  • the value of N exists in the mobile device, and the value of N may be 3, 2, etc., and the value of N is not limited in this embodiment.
  • the automatic device outputs all scene types.
  • the user terminal is communicatively connected to the self-mobile device, and in this case, outputting the top N scene types includes: sending the top N scene types to the user terminal, so that the user terminal displays the top N scene types.
  • an information output component such as a display screen or an audio playback component, is installed on the mobile device.
  • outputting the top N scene types includes: outputting the top N scene types through the output component.
  • the self-mobile device may also determine a corresponding work strategy according to the scene type of the current scene; and control the self-mobile device to perform cleaning work according to the cleaning strategy.
  • the self-moving device is a floor cleaning device.
  • the corresponding work strategy is determined to use the first cleaning component to clean the current scene, and control the mobile device to use the first cleaning component to clean the current scene; in the current scene
  • the type of the scene is the wet area type
  • it is determined that the corresponding work strategy is to use the second cleaning component to clean the current scene; use the second cleaning component to clean the current scene.
  • the types of dry areas include but are not limited to: bedroom type, study type and/or living room type; correspondingly, the first cleaning component is a component for cleaning dust, such as a brush and the like.
  • Types of wet areas include but are not limited to: kitchen type and/or toilet type; correspondingly, the second cleaning component is a component for cleaning liquids, such as an absorbent mop and the like.
  • the corresponding working strategy may also be other working strategies, which are not listed one by one in this embodiment.
  • the control method provided by this embodiment from a mobile device acquires the scene image collected by the image acquisition component; performs image recognition on the scene image to obtain object information, and the object information refers to the information of the object located in the current scene. attribute information; obtain the scene recognition model trained based on the sample object information and the sample scene type corresponding to the sample object information; input the control object information into the scene recognition model to obtain the scene type of the current scene; it can solve the scene where the mobile device cannot judge the current scene The problem of type; because by using the scene recognition model, the scene type of the scene where the mobile device is currently located can be determined, the scene type can be recognized, and the intelligence of the mobile device can be improved.
  • the accuracy of the output result can be increased.
  • FIG. 11 is a block diagram of an apparatus for controlling a self-mobile device provided by an embodiment of the present application. This embodiment is described by taking the apparatus applied to the control component 810 in the self-mobile device shown in FIG. 8 as an example.
  • the apparatus at least includes the following modules: an image acquisition module 1110 , an image recognition module 1120 , a model acquisition module 430 and a device control module 1140 .
  • An image acquisition module 1110 configured to acquire a scene image collected by the image acquisition component during the movement of the self-mobile device, where the scene image is an image of the current scene where the self-mobile device is located;
  • An image recognition module 1120 configured to perform image recognition on the scene image to obtain object information, where the object information refers to attribute information of objects located in the current scene;
  • a model acquisition module 1130 configured to acquire a scene recognition model, where the scene recognition model is obtained by training based on the sample object information and the sample scene type corresponding to the sample object information;
  • the device control module 1140 is configured to control the object information to be input into the scene recognition model to obtain the scene type of the current scene.
  • the self-mobile device provided in the above embodiment performs scene recognition
  • only the division of the above-mentioned functional modules is used as an example for illustration.
  • the above-mentioned functions can be allocated by different functional modules as required. , that is, dividing the internal structure of the self-mobile device into different functional modules to complete all or part of the functions described above.
  • the method for controlling a self-mobile device provided by the above embodiments belongs to the same concept as the embodiment of the control apparatus for a self-mobile device, and the specific implementation process thereof is detailed in the method embodiment, which will not be repeated here.
  • FIG. 12 is a schematic structural diagram of a self-mobile device provided by an embodiment of the present application. As shown in FIG. 12 , the system at least includes: a control component 1210 and an image acquisition component 1220 communicatively connected to the control component 1210 .
  • the image acquisition component 1220 is configured to acquire the environmental image 1230 during the movement of the mobile device; and send the environmental image 1230 to the control component 1210 .
  • the image acquisition component 1220 may be implemented as a camera, a video camera, or the like, and the implementation manner of the image acquisition component 1220 is not limited in this embodiment.
  • the field of view angle of the image acquisition component 1220 is 120° in the horizontal direction and 60° in the vertical direction; of course, the field of view angle can also be other values, and this embodiment does not set the field of view of the image acquisition component 120 The value of the angle is limited.
  • the field of view of the image capture component 1220 can ensure that the environment image 1230 in the direction of travel can be captured from the mobile device.
  • the number of image acquisition components 1220 may be one or more, and this embodiment does not limit the number of image acquisition components 1220 .
  • the control component 1210 is used to control the self-mobile device. For example: control the start and stop of the self-mobile device; control the start and stop of each component in the self-mobile device (eg, the image capture component 1220 ).
  • control component 1210 is communicatively connected to a memory; the memory stores a program, and the program is loaded and executed by the control component 1210 to implement at least the following steps: for acquiring the edge information 1240 of the work area where the mobile device is located; acquiring The environment image 1230 collected while moving; the access door information is obtained based on the environment image 1230 ; the independent area 1250 in the working area is divided based on the access door information and edge information 1240 .
  • the program is loaded and executed by the control component 1210 to implement the control method from the mobile device provided by the present application.
  • the independent area 1250 refers to an area in the work area that has different attributes from other areas. For example: bedroom area, living room area, dining area, kitchen area, etc. in the room. Since different independent areas are usually divided by access doors and walls, the boundary line between the wall and the ground can be obtained through the edge information, and the position of the access door can be obtained through the access door information. Therefore, the edge information and the access door can be obtained by The information is combined for regional division.
  • the edge information 1240 refers to the information of the ground boundary of each independent area, and the edge information includes the position and length of the corresponding ground boundary.
  • the access door information refers to the access door information in each independent area, and the access door information includes the position information of the door frame and/or the door in the work area.
  • An access door is a door that allows people, mobile devices, and/or other objects to enter or exit a separate area.
  • the access door may be an open virtual door (ie, a door without isolation objects such as door panels), or a solid door, and the type of the access door is not limited in this embodiment.
  • the position information of the door frame and/or door in the work area refers to the geographic location of the projected position of the door frame and/or door in the work area in the vertical projection direction to the ground.
  • control component 1210 may further implement the following steps: determine the scene prediction result 1260 corresponding to the independent area 1250 based on the environment image 1230; The scene type 1270 of the independent area is determined.
  • the scene type 1270 is used to indicate the type of the independent area where the self-mobile device is currently located, and the division method of the scene type 1270 is set according to the working area of the self-mobile device. For example, if the working area of the mobile device is a room, the scene type 1270 includes: bedroom type, kitchen type, study room type, bathroom type, etc. This embodiment does not limit the division method of the scene type 170 .
  • the self-moving device may further include other components, such as: a moving component (such as a wheel) for driving the self-moving device to move, a mobile driving component (such as a wheel) for driving the moving component to move : motor), etc., wherein, the mobile drive component is connected to the control component 110 in communication, and under the control of the control component 1210, the mobile drive component operates and drives the mobile component to move, thereby realizing the overall movement of the self-moving device.
  • a moving component such as a wheel
  • a mobile driving component such as a wheel
  • the mobile drive component operates and drives the mobile component to move, thereby realizing the overall movement of the self-moving device.
  • the components included in the mobile device are not listed one by one.
  • the self-moving device may be a sweeping robot, an automatic lawn mower, or other devices with an automatic driving function, and the present application does not limit the device type of the self-moving device.
  • the working area is divided into a plurality of independent areas by combining the edge information and the access door information, which can solve the problem that the existing technology cannot realize the division of the working area; The effect of individualized work in separate areas.
  • the access door can be an open virtual door
  • the information of the virtual door and the edge information can be combined to obtain the area boundary of each independent area, thereby dividing the corresponding independent area, which can realize the open door scene. Region division to improve the accuracy of region division.
  • FIG. 13 is a flowchart of a control method for a self-moving device.
  • the control method for a self-moving device is used in the self-moving device shown in FIG. 1 , and the execution subject of each step is the control component 1210 as an example for description.
  • the method includes at least the following steps:
  • Step 1301 Obtain the edge information of the working area where the mobile device is located.
  • the edge information refers to the information of the ground boundary of each independent area, and the edge information includes the position and length of the corresponding ground boundary.
  • the edge information is the path information obtained by driving along the edge during the movement of the mobile device.
  • the self-moving device has a function of driving along the edge. Under this function, if the self-moving device drives along the boundary formed by the wall and the ground, the edge information of the boundary can be obtained; if the self-moving device runs along the object ( For example, when driving along the boundary formed by a cabinet, table, bed, etc.) and the ground, the edge information of the boundary can be obtained.
  • Step 1302 Acquire an environmental image collected during movement.
  • the working time (or time stamp) when moving in the working area and the environmental image corresponding to the working time are recorded from the mobile device. After work on the work area is complete, the environment image is read.
  • the mobile device reads the environment images in the time period corresponding to the start time to the end time from the stored environment images according to the start time and end time of the current work.
  • Step 1303 obtain access door information based on the environment image.
  • the access door refers to a passage for entering or leaving the independent area from the mobile device.
  • the access door may be a door frame, a fence gate, etc., and the type of the access door is not limited in this embodiment.
  • the access door includes a door frame in the work area.
  • an image recognition model is stored in the self-mobile device; the self-mobile device inputs an environment image into the image recognition model to obtain an object recognition result, where the object recognition result includes attribute information of the target object.
  • the object recognition result includes the access door information, it is determined that the environment image includes the image of the access door; when the object recognition result does not include the access door information, it is determined that the environment image does not include the image of the access door.
  • the target object includes a passage door.
  • the image recognition model is trained using sample environment images and sample object results based on a small network model.
  • the computing resources occupied by the image recognition model when running are lower than the maximum computing resources provided by the self-mobile device.
  • the image recognition model is trained using training data based on a small network model.
  • the training data includes training images of various objects in the working area of the mobile device and the recognition result of each training image.
  • the small network model refers to a network model in which the number of model layers is less than the first value; and/or the number of nodes in each layer is less than the second value.
  • the first numerical value and the second numerical value are both small integers.
  • the small network model is a miniature YOLO model; or, the MobileNet model.
  • the small network model may also be other models, which are not listed one by one in this embodiment.
  • the self-mobile device may further perform model compression processing on the image recognition model.
  • Model compression processing includes, but is not limited to: model clipping, model quantization, and/or low-yield decomposition, etc.
  • the object recognition result includes attribute information of the target object, such as: the type of the object, the size of the object, and the position information of the object in the environmental image.
  • the target object may also include household items such as a bed, a table, and a sofa, and the type of the target object is not limited in this embodiment.
  • acquiring the position information of the access door in the working area includes: acquiring the first distance between the access door and the self-mobile device in the environmental image output by the image recognition model; The second distance between the borders; determine the position of the access door relative to the border indicated by the edge information according to the first distance and the second distance, and obtain the position information of the access door.
  • a positioning component is installed on the self-mobile device, and the first distance between the access door in the environmental image output by the image recognition model and the self-mobile device is obtained from the mobile device; the positioning information obtained by the positioning component when the environmental image is collected is obtained. ; Obtain the position information of the access door according to the first distance and the positioning information.
  • the manner of acquiring the position information of the access door from the mobile device may also be other manners, which are not listed one by one in this embodiment.
  • Step 1304 Divide the independent areas in the working area based on the access door information and the edge information.
  • one or more independent areas in the work area may not be completely closed, but are connected to other areas in the work area, such as areas such as open kitchens, at this time, only the edge information from the mobile device cannot be used. Distinguish this independent area from other areas. Based on this, in this embodiment, by combining the edge information with the access door information, an open independent area can be determined, thereby improving the accuracy of area division.
  • Divide the independent areas in the work area based on the access door information and edge information including: obtaining the position information of the corresponding access door in the work area indicated by the access door information; combining the edge information and the position information to obtain the combined boundary information ; Divide each enclosed area formed by the combined boundary information into corresponding independent areas.
  • the edge information is obtained from the mobile device driving along the edge.
  • the sweeper obtains the edge information of the house after cleaning the user's entire house; after that, obtains the environmental images collected during the cleaning process, and identifies each environmental image; the environmental images include The position information of the access door is obtained when the image of the access door is obtained; the position information and the edge information can be combined to obtain a plurality of closed figures, and each closed figure corresponds to an independent area.
  • the control method provided by this embodiment from a mobile device obtains the edge information of the working area where the mobile device is located; obtains the environmental image collected during movement; obtains access door information based on the environmental image; information and edge information to divide the independent areas in the work area; it can solve the problem that the existing technology cannot realize the division of the work area; it can realize the division of the work area, and achieve the effect of personalized work according to the divided independent areas .
  • the access door can be an open virtual door
  • the information of the virtual door and the edge information can be combined to obtain the area boundary of each independent area, thereby dividing the corresponding independent area, which can realize the open door scene. Region division to improve the accuracy of region division.
  • an image recognition model for identifying the access door is obtained, which can further reduce the computing resources occupied by the image recognition model when running, improve the recognition speed, and reduce the hardware requirements of mobile devices.
  • the self-mobile device may also identify the scene type of each independent area.
  • the control method of the self-mobile device further includes the following steps:
  • Step 1401 Determine the scene prediction result corresponding to the independent area based on the environment image.
  • the scene prediction result is used to indicate the scene type predicted from the mobile device based on the relevant information of the single independent area.
  • the scenario prediction result can be one or more scenario types.
  • the scene type is used to indicate the type of independent area in which the mobile device is currently located.
  • the division method of scene types is set according to the working area of the mobile device. For example, if the working area of the self-mobile device is a room, the scene types include: bedroom type, kitchen type, study room type, bathroom type, etc. This embodiment does not limit the division method of scene types.
  • determining the scene prediction result corresponding to the independent area based on the environmental image includes: acquiring an image recognition model; for each independent area, inputting the environmental image corresponding to the independent area into the image recognition model to obtain the object recognition result; acquiring the scene recognition model ; Input the object recognition result into the scene recognition model to obtain the scene prediction result, where the scene prediction result includes at least one prediction scene type of the independent area.
  • step 1303 For the relevant description of the image recognition model, refer to step 1303, which is not repeated here in this embodiment.
  • the scene recognition model is trained based on the probability model using the sample attribute information of the object and the sample scene.
  • the scene prediction result output by the scene recognition model includes multiple scene types.
  • the scene recognition model also outputs a confidence level corresponding to each scene type. Confidence is used to indicate how accurate each scene type is output.
  • Step 1402 Determine the scene type of the independent area according to the scene prediction result of the independent area.
  • determining the scene type of the independent area according to the scene prediction result of the independent area includes: obtaining the pose information of each independent area; combining the scene area result of each independent area and the pose information of each independent area,
  • the scene type of each independent area is determined according to the preset probability distribution strategy. Among them, the probability distribution strategy is used for each target scene type to determine from each independent area that the scene type is the independent area with the highest probability of the target scene type.
  • the pose information includes the position information and orientation information of the corresponding independent area in the working area.
  • the direction information may be the direction of the access door in the corresponding independent area.
  • the self-mobile device can determine the scene type of each independent area according to the scene prediction results of the multiple independent areas according to the preset probability distribution strategy;
  • the probability distribution strategy is: there is corresponding template pose information for each scene type; the pose information of the independent area is compared with the template pose information of each scene type to obtain a pose comparison result; A target scene type is obtained, and the sum of the scene prediction results and pose comparison results corresponding to the target scene type is multiplied by the corresponding weights to obtain a probability result; the type of the independent area with the highest probability result is determined as the scene type.
  • the edge information in the work area is obtained from the mobile device after the work is completed in the work area; the environmental image collected by the image acquisition component in the work process is input into the image recognition model 41 to obtain object information 42; combined with the object information 42
  • the access door information and edge information in the work area are divided into multiple independent areas 1250; the scene prediction results 1260 of each independent area are obtained after the object information is input into the scene recognition model; the scene prediction results 1260 of multiple independent areas are combined based on probability
  • the distribution strategy results in a scene type 1270 for each individual area.
  • the scene prediction result corresponding to the independent area is determined based on the environment image; the scene type of the independent area is determined according to the scene prediction result of the independent area,
  • the self-mobile device can be made to recognize the scene type of each independent area in the entire work area, provide more information for the user, and make the self-mobile device more intelligent.
  • FIG. 16 is a block diagram of an apparatus for controlling a self-mobile device provided by an embodiment of the present application. This embodiment is described by taking the device applied to the control component 1210 in the control system of the self-mobile device shown in FIG. 12 as an example.
  • the apparatus includes at least the following modules: a first information acquisition module 1610 , an environment image acquisition module 1620 , a second information acquisition module 1630 and a region division control module 1640 .
  • a first information acquisition module 1610 configured to acquire edge information of the work area where the self-mobile device is located
  • An environmental image acquisition module 1620 configured to acquire an environmental image collected during movement
  • a second information obtaining module 1630 configured to obtain access door information based on the environment image
  • the area division control module 1640 is configured to divide the independent areas in the working area based on the access door information and the edge information.
  • control device for self-mobile equipment when the control device for self-mobile equipment provided in the above-mentioned embodiments controls self-mobile equipment, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated as required. It is completed by different functional modules, that is, the internal structure of the control device of the self-mobile device is divided into different functional modules, so as to complete all or part of the functions described above.
  • the control apparatus for a self-mobile device provided by the above-mentioned embodiment and the control method for a self-mobile device belong to the same concept, and the specific implementation process thereof is detailed in the method embodiment, which will not be repeated here.
  • FIG. 17 is a block diagram of a control apparatus for a self-moving device provided by an embodiment of the present application.
  • the apparatus may be the self-moving device shown in FIG. 1 , FIG. 8 , or FIG. 12 , and of course, may also be other devices installed on the self-moving device.
  • the apparatus includes at least a processor 1701 and a memory 1702 .
  • the processor 1701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like.
  • the processor 1701 can use at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, programmable logic array) accomplish.
  • the processor 1701 may also include a main processor and a co-processor.
  • the main processor is a processor used to process data in the wake-up state, also called CPU (Central Processing Unit, central processing unit); the co-processor is A low-power processor for processing data in a standby state.
  • the processor 801 may further include an AI (Artificial Intelligence, artificial intelligence) processor, where the AI processor is used to process computing operations related to machine learning.
  • AI Artificial Intelligence, artificial intelligence
  • Memory 1702 may include one or more computer-readable storage media, which may be non-transitory.
  • Memory 802 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices.
  • a non-transitory computer-readable storage medium in the memory 1702 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1701 to implement the self-movement provided by the method embodiments in this application. Device control method.
  • control apparatus of the self-moving device may optionally further include: a peripheral device interface and at least one peripheral device.
  • the processor 1701, the memory 1702 and the peripheral device interface can be connected through a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface through bus, signal line or circuit board.
  • peripheral devices include, but are not limited to, radio frequency circuits, touch display screens, audio circuits, and power supplies.
  • control apparatus of the self-moving device may further include fewer or more components, which is not limited in this embodiment.
  • the present application further provides a computer-readable storage medium, where a program is stored in the computer-readable storage medium, and the program is loaded and executed by a processor to implement the control method from the mobile device according to the above method embodiment. .
  • the present application further provides a computer product, the computer product includes a computer-readable storage medium, and a program is stored in the computer-readable storage medium, and the program is loaded and executed by a processor to implement the above method embodiments The control method of the self-mobile device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Mechanical Engineering (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Electromagnetism (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Robotics (AREA)
  • Signal Processing (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Automatic Disk Changers (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Train Traffic Observation, Control, And Security (AREA)
  • Electric Vacuum Cleaner (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及一种自移动设备的控制方法、装置、存储介质及自移动设备，属于计算机技术领域，该方法包括：在所述自移动设备移动过程中，获取所述图像采集组件采集的环境图像；获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；控制所述环境图像输入所述图像识别模型得到对象识别结果，所述对象识别结果用于指示目标对象的类别；可以解决现有的图像识别算法对扫地机器人的硬件要求高，导致扫地机器人的对象识别功能应用范围受限的问题；通过使用消耗计算资源较少的图像识别模型来识别环境图像中的目标对象，可以降低对象识别方法对自移动设备的硬件要求，扩大对象识别方法的应用范围。

Description

自移动设备的控制方法、装置、存储介质及自移动设备 技术领域
本申请涉及自移动设备的控制方法、装置、存储介质及自移动设备，属于计算机技术领域。
背景技术
随着人工智能以及机器人行业的发展，扫地机器人等智能家用电器逐渐普及。
常见的扫地机器人通过固定在机身上方的摄像组件采集环境图片，使用图像识别算法识别采集图片中的物品。为了保证图像识别精度，该图像识别算法通常是基于神经网络模型等训练得到的。
然而，现有的图像识别算法需要图形处理器(Graphics Processing Unit，GPU)和神经网络处理器(Neural Processing Unit，NPU)的结合实现，对扫地机器人的硬件要求较高。
发明内容
本申请提供了一种自移动设备的控制方法、装置及存储介质，可以解决现有的图像识别算法对扫地机器人的硬件要求高，导致扫地机器人的对象识别功能应用范围受限的问题。本申请提供如下技术方案：
第一方面，提供了一种自移动设备的控制方法，所述自移动设备上安装有图像采集组件，所述方法包括：
获取所述图像采集组件采集的环境图像；
获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
控制所述环境图像输入所述图像识别模型得到对象识别结果，所述对象识别结果用于指示目标对象的类别。
可选地，所述图像识别模型是对小网络检测模型进行训练得到的。
可选地，所述获取图像识别模型之前，还包括：
获取小网络检测模型；
获取训练数据，所述训练数据包括所述自移动设备的工作区域中的各个对象的训练图像和每张训练图像的识别结果；
将所述训练图像输入所述小网络检测模型，得到模型结果；
基于所述模型结果与所述训练图像对应的识别结果之间的差异对所述小网络检测模型进行训练，得到所述图像识别模型。
可选地，所述基于所述模型结果与所述训练图像对应的识别结果之间的差异对所述小网络检测模型进行训练，得到所述图像识别模型之后，还包括：
对所述图像识别模型进行模型压缩处理，得到用于识别对象的图像识别模型。
可选地，所述小网络检测模型为：微型的YOLO模型；或者，MobileNet模型。
可选地，所述控制所述环境图像输入所述图像识别模型得到对象识别结果之后，还包括：
基于所述对象识别结果，控制所述自移动设备移动以完成对应的任务。
可选地，所述自移动设备上安装有液体清扫组件，所述基于所述对象识别结果，控制所述自移动设备移动以完成对应的任务，包括：
当所述对象识别结果指示所述环境图像包含液体图像时，控制所述自移动设备移动至所述液体图像对应的待清洁区域；
使用所述液体清扫组件清扫所述待清洁区域中的液体。
可选地，所述自移动设备中安装有供电组件，所述供电组件使用充电组件进行充电，所述基于所述对象识别结果，控制所述自移动设备移动以完成对应的任务，包括：
当所述供电组件的剩余电量小于或等于电量阈值、且所述环境图像包括所述充电组件的图像时，根据所述充电组件的图像位置确定所述充电组件的实际位置。
可选地，所述自移动设备上还安装有定位传感器，所述定位传感器用于定位所述充电组件上充电接口的位置；所述控制所述自移动设备向所述充电组件移动之后，还包括：
在向所述充电组件移动过程中，控制所述定位传感器对所述充电组件的位置进行定位得到定位结果；
控制所述自移动设备按照所述定位结果移动，以实现所述自移动设备与所述充电接口对接。
第二方面，提供了一种自移动设备的控制装置，所述自移动设备上安装有图像采集组件，所述装置包括：
图像获取模块，用于在所述自移动设备移动过程中，获取所述图像采集组件采集的环境图像；
模型获取模块，用于获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
设备控制模块，用于控制所述环境图像输入所述图像识别模型得到对象识别结果，所述对象识别结果用于指示目标对象的类别。
第三方面，提供了一种自移动设备的控制方法，所述自移动设备上安装有图像采集组件，所述方法包括：
在所述自移动设备移动过程中，获取所述图像采集组件采集的场景图像，所述场景图像为所述自移动设备所处的当前场景的图像；
对所述场景图像进行图像识别得到对象信息，所述对象信息是指位于所述当前场景中的对象的属性信息；
获取场景识别模型，所述场景识别模型是基于样本对象信息和所述样本对象信息对应的样本场景类型训练得到的；
控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型。
可选地，所述场景识别模型是对概率模型进行训练得到的。
可选地，所述获取场景识别模型之前，还包括：
获取所述概率模型；
获取训练数据，所述训练数据包括各个对象的样本属性信息和各个样本属性信息对应的样本场景类型；
将所述样本属性信息输入所述概率模型，得到模型结果；
基于所述模型结果与所述样本场景类型之间的差异对所述概率模型进行训练，得到所述场景识别模型。
可选地，所述对所述场景图像进行图像识别得到对象信息，还包括：
获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
将所述场景图像输入所述图像识别模型得到对象识别结果。
可选地，所述图像识别模型为基于小网络模型使用样本场景图像和样本对象结果训练得到的。
可选地，所述图像识别模型是经过模型压缩处理后的得到的。
可选地，所述场景识别模型还输出每个场景类型的置信度；所述控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型之后，还包括：
将所述场景类型按照置信度由高到低的顺序进行排序；
输出排序在前N位的场景类型，所述N为大于1的整数。
可选地，所述控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型之后，还包括：
根据所述场景类型确定对应的清扫策略；
控制所述自移动设备按照所述清扫策略执行清扫工作。
第四方面，提供了一种自移动设备的场景识别装置，所述自移动设备上安装有图像采集组件，所述装置包括：
图像获取模块，用于在所述自移动设备移动过程中，获取所述图像采集组件采集的场景图像，所述场景图像为所述自移动设备所处的当前场景的图像；
图像识别模块，用于对所述场景图像进行图像识别得到对象信息，所述对象信息是指位于所述当前场景中的对象的属性信息；
模型获取模块，用于获取场景识别模型，所述场景识别模型是基于样本对象信息和所述样本对象信息对应的样本场景类型训练得到的；
设备控制模块，用于控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型。
第五方面，提供了一种自移动设备的控制方法，方法包括：
获取所述自移动设备所处工作区域的边缘信息；
获取移动时采集到的环境图像；
基于所述环境图像获得通行门信息；
基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分。
可选地，所述基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分之后，还包括：
基于所述环境图像确定对应独立区域的场景预测结果；
根据所述独立区域的场景预测结果确定所述独立区域的场景类型。
可选地，所述基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分，包括：
获取所述通行门信息指示的对应通行门在所述工作区域中的位置信息；
将所述边缘信息和所述位置信息结合，得到结合后的边界信息；
将所述结合后的边界信息构成的各个封闭区域划分为对应的独立区域。
可选地，所述基于所述环境图像获得通行门信息，包括：
识别所述环境图像是否包括通行门的图像；
在所述环境图像包括通行门的图像时，获取所述通行门在所述工作区域中的位置信息。
可选地，所述通行门包括所述工作区域中的门框。
可选地，所述根据所述独立区域的场景预测结果确定所述独立区域的场景类型，包括：
获取每个独立区域的位姿信息，所述位姿信息包括对应独立区域在所述工作区域内的位置信息和方向信息；
结合每个独立区域的场景区域结果和每个独立区域的位姿信息，按照预先设置的概率分布策略确定各个独立区域的场景类型；
其中，概率分布策略用于对于每种目标场景类型，从各个独立区域中确定场景类型是所述目标场景类型的概率最高的独立区域。
可选地，所述基于所述环境图像确定对应独立区域的场景预测结果，包括：
获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
对于每个独立区域，将所述独立区域对应的环境图像输入所述图像识别模型获得对象识别结果，所述对象识别结果包括目标对象的属性信息；
获取场景识别模型，所述场景识别模型是使用对象的样本属性信息和样本场景类型训练得到的；
将所述对象识别结果输入所述场景识别模型，得到场景预测结果，所述场景预测结果包括所述独立区域的至少一种预测场景类型。
可选地，所述图像识别模型为基于小网络模型使用样本环境图像和样本对象结果训练得到的。
可选地，所述场景识别模型为基于概率模型使用对象的样本属性信息和样本场景训练得到的。
第六方面，提供了一种自移动设备的控制装置，装置包括：
第一信息获取模块，用于获取所述自移动设备所处工作区域的边缘信息；
环境图像获取模块，用于获取移动时采集到的环境图像；
第二信息获取模块，用于基于所述环境图像获得通行门信息；
区域划分控制模块，用于基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分
第七方面，提供一种自移动设备的控制装置，所述装置包括处理器和存储器；所述存储器中存储有程序，所述程序由所述处理器加载并执行以实现第一方面、第三方面或第五方面 所述的自移动设备的控制方法。
第八方面，提供一种计算机可读存储介质，所述存储介质中存储有程序，所述程序由所述处理器加载并执行以实现第一方面、第三方面或第五方面所述的自移动设备的控制方法。
第九方面，提供一种自移动设备，包括：
用于带动所述自移动设备移动的移动组件；
用于驱动所述移动组件运动的移动驱动组件；
安装在所述自移动设备上、用于采集行进方向上的环境图像的图像采集组件；
与所述移动驱动组件和所述图像采集组件通信相连的控制组件，所述控制组件与存储器通信相连；所述存储器中存储有程序，所述程序由所述控制组件加载并执行以实现第一方面、第三方面或第五方面所述的自移动设备的控制方法。
本申请的有益效果在于：通过在自移动设备移动过程中，获取图像采集组件采集的环境图像；获取图像识别模型，图像识别模型运行时占用的计算资源低于自移动设备提供的最大计算资源；控制环境图像输入图像识别模型得到对象识别结果，对象识别结果用于指示环境图像中目标对象的类别；可以解决现有的图像识别算法对扫地机器人的硬件要求高，导致扫地机器人的对象识别功能应用范围受限的问题；通过使用消耗计算资源较少的图像识别模型来识别环境图像中的目标对象，可以降低对象识别方法对自移动设备的硬件要求，扩大对象识别方法的应用范围。
上述说明仅是本申请技术方案的概述，为了能够更清楚了解本申请的技术手段，并可依照说明书的内容予以实施，以下以本申请的较佳实施例并配合附图详细说明如后。
附图说明
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是本申请一个实施例提供的自移动设备的结构示意图；
图2是本申请一个实施例提供的自移动设备的控制方法的流程图；
图3是本申请一个实施例提供的执行工作策略的流程图；
图4是本申请一个实施例提供的执行工作策略的示意图；
图5是本申请另一个实施例提供的执行工作策略的流程图；
图6是本申请另一个实施例提供的执行工作策略的示意图；
图7是本申请一个实施例提供的自移动设备的控制装置的框图；
图8是本申请一个实施例提供的自移动设备的结构示意图；
图9是本申请一个实施例提供的自移动设备的控制方法的流程图；
图10是本申请另一个实施例提供的自移动设备的控制方法的流程图；
图11是本申请一个实施例提供的自移动设备的控制装置的框图；
图12是本申请一个实施例提供的自移动设备结构示意图；
图13是本申请一个实施例提供的自移动设备的控制方法的流程图；
图14是本申请另一个实施例提供的自移动设备的控制方法的流程图；
图15是本申请一个实施例提供的自移动设备的控制的示意图；
图16是本申请一个实施例提供的自移动设备的控制装置的框图；
图17是本申请一个实施例提供的自移动设备的控制装置的框图。
具体实施方式
下面结合附图和实施例，对本申请的具体实施方式作进一步详细描述。以下实施例用于说明本申请，但不用来限制本申请的范围。
首先，以下对本申请涉及的若干名词进行介绍。
模型压缩：是指降低训练后的网络模型中的参数冗余，从而减少网络模型的存储占用、通信带宽和计算复杂度的方式。
模型压缩包括但不限于：模型裁剪、模型量化和/或低轶分解。
模型裁剪：是指最优网络结构的搜索过程。模型裁剪过程包括：1、训练网络模型；2、剪裁不重要的权重或通道；3、微调或再训练已剪裁的网络。其中，第2个步骤通常借助迭代式逐层剪裁、快速微调或者权重重建以保持精度。
量化：量化模型是一种模型加速方法的总称，它是以更少位数的数据类型表示有限范围(如32位)浮点型数据的过程，从而达到减少模型尺寸大小、减少模型内存消耗及加快模型推理速度等目标。
低秩分解：将网络模型的权重矩阵分解成多个小矩阵，小矩阵计算量比原始矩阵的计算量小，以达到减少模型运算量，降低模型占用内存的目的。
YOLO模型：基础网络模型之一，是通过卷积神经网络(Convolutional Neural Networks，CNN)网络就能够实现目标的定位和识别的神经网络模型。YOLO模型包括YOLO、YOLO v2和YOLO v3。其中，YOLO v3是YOLO和YOLO v2之后的YOLO系列的又一种目标检测算法，是基于YOLO v2的改进。而YOLO v3-tiny为YOLO v3的简化版本，在YOLO v3的基础上去掉 某些特征层，从而达到了模型运算量减少，运算更快的效果。
MobileNet模型：是基本单元是深度级可分离卷积(depthwise separable convolution)的网络模型。其中，深度级可分离卷积可以分解为深度可分卷积(Depthwise，DW)和逐点卷积(Pointwise，PW)。DW和标准卷积不同，对于标准卷积其卷积核是用在所有的输入通道上的，而DW针对每个输入通道采用不同的卷积核，就是说一个卷积核对应一个输入通道。而PW就是普通的卷积，只不过其采用1x1的卷积核。对于深度级可分离卷积，其首先是采用DW对不同输入通道分别进行卷积，然后采用PW将上面的输出再进行结合，这样其实整体计算结果和一个标准卷积过程的计算结果近似相同，但是会大大减少计算量和模型参数量。
实施例1
图1是本申请一个实施例提供的自移动设备的结构示意图，如图1所示，该系统至少包括：控制组件110、与控制组件110通信相连的图像采集组件120。
图像采集组件120用于采集自移动设备移动过程中的环境图像130；并将该环境图像130发送至控制组件110。可选地，图像采集组件120可以实现为照相机、摄像机等，本实施例不对图像采集组件120的实现方式作限定。
可选地，图像采集组件120的视场角在水平方向上为120°、竖直方向上为60°；当然，视场角也可以为其它数值，本实施例不对图像采集组件120的视场角的取值作限定。图像采集组件120的视场角可以保证能够采集到自移动设备的在行进方向上的环境图像130。
另外，图像采集组件120的数量可以是一个或多个，本实施例不对图像采集组件120的数量作限定。
控制组件110用于对自移动设备进行控制。比如：控制自移动设备的启动、停止；控制自移动设备中的各个组件(如图像采集组件120)的启动、停止等。
本实施例中，控制组件110与存储器通信相连；该存储器中存储有程序，该程序由控制组件110加载并执行至少实现以下步骤：在自移动设备移动过程中，获取图像采集组120件采集的环境图像130；获取图像识别模型；控制环境图像130输入图像识别模型得到对象识别结果140，该对象识别结果140用于指示环境图像130中目标对象的类别。换句话说，该程序由控制组件110加载并执行以实现本申请提供的自移动设备的控制方法。
在一个示例中，在环境图像中包括目标对象时，对象识别结果140为该目标对象的类型；在环境图像中包括目标对象时，对象识别结果140为空。或者，在环境图像中包括目标对象时，对象识别结果140为包括目标对象的指示(比如：通过“1”指示包括目标对象)和该目标对象的类型；在环境图像中不包括目标对象时，对象识别结果140为不包括目标对象的指示 (比如：通过“0”指示不包括目标对象)。
其中，图像识别模型运行时占用的计算资源低于自移动设备提供的最大计算资源。
可选地，对象识别结果140还可以包括但不限于：目标对象的图像在环境图像130中的位置、尺寸等信息。
可选地，目标对象是位于自移动设备的工作区域中的对象。比如：自移动设备的工作区域为房间时，目标对象可以为房间中的床、桌子、椅子、人等对象；自移动设备的工作区域为物流仓库时，目标对象可以为仓库中的箱子、人等，本实施例不对目标对象的类型作限定。
可选地，图像识别模型为模型层数小于第一数值；和/或，每层中的节点数量小于第二数值的网络模型。其中，第一数值和第二数值均为较小的整数，从而保证图像识别模型在运行时消耗较少的计算资源。
需要补充说明的是，本实施例中，自移动设备还可以包括其它组件，比如：用于带动自移动设备移动的移动组件(比如：车轮)、用于驱动移动组件运动的移动驱动组件(比如：电机)等，其中，移动驱动组件与控制组件110通信相连，在控制组件110的控制下移动驱动组件运行并带动移动组件运动，从而实现自移动设备的整体运动，本实施例在此对自移动设备包括的组件不再一一列举。
另外，自移动设备可以为扫地机器人、自动割草机或者其它具有自动行驶功能的设备，本申请不对自移动设备的设备类型作限定。
本实施例中，通过使用消耗计算资源较少的图像识别模型来识别环境图像130中的目标对象，可以降低对象识别方法对自移动设备的硬件要求，扩大对象识别方法的应用范围。
实施例2
下面对本申请提供的自移动设备的控制方法进行详细介绍。
图2中以该自移动设备的控制方法用于图1所示的自移动设备中、且各个步骤的执行主体为控制组件110为例进行说明，参考图2，该方法至少包括以下几个步骤：
步骤201，在自移动设备移动过程中，获取图像采集组件采集的环境图像。
可选地，图像采集组件用于采集视频数据，此时，环境图像可以为该视频数据中的一帧图像数据；或者，图像采集组件用于采集单张的图像数据，此时，环境图像为图像采集组件发送的单张的图像数据。
步骤202，获取图像识别模型，图像识别模型运行时占用的计算资源低于自移动设备提供的最大计算资源。
本实施例中，通过使用计算资源低于自移动设备提供的最大计算资源的图像识别模型， 可以降低图像识别模型对自移动设备的硬件要求，扩大对象识别方法的应用范围。
在一个示例中，自移动设备读取预先训练得到的图像识别模型。此时，图像识别模型是对小网络检测模型进行训练得到的。对小网络检测模型进行训练，包括：获取小网络检测模型；获取训练数据；将训练图像输入小网络检测模型，得到模型结果；基于模型结果与训练图像对应的识别结果之间的差异对小网络检测模型进行训练，得到图像识别模型。
其中，训练数据包括自移动设备的工作区域中的各个对象的训练图像和每张训练图像的识别结果。
本实施例中，小网络模型是指模型层数小于第一数值；和/或，每层中的节点数量小于第二数值的网络模型。其中，第一数值和第二数值均为较小的整数。比如：小网络检测模型为：微型的YOLO模型；或者，MobileNet模型。当然，小网络检测模型也可以是其它模型，本实施例在此不再一一列举。
可选地，为了进一步地压缩图像识别模型运行时占用的计算资源，在对小网络检测模型进行训练得到图像识别模型之后，自移动设备还可以对图像识别模型进行模型压缩处理，得到用于识别对象的图像识别模型。
可选地，模型压缩处理包括但不限于：模型裁剪、模型量化和/或低轶分解等。
可选地，在对模型进行压缩处理后，自移动设备可以再次使用训练数据对压缩后的图像识别模型进行训练，以提高图像识别模型的识别精度。
步骤203，控制环境图像输入图像识别模型得到对象识别结果，该对象识别结果用于指示目标对象的类别。
可选地，对象识别结果还包括但不限于：目标对象的图像在环境图像中的位置、和/或尺寸等信息。
综上所述，本实施例提供的自移动设备的控制方法，通过在自移动设备移动过程中，获取图像采集组件采集的环境图像；获取图像识别模型，图像识别模型运行时占用的计算资源低于自移动设备提供的最大计算资源；控制环境图像输入图像识别模型得到对象识别结果，对象识别结果用于指示环境图像中目标对象的类别；可以解决现有的图像识别算法对扫地机器人的硬件要求高，导致扫地机器人的对象识别功能应用范围受限的问题；通过使用消耗计算资源较少的图像识别模型来识别环境图像中的目标对象，可以降低对象识别方法对自移动设备的硬件要求，扩大对象识别方法的应用范围。
另外，通过采用小网络模型训练学习得到图像识别模型，不需要图形处理器(Graphics Processing Unit，GPU)和嵌入式神经网络处理器(Neural-network Processing Units，NPU) 的结合才能实现对象识别过程，因此，可以降低对象识别方法对设备硬件的要求。
另外，对图像识别模型进行模型压缩处理，得到用于识别对象的图像识别模型；可以进一步缩小图像识别模型运行时占用的计算资源，提高识别速度，扩大对象识别方法的应用范围。
可选地，基于上述实施例，本申请中，自移动设备得到对象识别结果后，还会基于该对象识别结果控制自移动设备移动以完成对应的任务。该任务包括但不限于：实现对某些物品进行避障的任务，例如对椅子、宠物粪便等进行避障；对某些物品进行定位的任务，例如对门窗、充电组件等进行定位的任务；对人进行监控和跟随的任务；对特定物品进行清扫的任务，如对液体进行清扫；和/或，自动回充任务。下面，对不同的对象识别结果对应的执行的任务进行介绍。
可选地，自移动设备上安装有液体清扫组件。此时，在步骤203之后，基于对象识别结果控制自移动设备移动以完成对应的任务，包括：当对象识别结果指示环境图像包含液体图像时，控制自移动设备移动至液体图像对应的待清洁区域；使用液体清扫组件清扫待清洁区域中的液体。
在一个示例中，液体清扫组件包括安装于自移动设备的轮体外围的吸水拖布。当环境图像中存在液体图像时，控制自移动设备向液体图像对应的待清洁区域移动，使得自移动设备的轮体经过待清洁区域，从而使得吸水拖布吸收地面的液体。自移动设备中还设置有清洗池和蓄水池；清洗池位于轮体下方；水泵抽吸蓄水池中的水，通过管道从喷嘴喷到轮体上，将吸水拖布上的污垢冲刷至清洗池。轮体上还设有压辊，以拧干吸水拖布。
当然，上述液体清扫组件仅是示意性的，在实际实现时，液体清扫组件也可以通过其它方式实现，本实施例在此不再一一列举。
为了更清楚地理解基于对象识别结果执行对应的工作策略的方式，参考图3和图4所示的执行清扫液体工作策略的示意图，根据图3和图4可知，自移动设备采集到环境图像后，使用图像识别模型得到环境图像的对象识别结果；在该对象识别结果为当前环境包括液体时，使用液体清扫组件31对该液体进行清扫。
可选地，本实施例中，自移动设备可以为扫地机器人，此时，该自移动设备具有干湿垃圾统一清除的功能。
本实施例中，通过在环境图像中存在液体图像时，启动液体清扫组件，可以避免自移动设备对液体绕行，导致清扫任务无法完成的问题；可以提高自移动设备的清扫效果。同时，可以避免液体进入自移动设备内部，造成电路损坏，可以降低自移动设备损坏风险。
可选地，基于上述实施例，自移动设备中安装有供电组件。基于对象识别结果控制自移动设备移动以完成对应的任务，包括：当供电组件的剩余电量小于或等于电量阈值、且环境图像包括充电组件的图像时，自移动设备根据充电组件的图像位置确定充电组件的实际位置；控制自移动设备向充电组件移动。
由于自移动设备拍摄到充电组件的图像之后，可以将根据该图像在环境图像中的位置，确定出充电组件相对于自移动设备的方向，因此，自移动设备可以根据大致确定出的方向向充电组件移动。
可选地，为了提高自移动设备向充电组件移动的准确度，自移动设备上还安装有定位传感器，该定位传感器用于定位充电组件上充电接口的位置。此时，自移动设备在控制自移动设备向充电组件移动过程中，控制定位传感器对充电组件的位置进行定位得到定位结果；控制自移动设备按照定位结果移动，以实现自移动设备与充电接口对接。
在一个示例中，定位传感器为激光传感器。此时，充电组件充电接口发射不同角度的激光信号，定位传感器基于接收到的激光信号的角度差确定充电接口的位置。
当然，定位传感器可以为其它类型的传感器，本实施例不对定位传感器的类型作限定。
为了更清楚地理解基于对象识别结果执行对应的工作策略的方式，参考图5和图6所示的执行清扫液体工作策略的示意图，根据图5和图6可知，自移动设备采集到环境图像后，使用图像识别模型得到环境图像的对象识别结果；在该对象识别结果为当前环境包括充电组件51时，使用定位传感器52定位充电组件51上的充电接口53的位置；向充电接口53移动，以使自移动设备通过充电接口与充电组件51电连接实现充电。
本实施例中，通过图像识别模型识别充电组件，并移动至充电组件附近；可以实现自移动设备自动回归充电组件进行充电，提高自移动设备的智能化。
另外，通过定位传感器确定充电组件上充电接口的位置，可以提高自移动设备自动回归充电组件时的准确性，提高自动充电效率。
实施例3
图7是本申请一个实施例提供的自移动设备的控制装置的框图，本实施例以该装置应用于图1所示的自移动设备中为例进行说明。该装置至少包括以下几个模块：图像获取模块710、模型获取模块720和设备控制模块730。
图像获取模块710，用于在所述自移动设备移动过程中，获取所述图像采集组件采集的环境图像；
模型获取模块720，用于获取图像识别模型，所述图像识别模型运行时占用的计算资源 低于所述自移动设备提供的最大计算资源；
设备控制模块730，用于控制所述环境图像输入所述图像识别模型得到对象识别结果，所述对象识别结果用于指示目标对象的类别。
相关细节参考上述方法实施例。
需要说明的是：上述实施例中提供的自移动设备的控制装置在进行自移动设备的控制时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将自移动设备的控制装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的自移动设备的控制装置与自移动设备的控制方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
实施例4
图8是本申请一个实施例提供的自移动设备的结构示意图，如图8所示，该系统至少包括：控制组件810和控制组件810通信相连的图像采集组件820。
图像采集组件820用于采集场景图像；并将该场景图像发送至控制组件810。可选地，图像采集组件820可以实现为照相机、摄像机等，本实施例不对图像采集组件820的实现方式作限定。
可选地，图像采集组件820的视场角在水平方向上为20°、竖直方向上为60°；当然，视场角也可以为其他数值，本实施例不对图像采集组件120的视场角的取值作限定。图像采集组件820的视场角可以保证能够采集到自移动设备的在行进方向上的场景图像。
另外，图像采集组件820的数量可以是一个或多个，本实施例不对图像采集组件820的数量作限定。
控制组件810用于对自移动设备进行控制。比如：控制自移动设备的启动、停止；控制自移动设备中的各个组件(如图像采集组件820)的启动、停止等。
本实施例中，控制组件810与存储器通信相连；该存储器中存储有程序，该程序由控制组件810加载并执行至少实现以下步骤：在自移动设备移动过程中，获取图像采集组件820采集的场景图像830；对场景图像830进行图像识别得到对象信息840，该对象信息840是指位于当前场景中的对象的属性信息；获取场景识别模型；控制对象信息840输入场景识别模型，得到当前场景的场景类型850。换句话说，该程序由控制组件810加载并执行以实现本申请提供的自移动设备的控制方法。
其中，场景识别模型是基于样本对象信息840和样本对象信息840对应的样本场景类型850训练得到的。
可选地，对象的属性信息包括但不限于：对象的类型、对象的图像在场景图像830中的位置。
场景类型850用于指示自移动设备当前所处的工作环境的类型，场景类型850的划分方式根据自移动设备的工作环境设置。比如：自移动设备的工作环境为房间，则场景类型850包括：卧室类型、厨房类型、书房类型、卫生间类型等，本实施例不对场景类型850的划分方式作限定。
需要补充说明的是，本实施例中，自移动设备还可以包括其它组件，比如：用于带动自移动设备移动的移动组件(比如：车轮)、用于驱动移动组件运动的移动驱动组件(比如：电机)等，其中，移动驱动组件与控制组件810通信相连，在控制组件810的控制下移动驱动组件运行并带动移动组件运动，从而实现自移动设备的整体运动，本实施例在此对自移动设备包括的组件不再一一列举。
另外，自移动设备可以为扫地机器人、自动割草机或者其它具有自动行驶功能的设备，本申请不对自移动设备的设备类型作限定。
本实施例中，通过使用场景识别模型来识别场景类型，可以使得自移动设备识别出当前的工作环境的场景类型，为用户提供更多的信息。
实施例5
图9是本申请一个实施例提供的自移动设备的控制方法的流程图，本实施例以该方法应用于图8所示的自移动设备中，且各个步骤的执行主体为该系统中的控制组件810为例进行说明。该方法至少包括以下几个步骤：
步骤901，在自移动设备移动过程中，获取图像采集组件采集的场景图像，该场景图像为自移动设备所处的当前场景的图像。
可选地，图像采集组件用于采集视频数据，此时，场景图像可以为该视频数据中的一帧图像数据；或者，图像采集组件用于采集单张的图像数据，此时，场景图像为图像采集组件发送的单张的图像数据。
步骤902，对场景图像进行图像识别得到对象信息，该对象信息是指位于当前场景中的对象的属性信息。
在一个示例中，自移动设备中存储有图像识别模型。在对场景图像进行图像识别时，自移动设备获取图像识别模型；将场景图像输入该图像识别模型得到对象信息。
可选地，为了降低对象识别过程对自移动设备的硬件要求，图像识别模型运行时占用的计算资源低于自移动设备提供的最大计算资源。
在一个示例中，图像识别模型为基于小网络模型使用样本场景图像和样本对象结果训练得到的。其中，小网络模型是指模型层数小于第一数值；和/或，每层中的节点数量小于第二数值的网络模型。其中，第一数值和第二数值均为较小的整数。比如：小网络模型为微型的YOLO模型；或者，MobileNet模型。当然，小网络模型也可以是其它模型，本实施例在此不再一一列举。
可选地，为了进一步地压缩图像识别模型运行时占用的计算资源，在对小网络模型进行训练得到图像识别模型之后，自移动设备还可以对图像识别模型进行模型压缩处理，得到用于识别对象的图像识别模型。其中，模型压缩处理包括但不限于：模型裁剪、模型量化和/或低轶分解等。
当然，在其它实施例中，图像识别模型也可以为基于深度网络神经模型建立的，比如：基于卷积神经网络建立。
可选地，对象为自移动设备的工作区域中的对象。比如：桌子、床、椅子、沙发等。对象的属性信息包括对象的类型信息。对象的类型信息是基于自移动设备的工作环境划分的，比如：在自移动设备的工作环境为房间时，对象的类型信息包括：桌子类型、椅子类型、沙发类型、液体类型、充电站类型等，本实施例不对对象的类型划分方式作限定。
当然，对象的属性信息还可以包括其它信息，比如：对象在场景图像中的位置信息、对象的尺寸等，本实施例不对对象的属性信息包括的具体内容作限定。
可选地，在对模型进行压缩处理后，自移动设备可以再次使用训练数据对压缩后的图像识别模型进行训练，以提高图像识别模型的识别精度。
步骤903，获取场景识别模型，场景识别模型是基于样本对象信息和样本对象信息对应的样本场景类型训练得到的。
在一个示例中，自移动设备读取预先训练得到的场景识别模型，该场景识别模型是对概率模型进行训练得到的。此时，在获取场景识别模型之前，需要对概率模型进行训练，训练概率模型包括：获取概率模型；获取训练数据，该训练数据包括各个对象的样本属性信息和各个样本属性信息对应的样本场景类型；将样本属性信息输入概率模型，得到模型结果；基于模型结果与样本场景类型之间的差异对概率模型进行训练，得到场景识别模型。
本实施例中，通过使用基于样本对象信息和样本对象信息对应的样本场景训练得到的场景识别模型，可以为自移动设备提供识别场景类型的功能，可以提高自移动设备的智能性。
步骤904，控制对象信息输入场景识别模型，得到当前场景的场景类型。
可选地，场景类型是基于自移动设备的工作环境预先划分得到的。在一个示例中，自移 动设备的工作环境为房间，场景类型包括但不限于：厨房类型、客厅类型、卧室类型、卫生间类型、储物间类型、和/或书房类型，此时，当前场景的场景类型为预先划分好的场景类型中的一个或多个。
为了更清楚地理解本申请提供的场景识别方法，下面对该场景识别方法进行举例说明。参考图10，自移动设备使用图像采集组件820采集到场景图像后，将该场景图像输入图像识别模型，得到对象信息32；将对象信息输出场景识别模型后得到当前场景的场景类型33。
可选地，场景识别模型还会输出每个场景类型的置信度，置信度用于指示所输出的每个场景类型的准确度。此时，在步骤904之后，自移动设备将场景类型按照置信度由高到低的顺序进行排序；输出排序在前N位的场景类型。N为大于1的整数。
可选地，N的取值存在自移动设备中，N的值可以为3、2等，本实施例不对N的取值作限定。在场景识别模型输出的场景类型的个数小于N时，自动设备将所有的场景类型输出。
在一个示例中，用户终端与自移动设备通信相连，此时，输出前N位场景类型包括：将前N位场景类型发送至用户终端，以使用户终端显示将前N位场景类型。
在另一个示例中，自移动设备上安装有信息输出组件，比如：显示屏或者音频播放组件，此时，输出前N位场景类型包括：通过输出组件输出前N位场景类型。
可选地，自移动设备在确定出当前场景的场景类型之后，还可以根据当前场景的场景类型确定对应的工作策略；控制自移动设备按照清扫策略执行清扫工作。
在一个示例中，自移动设备为地面清洁设备。此时，在当前场景的场景类型为干燥区域类型时，确定对应的工作策略为使用第一清扫组件对当前场景进行清扫，控制自移动设备使用第一清扫组件对当前场景进行清扫；在当前场景的场景类型为湿润区域类型时，确定对应的工作策略为使用第二清扫组件对当前场景进行清扫；使用第二清扫组件对当前场景进行清扫。
其中，干燥区域类型包括但不限于：卧室类型、书房类型和/或客厅类型；相应地，第一清扫组件为用于清扫灰尘的组件，比如：毛刷等。湿润区域类型包括但不限于：厨房类型和/或卫生间类型；相应地，第二清扫组件为用于清扫液体的组件，比如：吸水拖布等。
当然，根据自移动设备的工作环境不同，对应的工作策略也可以为其它工作策略，本实施例在此不再一一列举。
综上所述，本实施例提供的自移动设备的控制方法，通过获取所述图像采集组件采集的场景图像；对场景图像进行图像识别得到对象信息，对象信息是指位于当前场景中的对象的属性信息；获取基于样本对象信息和样本对象信息对应的样本场景类型训练得到的场景识别 模型；控制对象信息输入场景识别模型，得到当前场景的场景类型；可以解决自移动设备无法判断当前场景的场景类型的问题；由于通过使用场景识别模型，可以判断出自移动设备当前所处场景的场景类型，可以实现对场景类型的识别，提高自移动设备的智能性。
另外，通过获取场景识别模型输出的场景类型的置信度，并将置信度最高的N个场景类型输出，可以增加输出结果的准确性。
实施例6
图11是本申请一个实施例提供的自移动设备的控制装置的框图，本实施例以该装置应用于图8所示的自移动设备中的控制组件810为例进行说明。该装置至少包括以下几个模块：图像获取模块1110、图像识别模块1120、模型获取模块430和设备控制模块1140。
图像获取模块1110，用于在所述自移动设备移动过程中，获取所述图像采集组件采集的场景图像，所述场景图像为所述自移动设备所处的当前场景的图像；
图像识别模块1120，用于对所述场景图像进行图像识别得到对象信息，所述对象信息是指位于所述当前场景中的对象的属性信息；
模型获取模块1130，用于获取场景识别模型，所述场景识别模型是基于样本对象信息和所述样本对象信息对应的样本场景类型训练得到的；
设备控制模块1140，用于控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型。
相关细节参考上述方法实施例。
需要说明的是：上述实施例中提供的自移动设备在进行场景识别时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将自移动设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的自移动设备的控制方法与自移动设备的控制装置实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
实施例7
图12是本申请一个实施例提供的自移动设备的结构示意图，如图12所示，该系统至少包括：控制组件1210和与控制组件1210通信相连的图像采集组件1220。
图像采集组件1220用于采集自移动设备移动过程中的环境图像1230；并将该环境图像1230发送至控制组件1210。可选地，图像采集组件1220可以实现为照相机、摄像机等，本实施例不对图像采集组件1220的实现方式作限定。
可选地，图像采集组件1220的视场角在水平方向上为120°、竖直方向上为60°；当然， 视场角也可以为其它数值，本实施例不对图像采集组件120的视场角的取值作限定。图像采集组件1220的视场角可以保证能够采集到自移动设备的在行进方向上的环境图像1230。
另外，图像采集组件1220的数量可以是一个或多个，本实施例不对图像采集组件1220的数量作限定。
控制组件1210用于对自移动设备进行控制。比如：控制自移动设备的启动、停止；控制自移动设备中的各个组件(如图像采集组件1220)的启动、停止等。
本实施例中，控制组件1210与存储器通信相连；该存储器中存储有程序，该程序由控制组件1210加载并执行至少实现以下步骤：用于获取自移动设备所处工作区域的边缘信息1240；获取移动时采集到的环境图像1230；基于环境图像1230获得通行门信息；基于通行门信息和边缘信息1240，对工作区域内的独立区域1250进行划分。换句话说，该程序由控制组件1210加载并执行以实现本申请提供的自移动设备的控制方法。
独立区域1250是指工作区域中与其它区域的属性不同的区域。比如：房间中的卧室区域、客厅区域、餐厅区域、厨房区域等。由于不同的独立区域通常通过通行门和墙进行划分，其中，墙与地面之间的边界线可以通过边缘信息获得，通行门的位置可以通过通行门信息获得，因此，可以通过边缘信息和通行门信息结合进行区域划分。
其中，边缘信息1240是指每个独立区域的地面边界的信息，该边缘信息包括对应地面边界的位置和长度。通行门信息是指每个独立区域中通行门的信息，通行门信息包括门框和/或门在工作区域中的位置信息。通行门是指供人、自移动设备和/或其他对象出入某个独立区域的门。通行门可以是开放式的虚拟门(即不存在门板等隔离物体的门)、或者是实体门，本实施例不对通行门的类型作限定。
需要补充说明的是，门框和/或门在工作区域中的位置信息是指：在向地面的垂直投影方向上，门框和/或门的投影位置在工作区域中的地理位置。
可选地，在对工作区域内的独立区域1250进行划分之后，控制组件1210还可以实现以下步骤：基于环境图像1230确定对应独立区域1250的场景预测结果1260；根据独立区域1250的场景预测结果1260确定独立区域的场景类型1270。
场景类型1270用于指示自移动设备当前所处的独立区域的类型，场景类型1270的划分方式根据自移动设备的工作区域设置。比如：自移动设备的工作区域为房间，则场景类型1270包括：卧室类型、厨房类型、书房类型、卫生间类型等，本实施例不对场景类型170的划分方式作限定。
需要补充说明的是，本实施例中，自移动设备还可以包括其它组件，比如：用于带动自 移动设备移动的移动组件(比如：车轮)、用于驱动移动组件运动的移动驱动组件(比如：电机)等，其中，移动驱动组件与控制组件110通信相连，在控制组件1210的控制下移动驱动组件运行并带动移动组件运动，从而实现自移动设备的整体运动，本实施例在此对自移动设备包括的组件不再一一列举。
另外，自移动设备可以为扫地机器人、自动割草机或者其它具有自动行驶功能的设备，本申请不对自移动设备的设备类型作限定。
本实施例中，通过结合边缘信息和通行门信息将工作区域划分为多个独立区域，可以解决现有技术无法实现对工作区域进行划分的问题；可以实现对工作区域进行划分，达到按照划分出的独立区域个性化工作的效果。另外，由于通行门可以是开放式的虚拟门，因此，可以将该虚拟门的信息和边缘信息结合得到每个独立区域的区域边界，从而划分出对应的独立区域，可以实现对开放门场景的区域划分，提高区域划分的准确度。
实施例8
下面对本申请提供的自移动设备的控制方法进行详细介绍。
图13是自移动设备的控制方法的流程图，图13中以该自移动设备的控制方法用于图1所示的自移动设备中、且各个步骤的执行主体为控制组件1210为例进行说明，该方法至少包括以下几个步骤：
步骤1301，获取自移动设备所处工作区域的边缘信息。
边缘信息是指每个独立区域的地面边界的信息，该边缘信息包括对应地面边界的位置和长度。边缘信息为自移动设备移动过程中，沿边行驶所获得的路径信息。
可选地，自移动设备具有沿边行驶的功能，在该功能下，若自移动设备沿着墙与地面形成的边界行驶，则可以获取到该边界的边缘信息；若自移动设备沿着物体(比如：柜子、桌子、床等)与地面形成的边界行驶，则可以获取到该边界的边缘信息。
步骤1302，获取移动时采集到的环境图像。
自移动设备记录在工作区域内移动时的工作时间(或称时间戳)和该工作时间对应的环境图像。在对工作区域工作完成后，读取该环境图像。
在一个示例中，自移动设备在本次工作完成后，根据本次工作的开始时间和结束时间，从已存储的环境图像中读取开始时间至结束时间对应的时间段内的环境图像。
步骤1303，基于环境图像获得通行门信息。
识别环境图像是否包括通行门的图像；在环境图像包括通行门的图像时，获取通行门在工作区域中的位置信息。
其中，通行门是指供自移动设备进入独立区域或者离开独立区域的通道。通行门可以是门框、栅栏口等，本实施例不对通行门的类型作限定。在一个示例中，通行门包括工作区域中的门框。
在一个示例中，自移动设备中存储有图像识别模型；自移动设备将环境图像输入图像识别模型，得到对象识别结果，该对象识别结果包括目标对象的属性信息。在对象识别结果包括通行门信息时，确定环境图像包括通行门的图像；在对象识别结果不包括通行门信息时，确定环境图像不包括通行门的图像。其中，目标对象包括通行门。
图像识别模型为基于小网络模型使用样本环境图像和样本对象结果训练得到的。
可选地，为了降低图像识别过程对自移动设备的硬件要求，图像识别模型运行时占用的计算资源低于自移动设备提供的最大计算资源。图像识别模型是基于小网络模型使用训练数据训练得到的。其中，训练数据包括自移动设备的工作区域中的各个对象的训练图像和每张训练图像的识别结果。其中，小网络模型是指模型层数小于第一数值；和/或，每层中的节点数量小于第二数值的网络模型。其中，第一数值和第二数值均为较小的整数。比如：小网络模型为微型的YOLO模型；或者，MobileNet模型。当然，小网络模型也可以是其它模型，本实施例在此不再一一列举。
可选地，为了进一步地压缩图像识别模型运行时占用的计算资源，在训练得到图像识别模型之后，自移动设备还可以对图像识别模型进行模型压缩处理。模型压缩处理包括但不限于：模型裁剪、模型量化和/或低轶分解等。
其中，对象识别结果包括目标对象的属性信息，比如：对象的类型、对象的大小、对象在环境图像中的位置信息。目标对象除了包括通行门之外，还可以包括床、桌子、沙发等家居用品，本实施例不对目标对象的类型作限定。
可选地，获取通行门在工作区域中的位置信息，包括：获取图像识别模型输出的环境图像中的通行门与自移动设备之间的第一距离；获取采集该环境图像时与边缘信息指示的边界之间的第二距离；根据第一距离和第二距离确定通行门相对于边缘信息指示的边界之间的位置，得到通行门的位置信息。或者，自移动设备上安装有定位组件，自移动设备获取图像识别模型输出的环境图像中的通行门与自移动设备之间的第一距离；获取采集到该环境图像时定位组件获取的定位信息；根据第一距离和定位信息得到通行门的位置信息。当然，自移动设备获取通行门的位置信息的方式还可以为其他方式，本实施例在此不再一一列举。
步骤1304，基于通行门信息和边缘信息，对工作区域内的独立区域进行划分。
由于工作区域内的某个或多个独立区域可能不是完全封闭的，而是与工作区域中的其它 区域连通的，比如：开放式厨房等区域，此时，仅根据自移动设备的边缘信息无法将该独立区域与其它区域区分开。基于此，本实施例中，通过将边缘信息与通行门信息相结合，可以确定出开放式的独立区域，提高区域划分的准确性。
基于通行门信息和边缘信息对工作区域内的独立区域进行划分，包括：获取通行门信息指示的对应通行门在工作区域中的位置信息；将边缘信息和位置信息结合，得到结合后的边界信息；将结合后的边界信息构成的各个封闭区域划分为对应的独立区域。
其中，边缘信息是自移动设备沿边行驶得到的。
比如：自移动设备为扫地机，扫地机对用户的整个房屋清洁完成后获取到房屋的边缘信息；之后，获取清扫过程中采集到的环境图像，对每张环境图像进行识别；在环境图像包括通行门的图像时获取该通行门的位置信息；将该位置信息与边缘信息结合即可多个封闭图形，每个封闭图形对应一个独立区域。
综上所述，本实施例提供的自移动设备的控制方法，通过获取自移动设备所处工作区域的边缘信息；获取移动时采集到的环境图像；基于环境图像获得通行门信息；基于通行门信息和边缘信息，对工作区域内的独立区域进行划分；可以解决现有技术无法实现对工作区域进行划分的问题；可以实现对工作区域进行划分，达到按照划分出的独立区域个性化工作的效果。另外，由于通行门可以是开放式的虚拟门，因此，可以将该虚拟门的信息和边缘信息结合得到每个独立区域的区域边界，从而划分出对应的独立区域，可以实现对开放门场景的区域划分，提高区域划分的准确度。
另外，通过对图像识别模型压缩处理，得到用于识别通行门的图像识别模型，可以进一步的缩小图像识别模型运行时占用的计算资源，提高识别速度，降低自移动设备对硬件的要求。
可选地，在获取到工作区域内的多个独立区域后，自移动设备还可以对每个独立区域的场景类型进行识别。此时，在步骤1304之后，参考图14，自移动设备的控制方法还包括如下几个步骤：
步骤1401，基于环境图像确定对应独立区域的场景预测结果。
场景预测结果用于指示自移动设备基于单个独立区域的相关信息预测得到的场景类型。场景预测结果的可以为一个或多个场景类型。
场景类型用于指示自移动设备当前所处的独立区域的类型。场景类型的划分方式根据自移动设备的工作区域设置。比如：自移动设备的工作区域为房间，则场景类型包括：卧室类型、厨房类型、书房类型、卫生间类型等，本实施例不对场景类型的划分方式作限定。
在一个示例中，基于环境图像确定对应独立区域的场景预测结果，包括：获取图像识别模型；对于每个独立区域，将独立区域对应的环境图像输入图像识别模型获得对象识别结果；获取场景识别模型；将对象识别结果输入场景识别模型，得到场景预测结果，场景预测结果包括独立区域的至少一种预测场景类型。
图像识别模型的相关描述详见步骤1303，本实施例在此不再赘述。
场景识别模型为基于概率模型使用对象的样本属性信息和样本场景训练得到的。
可选地，场景识别模型输出的场景预测结果包括多个场景类型，此时，场景识别模型还会输出每个场景类型对应的置信度。置信度用于指示所输出的每个场景类型的准确度。
步骤1402，根据独立区域的场景预测结果确定独立区域的场景类型。
在一个示例中，根据独立区域的场景预测结果确定独立区域的场景类型，包括：获取每个独立区域的位姿信息；结合每个独立区域的场景区域结果和每个独立区域的位姿信息，按照预先设置的概率分布策略确定各个独立区域的场景类型。其中，概率分布策略用于对于每种目标场景类型，从各个独立区域中确定场景类型是目标场景类型的概率最高的独立区域。
位姿信息包括对应独立区域在工作区域内位置信息和方向信息。其中，方向信息可以是通行门在对应独立区域中的方向。
在确定独立区域的场景类型时，自移动设备可以按照预先设置的概率分布策略，根据多个独立区域的场景预测结果确定每个独立区域的场景类型；
示意性地，概率分布策略为：对于每种场景类型存在对应的模板位姿信息；将独立区域的位姿信息与每种场景类型的模板位姿信息进行比较，得到位姿比较结果；对于每种目标场景类型，将该目标场景类型对应的场景预测结果和位姿比较结果与对应的权重相乘之和，得到概率结果；将概率结果最高的独立区域的类型确定为该场景类型。
为了更清楚地理解本申请提供的自移动设备的控制方法，下面对该方法举一个实例进行说明。参考图15，自移动设备在工作区域内工作完成后获取到工作区域内的边缘信息；将工作过程中图像采集组件采集到的环境图像输入图像识别模型41，得到对象信息42；结合对象信息42中的通行门信息和边缘信息划分工作区域，得到多个独立区域1250；将对象信息输入场景识别模型后得到每个独立区域的场景预测结果1260；结合多个独立区域的场景预测结果1260基于概率分布策略得到每个独立区域的场景类型1270。
综上所述，本实施例提供的自移动设备的控制方法，通过基于所述环境图像确定对应独立区域的场景预测结果；根据所述独立区域的场景预测结果确定所述独立区域的场景类型，可以使得自移动设备识别出整个工作区域中每个独立区域的场景类型，为用户提供更多的信 息，可以使得自移动设备更加智能化。
实施例9
图16是本申请一个实施例提供的自移动设备的控制装置的框图，本实施例以该装置应用于图12所示的自移动设备的控制系统中的控制组件1210为例进行说明。该装置至少包括以下几个模块：第一信息获取模块1610、环境图像获取模块1620、第二信息获取模块1630和区域划分控制模块1640。
第一信息获取模块1610，用于获取所述自移动设备所处工作区域的边缘信息；
环境图像获取模块1620，用于获取移动时采集到的环境图像；
第二信息获取模块1630，用于基于所述环境图像获得通行门信息；
区域划分控制模块1640，用于基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分。
相关细节参考上述方法实施例。
需要说明的是：上述实施例中提供的自移动设备的控制装置在进行自移动设备的控制时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将自移动设备的控制装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的自移动设备的控制装置与自移动设备的控制方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
实施例10
图17是本申请一个实施例提供的自移动设备的控制装置的框图，该装置可以是图1、图8或图12所示的自移动设备，当然，也可以是其他安装在自移动设备上与自移动设备相互独立的设备。该装置至少包括处理器1701和存储器1702。
处理器1701可以包括一个或多个处理核心，比如：4核心处理器、8核心处理器等。处理器1701可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器1701也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。一些实施例中，处理器801还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1702可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非 暂态的。存储器802还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1702中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1701所执行以实现本申请中方法实施例提供的自移动设备的控制方法。
在一些实施例中，自移动设备的控制装置还可选包括有：外围设备接口和至少一个外围设备。处理器1701、存储器1702和外围设备接口之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口相连。示意性地，外围设备包括但不限于：射频电路、触摸显示屏、音频电路、和电源等。
当然，自移动设备的控制装置还可以包括更少或更多的组件，本实施例对此不作限定。
可选地，本申请还提供有一种计算机可读存储介质，所述计算机可读存储介质中存储有程序，所述程序由处理器加载并执行以实现上述方法实施例的自移动设备的控制方法。
可选地，本申请还提供有一种计算机产品，该计算机产品包括计算机可读存储介质，所述计算机可读存储介质中存储有程序，所述程序由处理器加载并执行以实现上述方法实施例的自移动设备的控制方法。
以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (32)

  1. 一种自移动设备的控制方法，其特征在于，所述自移动设备上安装有图像采集组件，所述方法包括：
    获取所述图像采集组件采集的环境图像；
    获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
    控制所述环境图像输入所述图像识别模型得到对象识别结果，所述对象识别结果用于指示目标对象的类别。
  2. 根据权利要求1所述的方法，其特征在于，所述图像识别模型是对小网络检测模型进行训练得到的。
  3. 根据权利要求1所述的方法，其特征在于，所述获取图像识别模型之前，还包括：
    获取小网络检测模型；
    获取训练数据，所述训练数据包括所述自移动设备的工作区域中的各个对象的训练图像和每张训练图像的识别结果；
    将所述训练图像输入所述小网络检测模型，得到模型结果；
    基于所述模型结果与所述训练图像对应的识别结果之间的差异对所述小网络检测模型进行训练，得到所述图像识别模型。
  4. 根据权利要求3所述的方法，其特征在于，所述基于所述模型结果与所述训练图像对应的识别结果之间的差异对所述小网络检测模型进行训练，得到所述图像识别模型之后，还包括：
    对所述图像识别模型进行模型压缩处理，得到用于识别对象的图像识别模型。
  5. 根据权利要求2所述的方法，其特征在于，所述小网络检测模型为：微型的YOLO模型；或者，MobileNet模型。
  6. 根据权利要求1至5任一所述的方法，其特征在于，所述控制所述环境图像输入所述图像识别模型得到对象识别结果之后，还包括：
    基于所述对象识别结果，控制所述自移动设备移动以完成对应的任务。
  7. 根据权利要求6所述的方法，其特征在于，所述自移动设备上安装有液体清扫组件，所述基于所述对象识别结果，控制所述自移动设备移动以完成对应的任务，包括：
    当所述对象识别结果指示所述环境图像包含液体图像时，控制所述自移动设备移动至所述液体图像对应的待清洁区域；
    使用所述液体清扫组件清扫所述待清洁区域中的液体。
  8. 根据权利要求6所述的方法，其特征在于，所述自移动设备中安装有供电组件，所述供电组件使用充电组件进行充电，所述基于所述对象识别结果，控制所述自移动设备移动以完成对应的任务，包括：
    当所述供电组件的剩余电量小于或等于电量阈值、且所述环境图像包括所述充电组件的图像时，根据所述充电组件的图像位置确定所述充电组件的实际位置。
  9. 根据权利要求8所述的方法，其特征在于，所述自移动设备上还安装有定位传感器，所述定位传感器用于定位所述充电组件上充电接口的位置；所述控制所述自移动设备向所述充电组件移动之后，还包括：
    在向所述充电组件移动过程中，控制所述定位传感器对所述充电组件的位置进行定位得到定位结果；
    控制所述自移动设备按照所述定位结果移动，以实现所述自移动设备与所述充电接口对接。
  10. 一种自移动设备的控制装置，其特征在于，所述自移动设备上安装有图像采集组件，所述装置包括：
    图像获取模块，用于在所述自移动设备移动过程中，获取所述图像采集组件采集的环境图像；
    模型获取模块，用于获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
    设备控制模块，用于控制所述环境图像输入所述图像识别模型得到对象识别结果，所述对象识别结果用于指示所述环境图像中目标对象的类别。
  11. 一种自移动设备的控制方法，其特征在于，所述自移动设备上安装有图像采集组件，所述方法包括：
    在所述自移动设备移动过程中，获取所述图像采集组件采集的场景图像，所述场景图像为所述自移动设备所处的当前场景的图像；
    对所述场景图像进行图像识别得到对象信息，所述对象信息是指位于所述当前场景中的对象的属性信息；
    获取场景识别模型，所述场景识别模型是基于样本对象信息和所述样本对象信息对应的样本场景类型训练得到的；
    控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型。
  12. 根据权利要求11所述的方法，其特征在于，所述场景识别模型是对概率模型进行训 练得到的。
  13. 根据权利要求12所述的方法，其特征在于，所述获取场景识别模型之前，还包括：
    获取所述概率模型；
    获取训练数据，所述训练数据包括各个对象的样本属性信息和各个样本属性信息对应的样本场景类型；
    将所述样本属性信息输入所述概率模型，得到模型结果；
    基于所述模型结果与所述样本场景类型之间的差异对所述概率模型进行训练，得到所述场景识别模型。
  14. 根据权利要求11所述的方法，其特征在于，所述对所述场景图像进行图像识别得到对象信息，还包括：
    获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
    将所述场景图像输入所述图像识别模型得到对象识别结果。
  15. 根据权利要求14所述的方法，其特征在于，所述图像识别模型为基于小网络模型使用样本场景图像和样本对象结果训练得到的。
  16. 根据权利要求14所述的方法，其特征在于，所述图像识别模型是经过模型压缩处理后的得到的。
  17. 根据权利要求11至16任一所述的方法，其特征在于，所述场景识别模型还输出每个场景类型的置信度；所述控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型之后，还包括：
    将所述场景类型按照置信度由高到低的顺序进行排序；
    输出排序在前N位的场景类型，所述N为大于1的整数。
  18. 根据权利要求11至16任一所述的方法，其特征在于，所述控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型之后，还包括：
    根据所述场景类型确定对应的清扫策略；
    控制所述自移动设备按照所述清扫策略执行清扫工作。
  19. 一种自移动设备的场景识别装置，其特征在于，所述自移动设备上安装有图像采集组件，所述装置包括：
    图像获取模块，用于在所述自移动设备移动过程中，获取所述图像采集组件采集的场景图像，所述场景图像为所述自移动设备所处的当前场景的图像；
    图像识别模块，用于对所述场景图像进行图像识别得到对象信息，所述对象信息是指位于所述当前场景中的对象的属性信息；
    模型获取模块，用于获取场景识别模型，所述场景识别模型是基于样本对象信息和所述样本对象信息对应的样本场景类型训练得到的；
    设备控制模块，用于控制所述对象信息输入所述场景识别模型，得到所述当前场景的场景类型。
  20. 一种自移动设备的控制方法，其特征在于，所述方法包括：
    获取所述自移动设备所处工作区域的边缘信息；
    获取移动时采集到的环境图像；
    基于所述环境图像获得通行门信息；
    基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分。
  21. 根据权利要求20所述的方法，其特征在于，所述基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分之后，还包括：
    基于所述环境图像确定对应独立区域的场景预测结果；
    根据所述独立区域的场景预测结果确定所述独立区域的场景类型。
  22. 根据权利要求21所述的方法，其特征在于，所述基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分，包括：
    获取所述通行门信息指示的对应通行门在所述工作区域中的位置信息；
    将所述边缘信息和所述位置信息结合，得到结合后的边界信息；
    将所述结合后的边界信息构成的各个封闭区域划分为对应的独立区域。
  23. 根据权利要求20所述的方法，其特征在于，所述基于所述环境图像获得通行门信息，包括：
    识别所述环境图像是否包括通行门的图像；
    在所述环境图像包括通行门的图像时，获取所述通行门在所述工作区域中的位置信息。
  24. 根据权利要求23所述的方法，其特征在于，所述通行门包括所述工作区域中的门框。
  25. 根据权利要求21所述的方法，其特征在于，所述根据所述独立区域的场景预测结果确定所述独立区域的场景类型，包括：
    获取每个独立区域的位姿信息，所述位姿信息包括对应独立区域在所述工作区域内的位置信息和方向信息；
    结合每个独立区域的场景区域结果和每个独立区域的位姿信息，按照预先设置的概率分 布策略确定各个独立区域的场景类型；
    其中，概率分布策略用于对于每种目标场景类型，从各个独立区域中确定场景类型是所述目标场景类型的概率最高的独立区域。
  26. 根据权利要求21所述的方法，其特征在于，所述基于所述环境图像确定对应独立区域的场景预测结果，包括：
    获取图像识别模型，所述图像识别模型运行时占用的计算资源低于所述自移动设备提供的最大计算资源；
    对于每个独立区域，将所述独立区域对应的环境图像输入所述图像识别模型获得对象识别结果，所述对象识别结果包括目标对象的属性信息；
    获取场景识别模型，所述场景识别模型是使用对象的样本属性信息和样本场景类型训练得到的；
    将所述对象识别结果输入所述场景识别模型，得到场景预测结果，所述场景预测结果包括所述独立区域的至少一种预测场景类型。
  27. 根据权利要求26所述的方法，其特征在于，所述图像识别模型为基于小网络模型使用样本环境图像和样本对象结果训练得到的。
  28. 根据权利要求26所述的方法，其特征在于，所述场景识别模型为基于概率模型使用对象的样本属性信息和样本场景训练得到的。
  29. 一种自移动设备的控制装置，其特征在于，装置包括：
    第一信息获取模块，用于获取所述自移动设备所处工作区域的边缘信息；
    环境图像获取模块，用于获取移动时采集到的环境图像；
    第二信息获取模块，用于基于所述环境图像获得通行门信息；
    区域划分控制模块，用于基于所述通行门信息和所述边缘信息，对所述工作区域内的独立区域进行划分。
  30. 一种自移动设备的控制装置，其特征在于，所述装置包括处理器和存储器；所述存储器中存储有程序，所述程序由所述处理器加载并执行以实现如权利要求1至9、11至18和20至28任一项所述的自移动设备的控制方法。
  31. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有程序，所述程序被处理器执行时用于实现如权利要求1至9、11至18和20至28中任一项所述的自移动设备的控制方法。
  32. 一种自移动设备，其特征在于，包括：
    用于带动所述自移动设备移动的移动组件；
    用于驱动所述移动组件运动的移动驱动组件；
    安装在所述自移动设备上、用于采集行进方向上的环境图像的图像采集组件；
    与所述移动驱动组件和所述图像采集组件通信相连的控制组件，所述控制组件与存储器通信相连；所述存储器中存储有程序，所述程序由所述控制组件加载并执行以实现如权利要求1至9、11至18和20至28任一项所述的自移动设备的控制方法。
PCT/CN2021/105792 2020-07-13 2021-07-12 自移动设备的控制方法、装置、存储介质及自移动设备 Ceased WO2022012471A1 (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US18/015,719 US12478241B2 (en) 2020-07-13 2021-07-12 Control method for self-moving device and self-moving device
CA3185243A CA3185243A1 (en) 2020-07-13 2021-07-12 Control method for self-moving device, apparatus, storage medium, and self-moving device
KR1020237004202A KR20230035610A (ko) 2020-07-13 2021-07-12 자율 이동 디바이스의 제어 방법, 및 자율 이동 디바이스의 제어 디바이스
EP21842796.1A EP4163819A4 (en) 2020-07-13 2021-07-12 Control method for self-moving device, apparatus, storage medium, and self-moving device
JP2023501666A JP2023534932A (ja) 2020-07-13 2021-07-12 自律移動機器の制御方法、装置、記憶媒体及び自律移動機器
AU2021308246A AU2021308246A1 (en) 2020-07-13 2021-07-12 Control method for self-moving device, apparatus, storage medium, and self-moving device

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202010666134.2A CN111539398B (zh) 2020-07-13 2020-07-13 自移动设备的控制方法、装置及存储介质
CN202010666140.8 2020-07-13
CN202010666135.7 2020-07-13
CN202010666134.2 2020-07-13
CN202010666135.7A CN111539399B (zh) 2020-07-13 2020-07-13 自移动设备的控制方法、装置、存储介质及自移动设备
CN202010666140.8A CN111539400A (zh) 2020-07-13 2020-07-13 自移动设备的控制方法、装置、存储介质及自移动设备

Publications (1)

Publication Number Publication Date
WO2022012471A1 true WO2022012471A1 (zh) 2022-01-20

Family

ID=79554308

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/105792 Ceased WO2022012471A1 (zh) 2020-07-13 2021-07-12 自移动设备的控制方法、装置、存储介质及自移动设备

Country Status (7)

Country Link
US (1) US12478241B2 (zh)
EP (1) EP4163819A4 (zh)
JP (1) JP2023534932A (zh)
KR (1) KR20230035610A (zh)
AU (1) AU2021308246A1 (zh)
CA (1) CA3185243A1 (zh)
WO (1) WO2022012471A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025143894A1 (ko) * 2023-12-29 2025-07-03 삼성전자 주식회사 로봇형 이동기기의 인식 모델을 업데이트하는 방법 및 이를 수행하기 위한 전자 장치
CN121445259A (zh) * 2024-07-31 2026-02-03 北京小米移动软件有限公司 机器人控制方法、装置、计算机设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106983449A (zh) * 2010-07-01 2017-07-28 德国福维克控股公司 具有区域划分的测绘制图
CN110458082A (zh) * 2019-08-05 2019-11-15 城云科技（中国）有限公司 一种城市管理案件分类识别方法
CN110897567A (zh) * 2018-12-13 2020-03-24 成都家有为力机器人技术有限公司 一种基于目标物识别的清洁方法及清洁机器人
CN111012261A (zh) * 2019-11-18 2020-04-17 深圳市杉川机器人有限公司 基于场景识别的清扫方法、系统、扫地设备及存储介质
CN111539398A (zh) * 2020-07-13 2020-08-14 追创科技(苏州)有限公司 自移动设备的控制方法、装置及存储介质
CN111539400A (zh) * 2020-07-13 2020-08-14 追创科技(苏州)有限公司 自移动设备的控制方法、装置、存储介质及自移动设备
CN111539399A (zh) * 2020-07-13 2020-08-14 追创科技(苏州)有限公司 自移动设备的控制方法、装置、存储介质及自移动设备

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100791384B1 (ko) 2006-07-05 2008-01-07 삼성전자주식회사 특징점을 이용한 영역 구분 방법 및 장치와 이를 이용한이동 청소 로봇
US8706297B2 (en) 2009-06-18 2014-04-22 Michael Todd Letsky Method for establishing a desired area of confinement for an autonomous robot and autonomous robot implementing a control system for executing the same
KR102158695B1 (ko) 2014-02-12 2020-10-23 엘지전자 주식회사 로봇 청소기 및 이의 제어방법
JP6532530B2 (ja) 2014-12-16 2019-06-19 アクチエボラゲット エレクトロルックス ロボット掃除機の掃除方法
DE102015119501A1 (de) 2015-11-11 2017-05-11 RobArt GmbH Unterteilung von Karten für die Roboternavigation
AU2017285019B2 (en) * 2016-06-15 2022-11-10 Irobot Corporation Systems and methods to control an autonomous mobile robot
CN106355151B (zh) 2016-08-30 2019-10-01 电子科技大学 一种基于深度置信网络的三维sar图像目标识别方法
KR102688528B1 (ko) 2017-01-25 2024-07-26 엘지전자 주식회사 이동 로봇 및 그 제어방법
CN107088883A (zh) 2017-07-03 2017-08-25 贵州大学 交互式服务机器人
US10857679B1 (en) * 2017-08-31 2020-12-08 Savioke, Inc. Apparatus and method for auxiliary mobile robot functionality
CN108143364B (zh) 2017-12-28 2021-02-19 湖南格兰博智能科技有限责任公司 一种自移动清洁机器人清洁地图区域划分的方法
US10878294B2 (en) 2018-01-05 2020-12-29 Irobot Corporation Mobile cleaning robot artificial intelligence for situational awareness
CN108763606B (zh) 2018-03-12 2019-12-10 江苏艾佳家居用品有限公司 一种基于机器视觉的户型图元素自动提取方法与系统
CN109272554A (zh) 2018-09-18 2019-01-25 北京云迹科技有限公司 一种识别目标的坐标系定位和语义地图构建的方法及系统
US12140954B2 (en) 2018-09-20 2024-11-12 Samsung Electronics Co., Ltd. Cleaning robot and method for performing task thereof
KR102228866B1 (ko) 2018-10-18 2021-03-17 엘지전자 주식회사 로봇 및 그의 제어 방법
CN109522803B (zh) 2018-10-18 2021-02-09 深圳乐动机器人有限公司 一种室内区域划分和识别方法、装置及终端设备
CN109784194B (zh) 2018-12-20 2021-11-23 北京图森智途科技有限公司 目标检测网络构建方法和训练方法、目标检测方法
CN109871420B (zh) 2019-01-16 2022-03-29 深圳乐动机器人有限公司 地图生成和分区方法、装置及终端设备
CN110059558B (zh) 2019-03-15 2023-08-25 江苏大学 一种基于改进ssd网络的果园障碍物实时检测方法
CN110070005A (zh) 2019-04-02 2019-07-30 腾讯科技（深圳）有限公司 图像目标识别方法、装置、存储介质及电子设备
CN110348318A (zh) 2019-06-18 2019-10-18 北京大米科技有限公司 图像识别方法、装置、电子设备及介质
CN110450152A (zh) 2019-06-24 2019-11-15 广东宝乐机器人股份有限公司 区域识别方法、机器人和存储介质
CN110251004B (zh) 2019-07-16 2022-03-11 深圳市杉川机器人有限公司 扫地机器人及其清扫方法和计算机可读存储介质
CN112155487A (zh) 2019-08-21 2021-01-01 追创科技(苏州)有限公司 扫地机器人、扫地机器人的控制方法及存储介质
CN110502019A (zh) 2019-09-06 2019-11-26 北京云迹科技有限公司 一种室内机器人的避障方法及装置
CN111150330B (zh) 2019-12-30 2024-12-10 深圳腾跃信息科技服务有限公司 清扫控制方法
CN111166247B (zh) 2019-12-31 2022-06-07 深圳飞科机器人有限公司 垃圾分类处理方法及清洁机器人

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106983449A (zh) * 2010-07-01 2017-07-28 德国福维克控股公司 具有区域划分的测绘制图
CN110897567A (zh) * 2018-12-13 2020-03-24 成都家有为力机器人技术有限公司 一种基于目标物识别的清洁方法及清洁机器人
CN110458082A (zh) * 2019-08-05 2019-11-15 城云科技（中国）有限公司 一种城市管理案件分类识别方法
CN111012261A (zh) * 2019-11-18 2020-04-17 深圳市杉川机器人有限公司 基于场景识别的清扫方法、系统、扫地设备及存储介质
CN111539398A (zh) * 2020-07-13 2020-08-14 追创科技(苏州)有限公司 自移动设备的控制方法、装置及存储介质
CN111539400A (zh) * 2020-07-13 2020-08-14 追创科技(苏州)有限公司 自移动设备的控制方法、装置、存储介质及自移动设备
CN111539399A (zh) * 2020-07-13 2020-08-14 追创科技(苏州)有限公司 自移动设备的控制方法、装置、存储介质及自移动设备

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
NING KAI: "Research on Garbage and Driving Area Detection of Sweeping Robot", MASTER THESIS, TIANJIN POLYTECHNIC UNIVERSITY, CN, no. 2, 15 February 2020 (2020-02-15), CN , XP055887537, ISSN: 1674-0246 *
See also references of EP4163819A4 *
ZHAO HAIPENG, ZHOU YANG, ZHANG LONG, PENG YANGZHAO, HU XIAOFEI, PENG HAOJIE, CAI XINYUE: "Mixed YOLOv3-LITE: A Lightweight Real-Time Object Detection Method", SENSORS, vol. 20, no. 7, 27 March 2020 (2020-03-27), XP055887540, DOI: 10.3390/s20071861 *

Also Published As

Publication number Publication date
US12478241B2 (en) 2025-11-25
US20230270308A1 (en) 2023-08-31
EP4163819A4 (en) 2023-12-06
AU2021308246A1 (en) 2023-02-16
CA3185243A1 (en) 2022-01-20
KR20230035610A (ko) 2023-03-14
EP4163819A1 (en) 2023-04-12
JP2023534932A (ja) 2023-08-15

Similar Documents

Publication Publication Date Title
CN111539399B (zh) 自移动设备的控制方法、装置、存储介质及自移动设备
CN113920451B (zh) 自移动设备的控制方法、装置及存储介质
CN111643010B (zh) 清洁机器人控制方法、装置、清洁机器人和存储介质
CN111657798A (zh) 基于场景信息的清扫机器人控制方法、装置和清扫机器人
US12282342B2 (en) Stationary service appliance for a poly functional roaming device
CN111543902B (zh) 地面清洁方法、装置、智能清洁设备和存储介质
CN111643014A (zh) 智能清洁方法、装置、智能清洁设备和存储介质
CN111643017B (zh) 基于日程信息的清扫机器人控制方法、装置和清扫机器人
EP3387980B1 (en) Apparatus and method for controlling cleaning in robotic cleaner
CN111568314A (zh) 基于场景识别的清洁方法、装置、清洁机器人和存储介质
CN111539400A (zh) 自移动设备的控制方法、装置、存储介质及自移动设备
CN111381590A (zh) 一种扫地机器人及其路线规划方法
CN113985866B (zh) 扫地机器人路径规划方法、装置、电子设备、存储介质
WO2022012471A1 (zh) 自移动设备的控制方法、装置、存储介质及自移动设备
CN116188849A (zh) 一种基于轻量化网络的目标识别方法、系统及扫地机器人
DE112023003580T5 (de) Lebenslanges roboterlernen für mobile Roboter
CN110315538A (zh) 一种在电子地图上显示障碍物的方法、装置及机器人
DE102022100849A1 (de) Situationsbewertung mittels objekterkennung in autonomen mobilen robotern
CN116883733B (zh) 交互控制方法及电子设备
CN112906642B (zh) 自移动机器人、自移动机器人的控制方法及存储介质
CN111419117A (zh) 视觉扫地机器人返航控制方法及视觉扫地机器人
Abdusalomov et al. Development of a Multi-Functional Robotic Vacuum Cleaner with Enhanced Intelligence Techniques and Computer Vision Approaches
US20260033690A1 (en) Method for controlling robot, computer program product, robot, and storage medium
EP4654152A1 (en) Temporal aggregation for online 3d object detection
CN110315537A (zh) 一种控制机器人运动的方法、装置及机器人

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21842796

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 3185243

Country of ref document: CA

ENP Entry into the national phase

Ref document number: 2023501666

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021842796

Country of ref document: EP

Effective date: 20230103

ENP Entry into the national phase

Ref document number: 20237004202

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021308246

Country of ref document: AU

Date of ref document: 20210712

Kind code of ref document: A

WWG Wipo information: grant in national office

Ref document number: 18015719

Country of ref document: US