WO2023020269A1 - 自移动机器人控制方法、装置、设备及可读存储介质 - Google Patents

自移动机器人控制方法、装置、设备及可读存储介质 Download PDF

Info

Publication number
WO2023020269A1
WO2023020269A1 PCT/CN2022/109522 CN2022109522W WO2023020269A1 WO 2023020269 A1 WO2023020269 A1 WO 2023020269A1 CN 2022109522 W CN2022109522 W CN 2022109522W WO 2023020269 A1 WO2023020269 A1 WO 2023020269A1
Authority
WO
WIPO (PCT)
Prior art keywords
self
mobile robot
target object
area
voice signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/109522
Other languages
English (en)
French (fr)
Inventor
丁民权
饶铸
高超
杨晓璐
陈茂勇
刘阳
林兴宝
岳鹏飞
于昊田
李大亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecovacs Robotics Suzhou Co Ltd
Original Assignee
Ecovacs Robotics Suzhou Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202110940990.7A external-priority patent/CN113793605B/zh
Priority claimed from CN202110959732.3A external-priority patent/CN113787517B/zh
Application filed by Ecovacs Robotics Suzhou Co Ltd filed Critical Ecovacs Robotics Suzhou Co Ltd
Priority to EP22857581.7A priority Critical patent/EP4390599A4/en
Priority to US18/681,526 priority patent/US20250127362A1/en
Publication of WO2023020269A1 publication Critical patent/WO2023020269A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/242—Means based on the reflection of waves generated by the vehicle
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4011—Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/16—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/22—Command input arrangements
    • G05D1/228—Command input arrangements located on-board unmanned vehicles
    • G05D1/2285—Command input arrangements located on-board unmanned vehicles using voice or gesture commands
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/246—Arrangements for determining position or orientation using environment maps, e.g. simultaneous localisation and mapping [SLAM]
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/617—Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards
    • G05D1/622—Obstacle avoidance
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/648—Performing a task within a working area or space, e.g. cleaning
    • G05D1/6482—Performing a task within a working area or space, e.g. cleaning by dividing the whole area or space in sectors to be processed separately
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/65—Following a desired speed profile
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/656—Interaction with payloads or external entities
    • G05D1/686—Maintaining a relative position with respect to moving targets, e.g. following animals or humans
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/04—Automatic control of the travelling movement; Automatic obstacle detection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2101/00—Details of software or hardware architectures used for the control of position
    • G05D2101/10—Details of software or hardware architectures used for the control of position using artificial intelligence [AI] techniques
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2105/00—Specific applications of the controlled vehicles
    • G05D2105/10—Specific applications of the controlled vehicles for cleaning, vacuuming or polishing
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2107/00—Specific environments of the controlled vehicles
    • G05D2107/40—Indoor domestic environment
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2109/00—Types of controlled vehicles
    • G05D2109/10—Land vehicles
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2111/00—Details of signals used for control of position, course, altitude or attitude of land, water, air or space vehicles
    • G05D2111/10—Optical signals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2111/00—Details of signals used for control of position, course, altitude or attitude of land, water, air or space vehicles
    • G05D2111/20—Acoustic signals, e.g. ultrasonic signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166—Microphone arrays; Beamforming

Definitions

  • the present application relates to the technical field of artificial intelligence, in particular to a control method, device, equipment and readable storage medium of a self-moving robot.
  • Voice control is a common robot control method.
  • the robot stores the environment map in advance, and each work area is marked in the environment map, such as Huawei's room and living room.
  • the robot determines the working area according to the voice command and works in the working area. For example, the user voice controls the sweeping robot to clean a certain room, clean around a certain piece of furniture, etc.
  • the user voice controls the mowing robot to mow grass in the target area.
  • the above voice control method needs to store the environment map in advance. If the user temporarily designates a working area, the working area needs to be marked on the environmental map with the help of APP, etc., which is cumbersome and inflexible.
  • Embodiments of the present application provide a self-mobile robot control method, device, equipment, and readable storage medium.
  • the self-mobile robot determines a working area by following a user, and the process is simple, highly flexible, and easy to implement.
  • the embodiment of the present application provides a method for controlling a self-mobile robot, including:
  • the embodiment of the present application provides a control device for a self-mobile robot, including:
  • a first determining module configured to determine the sound source direction according to the voice signal sent by the user
  • the second determining module is used to determine the moving objects around the self-mobile robot
  • a third determining module configured to determine a target object located in the direction of the sound source from the moving object
  • a processing module configured to determine a working area according to the target object
  • the executing module is used for moving to the working area and executing tasks in the working area.
  • an embodiment of the present application provides a self-moving robot, including: a processor, a memory, and a computer program stored on the memory and operable on the processor.
  • the processor executes the computer program
  • the The self-mobile robot implements the method described in the above first aspect or various possible implementation manners of the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when executed by a processor, the computer instructions are used to implement the above first aspect or the first The method described in various possible implementation manners of the aspect.
  • the embodiments of the present application provide a computer program product including a computer program, and when the computer program is executed by a processor, the method described in the above first aspect or various possible implementation manners of the first aspect is implemented.
  • the self-mobile robot control method, device, equipment, and readable storage medium allow the self-mobile robot to determine the direction of the sound source according to the voice signal sent by the user, and determine the moving objects around itself. After that, determine the target object located in the direction of the sound source from the moving objects around itself, determine the working area according to the target object, and move to the working area and execute the task.
  • the self-mobile robot determines the target object from the mobile object, the mobile object has an accurate spatial position. Therefore, the self-mobile robot can accurately determine the target object from multiple moving objects according to the direction of the sound source, and accurately arrive at the working area without using the client, and the process is simple and flexible.
  • this solution is applicable to all laser-type self-mobile robots, with low cost, simple algorithm, and low computing power required.
  • FIG. 1A is a schematic diagram of the implementation environment of the self-mobile robot control method provided by the embodiment of the present application.
  • Fig. 1B is a schematic structural diagram of the sweeping robot provided by the embodiment of the present application.
  • Fig. 1C is a structural schematic diagram of a sound signal acquisition device of a self-mobile robot
  • Fig. 1D is another structural schematic diagram of the sweeping robot provided by the embodiment of the present application.
  • Fig. 2A is a voice control flowchart of the self-mobile robot provided by the embodiment of the present application.
  • Fig. 2B is another voice control flowchart of the self-mobile robot provided by the embodiment of the present application.
  • Fig. 2C is another voice control flowchart of the self-mobile robot provided by the embodiment of the present application.
  • Fig. 3 is a flow chart of the self-mobile robot control method provided by the embodiment of the present application.
  • Fig. 4 is another flow chart of the self-mobile robot control method provided by the embodiment of the present application.
  • Fig. 5 is the flowchart of determining target object
  • FIG. 6 shows the LAM diagram
  • Figure 7 shows the DTOF scatter diagram
  • Fig. 8 is a follow-up flow chart based on AI camera in the self-mobile robot control method provided by the embodiment of the present application;
  • Fig. 9 is the flow chart that keeps following state from mobile robot
  • Fig. 10 is the structural representation of self-mobile robot
  • Fig. 11 is a schematic structural diagram of a self-mobile robot control device provided by an embodiment of the present application.
  • FIG. 12 is a flow chart of a method for voice control of an autonomous mobile device provided by an embodiment of the present application.
  • Fig. 13 is a schematic diagram of the speech recognition process in the embodiment of the present application.
  • Figure 14 is a schematic diagram of a carpet area
  • Fig. 15 is a schematic diagram of the furniture identification process
  • Fig. 16 is a schematic diagram of the process of identifying a door
  • Fig. 17 is a schematic diagram of the synchronization process between the autonomous mobile device and the speech recognition server
  • Fig. 18A is a schematic diagram of determining the position of the sound source relative to the center of the microphone array
  • 18B is a schematic diagram of a microphone array and an autonomous mobile device body
  • Fig. 18C is a schematic diagram of the process of training a speech recognition model and recognizing speech
  • Fig. 19 is a logic flow chart of the voice control of the autonomous mobile device provided by the embodiment of the present application.
  • FIG. 20 is another flow chart of the voice control method for an autonomous mobile device provided by an embodiment of the present application.
  • Fig. 21 is another flow chart of the voice control method for an autonomous mobile device provided by the embodiment of the present application.
  • Fig. 22 is another flow chart of the voice control method of autonomous mobile equipment provided by the embodiment of the present application.
  • Fig. 23 is a schematic structural diagram of a self-mobile robot provided by an embodiment of the present application.
  • the robot can reach the designated work area to work according to the user's voice command.
  • the sweeper pre-builds and stores an environmental map.
  • send a voice command including the area to the sweeper such as "clean Xiao Ming's room” and so on.
  • the user temporarily designates a work area. For example, the user expects the sweeper to clean the vicinity of his location, that is, where the user sweeps, and this function is commonly known as the "where to sweep" function.
  • the location of the user is random each time. If the user marks the working area on the environmental map with the help of APP every time, the process will be cumbersome and the flexibility will be poor.
  • embodiments of the present application provide a control method, device, device, and readable storage medium for a self-mobile robot.
  • the self-mobile robot determines a working area by following a user, and the process is simple, highly flexible, and easy to implement.
  • Fig. 1A is a schematic diagram of the implementation environment of the self-mobile robot control method provided by the embodiment of the present application.
  • the implementation environment includes self-moving robots, such as sweeping robots, self-moving air cleaning robots, automatic lawn mowers, window cleaning robots, solar panel cleaning robots, housekeeping robots, unmanned aerial vehicles, automatic Guided vehicles (Automated Guided Vehicle, AGV), security robots, welcome robots, care robots, etc.
  • self-moving robots such as sweeping robots, self-moving air cleaning robots, automatic lawn mowers, window cleaning robots, solar panel cleaning robots, housekeeping robots, unmanned aerial vehicles, automatic Guided vehicles (Automated Guided Vehicle, AGV), security robots, welcome robots, care robots, etc.
  • AGV Automatic Guided Vehicle
  • a sound signal collection device such as a microphone is installed on the mobile robot, which can collect the voice signal sent by the user. After the voice signal is collected by the mobile robot, the voice signal is recognized to obtain a voice command, and the task indicated by the voice command is executed.
  • the self-mobile robot can recognize the voice signal itself. Or, establish a network connection with the voice recognition server (not shown in the figure) from the mobile robot, after the voice signal is collected from the mobile robot, send the voice signal to the voice recognition server, so that the voice recognition server recognizes the voice signal, and The recognized voice commands are sent to the self-mobile robot.
  • Fig. 1B is a schematic structural diagram of a sweeping robot provided by an embodiment of the present application.
  • the sweeping robot is referred to as a robot for short.
  • ⁇ represents the propagation direction of the voice signal.
  • the robot includes a robot shell 1, a drive element, a protruding structure 2 and a voice signal acquisition device 3; wherein the drive element is arranged in the robot shell 1, and it is used to drive the robot shell 1 to move; the protruding structure 2 is arranged on the robot shell 1
  • the robot shell 1 includes a top plate, an annular side plate and a bottom plate.
  • the top plate, the annular vertical plate and the bottom plate are enclosed and assembled to form an accommodating chamber, and a control unit and a driving element are housed in the accommodating chamber.
  • the robot also includes functional elements such as a driving wheel 6, a side brush 7, a rolling brush or a fan arranged on the robot shell 1, wherein the driving wheel 6 is used to drive the robot to travel under the action of the driving element, and the side brush 7 and the rolling brush
  • the fan is used to form a negative pressure chamber in the dust box to suck the dust and debris on the working surface into the dust box to remove dust.
  • the upper surface 10 of the top plate of the robot housing 1 is protrudingly provided with a protruding structure 2 .
  • the protruding structure 2 and the top plate are integrally formed.
  • the protruding structure 2 and the top plate are separately processed and formed, and then the protruding structure 2 is fixedly connected to the upper surface 10 of the top plate by means of bonding, screwing, or the like.
  • a sound signal collecting device 3 is arranged on the protruding structure 2 .
  • the self noise of the robot is generated by functions such as the driving element, the side brush 7, the rolling brush and/or the blower fan, and these parts are located in the accommodating cavity or its bottom.
  • the sound signal acquisition device is arranged on the raised Be arranged on the raised structure 2 on the upper surface 10 of the robot shell 1, so that the sound signal collection device 3 is far away from the noise source of the robot, and reduce the interference of the noise that the robot itself sends to the sound signal collection device 3, so that the robot can be more Accurately collect user voice control instructions.
  • the user's voice control instructions include starting sweeping, playing music, stopping sweeping, recharging, etc. Those skilled in the art can set corresponding functions according to the actual needs of the robot.
  • Fig. 1C is a schematic structural diagram of the sound signal collection device of the self-mobile robot.
  • the sound signal collection device 3 includes a microphone (MIC).
  • the sound signal acquisition device 3 includes a PCB board 30 (printed circuit board), a shock-absorbing case 31 and a microphone chip 32; wherein the shock-absorbing case 31 is arranged on the PCB board 30 and The external packaging of the sound signal acquisition device 3 with a housing cavity is surrounded by the PCB board 30, the microphone chip 32 is arranged in the housing cavity, and the central area of the top of the shock-absorbing case 31 is provided with a sound pickup hole 310 communicating with the outside and the housing cavity .
  • PCB board 30 printed circuit board
  • the shock-absorbing case 31 is arranged on the PCB board 30
  • the external packaging of the sound signal acquisition device 3 with a housing cavity is surrounded by the PCB board 30, the microphone chip 32 is arranged in the housing cavity, and the central area of the top of the shock-absorbing case 31 is provided with a sound pickup hole 310 communicating with the outside and
  • the PCB board 30 is communicatively connected with the microphone chip 32 and the control unit of the robot.
  • the microphone chip 32 collects an external sound signal from the sound pickup hole 310 and transmits it to the control unit through the PCB board 30.
  • the control unit controls the robot to execute the user's voice included in the sound signal. Control instruction.
  • the shock-absorbing cover 31 of the sound signal collecting device 3 can reduce the impact of the vibration generated in the working process of the robot on the sound signal collecting device 3 on the one hand, and the shock-absorbing cover 31 can absorb vibrations from the robot on the other hand.
  • the sound pickup hole 310 is set in the central area of the top of the shock-absorbing case 31, and it only collects the sound signal from the top (usually a voice control command issued by the user). Especially for the sweeping robot, the sweeping robot generally works on the ground and the user sends out voice control from a height.
  • the sound pickup hole 310 located in the top center area of the shock-absorbing casing 31 can easily collect the sound signal of the user's voice control.
  • the noise emitted by the robot itself can be blocked by the shock-absorbing casing 31 surrounding the sound pickup hole 310 , which can reduce its interference on the signal collected by the sound signal collecting device 3 .
  • the shock-absorbing shell 31 includes shock-absorbing foam. It can be understood that the shock-absorbing foam can not only prevent the noise from the robot itself from entering the sound pickup hole 310, but also absorb part of the noise.
  • the sound signal acquisition device 3 also includes a waterproof and dustproof film 33, which is arranged on the shock-absorbing casing 31 and covers the sound pickup hole 310 to prevent water or dust from passing through the sound pickup hole 310 Falls on the microphone chip 32, and influences the effect that the microphone chip 32 collects the sound signal.
  • the sound signal acquisition device 3 also includes an upper cover 34, the upper cover presses the shockproof cover 31 on the PCB board, and is fixedly connected by connectors such as screws (not shown in the figure)
  • the protruding structure 2 or on the distance sensor 3 On the protruding structure 2 or on the distance sensor 3, a fixed connection relationship between the sound signal collection device 3 and the robot shell 1 is realized, so as to prevent the sound signal collection device 3 from falling off from the robot shell 1 during the driving process of the robot.
  • a sound pickup hole is also provided on the top center area of the upper cover 34 corresponding to the sound pickup hole of the shock-absorbing case 31 .
  • the above-mentioned purpose is achieved by limiting the aperture-to-depth ratio of the sound pickup hole 310, specifically , the ratio of the diameter (d1) to the depth (d2) of the sound pickup hole 310 is greater than 1 as much as possible. In a more specific embodiment, the ratio of the diameter (d1) to the depth (d2) of the sound pickup hole 310 is greater than 2:1.
  • the robot includes at least three sound signal collection devices 3 , and these sound signal collection devices 3 are uniformly distributed in a ring.
  • a plurality of ring-shaped and evenly distributed sound signal acquisition devices 3 can evenly collect sound signals transmitted from various angles, so as to ensure the accuracy and consistency of the collected user voice control signals.
  • Fig. 1D is another schematic structural view of the sweeping robot provided by the embodiment of the present application.
  • the robot includes three sound signal acquisition devices 3, and these three sound signal acquisition devices 3 are evenly distributed in a ring, that is, the three sound signal acquisition devices 3 are located on a circle, and each sound signal acquisition device 3 reaches the center of the circle.
  • the distances are all the radius of the circle, and the central angle between two adjacent sound signal collecting devices 3 is 120° (degree).
  • the diameters of the circles in which the at least three sound signal collection devices 3 are uniformly distributed in a ring are within the range of 60 mm to 100 mm.
  • the robot includes three sound signal collection devices 3, and the three sound signal collection devices 3 are distributed in a triangle, and one of the three sound signal collection devices 3 is located on the robot shell 1 relative to the other two. the front of the surface 10.
  • These three sound signal acquisition devices 3 can be uniformly distributed in a ring, that is to say that these three sound signal acquisition devices 3 are located on the circumscribed circle of the triangle and the central angle between two adjacent sound signal acquisition devices 3 is 120 ° ( Spend).
  • the three sound signal collection devices 3 do not need to be evenly distributed in a ring, and only need to ensure that they are arranged in a front-to-back arrangement.
  • the advantage of this arrangement is that when the sweeping robot is moving forward, the voice control command issued by the user is delayed due to transmission in air and other media, and the front sound signal acquisition device 3 on the upper surface 10 of the robot shell 1 will only collect a small amount. sound signal, and most of the sound signals need to be collected by the sound signal acquisition device 3 located at the rear, and more sound signal acquisition devices 3 are set at the rear to better collect the sound signal and ensure the accuracy of the collected sound signal.
  • the selection criteria for the sound signal acquisition device 3 are also provided, specifically: select an omnidirectional digital microphone, and its signal-to-noise ratio (Signal-to-noise ratio, SNR) greater than 64dB(A), sensitivity guaranteed -26+3dBFS, acoustic overload point (Acoustic Overload Point, AOP) guaranteed 120Db SPL, total harmonic distortion (total harmonic distortion, THD) 94dB SPL Preferably less than 0.5% at @1kHz.
  • SNR Signal-to-noise ratio
  • SNR signal-to-noise ratio
  • AOP acoustic overload point
  • THD total harmonic distortion
  • the robot also includes a distance sensor 4, the distance sensor 4 is arranged on the robot shell 1, and it is used to measure the distance between the obstacle in front of the moving direction of the robot and the robot, so that the distance between the two When the distance reaches the set threshold, the robot can stop moving or change the moving path to prevent the robot from colliding with obstacles.
  • the distance sensor 4 is rotatably arranged on the robot shell 1, which can rotate 360 degrees relative to the robot shell to detect the layout of furniture, walls, etc. in the workspace, and then draw a map of the workspace, And work according to the drawn map to improve work efficiency.
  • the distance sensor 4 includes DTOF and LDS.
  • the distance sensor 4 is disposed on the above-mentioned protruding structure 2
  • the sound signal collecting device 3 is disposed on the distance sensor 4 . It can be seen that the distance sensor 4 and the sound signal collecting device 3 can utilize the protrusion structure 2, and there is no need to separately provide protrusions for each, which can simplify the structure of the robot as much as possible and reduce its manufacturing cost.
  • the protruding structure 2 includes a distance sensor 4, that is to say, the distance sensor 4 is directly arranged on the upper surface of the robot shell 1 to form a protruding structure 2, and the sound signal collecting device 3 is arranged on The distance sensor 4 , that is, the sound signal collecting device 3 is arranged on the protruding structure 2 formed by the distance sensor 4 .
  • the distance sensor 4 is directly arranged on the upper surface of the robot shell 1 to form a raised structure 2, and the sound signal acquisition device 3 is set on the robot shell 1 by using the characteristic of the distance sensor 4 itself, and there is no need for additional raised structures, and the overall structure is simple ,low cost.
  • the distance sensor 4 is on the upper surface 10 of the robot shell 1, which can well avoid other structures of the robot itself, so as to accurately sense the position of obstacles.
  • the sound signal acquisition device 3 can be as far away from noise-generating parts as possible, such as the driving motor of the robot, the roller brush, the side brush 7 and the blower fan, and can reduce the interference of the noise generated by the robot itself on the sound signal.
  • the robot also includes a sound signal playing device 5, which can be a loudspeaker (speaker), and the sound signal playing device 5 is arranged on the robot shell 1, and the sound signal playing device 5 and
  • the control unit of the robot is connected by communication, and the control unit is provided with a broadcasting working mode of the robot, such as playing music. After the user controls the robot to enter the broadcasting working mode through the remote controller or APP, the music stored in the control unit will be played out through the sound signal playing device 5 .
  • the sound pickup hole 310 of the sound signal collection device 3 and the amplifier of the sound signal playback device 5 face in different directions. More specifically, the sound pickup hole 310 of the sound signal collecting device 3 is oriented towards the upper surface 10 perpendicular to the robot shell 1, while the sound emitting hole of the sound signal playing device 5 is oriented towards the outer surface perpendicular to the robot shell 1. 11, that is to say, the direction of the sound pickup hole 310 of the sound signal collecting device 3 and the sound emitting hole of the sound signal playing device 5 are set at an angle of 90° (degrees).
  • the upper surface 10 and the outer facade 11 of the robot shell 1 are arranged perpendicular to each other. In the case of facing different directions, the upper surface 10 and the outer surface 11 of the robot housing 1 are arranged at other angles.
  • the sound signal playing device 5 is located at the front of the robot housing 1
  • the sound signal collecting device 3 is located at the rear of the robot housing 1
  • the sound signal playing device 5 is located at the rear of the robot shell 1
  • the sound signal collecting device 3 is located at the rear of the robot shell 1 .
  • the division standard of the front part and the rear part of the robot shell 1 is based on the shape of the robot shell 1 and divides it into two along the front and back, wherein, the area located on the front side of the robot shell 1 is the front part, and the area located on the rear side of the robot shell 1 for the rear. For example: taking the embodiment shown in FIG. 1C as an example, the circular robot shell 1 is divided into a front semicircle area and a rear semicircle area along the front and rear directions, the front semicircle area is defined as the front part, and the rear semicircle area is defined as the rear part.
  • one of the sound signal collecting device 3 and the sound signal playing device 5 is positioned at the front portion of the robot casing 1, and the other is positioned at the rear portion of the robot casing 1, so that a sufficient distance is kept between the two, thereby further
  • the robot can more accurately collect the user's voice control command and execute the command accurately, thereby providing a better user experience for the user.
  • the robot also includes a sound signal recovery device, which is connected with the control unit of the robot and the sound signal player.
  • the device 5 is connected in communication, and it is used for the sound signal of the sound signal playing device 5, and the control unit accepts the sound signal of the sound signal recovery device, and filters the sound signal of the sound signal from the sound signal collected by the sound signal acquisition device 3, and then The instruction contained in the filtered sound signal is transmitted to the actuator, and the actuator is controlled to execute the instruction.
  • the sound signal recovery device includes a filter-type recovery circuit, and the filter-type recovery circuit is electrically connected to the control unit of the robot body through a wire, and is electrically connected to the sound signal playing device through a wire.
  • the robot also includes a sound signal noise reduction device, which is connected in communication with the sound signal collection device 3 and the control unit, and is used to control the sound signal collection device 3
  • the collected sound signal is subjected to noise reduction processing, so as to eliminate noise or invalid sound signal part of the collected sound signal.
  • the present invention also provides a kind of control method that is applicable to above-mentioned robot, to eliminate the invalid sound signal that sound signal collection device 3 gathers, especially will eliminate the sound signal that robot itself sends to the signal collection of sound collection signal caused by the interference.
  • a kind of control method that is applicable to above-mentioned robot, to eliminate the invalid sound signal that sound signal collection device 3 gathers, especially will eliminate the sound signal that robot itself sends to the signal collection of sound collection signal caused by the interference.
  • FIG. 2A please refer to FIG. 2A.
  • Fig. 2A is a voice control flow chart of the self-mobile robot provided by the embodiment of the present application. This example includes:
  • the sound signal collected by the sound signal collection device 3 mainly includes the user's voice control instructions to the robot, for example, the robot uses the sound signal collection device 3 and other sound signal collection devices 3 to collect the sound signals included in the user's voice control.
  • the functional components such as the driving motor, side brush 7, rolling brush and/or fan of the robot can also generate sound signals during the working process of the robot, or the robot itself also has the ability to generate sound signals, such as the robot in the working process Music can be played, books can be read aloud, etc. in the middle or shutdown state.
  • the main function of the sound signal acquisition device 3 is to collect the user's voice control, these sound signals generated by the robot itself are collectively referred to as "invalid sound signals" in this paper. Based on this, in order to eliminate the interference of these invalid sound signals on the signal collected by the sound signal acquisition device 3, the control method of the robot of the present invention also includes the following steps:
  • Fig. 2B is another voice control flowchart of the self-mobile robot provided by the embodiment of the present application. Please refer to FIG. 2B.
  • the method for implementing step S2 in the control method includes the following steps:
  • a sound signal playback device 5 is set in the robot, and the sound signal playback device 5 can be a loudspeaker (horn), the sound signal playback device 5 is arranged on the robot shell 1, and the control of the sound signal playback device 5 and the robot
  • the unit is connected by communication, and the control unit is provided with a working mode of the robot, such as playing music, etc. After the user controls the robot to enter the control mode through a remote control or APP, the music stored in the control unit is played through the sound signal playback device 5 .
  • This robot also comprises sound signal recovery device, and this sound signal recovery device is connected with the control unit of robot and sound signal playback device 5 communication, and it is used for the sound signal of recovery sound signal playback device 5, and control unit accepts the sound signal of sound signal recovery device recovery. Sound signal and filter the recovered sound signal from the sound signal collected by the sound signal collecting device 3, and then transmit the instruction contained in the filtered sound signal to the executive element, and control the executive element to execute the instruction.
  • Fig. 2C is another voice control flowchart of the self-mobile robot provided by the embodiment of the present application.
  • the method for implementing step S2 in the control method includes the following steps:
  • the sound signal acquisition device 3 is used to collect the sound signal in the control method of the present invention
  • the sound signal is denoised first, and then the sound signal played by the robot is filtered to obtain an effective sound. signal to further eliminate the influence of other voice signals other than the user's voice control instructions.
  • control method After obtaining the effective sound signal from step S2, the control method then performs the following steps:
  • the sweeping robot is cleaning the ground, and the user sends out the voice control command of "play music", and the robot starts to play the stored music after collecting the command.
  • the user can also order the desired music according to the audio data stored in the robot, and the voice control command only needs to include the name of the music.
  • the current sweeping robot is in the shutdown or standby state.
  • the user sends out the voice control command of "sweeping the floor”. After collecting the command, the robot starts to clean the ground according to the predetermined route.
  • the sweeping robot is cleaning the ground and playing music at the same time.
  • the user sends out the voice control command of "stop playing music", and the robot stops playing music after collecting the command and filtering out the invalid sound signal generated by playing music.
  • Fig. 3 is a flow chart of the control method of the self-mobile robot provided by the embodiment of the present application.
  • the execution subject of this embodiment is a self-mobile robot.
  • This example includes:
  • the microphone array on the self-mobile robot includes multiple microphones, and the self-mobile robot can determine the direction of the sound source according to the time difference or sound intensity of the voice signals received by each microphone.
  • Speech signals usually include location keywords, such as “come here and scan”, “scan here”, “come here” and so on.
  • the self-mobile robot After the self-mobile robot determines the sound source, it rotates at a certain angle so that the front of the self-mobile robot faces the user.
  • the self-mobile robot facing the user means that the camera of the self-mobile robot faces the user.
  • the self-mobile robot can construct an environmental map and plan a path according to the Simultaneous Localization and Mapping (SLAM) algorithm during the moving process
  • SLAM Simultaneous Localization and Mapping
  • the environmental map obtained based on the SLAM algorithm only contains stationary objects.
  • a 3D sensor such as a Direct Time-of-Flight (DTOF) sensor or an AI camera is installed on the mobile robot, and images collected by the 3D sensor or AI camera can be used to determine the location of the mobile robot from around the mobile robot. moving objects.
  • DTOF Direct Time-of-Flight
  • AI camera an AI camera
  • the DTOF sensor quickly and continuously scans the surrounding environment 360 degrees, and uses the difference between the two or several frames before and after to extract the moving object, and according to the moving object's trajectory, motion mode, etc. from multiple Separate the pedestrians from the moving objects, take the pedestrians in the direction of the sound source as the target object, and then track the target object.
  • the self-moving device is a sweeper, and the sweeper works in the living room, and the moving objects in the living room include children, adults, kittens and puppies, and balls.
  • the moving objects in the living room include children, adults, kittens and puppies, and balls.
  • 303. Determine a target object located in the direction of the sound source from the moving objects.
  • the mobile object may be located in any direction within 360 degrees around the mobile robot.
  • the direction of each mobile object relative to the self-mobile robot is further determined.
  • the moving object whose direction is the same as that of the sound source is taken as the target object, and the target object is the user who sent the voice signal in step 301 . If the direction of each object does not coincide with the direction of the sound source, the moving object whose direction is close to the direction of the sound source is used as the target object.
  • the self-mobile robot can determine the position of the mobile object in space, and then determine the initial distance between the self-mobile robot and the target object.
  • the mobile robot After the mobile robot determines the target object, it moves towards the target object. If the target object has not moved since the voice signal was sent, the working area is determined based on the initial position of the target object. For example, take the initial position of the target object as the center, draw a circle with a radius of 2 meters, and use the circular area as the working area. It can be understood that if an object such as a wall is encountered in the process of drawing a circle, the working area is determined by combining the outline of the object and the circle. By adopting this scheme, the self-mobile robot can accurately reach the user's designated area.
  • the self-mobile robot follows the target object until the target object stops moving. Afterwards, the self-mobile robot determines the work area according to the position of the target object when it stops moving. In this scheme, the purpose of guiding the self-mobile robot to reach the designated position and perform the task is realized.
  • the self-mobile robot plans a path according to its own location and the position of the target object, and controls the self-mobile robot to move to the vicinity of the target object according to the path. Afterwards, perform tasks within the work area. where the length of the path is approximately the length of the initial distance between the mobile robot and the target object.
  • the target object If the target object is displaced, after the self-movement moves to the vicinity of the target object according to the path, it will continue to move with the target object until the target object stops moving. Afterwards, perform tasks within the work area.
  • a 3D sensor such as a laser sensor
  • the laser sensor is, for example, a DTOF sensor.
  • Each mobile object has depth information based on the 3D sensor, so that the self-mobile robot can determine the position of the mobile object in space, and then determine the initial distance between the self-mobile robot and the target object. Therefore, during the traveling process, the self-mobile robot can accurately reach the working area.
  • the self-mobile robot determines the direction of the sound source according to the voice signal sent by the user, and determines the moving objects around itself. After that, determine the target object located in the direction of the sound source from the moving objects around itself, determine the working area according to the target object, and move to the working area and execute the task.
  • the self-mobile robot determines the target object from the mobile object, the mobile object has an accurate spatial position. Therefore, the self-mobile robot can accurately determine the target object from multiple moving objects according to the direction of the sound source, and accurately arrive at the working area without using the client, and the process is simple and flexible.
  • this solution is applicable to all laser-type self-mobile robots, with low cost, simple algorithm, and low computing power required.
  • Scenario 1 There are no obstacles in the direction of the sound source, and the self-mobile robot uses the AI camera to determine the target object.
  • the self-mobile robot is located in a relatively open area with no obstacles around it.
  • the user only needs to send out a voice signal without stepping on the ground twice.
  • the self-mobile robot uses the AI camera to determine the target object.
  • the self-mobile robot and the user are located in the same space.
  • the self-mobile robot determines the direction of the sound source according to the voice signal, it uses an AI camera to collect images of the direction of the sound source, and determines whether there are objects other than pedestrians in the direction of the sound source based on the image. If there is no object other than pedestrians in the direction of the sound source, it is considered that there is no obstacle in the direction of the sound source, continue to use the AI camera to capture images in the direction of the sound source, and use the image captured by the AI camera to determine the target object. During this process, the user does not need to make actions such as lightly stepping on the ground twice.
  • Scenario 2 There are obstacles that can pass through below in the direction of the sound source, and the self-mobile robot uses the DTOF sensor to determine the target object.
  • Self-mobile robots use DTOF sensors to determine target objects. If the user only sends out a voice signal, the self-mobile robot prompts the user to make actions such as stepping on the ground twice.
  • a user sits on a sofa, a coffee table is placed in front of the sofa, and the self-mobile robot is placed in front of the coffee table.
  • the coffee table obscures part of the user's body.
  • the mobile robot determines the direction of the sound source according to the voice signal, it uses the AI camera to collect the image of the direction of the sound source, and determines the direction of the sound source based on the image.
  • the self-mobile robot determines the target object according to the SLAM graph and DTOF scatter graph collected by the DTOF sensor.
  • the self-mobile robot determines that the current scene is scene two, if the self-mobile robot does not find a moving object by using the DTOF sensor. At this time, the self-mobile robot can prompt the user to make actions such as stepping on the ground twice, so that the self-mobile robot can determine the target object.
  • Fig. 4 is another flow chart of the control method of the self-mobile robot provided by the embodiment of the present application.
  • the self-mobile robot is specifically a sweeper, and this embodiment includes:
  • the purpose of the user sending out the voice signal is to make the sweeper determine the direction of the sound source.
  • the user steps on the ground lightly in order to make the sweeper recognize the target object, determine the specific position of the target object in space, and this specific position is also called the initial position.
  • the sweeper navigates to the area near the ground where human legs lightly step on the ground.
  • the sweeper navigates to the vicinity where the user treads lightly on the ground.
  • the area where the user steps on the ground is the initial position of the target object in space. After that, the sweeper navigates to the foot of the target object according to the DTOF tracking algorithm. If the user does not move, the working area is determined according to the initial position.
  • the target object If the target object is displaced, it will follow the target object to move. That is to say, if the target object keeps moving when the sweeper moves to the target object, or if the sweeper moves to the front and back of the target object, and the target object moves to cause displacement, the sweeper follows the target object to the designated position,
  • the specified position is the position when the target object stops walking. Afterwards, the sweeper determines the working area according to the position of the target object when it stops moving.
  • Scenario 3 There are obstacles in the direction of the sound source, and the obstacles completely block pedestrians.
  • the self-mobile robot cannot pass under the obstacle.
  • the obstacle is a refrigerator.
  • the self-mobile robot is located in one room and the user is located in another room.
  • the self-mobile robot determines the direction of the sound source according to the voice signal, it uses the AI camera to collect the image of the direction of the sound source, and determines whether there is an obstacle blocking pedestrians in the direction of the sound source according to the image. If there are obstacles blocking pedestrians in the direction of the sound source, determine the approximate navigation path, continuously collect images during the movement process according to the navigation path, and adjust the navigation path.
  • the mobile robot after the mobile robot has determined the direction of the sound source, it is necessary to further determine the target object from multiple mobile objects. If the target object is displaced, the target object needs to be tracked.
  • Self-mobile robots can track pedestrians through visual tracking, use cameras to capture pedestrian images, and then use image processing algorithms to extract pedestrians from the images and lock target objects for tracking.
  • the camera has relatively high requirements on the environment, and the intensity of the ambient light must meet certain conditions. If the intensity of ambient light is relatively low, such as when the screen is completely black, high-quality images cannot be collected.
  • the image processing algorithm is relatively complex, which requires relatively high computing power of the chip, and it is difficult to realize dynamic tracking. Equipping a large number of autonomous mobile robots with high-quality cameras is costly.
  • the embodiment of the present application may also determine the target object and track the target object.
  • the 3D sensor specifically as the DTOF sensor as an example, how the self-mobile robot determines and tracks the target object will be described in detail.
  • Fig. 5 is a flowchart of determining a target object. This example includes:
  • the SLAM graphs in the multiple SLAM graphs are in one-to-one correspondence with the DTOF scatter graphs in the multiple DTOF scatter graphs.
  • the self-mobile robot uses the DTOF sensor to scan the surrounding environment, detect the surrounding environment, and obtain multiple SLAM images and multiple DTOF scatter images. For example, if a mobile robot collects SLAM graphs and DTOFS scatter graphs synchronously, and collects 5 frames of SLAM graphs and 5 frames of DTOF scatter graphs in one second, then the SLAM graphs in 5 frames of SLAM graphs and the DTOF scatter graphs in 5 frames of DTOF scatter graphs One-to-one correspondence of point diagrams.
  • Figure 6 shows the LAM diagram. Please refer to Figure 6. Only stationary objects, such as walls, are marked in the SLAM diagram. When the self-mobile robot builds an environmental map based on the SLAM algorithm, it can recognize and label the outlines of objects in the surrounding environment, such as walls, sofas, coffee tables, beds, etc. In Figure 6, only the wall is marked, as shown by the thick black solid line in the figure.
  • Figure 7 shows the DTOF scatter plot. Please refer to Figure 7. Unlike the SLAM diagram, the DTOF scatter diagram has both pixels representing static objects and pixels representing moving objects.
  • the thick black solid line in the figure shows the wall, and the solid ellipse represents pedestrians and stray points respectively.
  • the SLAM diagram it is possible to identify which points represent the wall and which points represent the sofa, coffee table, bed, etc. from the DTOF scatter diagram, that is, the pixels representing static objects in the DTOF scatter diagram can be identified. Afterwards, the pixels representing static objects are filtered out from the DTOF scatter diagram to obtain a dynamic point set.
  • the dynamic point set includes some stray points and points corresponding to moving objects.
  • the points used to represent objects such as walls and sofas in the DTOF scattergram are determined according to the corresponding SLAM diagram. After that, for any two adjacent frames of DTOF images, the points on the two frames of DTOF scatter diagrams are all drawn in the same blank image. If an object is a stationary object, the points representing the stationary object in the two frames of DTOF scatter diagrams are located at the same position; if an object is a moving object, the points representing the moving object in the two DTOF scatter diagrams are located at different positions. location and are relatively similar. Therefore, after drawing the pixels in two adjacent DTOF scatter diagrams in the same blank image, the dynamic point set can be determined.
  • the dynamic point set includes some stray points and points corresponding to moving objects.
  • the purpose of collecting SLAM diagrams and DTOF scatter diagrams from the mobile robot is to find and follow the target object, and the target object is usually a pedestrian. Therefore, in order to reduce the amount of calculation, there is no need to consider other moving objects, such as rolling balls, etc. .
  • the self-mobile robot determines the moving objects around the self-mobile robot, before determining the target object located in the direction of the sound source from the moving objects, according to the characteristics such as gait and movement speed of human beings when walking, from A target object that may be a pedestrian is determined from multiple moving objects, thereby filtering out some stray points.
  • the positions of the stray points in different DTOF images are different, and there is no rule to be found. Even if the stray points in two adjacent frames of DTOF images are drawn into the same blank image, no rules can be concluded.
  • the moving object is different. If the same moving object is drawn in the same blank image in two adjacent frames of DTOF images, the moving object is located in two different positions, and the distance between the two positions satisfies certain conditions and the two The number of points in the point set of positions is close.
  • the point set representing the ball is located at position A in the blank image
  • the point set representing the ball is located at position B in the blank image
  • the number of pixels in the point set at position A is approximately equal to the number of pixels in the point set at position B
  • the shape formed by the point set at position A is the same as that formed by the point set at position B
  • the points are similar in shape.
  • the purpose of determining the moving objects and stationary objects from around the mobile robot is realized according to the DTOF scatter diagram and SLAM diagram of the front and rear frames.
  • the self-mobile robot determines whether there is a second subset in the second dynamic point set of the second DTOF scatter diagram, the first position indicated by the first subset is the same as the second position indicated by the second subset The distance between them is greater than a preset distance, and the difference between the number of pixels in the first subset and the second subset is less than a preset difference, the first DTOF scattergram and the second DTOF scattergram
  • the point diagram is any two adjacent DTOF scatter diagrams in the plurality of DTOF scatter diagrams. If the second subset exists in the second dynamic point set, the first subset and the second subset are determined. The subsets represent the same object and the object is a moving object.
  • the preset distance is the minimum distance between the first position and the second position when an object is a moving object.
  • the dynamic point set of each frame of DTOF scatter diagram may contain one or more point sets corresponding to moving objects and some stray points.
  • a first subset is determined from the first dynamic point set of the first DTOF scattergram by the mobile robot, and the first subset includes a plurality of pixel points in the comparison set.
  • the self-mobile robot determines whether there is a second set of points in the second set of dynamic points in the second DTOF scattergram. If the second point set exists in the second dynamic point set, it means that the first point set and the second point set represent the same object and the object is a moving object.
  • the mobile robot estimates the walking speed of the target object, etc., and filters out the objects that do not meet the conditions such as the walking speed of the target object.
  • the position coordinates of the ball in the first DTOF scatter diagram are the same as those of the ball in the second DTOF
  • the distance of the position coordinates in the scatter plot is about 20 cm. Therefore, if the distance between the position A corresponding to the first subset and the position B corresponding to the second subset is 20cm, and the number of pixels in the first subset and the second subset is close, it means that the first subset and the second subset
  • the subsets represent the same object and the object is a moving object.
  • the self-mobile robot can determine the surrounding moving objects according to the dynamic point set.
  • the purpose of collecting SLAM graphs and DTOF scattergrams from the mobile robot is to find the target object and follow the target object, and the target object is usually a pedestrian. Therefore, in order to reduce the amount of calculation, there is no need to consider other moving Objects such as rolling balls etc.
  • the self-mobile robot determines the moving objects around the self-mobile robot, before determining the target object located in the direction of the sound source from the moving objects, according to the characteristics such as gait and movement speed of human beings when walking, from A target object that may be a pedestrian is determined from multiple moving objects, and then the target object located in the direction of the sound source is determined from the pedestrian.
  • the mobile robot determines the moving object whose foot moves and is located in the direction of the sound source from the moving objects, so as to obtain the target object .
  • the height of the self-mobile robot is usually limited. Taking a sweeper as an example, the height of the sweeper is usually 10 centimeters, and the sweeper can only collect DTOF images within a height range of 10 centimeters.
  • the user needs to make movements when issuing voice commands, such as lightly stepping on the ground twice, switching from keeping the left and right feet together to opening the left and right feet at a certain angle, and opening the left and right feet to a certain angle. The angle is switched to the left and right feet close together, etc. If the user makes actions such as waving, applauding, shaking the head, etc., although the user is exercising, it cannot be collected by the DTOF sensor because it is not in the field of view of the DTOF sensor. Therefore, these actions cannot be realized. plan.
  • a pre-trained model is deployed on the mobile robot, which can recognize the user's action of lightly stepping on the ground according to the DTOF scatter diagram.
  • the action of the moving object represented by the first subset and the second subset is determined according to the model as stepping on the ground lightly, if the If the moving object is located in the direction of the sound source, then the moving object is determined to be the target object.
  • a moving object must be lightly stepped on the ground and the moving object is located in the direction of the sound source before the moving object is determined as the target object, so as to achieve the purpose of accurately determining the target object.
  • the above is to determine the moving objects around the self-mobile robot and determine the target object therefrom. Next, how to follow the target object will be described in detail.
  • the self-mobile robot determines the target object and moves towards the target object. After that, if the target object moves, that is, when the target object moves, the navigation technology is used to follow the target object until the target object stops moving. Afterwards, the working area is determined according to the position of the target object when it stops moving.
  • the self-mobile robot can follow according to local planning algorithms.
  • Local programming algorithms include: Vector Field Histogram (Vector Field Histogram, VFH) algorithm, dynamic window approach (DWA), etc. From the perspective of the hardware used, following includes DTOF sensor-based following and AI camera-based following.
  • the self-mobile robot determines whether the target object appears in the adjacent two DTOF scatter diagrams, if the target object appears in the adjacent two DTOF scatter diagrams, then Determine the difference between the distance of the target object in two adjacent DTOF scatter diagrams and the preset distance. Afterwards, the speed is adjusted according to the difference to follow the movement of the target object.
  • the distance between the position coordinates of the target object in two adjacent frames of DTOF scattergrams is about 20 centimeter. Therefore, during the following process, for any two adjacent frames of DTOF scattergrams, the self-mobile robot determines whether the target object appears in the two frames of DTOF scattergrams. If the target object appears in the two frames of DTOF scatter diagrams, it means that there is no tracking loss.
  • the self-mobile robot determines the position coordinates of the target object in the first frame of the DTOF scatter diagram (the previous frame). Afterwards, according to the direction of travel, determine whether there is a target object at a position 20 cm away from the position coordinates in the second frame of the DTOF scatter diagram (the next frame). If there is a target object, it means that the user is not lost and continues to follow the user.
  • the mobile robot collects the DTOF scattergram each time, calculate the distance of the target object according to the DTOF scattergram and the previous frame DTOF scattergram, and compare the distance with the previous distance. If the distance increases, it means that the moving speed of the target object is accelerated, and the self-mobile robot increases its speed and moves with the target object. If the distance decreases, it means that the moving speed of the target object slows down, and the self-mobile robot slows down and moves with the target object. If the distance remains unchanged, it means that the moving speed of the target object remains unchanged, and the self-mobile robot maintains the current speed and moves with the target object.
  • Fig. 8 is a follow-up flow chart based on an AI camera in the self-mobile robot control method provided by the embodiment of the present application. This embodiment includes:
  • the user wakes up or summons the mobile robot.
  • the user speaks a wake-up word to the self-mobile robot to wake up the self-mobile robot.
  • the wake-up word is "Xiao Q Xiao Q”
  • the voice control function of the self-mobile robot is in the wake-up state, after the user sends out the voice signal of "Xiao Q Xiao Q”
  • the voice control function of the self-mobile robot wakes up, after that, the user Voice interaction with mobile robots.
  • the user speaks a conversion keyword to the self-mobile robot to summon the self-mobile robot.
  • the calling keyword is "Little Q, come here to scan”.
  • the voice control function of the self-mobile robot is awakened, after the user sends out the voice signal of "Small Q, come here to scan", the self-mobile robot recognizes the call and marches towards the user.
  • the summoning keyword also has the function of awakening and summoning.
  • the voice control function of the self-mobile robot is in the waiting state. After the user sends out the voice signal of "Small Q, come here to scan", the voice control function of the self-mobile robot is awakened. At the same time, the self-mobile robot recognizes the call and sends a message to the user March.
  • the self-mobile robot After the self-mobile robot is woken up or recognizes the call, use the ring microphone to position and rotate a certain angle to make the self-mobile robot basically face the user, continue to rotate until the AI camera can capture the user, and use the portrait captured by the AI camera to position the user, so that Make the AI camera of the self-mobile robot face the user.
  • the self-mobile robot utilizes AI's humanoid positioning to precisely face the user.
  • AI camera captures multiple users, it will issue a prompt message to make the user make specific actions, such as waving hands, shaking the head, etc., so as to determine the target object again.
  • the AI humanoid positioning algorithm provides the angle of the target object relative to the self-mobile robot and the skeleton map of the target object.
  • Step 804 and 806 are performed in the following process.
  • the self-mobile robot judges whether it is necessary to circumvent the obstacle, and if it is necessary to circumvent the obstacle, perform step 805; if it does not need to circumvent the obstacle, perform step 808.
  • step 808 is performed.
  • step 802 is performed.
  • the self-mobile robot judges whether it is necessary to overcome obstacles, and if it is necessary to overcome obstacles, perform step 807; if it does not need to overcome obstacles, perform step 808.
  • the self-mobile robot overcomes obstacles, and then execute step 808.
  • the self-mobile robot can obtain ground information by using line laser sensors, etc., and then recognize whether there are steps or cliffs in front of the self-mobile robot. If there are steps and cliffs, you need to overcome obstacles.
  • the line laser sensor of the self-mobile robot adopts a dynamic exposure scheme, that is, the exposure value is increased on materials with low reflectivity to obtain more effective data; Lower the exposure value on materials with higher reflectivity to get more accurate data.
  • the self-mobile robot recognizes the height of the steps and the height of the cliff according to the data raised by the line laser sensor, and determines the obstacle-crossing strategy according to the height of the steps and the height of the cliff.
  • the self-mobile robot judges whether to end the following, and if so, end the following; if not end the following, execute step 809.
  • step 809 Determine whether the self-mobile robot has lost track, and if the target object is not lost, perform step 802; if the target object is lost, perform step 810.
  • the self-mobile robot determines a search range according to the last position coordinates of the target object, and searches for the target object within the search range.
  • the search range is, for example, a circular range delineated with a preset radius as the center and a preset radius, and the circular range is, for example, x square meters.
  • the target object If the target object is found, it will continue to follow the target object. If the target object is not found, it will enter the waiting call state and wait for the user to call again. In addition, if the self-mobile robot cannot find the user, it can also issue a prompt message: "I lost track, please guide me", etc., to guide the target object to come to the self-mobile robot.
  • Fig. 9 is a flow chart of keeping following state from the mobile robot. This example includes:
  • the self-mobile robot uses an AI camera to capture a skeleton diagram of a target object.
  • the self-mobile robot determines whether the skeleton diagram is complete, and if the skeleton diagram is complete, execute step 903; if the skeleton diagram is incomplete, execute step 905.
  • the self-mobile robot judges whether there is an obstacle between the self-mobile robot and the target object according to whether the skeleton is complete.
  • the AI camera captures the complete skeleton image, it indicates that there are no obstacles between the self-mobile robot and the target object.
  • the self-mobile robot determines the position coordinates of the target object at the point where the target object is closest to the self-mobile robot in the direction of the AI camera according to the laser sensor data, and then moves to the target object and maintains a fixed distance from the target object.
  • the laser sensor is, for example, a DTOF sensor, a laser radar (Laser Direct Structuring, LDS) and the like.
  • the skeleton image of the target object captured by the AI camera is incomplete, it indicates that there is an obstacle between the self-mobile robot and the target object. There is a high probability that the laser sensor data will be blocked by obstacles, so that the self-mobile robot cannot obtain the laser sensor data, and thus cannot follow. At this time, the laser sensor needs to avoid or overcome obstacles to try to avoid obstacles until the AI camera can capture the complete skeleton image of the target object. After that, move according to the target object.
  • the target object keeps moving, it indicates that the following has not ended; if the target object stops moving, the digital end follows.
  • the self-mobile robot only moves with the target object and does not work during the following process.
  • the sweeping machine does not perform the functions of sweeping and mopping the floor during the following process, until the target object stops moving, the working area is determined according to the position of the target object when it stops moving and works in the working area.
  • the self-mobile robot performs the music playing function.
  • the target object calls the self-mobile robot, and the self-mobile robot follows the target object.
  • the self-mobile robot keeps playing the music. If the target object stops moving, the self-mobile robot keeps a certain distance from the target object and always faces the target object to obtain the best music playback effect.
  • Fig. 10 is a schematic diagram of the structure of the self-mobile robot.
  • the self-mobile robot is, for example, an air purification robot, on which a DTOF sensor, an LDS sensor, a ring microphone, an AI camera, a line laser sensor, etc. are installed.
  • Fig. 11 is a schematic structural diagram of a self-mobile robot control device provided by an embodiment of the present application.
  • the self-mobile robot control device 1100 includes: a first determination module 1101 , a second determination module 1102 , a third determination module 1103 , a processing module 1104 and an execution module 1105 .
  • the first determination module 1101 is configured to determine the sound source direction according to the voice signal sent by the user;
  • the second determining module 1102 is used to determine the moving objects around the self-mobile robot
  • a third determining module 1103, configured to determine a target object located in the direction of the sound source from the moving object;
  • a processing module 1104 configured to determine a working area according to the target object
  • Executing module 1105 configured to move to the work area and execute tasks in the work area.
  • the second determination module 1102 is configured to obtain multiple instant positioning and map construction SLAM diagrams and multiple direct time-of-flight DTOF scatter diagrams, and the SLAM diagrams in the multiple SLAM diagrams and The DTOF scatter diagrams in the multiple DTOF scatter diagrams correspond one-to-one; for each DTOF scatter diagram in the multiple DTOF scatter diagrams, according to the corresponding SLAM diagram, from the DTOF scatter diagram Filter out the pixels representing the static objects to obtain a dynamic point set; determine the moving objects around the self-mobile robot according to the dynamic point sets of multiple adjacent DTOF scatter diagrams.
  • the second determining module 1102 determines the moving objects around the self-mobile robot according to the dynamic point sets of multiple adjacent DTOF scattergrams, it is used to obtain the first DTOF scattergram Determine the first subset in the first dynamic point set of the second DTOF scatter diagram; determine whether there is a second subset in the second dynamic point set of the second DTOF scatter diagram, the first position indicated by the first subset and the first position indicated by the first subset The distance between the second positions indicated by the two subsets is greater than a preset distance, and the difference between the number of pixels in the first subset and the second subset is less than a preset difference, the first DTOF scatter point Figure and the second DTOF scatter diagram are any adjacent two DTOF scatter diagrams in the multiple DTOF scatter diagrams; if the second subset exists in the second dynamic point set, then determine the The first subset and the second subset represent the same object and the object is a moving object.
  • the third determining module 1103 is configured to determine, from the moving objects, a moving object whose foot moves and is located in the direction of the sound source, so as to obtain the target object.
  • the processing module 1104 is configured to move to a position at a preset distance from the target object, and if the target object does not move, determine the target object according to the initial position of the target object. Work area.
  • the processing module 1104 is configured to control the self-mobile robot to follow the target object if the target object is displaced after moving to a position at a preset distance from the target object moving: when the target object stops moving, determine the working area according to the position of the target object when it stops moving.
  • the processing module 1104 is used to determine whether the target object appears in two adjacent DTOF scatter diagrams when controlling the self-mobile robot to move with the target object;
  • the processing module 1104 controls the self-mobile robot to capture the skeleton diagram of the target object with an artificial intelligence AI camera when it moves with the target object; when the skeleton diagram is complete, keep Follow state to follow the movement of the target object; when the skeleton graph is incomplete, avoid or overcome obstacles between the mobile device and the target object until the AI camera captures a complete skeleton After the image is displayed, keep the following state to follow the movement of the target object.
  • the processing module 1104 is also used to wake up the self-mobile robot;
  • the control instruction is used to control the self-mobile robot to determine the working area according to the target object in real time.
  • the processing module 1104 is further configured to determine whether to follow the target object; if the target object is lost, determine the search range according to the position coordinates of the last occurrence of the target object; Search for the target object within the search range; if the target object is not found, enter the waiting call state.
  • the self-mobile robot control device provided in the embodiment of the present application can execute the actions of the self-mobile robot in the above-mentioned embodiments, and its implementation principle and technical effect are similar, and will not be repeated here.
  • the current voice commands can only control the robot to turn on, perform tasks, stop, charge, etc. Precise control of the robot is not possible. For example, if the user wants the robot to perform tasks in a specific area, the robot needs to be transported to the specific area and then controlled by voice. For another example, if some areas are prohibited areas, such as toilets, the user must close the toilets when the robot is working to prevent the robot from entering the prohibited areas.
  • the user wants the robot to perform tasks in multiple areas, and needs to move the robot to one of the areas, and then move the robot to another area after the robot finishes performing tasks in this area.
  • some robots can recognize the work area indicated by the user, the premise is that the voice uttered by the user can only indicate one work area. If the user wants to perform tasks on multiple areas, the user needs to indicate the next working area by voice after each task performed by the robot, which is a troublesome process.
  • the embodiment of the present application also provides a voice control method, device, device, and readable storage medium for a self-mobile robot, which indicates multiple work areas to the self-mobile robot through a voice signal, so that the self-mobile robot can sequentially control multiple The work area performs tasks with high precision and simple process.
  • Fig. 12 is a flow chart of the voice control method for the self-mobile robot provided by the embodiment of the present application.
  • the execution subject of this embodiment is a self-mobile robot, and this embodiment includes:
  • a sound signal collection device is provided on the mobile robot, and the sound signal collection device is, for example, a microphone, a microphone array, and the like.
  • the sound signal acquisition device continuously collects the first voice signal in the surrounding environment, and recognizes the first voice signal. If the first voice signal matches the wake-up instruction of the self-mobile robot, Then wake up the voice control function of the self-mobile robot; if the first voice signal does not match the wake-up instruction of the self-mobile robot, then keep the voice control function in a waiting wake-up state.
  • the voice control function wakes up, after the second voice signal is collected from the mobile robot, the second voice signal is sent to the voice recognition server in the cloud, so that the voice recognition server determines whether the second voice signal matches the control instruction; when the voice control When the function is in the waiting wake-up state, the self-mobile robot locally identifies whether the collected first voice signal matches the wake-up instruction.
  • the self-mobile robot can use the sound signal acquisition device to collect voice signals in the surrounding environment. For example, the second voice signal sent by the user is collected.
  • the self-mobile robot When the self-mobile robot itself has a voice recognition function, the self-mobile robot recognizes the second voice signal, thereby determining at least two working areas.
  • the self-mobile robot sends the second voice signal to the voice recognition server, and the voice recognition server determines at least two working areas and indicates them to the self-mobile robot.
  • the text content corresponding to the second voice signal is "cleaning Xiaoming's room and study”, then at least two work areas are Xiaoming's room and study; for another example, the text content corresponding to the second voice signal is "cleaning all carpet areas", Then the working area is a plurality of areas centered on the carpet and containing the carpet.
  • Fig. 13 is a schematic diagram of the speech recognition process in the embodiment of the present application.
  • the mobile robot collects the second speech signal, performs noise reduction processing on the second speech signal, and uploads the noise reduction processed second speech signal to the speech recognition server.
  • the voice recognition server performs semantic recognition on the second voice signal to obtain a voice command, and sends the voice command back to the self-mobile robot.
  • the mobile robot After the mobile robot receives the voice command, it executes the task indicated by the voice command. Tasks can be sweeping, mopping, mowing, purifying the air, etc.
  • FIG. 13 takes the collection of the second voice signal from the mobile robot as an example for illustration.
  • this embodiment of the present application is not limited.
  • the user can also turn on the client on the terminal device and then send out the second voice signal, and the terminal device collects the second voice signal, and degrades the second voice signal.
  • the second speech signal after the noise reduction processing is uploaded to the speech recognition server.
  • the self-mobile robot performs tasks on each work area in a certain order. For example, the self-mobile robot randomly sorts each work area to obtain a random queue, and performs tasks on each work area in sequence according to the order indicated by the random queue.
  • the second voice signal may be sent out in order of priority.
  • the self-mobile robot determines the sequence in which each of the at least two working areas appears in the second voice signal. Afterwards, the tasks indicated by the second voice signal are sequentially executed on the at least two working areas according to the sequence.
  • the self-mobile robot as an air purification robot as an example
  • the second voice signal is "Please clean the baby room and study room”.
  • the self-mobile robot determines that the working area is a baby room and a study room, and the baby room is given priority. At this time, even if the study room is closer to the self-mobile robot and the baby room is farther away from the self-mobile robot, the self-mobile robot will first go to the baby room, complete the air purification of the baby room, and then go to the study room to purify the air in the study room .
  • the self-mobile robot performs tasks on multiple work areas in sequence according to the priority, which meets the needs of users to a great extent and is more humane.
  • the self-mobile robot determines a plurality of working areas, it determines the distance between itself and each of the at least two working areas. Afterwards, a queue is obtained by sorting the at least two working areas in the order of the distance from the closest to the farthest; and performing the tasks indicated by the second voice signal on the at least two working areas in sequence according to the queue.
  • Figure 14 is a schematic view of a carpeted area.
  • FIG. 15 there are three carpet areas in the figure, which are respectively a carpet area 41 , a carpet area 42 and a carpet area 43 . From the mobile robot 40 it is determined that the carpet area 41 is closest to it, followed by the carpet area 42 and finally the carpet area 43 . Therefore, the cleaning order is carpet area 41 , carpet area 42 and carpet area 43 .
  • the self-mobile robot performs tasks on each work area in the order of distance from near to far, which can maximize energy consumption and increase speed.
  • the self-mobile robot may also be operated in the form of voice operation to perform tasks on a single work area.
  • the voice control method of the self-mobile robot provided in the embodiment of the present application, after the self-mobile robot collects the second voice signal, at least two work areas are determined according to the second voice signal, and the tasks indicated by the second voice signal are executed for each work area in sequence .
  • multiple work areas can be indicated to the self-mobile robot through a single voice signal, so that the self-mobile robot can perform tasks on multiple work areas in turn, and the user interacts with the self-mobile robot through natural language to make the self-mobile robot
  • the task is completed on one of the at least two working areas. After a task, stop executing the task before proceeding to the next work area.
  • the mobile robot After the mobile robot has determined at least two work areas, it performs tasks on each work area in a certain order. If the distance between two adjacent working areas is long, the working module can be closed when the self-mobile robot travels from one working area to the other according to the driving path. That is to say, the self-mobile robot does not perform tasks such as cleaning and mowing while traveling on the driving path.
  • the second voice signal is used to clean the carpet area 43 , the carpet area 42 and the carpet area 41 .
  • the self-mobile robot 40 travels from the current location to the Ditan area 43 .
  • the working module of the self-mobile robot 40 is in a closed state.
  • the self-mobile robot 40 finishes cleaning the carpet area 43 in the process of advancing from the carpet area 43 to the carpet area 42, and when the self-mobile robot 40 completes the cleaning of the carpet area 42, proceeds from the carpet area 42 to the carpet area 41 During the process, the working modules are closed on the driving path, which is shown by the dotted arrow in the figure.
  • the self-mobile robot does not need to open the working module during the process of moving from one working area to the next, which saves energy and increases the speed of travel.
  • the self-mobile robot moves from the current position to the first working area, or before moving from the current working area to the next working area, it is further determined whether the length of the traveling path is greater than a preset threshold. If the length of the travel path is greater than the preset length, then close the working module and proceed to the work area according to the travel path. If the length of the traveling path is less than or equal to the preset length, then travel to the working area with the working module turned on.
  • the driving path between two working areas is relatively short, for example, the length of the driving path between the bedroom and the living room is almost negligible.
  • the driving path is short, there is no need to close the working module.
  • the self-mobile robot determines at least two working areas according to the second voice signal
  • the area category is determined according to the second voice signal.
  • at least two working areas are determined from the area set corresponding to the environment map according to the area category.
  • the self-mobile robot when it is in a completely unknown environment, it will construct an environmental map, or receive an environmental map sent by other robots, and then use a partition algorithm to divide the environmental map into regions to obtain multiple working areas.
  • This enables the environment map to represent various work areas, such as kitchens, bathrooms, bedrooms, etc.
  • the environment map can also represent the actual position of different objects in the environment, so that the self-mobile robot can judge the placement status of objects in each working area.
  • the category information is added to the first voice instruction. For example, "Clean all bedrooms.” Only each bedroom is cleaned since the mobile robot recognizes the voice signal.
  • the self-mobile robot determines the living room from the environment map. Further, the voice signal also indicates the target object "furniture”. Therefore, the self-mobile robot determines the furniture in the living room, such as sofa, coffee table, etc. For each piece of furniture, with the furniture as the center, determine an area containing the furniture, and clean the area.
  • the self-mobile robot determines the bedroom from the environment map. Further, the voice signal also indicates the target object "bed”. Thus, the self-mobile robot continues to determine the positions of the beds within the various bedrooms. For each bed, with the bed as the center, determine an area containing the bed, and clean the area.
  • the behavior of the self-mobile robot is: walk to the location of the sofa, draw a rectangular frame larger than the sofa with the center of the sofa as the origin, use the rectangular frame as the working area and clean it. After that, go to the area where the dining table is located, and draw a rectangular frame larger than the dining table as the working area with the center of the dining table as the origin and clean it.
  • start cleaning the sofa area the behavior of the self-mobile robot is: walk to the location of the sofa, draw a rectangular frame larger than the sofa with the center of the sofa as the origin, use the rectangular frame as the working area and clean it.
  • the user can control the self-mobile robot to perform tasks such as cleaning only a certain type of work area through voice, which is highly intelligent.
  • the target object can also be indicated in the voice signal, so that the self-mobile robot can perform tasks such as cleaning a local area, and further improve the intelligence of the self-mobile robot.
  • the self-mobile robot can divide the environmental map into multiple working areas according to the environmental map, the position information of objects in the environmental map, or the position information of doors in the environmental map to get the set of regions. Afterwards, the identification of each working area in the area set is updated, and update information is sent to the voice recognition server, so that the voice recognition server updates the identification of each working area.
  • a photographing device such as a camera is installed on the mobile robot.
  • Fig. 15 is a schematic diagram of the furniture identification process. Please refer to FIG. 15 , the self-mobile robot constructs an environment map or continuously captures images for image acquisition during the process of traveling. After the image is collected, the image is preprocessed, and the preprocessing includes one or more of contrast enhancement, lossless enlargement, and feature extraction.
  • the self-mobile robot uses the pre-deployed training model to perform AI recognition on the preprocessed image so that the training model outputs the recognition results such as the type and position coordinates of the furniture in the image, and the recognition results are obtained from the (three-dimensional, 3D) environment
  • the map is stored and displayed in a two-dimensional (2D) environment map.
  • the training model is, for example, an AI model trained by taking various furniture as samples.
  • Fig. 16 is a schematic diagram of the process of identifying a door.
  • the training model is an AI model pre-trained with various doors as samples.
  • the recognition results such as the position coordinates of the door are output, and the recognition results are mapped from the 3D environment map to Save and display in 2D environment map.
  • the mobile robot After the mobile robot obtains the 2D environment map, it fuses the recognition results of furniture and doors in the 2D environment map, and uses the partition algorithm to partition the 2D map, thereby dividing the 2D environment map into multiple regions.
  • the self-mobile robot sends the partition result to the APP server and the voice recognition server, and the APP server sends the partition result to the terminal device, and the terminal device is installed with an APP for controlling the self-mobile robot.
  • the terminal device After the terminal device receives the partition result, it displays the partition result. For example, please refer to FIG. 17 .
  • Fig. 17 is a schematic diagram of the synchronization process between the mobile robot and the speech recognition server.
  • the area set of the environment map includes area 1 , area 2 and area 3 .
  • Users can customize and edit the logos of each area on the client side. For example, the respective identifications of the above-mentioned area 1, area 2, and area 3 are changed to Xiao Ming's room, living room, and kitchen in order.
  • the terminal device sends the update information to the APP server, and the APP server sends the update information to the self-mobile robot.
  • the identification of each work area in the local environment map is updated.
  • the mobile robot since the mobile robot updates the identifications of each working area, it also updates the identifications of the working areas synchronously to the speech recognition server. Since the mobile robot received the update information from the APP server, it was found that the logo of the working area had changed. Afterwards, while updating the local area, the self-mobile robot sends the update information to the speech recognition server, so that the speech recognition server updates and saves the identification of each working area. Afterwards, the user is able to interact with the self-mobile robot according to the custom naming.
  • the behavior of the self-mobile robot is: first walk to the area named "living room” in the order of the user's shouting, and then walk to the custom-named "living room” after cleaning the area.
  • the identifications of each working area stored on the voice recognition server are consistent with the identifications of the corresponding working areas stored on the self-mobile robot, which improves the accuracy of voice recognition by the voice recognition server.
  • the self-mobile robot determines the working area conforming to the area category from the pre-built area set according to the area category.
  • the embodiment of the present application is not limited.
  • the self-mobile robot can also real-time Determine the work area according to the area category.
  • the specific area is the area with water damage in the home. Obviously, the areas with water damage in the home are different at different times.
  • the self-mobile robot uses a camera to collect images, and recognizes the images to determine at least two working areas that meet the area category.
  • the second voice signal is "check where there is water in the home and wipe it dry".
  • the mobile robot collects the voice signal and performs semantic recognition, it continuously takes images and recognizes the images during the traveling process. If there is water in the image, the place with water is wiped dry. After that, go ahead and take images, wiping dry each time water is identified.
  • the second voice signal is "mop the kitchen with oil stains".
  • the mobile robot collects the voice signal and performs semantic recognition, it continuously takes images and recognizes the images during the process of traveling in the kitchen, and mops the floor every time it recognizes a place with oil stains.
  • This scheme is used to achieve the purpose of performing tasks in a specific area.
  • the self-mobile robot when the self-mobile robot sequentially executes the tasks indicated by the second voice signal on the at least two working areas, it can determine the operation mode according to the area category, and indicate two tasks at a time according to the operation mode.
  • the work area performs tasks.
  • the user when the user sends out the second voice signal, the user may not indicate a specific operation method, but the self-mobile robot may autonomously determine and execute the operation method. For example, a user says: "clean the grease in the kitchen". After the mobile robot continuously collects images to determine the oily area, the operation method is determined to be: add cleaning fluid and increase the mopping frequency. Afterwards, the self-mobile robot sprays cleaning fluid over the oily areas and mops vigorously.
  • the user says: "clean the water stains in the living room".
  • the mobile robot continuously collects images to determine the water-stained area, if there is more water in the water-stained area, then determine the operation method as follows: perform three times of mopping the floor. Afterwards, the self-mobile robot mopped the water-damaged area three times. If there is less water in the water-stained area, then determine the operation method as follows: mop the floor once. After that, the self-mobile robot mopped the water-damaged area once.
  • the self-mobile robot can automatically determine a more suitable operation method to achieve the purpose of improving the efficiency of task execution.
  • the mobile robot is located in an initial area when collecting the second voice signal, and the initial area is not any working area in the second voice signal. Then, before the self-mobile robot moves from the initial area to the working area, the task execution in the initial area is recorded. Afterwards, perform the tasks on each work area in turn. After the task is performed, it is determined from the record whether the task has not been performed on the initial area. If the self-mobile robot has not completed the task in the initial area, it will return to the initial area and perform the task.
  • the self-mobile robot After the self-mobile robot completes the task in the area indicated by the second voice signal, it returns to the initial area to continue performing the task, so as to avoid being unable to complete the task in the initial area.
  • the second voice signal may further include task parameters and the like.
  • the second voice signal is: "Clean Xiao Ming's room and study in the forced mopping mode for 10 minutes respectively", “mop the oil-stained area twice", etc.
  • the voice control function of the self-mobile robot in order to prevent the self-mobile robot from continuously recognizing voice signals when the user has no interaction requirement, the voice control function of the self-mobile robot is usually in a silent state. Only after the user issues a specific wake-up word, the voice control function of the self-mobile robot can be woken up. When the voice control function is silent, the self-mobile robot can be stationary or working.
  • the working state after waking up from the mobile robot is referred to as the second working state below, and the working state before waking up from the mobile robot is called the first working state, and the sound generated by the self-mobile robot in the second working state
  • the volume is smaller than the volume of the sound generated in the first working state.
  • the self-mobile robot After the self-mobile robot collects the second voice command in the first working state, if the first voice signal matches the wake-up command of the self-mobile robot, the self-mobile robot automatically switches to the second working state, that is, the self-mobile robot passes through the lowering Switch to the second working state by means of output power consumption, etc., and collect the above-mentioned second voice signal in the second working state.
  • the microphone on the self-mobile robot find a place with the lowest noise and a stable place to install the microphone. Moreover, training the wake-up model through a large number of samples improves the wake-up rate of the self-mobile robot in various operating states. Afterwards, when the voice control function of the self-mobile robot is awakened in the running state, the self-mobile robot reduces the volume of the noise generated by itself by changing its own running state. Afterwards, the user sends control voice commands through the normal volume, and receives and executes corresponding tasks from the mobile robot.
  • the self-mobile robot is a sweeping robot, and when the self-mobile robot works in the first working state, the traveling speed is 0.2 m/s.
  • the user sends out the first voice signal, and if the first voice signal matches the wake-up command, the self-mobile robot switches to the second working state with a traveling speed of 0.1 m/s and low noise generation. Afterwards, the user sends out a second voice signal, and the mobile robot collects the second voice signal and performs related tasks.
  • the volume of the second voice signal may be smaller than the volume of the first voice signal.
  • the wake-up rate of the self-mobile robot in the running state can be improved through multiple trainings and algorithms in advance. For example, when the self-mobile robot is within 5 meters of the user and is in the first working state, the normal human voice awakening rate can reach 85%. After waking up, the self-mobile robot switches to the first working state, and the speech recognition accuracy in the first working state is almost at the same level as that of a smart speaker.
  • the self-mobile robot determines the sound source position of the first voice signal. Afterwards, the self-mobile robot controls the self-mobile robot to switch from the first pose to the second pose according to the position of the sound source, and the distance between the microphone and the sound source position when the self-mobile robot is in the second pose is, The distance between the microphone and the sound source is smaller than when the self-mobile robot is in the first pose, and the microphone is a microphone arranged on the self-mobile robot.
  • intelligent voice technology has been widely used in human-computer interaction, intelligent control, online services and other fields. With the expansion of more application scenarios, intelligent voice technology has become the most convenient and effective way for people to obtain information and communicate.
  • intelligent speech technology includes speech recognition technology and speech synthesis technology.
  • Sound source localization is a method of locating sound sources based on a microphone array. The implementation methods can be divided into directional wave velocity formation and time delay estimation.
  • Combining intelligent voice technology, microphone sound source localization technology and self-mobile robots can design a very rich application scenario, such as issuing voice commands to self-mobile robots to perform tasks, interacting with self-mobile robots to obtain corresponding guidance, and controlling Self-mobile robot steering, etc.
  • a microphone array is designed on the self-mobile robot to receive sound source information for sound source localization, and control the self-mobile robot to turn to the direction of sound source localization according to the positioning, increasing the fun of interaction and the next speech recognition accuracy.
  • the disadvantage of this type of application is that the positioning error of the microphone array is large, and the general error is about ⁇ 45°.
  • the root cause of the error is that the estimation accuracy of the time difference between the sound source and the microphone is not enough. Due to the existence of errors, the effect of turning from the mobile robot to the sound source may be inaccurate, resulting in a poor user experience.
  • the self-mobile robot when the voice control function of the self-mobile robot is awakened, the self-mobile robot can adjust its pose so that the microphone on the self-mobile robot is close to the user, thereby improving the accuracy of voice collection.
  • sound source positioning technology speech recognition technology and AI recognition technology to precisely control the self-mobile robot to turn to the speaker, that is, to turn to the user.
  • the self-mobile robot captures the sound source through the microphone array, and after the signal is converted, it is recognized as the predetermined wake-up word. After that, determine the position of the sound source relative to the microphone array, and then determine the position of the sound source relative to the body, so as to determine the approximate rotation angle, that is, locate the approximate position of the sound source. Finally, the self-mobile robot rotates according to the rotation angle.
  • AI recognition is combined to accurately determine the specific position of the sound source, and then the self-mobile robot is controlled to stop at a position facing the user.
  • a first position is determined, and the first position is the position of the sound source relative to the center of the microphone array.
  • the mobile robot picks up the voice signal through the microphone array, and uses the computing unit to process the voice signal to obtain the voice recognition result. If the speech recognition result matches the wake-up word, then determine the first position; if the speech recognition result does not match the wake-up word, then keep waiting for the wake-up state.
  • Fig. 18A is a schematic diagram of determining the position of a sound source relative to the center of a microphone array.
  • the microphone array includes 6 microphones, which are respectively located at S1-S6, and the 6 microphones are evenly distributed on a circle with a radius of L1, and the origin O of the space coordinate system is the center of the microphone array.
  • the self-mobile robot can determine the first position according to the propagation speed of the sound, the time delay, the position of each microphone, and the like.
  • the time delay refers to a difference in duration of receiving sound by different microphones.
  • the second position is determined according to the first position, and the rotation angle is determined according to the second position, wherein the second position is the position of the sound source relative to the center of the self-mobile robot.
  • the microphone array is located at a fixed position of the body of the self-mobile robot. After determining the first position, the self-mobile robot can determine the second position according to the position of the microphone array and the first position.
  • Figure 18B is a schematic diagram of a microphone array and a self-mobile robot body. Please refer to FIG. 18B , the center of the body is the center of the great circle, and the center of the body and the center of the microphone array do not coincide, but the relative positions of the two are known. Therefore, after the first position is determined by the self-mobile robot, the second position can be determined. After the second position is determined, the rotation angle can be determined, and the rotation angle refers to the rotation angle of the self-mobile robot during the process from the first pose to the second pose.
  • the first pose is the pose of the self-mobile robot before waking up
  • the second pose is the pose of the self-mobile robot’s microphone facing the user
  • the second pose can also be understood as the pose of the self-mobile robot facing the user.
  • the pose of the self-mobile robot facing the user refers to the pose of the camera of the autonomous mobile device facing the user.
  • the electronic device divides the rotation angle into a first angle and a second angle; rotating at a second speed within said second angle, said first speed being greater than said second speed.
  • the second angle is, for example, ⁇ degrees.
  • ⁇ is related to the statistical error of the self-mobile robot, which can be 30 degrees, 60 degrees, etc.
  • the embodiment of this application is not limited.
  • the functional components of the sweeping robot include a camera, a microphone array, a laser ranging sensor, an infrared receiving sensor, a side brush, and a driving wheel.
  • the sweeping robot also includes edge sensors, anti-drop sensors, vacuum fans, motion motors, rolling brushes, computing storage units, battery modules, wifi modules, etc. not shown in the figure.
  • the voice control usage scenario the sweeping robot is in any working state, the user sends a voice wake-up command to the sweeper, the sweeping robot suspends the current work, turns to the sender of the wake-up command, and waits for the user's next interaction command.
  • the sweeping robot is cleaning the living room, and the user sends out the wake-up word "Xiao Q, Xiao Q", the sweeping robot suspends cleaning work, turns to the user, and at the same time responds with voice broadcast "I am", waiting for further instructions from the user, such as "Please leave the living room and go Clean other rooms", the sweeping robot will answer "OK” by voice broadcast, and at the same time leave the living room and enter the bedroom and other rooms to continue cleaning.
  • the sweeping robot is required to accurately recognize the user's wake-up command, and at the same time turn to the user accurately and quickly, waiting for the next control command. If the position of the person who issued the voice command cannot be accurately located, this kind of interaction scene will become very bad.
  • the first case is that the positioning is not accurate, and the robot turns to another direction and does not accurately face the voice controller; the second case It is positioned in the direction of the voice operator, but the rotation process is slow, the action lasts for a long time, and the interactive experience is poor.
  • the sweeping robot suspends cleaning work, and in the process of turning to the user, first determines the first position of the sound source relative to the center of the microphone array, and then determines the second position and rotation angle according to the first position and the position of the microphone array relative to the body. After that, first rotate ⁇ - ⁇ degrees quickly, and then rotate ⁇ degrees at a constant speed.
  • the camera is used to continuously capture images and perform AI recognition. If the user is recognized, the rotation will stop; if the user is not recognized, the rotation will stop after rotating ⁇ degree.
  • the rotation angle ⁇ 180 degrees. If ⁇ is 60 degrees, the self-mobile robot first rotates 120 degrees clockwise quickly, and then rotates at a constant speed to collect images. If the user is recognized based on the image when it rotates to 170 degrees, it stops rotating. If no user is recognized, it will rotate 60 degrees at a constant speed and then stop.
  • the speech recognition technology is a pattern recognition based on speech feature parameters
  • the speech recognition server can classify the input speech according to a certain pattern, and then find the best matching result according to the judgment criterion.
  • the principle frame diagram is shown in Fig. 18C.
  • Fig. 18C is a schematic diagram of the process of training a speech recognition model and recognizing speech. Please refer to FIG. 18C , during the training process, the input speech signal is preprocessed and then feature extraction is performed, and the extracted features are used for model training, and the speech recognition model is generated and then saved.
  • the trained speech recognition model is deployed on the speech recognition server.
  • the speech signal sent by the user is preprocessed and then feature extraction is performed, and the speech recognition server inputs the extracted features to the speech recognition model to obtain a speech recognition result.
  • Fig. 19 is a flow chart of the voice control logic of the self-mobile robot provided by the embodiment of the present application. This example includes:
  • the self-mobile robot is in the first working state, and the voice control function is in the waiting state for waking up.
  • the user sends out a first voice signal
  • the mobile robot collects the first voice signal by using a voice signal collection device or the like.
  • step 1903 Whether the first voice signal matches the wake-up instruction of the self-mobile robot. If the first voice signal matches the wake-up instruction, it means that the self-mobile robot is successfully awakened and step 1904 is performed. If the first voice signal does not match the wake-up instruction, then It means that the self-mobile robot cannot be woken up, and the self-mobile robot performs step 1911 .
  • the self-mobile robot uses its own voice recognition function to determine whether the first voice signal matches the wake-up instruction, or the self-mobile robot sends the first voice signal to the voice recognition server, and the voice recognition server determines the first voice signal Whether it matches the wake-up command.
  • the self-mobile robot switches from the first working state to the second working state.
  • the power consumption of the self-mobile robot in the second working state is relatively small, and the noise is relatively small.
  • the driving wheels roll normally, and the rest of the pronunciation components operate normally; and
  • the driving wheel rolls at a reduced speed, and the operating power of the remaining sounding components is reduced.
  • the self-mobile robot determines whether the second voice signal is collected within the preset time period. If the self-mobile robot collects the second voice signal within the preset time period, execute step 1906; if the self-mobile robot does not collect the second voice signal within the preset time period. For the second voice signal, go to step 1912.
  • the second voice signal is parsed from the mobile robot itself or the voice server, and if the second voice signal is successfully parsed, step 1907 is performed; if the second voice signal fails to be parsed, step 1913 is performed.
  • the self-mobile robot determines whether the analysis result matches the control instruction, and if the analysis result matches the control instruction, execute step 1908; if the analysis result does not match the control instruction, execute step 1914.
  • the analysis result is tasks such as cleaning, sweeping, and mowing of the machine itself.
  • the self-mobile robot determines whether its own state meets the requirements for executing the task. If its own state meets the requirements for executing the task, execute step 1909; if its own state does not meet the requirements for executing the task, execute step 1915.
  • the self-mobile robot determines whether its own power, remaining space in the dust box, remaining water in the water box, etc. meet the task requirements.
  • step 1910 The self-mobile robot continues to work in the first working state, and after a preset period of time, step 1910 is executed.
  • the self-mobile robot feedback command times out, and resumes the first working state, and then executes step 1910.
  • the self-mobile robot sends a voice feedback to the user: "Voice interaction timed out, please wake up again”.
  • the self-mobile robot re-enters the first working state. After that, step 1910 is executed.
  • step 1910 is executed.
  • the self-mobile robot sends a voice feedback to the user: "I did not receive the correct command, please wake up again”; or "I didn't hear what you said clearly, please wake up again” voice feedback. Simultaneously, self-mobile robot re-enters the first working state. After that, step 1910 is executed.
  • step 1910 The self-mobile robot continues to work in the second working state, and performs voice interaction and answering with the user. After the preset duration, step 1910 is executed.
  • the self-mobile robot sends out to the user: "This task is too difficult, I can't perform it, please describe it differently”, “Do you want to clean under the bed in the bedroom”, guide the user to interact with it to understand the user's intention .
  • the voice feedback of the self-mobile robot cannot perform the task, and continues to work in the second working state.
  • step 1910 is executed after the preset time period elapses.
  • the self-mobile robot performs tasks such as cleaning.
  • the self-mobile robot first judges its own state before executing the task, and determines whether to perform the task immediately or after charging and replenishing water according to its own state, which can avoid interruptions during the execution of the task.
  • the voice control function after the voice control function is woken up, the voice control function re-enters the waiting wake-up state after a preset period of time. For example, after the voice control function is woken up, if the second voice signal is not collected after a preset period of time, it will automatically enter the waiting wake-up state. For another example, after executing a cycle of wake-up and command issuance, it will automatically enter the waiting wake-up state.
  • the second voice signal may also indicate a task restricted zone.
  • the self-mobile robot determines the task restricted area from the environment map, and determines at least two working areas from the areas outside the task restricted area.
  • the user can indicate the task restricted area in the second voice signal.
  • the self-mobile robot determines the restricted area of the task according to the second voice signal, and then takes other areas as the working area and executes the task. For example, if the user says: "Do not clean the bottom of the bed", the mobile robot will clean the area other than the bottom of the bed.
  • the voice signal is mainly used for the control of the working area.
  • the embodiments of the present application are not limiting.
  • Fig. 20 is another flow chart of the voice control method for the self-mobile robot provided by the embodiment of the present application. This example includes:
  • the self-mobile robot is in the first working state, and the voice control function is in the waiting state for waking up.
  • the user wakes up the voice control function through a first voice signal, and sends out a second voice signal.
  • the self-mobile robot wakes up the voice control function, it also automatically switches to the second working state.
  • the voice control function For details, please refer to the above description, which will not be repeated here.
  • the self-mobile robot obtains a control instruction according to the second voice signal, where the control instruction is used to instruct the self-mobile robot to control the designated device.
  • the second voice signal is parsed by the mobile robot itself, or the voice signal is parsed by the voice recognition server to obtain the control instruction.
  • the control instruction is used to instruct the automatic mobile device to control the designated device in the designated area.
  • the specified device is, for example, an air conditioner, a refrigerator, a curtain, or other home appliances or home appliances.
  • the self-mobile robot moves to the designated area to complete the control of the designated equipment.
  • the self-mobile robot when the self-mobile robot is turned on, the user is located within the voice signal collection range of the self-mobile robot, for example, the user and the self-mobile robot are in the object at the same time, and the self-mobile robot is 5 meters away from the user.
  • the second voice signal sent by the user is "turn on the air conditioner in the master bedroom and cool it down to 25°C".
  • the mobile robot uses its own hardware remote control module to turn on the air conditioner in the master bedroom, and sets the mode to cooling mode and the temperature to 25°C.
  • the value-added function is realized through voice control combined with other hardware and algorithms of the self-mobile robot, making the self-mobile robot more intelligent.
  • Fig. 21 is another flow chart of the voice control method for the self-mobile robot provided by the embodiment of the present application. This example includes:
  • the self-mobile robot is in a first working state, and the voice control function is in a waiting state for waking up.
  • the user wakes up the voice control function by using the first voice signal, and sends out a second voice signal.
  • the self-mobile robot wakes up the voice control function, it also automatically switches to the second working state.
  • the voice control function For details, please refer to the above description, which will not be repeated here.
  • the self-mobile robot obtains a control instruction according to the second voice signal, and the control instruction is used to instruct the self-mobile robot to patrol at a fixed point.
  • the second voice signal is parsed by the mobile robot itself, or the voice signal is parsed by the voice recognition server to obtain the control instruction.
  • the control instruction is used to instruct the automatic mobile device to patrol, monitor or care at fixed points.
  • the self-mobile robot performs fixed-point patrol, monitoring or care.
  • the second voice signal sent by the user is "go to dad's room to inspect".
  • the mobile robot plans the driving path according to the environment map and enters the father's room, it travels to the previously set monitoring point, turns on the camera to shoot video, and sends the video back to the client on the user's mobile phone, realizing the function of monitoring the elderly across rooms .
  • the self-mobile robot can carry out fixed-point patrols in the home according to the user's intention, monitor designated areas, and provide care for specific rooms.
  • Fig. 22 is another flow chart of the voice control method for the self-mobile robot provided by the embodiment of the present application. This example includes:
  • the self-mobile robot is in a first working state, and the voice control function is in a waiting state for waking up.
  • the user wakes up the voice control function through the first voice signal, and sends out a second voice signal.
  • the self-mobile robot obtains a control instruction according to the second voice signal, where the control instruction is used to instruct the self-mobile robot to find the target object.
  • the self-mobile robot determines whether the location coordinates of the target object are marked in the environment map. If the target object has been marked in the environment map, perform step 1205; if the target object is not marked in the environment map, perform step 1208.
  • the collected images are input into the AI training model to obtain the position coordinates, name, type, etc. of the object, and the information is recorded in the environment map for subsequent intelligent object finding.
  • the location coordinates of these objects may not be displayed on the environment map.
  • the client side displays the position of the target object in the environment map, etc.
  • the self-mobile robot asks the user whether it is necessary to find the target object now, and if the user feedbacks that the target object is to be found now, perform step 1206; if the user feedback does not need to find the target object now, perform step 1207.
  • the location in the environment map of the target object found during the search run is the location in the environment map of the target object found during the search run.
  • the first voice signal is: "Please help me find socks”.
  • the mobile robot recognizes the first voice signal, it is determined whether the location coordinates of the socks have been marked locally. If the sock is not marked, prompt the user that the sock cannot be found. If the position coordinates of the socks are marked, an inquiry voice is issued: "Do you want to find the socks now?" If the user's reply is "yes", “okay” and other affirmative answers, then the self-mobile robot travels to guide the user to the location of the socks, and displays the location coordinates of the socks on the interface of the client environment map. If the user's reply is a negative answer such as "no need", the self-mobile robot only needs to instruct the client to display the location coordinates of the socks.
  • the AI training model is trained through machine learning, and the AI training model is used to identify specific objects, so that the user can control the self-mobile robot to search for some specific objects through voice, and mark them on the map or take the user to the object site to realize intelligence.
  • the utility of the self-mobile robot is expanded, and the intelligence of the self-mobile robot is improved.
  • Fig. 23 is a schematic structural diagram of a self-mobile robot provided by an embodiment of the present application. As shown in Figure 23, the self-mobile robot 1200 includes:
  • the memory 1202 stores computer instructions
  • the processor 1201 executes the computer instructions stored in the memory 1202, so that the processor 1201 executes the method implemented by the mobile robot as above.
  • the autonomous mobile robot 1200 also includes a communication component 1203 .
  • the processor 1201 , the memory 1202 and the communication unit 1203 may be connected through a bus 1204 .
  • the embodiment of the present application also provides a computer-readable storage medium, wherein computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed by a processor, they are used to implement the above method implemented by the self-mobile robot.
  • the embodiment of the present application also provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the above method implemented by the self-mobile robot is realized.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Electromagnetism (AREA)
  • Manipulator (AREA)

Abstract

一种自移动机器人控制方法、装置、设备及可读存储介质，自移动机器人根据用户发出的语音信号确定声源方向(301)，确定自移动机器人周围的移动对象(302)。从移动对象中确定出位于声源方向的目标对象(303)，根据目标对象确定工作区域(304)，移动至工作区域并在工作区域内执行任务(305)。采用本方案，由于自移动机器人从移动对象中确定出目标对象，移动对象具有准确的空间位置。因此，自移动机器人能够根据声源方向从多个移动对象中准确确定出目标对象，并准确到达工作区域，无需借助客户端，过程简单、灵活。而且，该方案适用于所有激光类自移动机器人，成本低、算法简单、需要的算力低。

Description

自移动机器人控制方法、装置、设备及可读存储介质 技术领域
本申请涉及人工智能技术领域，特别涉及一种自移动机器人控制方法、装置、设备及可读存储介质。
背景技术
随着人工智能技术的发展，各种机器人越来越多地进入人们的生活，比如物流机器人、扫地机器人、割草机器人、迎宾机器人等。
语音控制方式是一种常见的机器人控制方式。采用语音控制方式时，机器人预先存储环境地图，环境地图中标识出各个工作区域，如小明的房间、客厅等。用户发出指示工作区域的语音指令后，机器人根据语音指令确定工作区域并在工作区域内工作。例如，用户语音控制扫地机器人清扫某个房间、清扫某件家具周围等。再如，用户语音控制割草机器人在目标区域内割草。
上述语音控制方法需要预先存储环境地图。若用户临时指定工作区域，则需要借助APP等在环境地图中标记该工作区域，过程繁琐、灵活度差。
发明内容
本申请实施例提供一种自移动机器人控制方法、装置、设备及可读存储介质，自移动机器人通过跟随用户确定出工作区域，过程简单、灵活度高且容易实现。
第一方面，本申请实施例提供一种自移动机器人控制方法，包括：
根据用户发出的语音信号确定声源方向；
确定自移动机器人周围的移动对象；
从所述移动对象中确定出位于所述声源方向的目标对象；
根据所述目标对象确定工作区域；
移动至所述工作区域并在所述工作区域内执行任务。
第二方面，本申请实施例提供一种自移动机器人控制装置，包括：
第一确定模块，用于根据用户发出的语音信号确定声源方向；
第二确定模块，用于确定自移动机器人周围的移动对象；
第三确定模块，用于从所述移动对象中确定出位于所述声源方向的目标对象；
处理模块，用于根据所述目标对象确定工作区域；
执行模块，用于移动至所述工作区域并在所述工作区域内执行任务。
第三方面，本申请实施例提供一种自移动机器人，包括：处理器、存储器及存储在所述存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时使得所述自移动机器人实现如上第一方面或第一方面各种可能的实现方式所述的方法。
第四方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机指令，所述计算机指令在被处理器执行时用于实现如上第一方面或第一方面各种可能的实现方式所述的方法。
第五方面，本申请实施例提供一种包含计算程序的计算机程序产品，所述计算机程序被处理器执行时实现如上第一方面或第一方面各种可能的实现方式所述的方法。
本申请实施例提供的自移动机器人控制方法、装置、设备及可读存储介质，自移动机器人根据用户发出的语音信号确定声源方向，并确定自身周围的移动对象。之后，从自身周围的移动对象中确定出位于声源方向的目标对象，根据目标对象确定工作区域，并移动至工作区域并执行任务。采用该种方案，由于自移动机器人从移动对象中确定出目标对象，移动对象具有准确的空间位置。因此，自移动机器人能够根据声源方向从多个移动对象中准确确定出目标对象，并准确到达工作区域，无需借助客户端，过程简单、灵活。而且，该方案适用于所有激光类自移动机器人，成本低、算法简单、需要的算力低。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1A是本申请实施例提供的自移动机器人控制方法的实施环境示意图；
图1B是本申请实施例提供的扫地机器人的一个结构示意图；
图1C是自移动机器人的声音信号采集装置的结构示意图；
图1D是本申请实施例提供的扫地机器人的另一个结构示意图；
图2A是本申请实施例提供的自移动机器人的一个语音控制流程图；
图2B是本申请实施例提供的自移动机器人的另一个语音控制流程图；
图2C是本申请实施例提供的自移动机器人的另一个语音控制流程图；
图3是本申请实施例提供的自移动机器人控制方法的流程图；
图4是本申请实施例提供的自移动机器人控制方法的另一个流程图；
图5是确定目标对象的流程图；
图6所示为LAM图；
图7所示为DTOF散点图；
图8是本申请实施例提供的自移动机器人控制方法中基于AI相机的跟随流程图；
图9是自移动机器人保持跟随状态的流程图；
图10是自移动机器人的结构示意图；
图11为本申请实施例提供的一种自移动机器人控制装置的结构示意图；
图12是本申请实施例提供的自主移动设备语音控制方法的流程图；
图13是本申请实施例中语音识别的过程示意图；
图14是地毯区域的示意图；
图15是家具识别过程示意图；
图16是识别门的过程示意图；
图17是自主移动设备和语音识别服务器同步过程示意图；
图18A是确定声源相对于麦克风阵列中心的位置的示意图；
图18B是确麦克风阵列和自主移动设备机体的示意图；
图18C是训练语音识别模型和识别语音的过程示意图；
图19是本申请实施例提供的自主移动设备语音控制逻辑流程图；
图20是本申请实施例提供的自主移动设备语音控制方法的另一个流程图；
图21是本申请实施例提供的自主移动设备语音控制方法的又一个流程图；
图22是本申请实施例提供的自主移动设备语音控制方法的又一个流 程图；
图23为本申请实施例提供的一种自移动机器人的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
随着科学技术的进步，机器人已经走进越来越多人的生活，在人们的生活中扮演着重要的角色。目前，机器人可根据用户的语音指令到达指定工作区域工作。以扫地机为例，扫地机预先构建并存储环境地图。当用户想要清扫某个区域时，向扫地机发出包含该区域的语音指令，例如“清扫小明的房间”等。
另外，有时候用户会临时指定工作区域。例如，用户期望扫地机清扫自己所在位置附近，即用户在哪扫地机扫哪，该种功能即为俗称的在哪扫哪功能。用户每次所处的位置具有随机性，若用户每次借助APP等在环境地图中标记工作区域，则过程繁琐、灵活度差。
基于此，本申请实施例提供一种自移动机器人控制方法、装置、设备及可读存储介质，自移动机器人通过跟随用户确定出工作区域，过程简单、灵活度高且容易实现。
图1A是本申请实施例提供的自移动机器人控制方法的实施环境示意图。请参照图1A，该实施环境包括自移动机器人，自移动机器人例如为扫地机器人、自移动空气净化机器人、自动割草机、擦窗机器人、太阳能电池板清洁机器人、管家机器人、无人飞行器、自动引导车(Automated Guided Vehicle，AGV)、安防机器人、迎宾机器人、看护机器人等。
自移动机器人上设置麦克风等声音信号采集装置，能够采集用户发出的语音信号。自移动机器人采集到语音信号后，对语音信号进行识别得到语音指令，并执行语音指令指示的任务。实际中，自移动机器人自身可以对语音信号进行识别。或者，自移动机器人与语音识别服务器(图中未示意出)建立网络连接，自移动机器人采集到语音信号后，向语音识别服务器发送语音信号，以使得语音识别服务器对语音信号进行识别，并将识别到的语音指令发送给自移动机器人。
下面，以自移动机器人为扫地机器人为例，对自移动机器人的结构进行 详细说明。
图1B是本申请实施例提供的扫地机器人的一个结构示意图。以下将扫地机器人简称为机器人，请参照图1B，“→”代表语音信号的传播方向。该机器人包括机器人外壳1、驱动元件、凸起结构2和语音信号采集装置3；其中，驱动元件设置在机器人外壳1内，其用于驱动机器人外壳1移动；凸起结构2设置在机器人外壳1的上表面10，语音信号采集装置3设置在该凸起结构2上。
再请参照图1B，机器人外壳1包括顶板、环形侧板和底板，该顶板、环形立板和底板围合拼装形成容纳腔室，容纳腔室内容置有控制单元和驱动元件。另外，机器人还包括设置在机器人外壳1上的驱动轮6、边刷7、滚刷或风机等功能元件，其中，驱动轮6用于在驱动元件作用下带动机器人行驶，边刷7和滚刷在收到控制单元的信号后清扫工作面，风机则是用于在尘盒内形成负压腔，以将工作面上的灰尘、杂物等吸入尘盒除尘。需要说明的是，上述这些功能元件的结构及工作原理与现有扫地机器人基本相同，本领域技术人员基于现有技术完全可以实现，故而本文在此不再赘述。
机器人外壳1的顶板的上表面10凸起设置有凸起结构2。在一些实施例中，凸起结构2和顶板一体加工成型。另一些实施例中，凸起结构2和顶板分体独立加工成型，然后凸起结构2通过粘接、螺纹连接等方式固定连接在顶板的上表面10。该凸起结构2上设置了声音信号采集装置3。
通常情况下，机器人的自身的噪音由驱动元件、边刷7、滚刷和/或风机等功能产生，并且这些部件位于容纳腔内或其底部，本发明中将声音信号采集装置设置在凸起设置于机器人外壳1的上表面10的凸起结构2上，以使声音信号采集装置3远离机器人的噪音源，来降低机器人自身发出的噪音对声音信号采集装置3的干扰，以便使机器人能更加准确的采集到用户语音控制指令。该用户语音控制指令包括开始扫地、播放音乐、停止扫地、去充电等，本领域技术人员可以根据机器人实际需求设定相应的功能。
图1C是自移动机器人的声音信号采集装置的结构示意图。请参照图1C，该声音信号采集装置3包括麦克风(MIC)。详细地，在一些实施例中，该声音信号采集装置3包括PCB板30(印制电路板)、减震罩壳31和麦克风芯片32；其中，减震罩壳31设置在PCB板30上并与PCB板30围成了具有容纳腔的声音信号采集装置3外部封装，麦克风芯片32设置在该容纳腔中，减 震罩壳31顶部的中心区域设置有连通外部与容纳腔的拾音孔310。PCB板30与麦克风芯片32和机器人的控制单元都通信连接，麦克风芯片32从拾音孔310采集外部的声音信号后通过PCB板30传输给控制单元，控制单元控制机器人执行声音信号包含的用户语音控制指令。
需要说明的是，其中，声音信号采集装置3的减震罩壳31一方面可以减少机器人工作过程中产生的震动对声音信号采集装置3的影响，另一方面减震罩壳31可以吸收来自机器人自身的噪音，而拾音孔310开设在减震罩壳31顶部的中心区域，其仅采集来自于顶部的声音信号(通常为用户发出的语音控制指令)。尤其是对于扫地机器人来说，扫地机器人一般在地面工作而用户则从高处发出语音控制，位于减震罩壳31顶部中心区域的拾音孔310能比较容易采集到用户语音控制的声音信号，而机器人自身发出的噪音能被环绕拾音孔310的减震罩壳31阻隔，能够减少其对声音信号采集装置3采集信号的干涉。在另一些实施例中，该减震罩壳31包括减震泡棉，可以理解减震泡棉不仅可以阻挡来自机器人自身的噪音进入拾音孔310，还可以吸收部分噪音。
继续参见图1C，该声音信号采集装置3还包括防水防尘膜33，该防水防尘膜33设置在减震罩壳31上并且遮盖拾音孔310，以防止水或灰尘通过拾音孔310落到麦克风芯片32上，而影响到麦克风芯片32的采集声音信号的效果。
继续参见图1C，本实施例中，声音信号采集装置3还包括上盖34，上盖将防震罩盖31压紧在PCB板上，并且通过螺钉(图中未示出)等连接件固定连接在凸起结构2上或距离传感器3上，从而实现声音信号采集装置3和机器人外壳1之间的固定连接关系，以防止机器人行驶过程中声音信号采集装置3从机器人外壳1上脱落。另外，上盖34的顶部中心区域上与减震罩壳31的拾音孔的对应位置也开设了拾音孔。
进一步地，为了增强声音信号采集装置3采集声音信号的能力，要尽量保证声音信号传播路径短宽，在一些实施例中，通过限定拾音孔310的孔径孔深比来实现上述目的，具体地，拾音孔310的孔径(d1)孔深(d2)比尽量大于1。在更加具体的实施例中，拾音孔310的孔径(d1)孔深(d2)比大于2:1。
为了能使机器人更好的采集到用户语音控制的声音信号，在一些实施例中，机器人至少包括三个声音信号采集装置3，并且这些声音信号采集装置3 环形均匀分布。环形均布的多个声音信号采集装置3可以均衡的采集到从各个角度传输过来的声音信号，以保证所采集的用户语音控制信号的准确性和连贯性。
图1D是本申请实施例提供的扫地机器人的另一个结构示意图。请参照图1D，机器人包括三个声音信号采集装置3，这三个声音信号采集装置3环形均匀分布，即三个声音信号采集装置3位于一个圆上，每个声音信号采集装置3至圆心的距离均为该圆的半径，并且相邻两个声音信号采集装置3的之间的圆心角为120°(度)。并且，为了使多个声音信号采集装置3的声音信号采集能力最佳，至少三个声音信号采集装置3环形均匀分布的圆的直径位于60mm～100mm范围内。
在另一些实施例中，机器人包括三个声音信号采集装置3，这三个声音信号采集装置3呈三角形分布，并且三个声音信号采集装置3中一个相对于另外两个位于机器人外壳1的上表面10的前部。这三个声音信号采集装置3可以环形均布，也就是说这三个声音信号采集装置3位于三角形的外切圆上并且相邻两个声音信号采集装置3之间的圆心角为120°(度)。
当然，在另一些实施例中，这三个声音信号采集装置3无需环形均匀分布，只需保证其以一前两后的排布方式分布即可。这种排布方式的优势在于，扫地机器人向前行驶时，用户发出的语音控制指令因在空气等介质中传输延迟，机器人外壳1上表面10的前部声音信号采集装置3仅会采集到少量声音信号，而大部分声音信号需要由位于后部的声音信号采集装置3采集，在后部多设置声音信号采集装置3可以能更好的采集到声音信号，保证采集的声音信号的准确性。
进一步地，为了能使声音信号采集装置3采集声音信号效果最佳，在一些实施例中还给出了声音信号采集装置3选型标准，具体为：选择全向型数字麦克风，其信噪比(Signal-to-noise ratio，SNR)大于64dB(A)，灵敏度保证-26+3dBFS，声学过载点(Acoustic Overload Point,AOP)保证120Db SPL，总谐波失真(total harmonic distortion,THD)94dB SPL@1kHz处最好低于0.5％。
进一步地，在一些实施例中，机器人还包括距离传感器4，该距离传感器4设置在机器人外壳1上，并且其用于测量机器人移动方向前方障碍物和机器人之间的距离，以便两者之间的距离达到设定阈值时，机器人可以停止运动或者改变运动路径，以防止机器人和障碍物相撞。在另一些实施例中，该距 离传感器4可转动的设置在机器人外壳1上，其可以相对于机器人外壳360度旋转以检测工作空间内家具、墙面等布局，继而描绘出工作空间内地图，并且根据描绘出的地图工作，以提高工作效率。
该距离传感器4包括DTOF和LDS。在一些实施例中，该距离传感器4设置在上述凸起结构2上，声音信号采集装置3则设置在距离传感器4上。由此可见，距离传感器4和声音信号采集装置3可以利用凸起结构2，无需为各自单独设置凸起，可以尽可能的简化机器人的结构，降低其制造成本。
而在另一些实施例中，凸起结构2包括距离传感器4，也就是说，距离传感器4直接设置在机器人外壳1的上表面上形成一个凸起结构2，而声音信号采集装置3则设置在距离传感器4，也即声音信号采集装置3设置在距离传感器4形成的凸起结构2上。距离传感器4直接设置在机器人外壳1的上表面形成凸起结构2，而声音信号采集装置3利用其距离传感器4自身特点凸起设置在机器人外壳1上，无需另外设置凸起结构，整体结构简单，成本低。
另一方面，距离传感器4在机器人外壳1的上表面10，可以很好地避开机器人自身的其他结构，从而能准确感应到障碍物的位置。而，声音信号采集装置3则可以尽可能的远离机器人的驱动电机、滚刷、边刷7和风机等产生噪音的部件，能降低机器人自身产生的噪音对声音信号的干涉。
另一些实施例中，该机器人还包括声音信号播放装置5，该声音信号播放装置5可以为扬声器(喇叭)，该声音信号播放装置5设置在机器人外壳1上，并且该声音信号播放装置5和机器人的控制单元通信连接，控制单元设置有机器人的播音工作模式，比如播放音乐等。当用户通过遥控器或APP控制机器人进入该播音工作模式后，存储在控制单元内的音乐通过声音信号播放装置5播放出来。
为了防止声音信号播放装置5播放的声音信号干涉声音信号采集装置3采集用户发出的语音控制的声音信号，在一些实施例中声音信号采集装置3的拾音孔310和声音信号播放装置5的放音孔的朝向不同方向。更为具体地，声音信号采集装置3的拾音孔310朝向垂直于所述机器人外壳1的上表面10，而声音信号播放装置5的放音孔朝向垂直于所述机器人外壳1的外立面11，也就是说，声音信号采集装置3的拾音孔310和声音信号播放装置5的放音孔的朝向成90°(度)夹角设置。
需要说明的是，通常情况下，机器人外壳1的上表面10和外立面11相 互垂直设置，当然，在满足声音信号采集装置3的拾音孔310和声音信号播放装置5的放音孔的朝向不同方向的情况下，机器人外壳1的上表面10和外立面11成其他夹角设置。
进一步地，在一些实施例中，声音信号播放装置5位于机器人外壳1的前部，而声音信号采集装置3位于机器人外壳1的后部。而，在另一些实施例中，声音信号播放装置5位于机器人外壳1的后部，而声音信号采集装置3位于机器人外壳1的后部。机器人外壳1的前部和后部的划分标准是基于机器人外壳1的形状沿前后将其一分为二，其中，位于机器人外壳1前侧的区域为前部，位于机器人外壳1后侧的区域为后部。例如：以图1C所示的实施例为例，该圆形机器人外壳1沿前后方向划分为前半圆区域和后半圆区域，将前半圆区域界定前部，将后半圆区域界定为后部。
可以理解，声音信号采集装置3和声音信号播放装置5中一者位于机器人外壳1的前部，另一者位于机器人外壳1的后部，以使两者之间保持足够远的距离，从而进一步地减少机器人自身播放的声音信号对声音信号采集装置3的干涉，机器人能够更加精准的采集到用户的语音控制指令并准确的执行该指令，继而能给用户提供更好的使用体验。
更进一步的，为了减少机器人自身播放的声音信号对声音信号采集装置3的干涉，在一些实施例中，该机器人还包括声音信号回采装置，该声音信号回采装置与机器人的控制单元和声音信号播放装置5通信连接，其用于回采声音信号播放装置5的声音信号，控制单元接受声音信号回采装置回采的声音信号，并且从声音信号采集装置3采集的声音信号中过滤回采的声音信号，再将过滤后的声音信号包含的指令传动给执行元件，控制执行元件执行该指令。
在一些实施例中，声音信号回采装置包括滤波式回采电路，该滤波式回采电路通过导线和机器人本体的控制单元电连接，并且通过导线和声音信号播放装置电连接。
除了声音信号回采装置外，在一些实施例中，机器人还包括声音信号降噪装置，该声音信号降噪装置与声音信号采集装置3和控制单元均通信连接，其用于对声音信号采集装置3采集到的声音信号进行降噪处理，以消除采集到的声音信号的杂音或无效声音信号部分。
除了上述机器人外，本发明还提供一种适用于上述机器人的控制方法， 以消除声音信号采集装置3采集到的无效声音信号，尤其是要消除机器人本身发出的声音信号对声音采集信号的信号采集造成的干涉。示例性的，请参照图2A。
图2A是本申请实施例提供的自移动机器人的一个语音控制流程图。本实施例包括：
S1、使用声音信号采集装置3采集声音信号；
其中，声音信号采集装置3采集的声音信号主要包括用户对机器人的语音控制指令，例如机器人使用声音信号采集装置3等声音信号采集装置3采集用户的语音控制所包含的声音信号。但是，实践中，机器人在工作过程中其驱动电机、边刷7、滚刷和/或风机等功能元件也能产生声音信号，或者是机器人本身也具备产生声音信号的能力，比如机器人在工作过程中或停机状态下可以播放音乐、朗读书籍等，由于声音信号采集装置3主要功能是采集用户语音控制而言，本文将机器人自身产生的这些声音信号统称为“无效声音信号”。基于此，为了消除这些无效声音信号对声音信号采集装置3采集信号的干涉，本发明的机器人的控制方法还包括如下步骤：
S2、过滤声音信号采集装置3采集的声音信号中由机器人自身播放的声音信号后得到有效声音信号。
图2B是本申请实施例提供的自移动机器人的另一个语音控制流程图。请参照图2B，在一些实施例中，该控制方法中实现步骤S2的方法包括如下步骤：
S20、回采机器人自身播放的声音信号作为无效声音信号；
S21、从声音信号采集装置3采集到的声音信号中过滤该无效声音信号后得到有效声音信号。
详细地，在机器人中设置声音信号播放装置5，该声音信号播放装置5可以为扬声器(喇叭)，该声音信号播放装置5设置在机器人外壳1上，并且该声音信号播放装置5和机器人的控制单元通信连接，控制单元设置有机器人的工作模式，比如播放音乐等，当用户通过遥控器或APP控制机器人进入该控制模式后，存储在控制单元内的音乐通过声音信号播放装置5播放出来。
该机器人还包括声音信号回采装置，该声音信号回采装置与机器人的控制单元和声音信号播放装置5通信连接，其用于回采声音信号播放装置5的声音信号，控制单元接受声音信号回采装置回采的声音信号并且从声音信号 采集装置3采集的声音信号中过滤回采的声音信号，再将过滤后的声音信号包含的指令传动给执行元件，控制执行元件执行该指令。
图2C是本申请实施例提供的自移动机器人的另一个语音控制流程图。本实施例中，该控制方法中实现步骤S2的方法包括如下步骤：
S20'、判断机器人是否处于播音工作模式；
S21'、若是，则获取机器人在播音工作模式下播放的声音信号作为无效声音信号；
S22'、从声音信号采集装置3采集的声音信号中过滤该无效声音信号后得到有效声音信号。
此外，在另一些实施例中，本发明的控制方法中使用声音信号采集装置3采集声音信号后，先对声音信号降噪处理，然后再过滤声音信号中由机器人播放的声音信号后得到有效声音信号，以进一步地消除处用户语音控制指令外其他声音信号的影响。
从步骤S2中得到有效声音信号后，该控制方法则执行如下步骤：
S3、执行该有效声音信号包含的控制指令，实现机器人与用户之间的语音互动，从而提高用户的使用体验。
应用场景举例如下：
一、当前扫地机器人正在清扫地面，用户发出“播放音乐”的语音控制指令，机器人采集到该指令后开始播放存储的音乐。当然，用户也可以根据机器人内存储的音频资料点播需要的音乐，语音控制指令中只需包含该音乐名称即可。
二、当前扫地机器人处于停机或待机状态，用户发出“扫地”的语音控制指令，机器人采集到该指令后开始根据预定路线清扫地面。
三、当前扫地机器人正在清扫地面并且同时在播放音乐，用户发出“停止播放音乐”的语音控制指令，机器人采集到该指令并且过滤掉播放音乐产生的无效声音信号后停止播放音乐。
图3是本申请实施例提供的自移动机器人控制方法的流程图。本实施例的执行主体是自移动机器人。本实施例包括：
301、根据用户发出的语音信号确定声源方向。
示例性的，自移动机器人上的麦克风阵列包含多个麦克风，自移动机器人可根据各麦克风接收语音信号的时间差或声音强度等确定出声源方向。
语音信号通常包括位置关键字，如“来这里扫”、“扫这里”、“过来这里”等。
自移动机器人确定出声源后，旋转一定角度使得自移动机器人的正面朝向用户。自移动机器人的正面朝向用户是指自移动机器人的摄像头朝向用户。
302、确定自移动机器人周围的移动对象。
虽然自移动机器人行进过程中，能够根据即时定位与地图构建(Simultaneous Localization and Mapping，SLAM)算法构建环境地图、规划路径等，但是基于SLAM算法得到的环境地图中仅包含静止物体。
本申请实施例中，自移动机器人上设置直接飞行时间法(Direct Time-of-Flight，DTOF)传感器等3D传感器或AI相机，利用3D传感器或AI相机采集的图像等，能够确定出自移动机器人周围的移动对象。
自移动机器人行进过程中，DTOF传感器快速、连续地对周围环境进行360度扫描，利用前后两帧或几帧之间的差异来提取移动对象，并根据移动对象的移动轨迹、运动模式等从多个移动对象中分离出行人，将位于声源方向的行人作为目标对象，进而对目标对象进行跟踪。
自移动设备周围可能有一个或多个移动对象。例如，自移动设备为扫地机，扫地机在客厅里工作，客厅里移动的对象包括小孩、大人、小猫小狗、皮球等。303、从所述移动对象中确定出位于所述声源方向的目标对象。
示例性的，移动对象可能位于自移动机器人周围360度任意一个方向。自移动机器人确定出移动对象后，进一步确定每个移动对象相对于自移动机器人的方向。确定出每个移动对象的方向后，将方向与声源方向相同的移动对象作为目标对象，目标对象即为步骤301中发出语音信号的用户。若每个对象的方向与声源方向均不重合，则将方向与声源方向相近的移动对象作为目标对象。
由于基于3D传感器确定出的目标对象具有深度信息，使得自移动机器人能够确定出移动对象在空间中的位置，进而确定出自移动机器人和目标对象之间的初始距离。
304、根据所述目标对象确定工作区域。
自移动机器人确定出目标对象后，向着目标对象行进。若目标对象自发出语音信号后一直未发生位移，则根据目标对象的初始位置确定工作区 域。例如，以目标对象的初始位置为中心，2米为半径画圆，将圆形区域作为工作区域。可以理解的是，如果画圆过程中遇到墙壁等物体，则结合物体轮廓和圆确定工作区域。采用该种方案，实现自移动机器人精准到达用户的指定区域的目的。
若目标对象自发出语音信号后发生位移，或者，当自移动机器人行进至目标对象附近后目标对象发生位移，则自移动机器人跟随目标对象移动，直到目标对象停止移动。之后，自移动机器人根据目标对象停止移动时的位置确定所述工作区域。该方案中，实现引导自移动机器人到达指定位置并执行任务的目的。
305、移动至所述工作区域并在所述工作区域内执行任务。
示例性的，若目标对象未发生位移，则自移动机器人根据自身所在位置和目标对象的位置规划路径，并控制自移动机器人按照该路径移动至目标对象附近。之后，在工作区域内执行任务。其中，路径的长度约为自移动机器人和目标对象之间的初始距离的长度。
若目标对象发生位移，则自移动按照路径移动至目标对象附近后，跟随目标对象继续移动，直到目标对象停止移动。之后，在工作区域内执行任务。
需要说明的是，虽然利用传统视觉传感器导航也能够实现在哪扫哪功能。例如，用户对扫地机器人说：“小Q，来这里扫”。扫地机器人识别语音信号后转向用户，然后根据视觉跟踪导航到达用户所在位置，在用户所在位置的附近区域执行清扫任务。但是该种实现方式有如下弊端：视觉跟踪导航没有深度信息，只能确定出用户在平面的位置，无法确定出用户在空间内的准确位置，导航过程非常的不流畅、体验不好。而且，若用户发生位移则容易跟丢，体验不好。其中，传统的视觉传感器例如为平面相机等。
然而，本申请实施例中，并未采用传统视觉导航方式，而是采用3D传感器，如激光类传感器，激光传感器例如为DTOF传感器。基于3D传感器每个移动对象具有深度信息，使得自移动机器人能够确定出移动对象在空间中的位置，进而确定出自移动机器人和目标对象之间的初始距离。因此，行进过程中，自移动机器人能够准确的到达工作区域。
本申请实施例提供的自移动机器人控制方法，自移动机器人根据用户 发出的语音信号确定声源方向，并确定自身周围的移动对象。之后，从自身周围的移动对象中确定出位于声源方向的目标对象，根据目标对象确定工作区域，并移动至工作区域并执行任务。采用该种方案，由于自移动机器人从移动对象中确定出目标对象，移动对象具有准确的空间位置。因此，自移动机器人能够根据声源方向从多个移动对象中准确确定出目标对象，并准确到达工作区域，无需借助客户端，过程简单、灵活。而且，该方案适用于所有激光类自移动机器人，成本低、算法简单、需要的算力低。
下面，用几个场景对上述的自移动机器人控制方法进行详细说明。
场景一、声源方向不存在障碍物，自移动机器人利用AI相机确定目标对象。
该种场景中，自移动机器人位于比较空旷区域、周围没有障碍物，用户只需要发出语音信号无需做轻踩地面两下等动作，自移动机器人利用AI相机确定目标对象。
例如，自移动机器人和用户位于同一空间，自移动机器人根据语音信号确定声源方向后，利用AI相机采集声源方向图像，根据图像确定声源方向是否存在行人以外的对象。若声源方向不存在行人以外的对象，则认为声源方向无障碍物，继续利用AI相机捕捉声源方向的图像，利用AI相机捕捉的图像确定目标对象。该过程中，用户无需做出轻踩地面两下等动作。
场景二、声源方向存在可从下面穿过的障碍物，自移动机器人利用DTOF传感器确定目标对象。
该种场景中，自移动机器人周围有茶几、桌子等无需绕行、可从下面穿过的障碍物，用户发出语音信号同时需要做轻踩地面两下等动作。自移动机器人利用DTOF传感器确定目标对象。如用户仅发出语音信号，则自移动机器人提示用户做出轻踩地面两下等动作。
例如，用户坐在沙发上，沙发前放置茶几，自移动机器人位于茶几之前。从自移动机器人的AI相机的角度，茶几遮挡了用户身体的一部分。自移动机器人根据语音信号确定声源方向后，利用AI相机采集声源方向图像，根据图像确定声源方向存在茶几，茶几是一个可从下面直接穿过的障碍物。之后，自移动机器人根据DTOF传感器采集的SLAM图和DTOF散点图等确定目标对象。
当自移动机器人确定当前场景为场景二时，若自移动机器人利用DTOF传感器没有发现移动对象。此时，自移动机器人可提示用户做出轻踩地面两下等动作，以便自移动机器人确定出目标对象。
图4是本申请实施例提供的自移动机器人控制方法的另一个流程图。本实施例中自移动机器人具体为扫地机，本实施例包括：
401、用户走到想要清扫的地方。
402、用户轻踩地面两下，并发出：“来这里扫”的语音信号。
用户发出语音信号的目的是为了使得扫地机确定声源方向。用户轻踩地面时为了使得扫地机识别目标对象后，确定目标对象在空间中的具体位置，该具体位置也称之为初始位置。
403、扫地机导航到人腿轻踩地面附近的区域。
扫地机根据目标对象在空间中的具体位置导航到用户轻踩地面的附近区域。用户轻踩地面的区域即为目标对象在空间中的初始位置。之后，扫地机根据DTOF跟踪算法导航到目标对象的脚下。若用户未发生位移，则根据初始位置确定工作区域。
若目标对象发生位移，则跟随目标对象移动。也就是说，若扫地机移动到目标对象跟前的过程中目标对象一直在移动，或者，若扫地机移动到目标对象跟前后，目标对象走动从而发生位移，则扫地机跟随目标对象到达指定位置，该指定位置即为目标对象停止行走时的位置。之后，扫地机根据目标对象停止移动时的位置确定所述工作区域。
404、开始自动清扫。
场景三、声源方向存在障碍物，且障碍物完全遮挡行人。
该种场景中，声源方向存在比较高的障碍物，且自移动机器人无法从该障碍物下面穿过。例如，障碍物为冰箱。再如，自移动机器人位于一个房间，用户位于另外房间。
自移动机器人根据语音信号确定声源方向后，利用AI相机采集声源方向图像，根据图像确定声源方向是否存在遮挡行人的障碍物。若声源方向存在遮挡行人的障碍物，则确定大致导航路径，按照导航路径移动过程中不断的采集图像，并调整导航路径。
上述实施例中，自移动机器人确定出声源方向后，需要对进一步的从多个移动对象中确定出目标对象。若目标对象发生位移，则需要跟踪目标对象。 自移动机器人可通过视觉追踪，利用摄像头捕捉行人画面，再利用图像处理算法从画面中提取出行人并且锁定目标对象从而进行跟踪。然而，摄像头对环境要求比较高，要求环境光的强度必须满足一定条件。若环境光的强度比较低时，比如画面全黑则无法采集到高质量图像。而且，图像处理算法较为复杂，对芯片算力要求比较高，实现动态跟踪比较困难。若为大量自移动机器人配置高质量摄像头，则成本很高。
为此，本申请实施例还可确定出目标对象并追踪目标对象。下面，以3D传感器具体为DTOF传感器为例，对自移动机器人如何确定并追踪目标对象进行详细说明。
图5是确定目标对象的流程图。本实施例包括：
501、获取多幅即时定位与地图构建SLAM图和多幅直接飞行时间DTOF散点图。
其中，所述多幅SLAM图中的SLAM图和所述多幅DTOF散点图中的DTOF散点图一一对应。
示例性的，自移动机器人利用DTOF传感器扫描周围环境，检测周围环境，得到多幅SLAM图和多幅DTOF散点图。例如，自移动机器人同步采集SLAM图和DTOFS散点图，一秒采集5帧SLAM图和5帧DTOF散点图，则5帧SLAM图中的SLAM图和5帧DTOF散点图中的DTOF散点图一一对应。
图6所示为LAM图。请参照图6，SLAM图中仅标记出静止对象，如墙面等。自移动机器人基于SLAM算法构建环境地图时，能够识别出周围环境中物体的轮廓并进行标注，如墙面、沙发、茶几、床等。图6中仅标识出墙面，如图中粗黑实线所示。
图7所示为DTOF散点图。请参照图7，不同于SLAM图，DTOF散点图中既有表征静态对象的像素点，也有表征移动对象的像素点。图中粗黑实线所示为墙面，实线椭圆分别表示行人和杂散点。
502、对于所述多幅DTOF散点图中的每一幅DTOF散点图，根据对应的SLAM图，从所述DTOF散点图中过滤掉表征静态对象的像素点以得到动态点集。
示例性的，如果仅有DTOF散点图，则无法识别出哪些点表征墙面、哪些点表征沙发、茶几、床等。因此，一种方式中，对于每一对SLAM图 和DTOF散点图，由于该俩幅图的采集时间相同、采集角度相同。因此，根据SLAM图，能够从DTOF散点图中识别出哪些点表征墙面、哪些点表征沙发、茶几、床等，即能够识别出DTOF散点图中表征静态对象的像素点。之后，从DTOF散点图中过滤掉表征静态对象的像素点以得到动态点集。该动态点集包括一些杂散点和移动对象对应的点。
另一种方式中，对于每一帧DTOF散点图，根据对应的SLAM图确定出该DTOF散点图中用于表征墙面、沙发等对象的点。之后，对于任意相邻的两帧DTOF图像，将该两帧DTOF散点图上的点都画在同一幅空白图像中。若一个对象为静止对象，则两帧DTOF散点图中表征该静止对象的点位于同一个位置，若一个对象为移动对象，则该两幅DTOF散点图中表征该移动对象的点位于不同位置且比较相似。因此，将相邻两幅DTOF散点图中的像素点画在同一幅空白图像中后，能够确定出动态点集。该动态点集包括一些杂散点和移动对象对应的点。
实际实现过程中，若两帧DTOF散点图中同一个对象映射在空白图像的同一个位置，则将该对象所有点用颜色a表示，该对象为静止对象。其余对象的像素点用户另外一种颜色b表示。显然，只有移动对象和杂散点才会使用颜色b，而静止对象使用颜色a。经过一些简单的过滤后，很容易识别出移动对象和静止对象。
另外，自移动机器人采集SLAM图和DTOF散点图的目的是为了找到目标对象并跟随目标对象，而目标对象通常为行人，因此，为了减少计算量，无需考虑其他移动对象，如滚动的球等。这种情况下，自移动机器人确定自移动机器人周围的移动对象之后，从移动对象中确定出位于所述声源方向的目标对象之前，还根据人类行走时的步态、运动速度等特征，从多个移动对象中确定出可能是行人的目标对象，从而过滤掉一部分杂散点。
503、根据相邻的多幅DTOF散点图的动态点集确定所述自移动机器人周围的移动对象。
示例性的，不同DTOF图中杂散点的位置不同，无规律可寻。即使将相邻两帧DTOF图中的杂散点画到同一幅空白图像中，也总结不出任何规律。而移动对象不同，若将相邻两帧DTOF图中同一个移动对象画到同一幅空白图像中，则该移动对象位于两个不同位置，该俩个位置之间的距离满足一定条件且该两个位置的点集中点的数量接近。
例如，若一个球正在滚动，则SLAM图中不存在用于表征球的像素点。但是，相邻两幅DTOF散点图中均存在代表该球的点集，前一帧DTOF散点图中，代表球的点集位于空白图像中的A位置，后一帧DTOF散点图中，代表球的点集位于空白图像中的B位置，A位置的点集中像素点的数量约等于B位置的点集中像素点的数量，且A位置点集形成的形状与B位置点集形成的点的形状相似。
采用该种方案，实现根据前后帧DTOF散点图和SLAM图确定出自移动机器人周围的移动对象和静止对象的目的。
可选的，上述实施例中，自移动机器人根据相邻的多幅DTOF散点图的动态点集确定所述自移动机器人周围的移动对象的过程中，首先，从第一DTOF散点图的第一动态点集中确定出第一子集。之后，自移动机器人确定所述第二DTOF散点图的第二动态点集中是否存在第二子集，所述第一子集指示的第一位置与所述第二子集指示的第二位置之间的距离大于预设距离，且所述第一子集和所述第二子集中像素点数量的差值小于预设差值，所述第一DTOF散点图和所述第二DTOF散点图是所述多幅DTOF散点图中任意相邻的两幅DTOF散点图，若第二动态点集中存在所述第二子集，则确定所述第一子集和所述第二子集表征同一个对象且所述对象为移动对象。
示例性的，预设距离是能表征一个对象是移动对象时，第一位置和第二位置的最小距离。每帧DTOF散点图的动态点集中可能包含一个或多个移动对象对应的点集以及一些杂散点。自移动机器人从第一DTOF散点图的第一动态点集中确定出第一子集，该第一子集包含比较集中的多个像素点。之后，自移动机器人确定第二DTOF散点图中的第二动态点集中是否存在第二点集。若第二动态点集中存在的第二点集，则说明第一点集和第二点集表征同一个对象且该对象为移动对象。若第二动态点集中不存在第二点集，则说明第一点集中的像素点是一些杂散点。另外，若一些像素点数量比较小，无法表征一个对象，则该些点为杂散点，如一个或数个点。或者，由于目标对象通常为行人，自移动机器人估算目标对象的行走速度等，将不符合目标对象的行走速度等条件的对象过滤掉。
继续以上述滚动的球为例，若球的速度为1米每秒，DTOF传感器的采集速率是5帧每秒，则球在第一DTOF散点图中的位置坐标，与球在第二DTOF 散点图中的位置坐标的距离约为20cm。因此，若第一子集对应的位置A和第二子集对应的位置B之间距离20cm，且第一子集和第二子集中像素点的数量接近，则说明第一子集和第二子集表征同一个对象且所述对象为移动对象。
采用该种方案，实现自移动机器人依据动态点集确定出周围的移动对象的目的。
可选的，上述实施例中，自移动机器人采集SLAM图和DTOF散点图的目的是为了找到目标对象并跟随目标对象，而目标对象通常为行人，因此，为了减少计算量，无需考虑其他移动对象，如滚动的球等。这种情况下，自移动机器人确定自移动机器人周围的移动对象之后，从移动对象中确定出位于所述声源方向的目标对象之前，还根据人类行走时的步态、运动速度等特征，从多个移动对象中确定出可能是行人的目标对象，进而从行人中确定出位于声源方向的目标对象。
自移动机器人从行人中确定出位于声源方向的目标对象的过程中，从所述移动对象中确定出脚部发生动作、且位于所述声源方向上的移动对象，以得到所述目标对象。
示例性的，自移动机器人的高度通常有限，以扫地机为例，扫地机的高度通常为10厘米，则扫地机只能采集10厘米高度范围内的DTOF图像。考虑到自移动机器人的DTOF传感器的视野的限制，用户发出语音指令时需要脚步做出动作，如轻踩地面两下、从左右脚并拢切换为左右脚张开一定角度、从左右脚张开一定角度切换为左右脚并拢等，若用户做出招手、鼓掌、摇头等动作，虽然说用户在运动，但是由于不在DTOF传感器的视野内，无法被DTOF传感器采集到，因此，这些动作无法实现本申请方案。
本申请实施例中，自移动机器人上部署一个预先训练好的模型，该模型能够根据DTOF散点图识别用户做出轻踩地面的动作。当相邻两帧DTOF散点图中第一子集和第二子集输入至模型，根据模型确定出第一子集和第二子集表征的移动对象的动作是轻踩地面时，若该移动对象位于声源方向，则确定移动对象为目标对象。
采用该种方案，一个移动对象必须做出轻踩地面的动作、且该移动对象位于声源方向时，才将该移动对象确定为目标对象，实现准确确定出目标对象的目的。
以上是确定自移动机器人周围的移动对象并从中确定出目标对象。下面， 对如何跟随目标对象进行详细说明。
可选的，上述实施例中，自移动机器人确定出目标对象，并朝向目标对象行进。之后，若目标对象发生位移，即目标对象走动时，则利用导航技术对目标对象进行跟随，直到目标对象停止移动。之后，根据目标对象停止移动时的位置确定所述工作区域。
利用导航技术跟随过程中，自移动机器人可根据局部规划算法等进行跟随。局部规划算法包括：向量场直方图(Vector Field Histogram，VFH)算法、动态窗口算法(dynamic window approach，DWA)等。从使用的硬件的角度，跟随包括基于DTOF传感器的跟随和基于A I相机的跟随。
自移动机器人基于DTOF传感器的跟随方式中，自移动机器人确定相邻两幅DTOF散点图中是否均出现所述目标对象，若相邻两幅DTOF散点图中均出现所述目标对象，则确定相邻两幅DTOF散点图中目标对象的距离与预设距离的差值。之后，根据所述差值调整速度以跟随所述目标对象移动。
以用户的正常行走速度为1米每秒、自移动机器人采集DTOF散点图的速度为200毫秒/帧为例，则相邻两帧DTOF散点图中目标对象的位置坐标的距离大约为20厘米。因此，跟随过程中，对于任意相邻的两帧DTOF散点图，自移动机器人确定该两帧DTOF散点图中是否均出现目标对象。若两帧DTOF散点图中均出现目标对象，则说明没有跟丢。
或者，自移动机器人在第一帧DTOF散点图(前一帧)中确定出目标对象的位置坐标。之后，按照行进方向，在第二帧DTOF散点图(后一帧)中距离该位置坐标20厘米的位置确定是否存在目标对象，若存在目标对象，则说明没有跟丢，继续跟随用户。
若跟丢，则根据最后一帧出现目标对象的DTOF散点图以及该帧之前的DTOF散点图寻找目标对象。或者，发出“哎呀，我迷路了，请来引导我前进吧”等语音提示。
若没有跟丢，则自移动机器人每次采集到DTOF散点图后，根据该DTOF散点图和前一帧DTOF散点图计算目标对象的距离，并比较该距离和上一个距离。若距离增大，说明目标对象移动速度加快，自移动机器人增大速度并跟随目标对象移动。若距离减小，说明目标对象移动速度放慢，自移动机器人降低速度并跟随目标对象移动。若距离不变，则说明目标对 象移动速度不变，自移动机器人保持当前速度跟随目标对象移动。
采用该种方案，实现基于DTOF传感器跟随目标对象移动的目的。
图8是本申请实施例提供的自移动机器人控制方法中基于AI相机的跟随流程图，本实施例包括：
801、用户唤醒或召唤自移动机器人。
用户对自移动机器人说出唤醒词从而唤醒自移动机器人。例如，唤醒词为“小Q小Q”，自移动机器人的语音控制功能处于待唤醒状态时，用户发出“小Q小Q”的语音信号后，自移动机器人的语音控制功能唤醒，之后，用户和自移动机器人进行语音交互。
用户对自移动机器人说出转换关键字从而召唤自移动机器人。例如，召唤关键字为“小Q，来这里扫”。自移动机器人的语音控制功能被唤醒后，用户发出“小Q，来这里扫”的语音信号后，自移动机器人识别出召唤并向用户行进。
另外，召唤关键字还兼具唤醒和召唤功能。例如，自移动机器人的语音控制功能处于待唤醒状态，用户发出“小Q，来这里扫”的语音信号后，自移动机器人的语音控制功能被唤醒，同时，自移动机器人识别出召唤并向用户行进。
自移动机器人被唤醒或识别出召唤后，利用环麦定位旋转一定角度使得自移动机器人基本朝向用户，继续旋转直到AI相机能够捕捉到用户，并利用AI相机捕捉到的人像对用户进行定位，从而使得自移动机器人的AI相机朝向用户。
802、自移动机器人确定目标对象。
自移动机器人利用AI的人形定位精确的面向用户。当AI相机捕捉到多个用户时，发出提示信息使得用户做出特定动作，如挥手、摇头等，以便二次确定目标对象。AI人形定位算法提供目标对象相对于自移动机器人的角度以及目标对象的骨架图。
803、自移动机器人保持跟随状态跟随目标对象移动。跟随过程中执行步骤804和806。
804、自移动机器人判断是否需要绕障，若需要绕障则执行步骤805；若不需要绕障则执行步骤808。
805，自移动机器人绕障，之后执行步骤808。
自移动机器人发现和目标对象之间存在障碍物时，切换为避障模式，利用LDS及线激光数据等确定最近的障碍物，确定障碍物的轮廓，进而选择更容易绕行的方向绕障。之后，执行步骤802。
806、自移动机器人判断是否需要越障，若需要越障，则执行步骤807；若不需要越障，则执行步骤808。
807、自移动机器人越障，之后执行步骤808。
自移动机器人利用线激光传感器等能够得到地面信息，进而识别出自移动机器人前方是否有台阶或悬崖。若有台阶和悬崖则需要越障。
为了保证自移动机器人的线激光传感器能够在不同材质地面上得到同样的有效距离，线激光传感器采用动态曝光方案，即在反射率较低的材质上提高曝光值，以得到更多有效数据；在反射率较高的材质上降低曝光值，以得到精度更高的数据。自移动机器人根据线激光传感器提高的数据识别台阶的高度、悬崖的高度等，根据台阶的高度、悬崖的高度等确定越障策略。
808、自移动机器人判断是否结束跟随，若结束跟随则结束；若未结束跟随则执行步骤809。
809、自移动机器人判断是否跟丢，若未跟丢目标对象，则执行步骤802；若跟丢目标对象，则执行步骤810。
810、寻找目标对象，若未找到目标对象则执行步骤801。
示例性的，由于自移动机器人移动能力的限制，存在一定概率的跟丢问题。若出现跟丢，则自移动机器人根据所述目标对象最后一次出现的位置坐标确定寻找范围，在所述寻找范围内寻找所述目标对象。其中，寻找范围例如为以最后一次出现的位置坐标为圆心，以预设半径为半径圈定的圆形范围，圆形范围例如为x平方米，
若寻找到目标对象，则继续跟随目标对象，若没有寻找到目标对象，则进入等待召唤状态，等待用户重新召唤。另外，若自移动机器人无法找到用户，还可以发出提示信息：“我跟丢了，来引导我吧”等，以引导目标对象来到自移动机器人跟前。
采用该种方案，利用AI相机跟随，响应灵敏、成本低，能够有效防止跟丢、跟错，同时，兼具避障和越障能力。而且，自移动机器人跟丢时执行跟丢逻辑以便自动寻找目标对象或提示用户以使得用户重新引导自移动机器人，过程简单。
图9是自移动机器人保持跟随状态的流程图。本实施例包括：
901、自移动机器人利用AI相机捕捉目标对象的骨架图。
902、自移动机器人确定骨架图是否完整，若骨架图完整，则执行步骤903；若骨架图不完整，则执行步骤905。
示例性的，自移动机器人根据骨架是否的完整性判断自移动机器人与目标对象之间是否存在障碍物。当AI相机捕捉到完整的骨架图时，表明自移动机器人和目标对象之间没有任何障碍物。此时，自移动机器人根据激光传感器数据，确定AI相机所在方向上，目标对象距离自移动机器人最近的点为目标对象的位置坐标，进而向目标对象移动并与目标对象保持固定距离。其中，激光传感器例如为DTOF传感器、激光雷达(Laser Direct Structuring，LDS)等。
当AI相机捕捉的目标对象的骨架图不完整时，表明自移动机器人和目标对象之间存在障碍物。激光传感器数据大概率会被障碍物遮挡，使得自移动机器人无法获取到激光传感器数据，进而无法跟随。此时，激光传感器需要避障或越障来尝试绕开障碍物，直到AI相机能够捕捉到目标对象完整的骨架图。之后，根据目标对象移动。
903、跟随目标对象移动。
904、判断是否结束跟随，若结束跟随则结束；若未结束跟随则执行步骤901。
示例性的，若目标对象一直移动，则说明未结束跟随；若目标对象停止移动，则数码结束跟随。
905、确定障碍物的轮廓。
906、根据轮廓避障，之后执行步骤901。
上述实施例中，一种方式中，跟随过程中自移动机器人仅跟随目标对象移动而不工作。例如，扫地机跟随过程中不执行扫地、拖地功能，直到目标对象停止移动时，根据目标对象停止移动时的位置确定出工作区域并在工作区域内工作。
另一种方式中，自移动机器人跟随过程中，保证自移动机器人和目标对象保持一定距离的同事，自移动机器人与目标对象能够进行互动。以边跟随边播放音乐为例，自移动机器人执行音乐播放功能，此时，目标对象召唤自移动机器人，自移动机器人跟随目标对象。跟随过程中，自移动机器人保持 音乐播放。若目标对象停止运动，自移动机器人与目标对象保持一定距离范围，并始终面向目标对象，以得到最佳音乐播放效果。
图10是自移动机器人的结构示意图。请参照图10，自移动机器人例如为空气净化机器人，其上设置有DTOF传感器、LDS传感器、环麦、AI相机、线激光传感器等。
下述为本申请装置实施例，可以用于执行本申请方法实施例。对于本申请装置实施例中未披露的细节，请参照本申请方法实施例。
图11为本申请实施例提供的一种自移动机器人控制装置的结构示意图。该自移动机器人控制装置1100包括：第一确定模块1101、第二确定模块1102、第三确定模块1103、处理模块1104和执行模块1105。
第一确定模块1101，用于根据用户发出的语音信号确定声源方向；
第二确定模块1102，用于确定自移动机器人周围的移动对象；
第三确定模块1103，用于从所述移动对象中确定出位于所述声源方向的目标对象；
处理模块1104，用于根据所述目标对象确定工作区域；
执行模块1105，用于移动至所述工作区域并在所述工作区域内执行任务。
一种可行的实现方式中，所述第二确定模块1102，用于获取多幅即时定位与地图构建SLAM图和多幅直接飞行时间DTOF散点图，所述多幅SLAM图中的SLAM图和所述多幅DTOF散点图中的DTOF散点图一一对应；对于所述多幅DTOF散点图中的每一幅DTOF散点图，根据对应的SLAM图，从所述DTOF散点图中过滤掉表征静态对象的像素点以得到动态点集；根据相邻的多幅DTOF散点图的动态点集确定所述自移动机器人周围的移动对象。
一种可行的实现方式中，所述第二确定模块1102根据相邻的多幅DTOF散点图的动态点集确定所述自移动机器人周围的移动对象时，用于从第一DTOF散点图的第一动态点集中确定出第一子集；确定所述第二DTOF散点图的第二动态点集中是否存在第二子集，所述第一子集指示的第一位置与所述第二子集指示的第二位置之间的距离大于预设距离，且所述第一子集和所述第二子集中像素点数量的差值小于预设差值，所述第一DTOF散点图和所述第二DTOF散点图是所述多幅DTOF散点图中任意相 邻的两幅DTOF散点图；若所述第二动态点集中存在所述第二子集，则确定所述第一子集和所述第二子集表征同一个对象且所述对象为移动对象。
一种可行的实现方式中，所述第三确定模块1103，用于从所述移动对象中确定出脚部发生动作、且位于所述声源方向上的移动对象，以得到所述目标对象。
一种可行的实现方式中，所述处理模块1104，用于移动至距所述目标对象预设距离的位置，若所述目标对象未发生位移，则根据所述目标对象的初始位置确定所述工作区域。
一种可行的实现方式中，所述处理模块1104，用于移动至距所述目标对象预设距离的位置后，若所述目标对象发生位移，则控制所述自移动机器人跟随所述目标对象移动；当所述目标对象停止移动时，根据所述目标对象停止移动时的位置确定所述工作区域。
一种可行的实现方式中，所述处理模块1104控制所述自移动机器人跟随所述目标对象移动时，用于确定相邻两幅DTOF散点图中是否均出现所述目标对象；
若相邻两幅DTOF散点图中均出现所述目标对象，则确定相邻两幅DTOF散点图中目标对象的距离；
根据所述距离调整速度以跟随所述目标对象移动。
一种可行的实现方式中，所述处理模块1104控制所述自移动机器人跟随所述目标对象移动时，用于利用人工智能AI相机捕捉目标对象的骨架图；当所述骨架图完整时，保持跟随状态以跟随所述目标对象移动；当所述骨架图不完整时，对所述自移动设备和所述目标对象之间的障碍物避障或越障直至所述AI相机捕捉到完整的骨架图像后，保持跟随状态以跟随所述目标对象移动。
一种可行的实现方式中，所述处理模块1104在所述第一确定模块1101根据用户发出的语音信号确定声源方向之前，还用于唤醒所述自移动机器人；确定所述语音信号对应的控制指令用于控制所述自移动机器人即时根据所述目标对象确定工作区域。
一种可行的实现方式中，所述处理模块1104，还用于确定是否跟丢所述目标对象；若跟丢所述目标对象，则根据所述目标对象最后一次出现的位置坐标确定寻找范围；在所述寻找范围内寻找所述目标对象；若未寻找 到所述目标对象，则进入等待召唤状态。
本申请实施例提供的自移动机器人控制装置，可以执行上述实施例中自移动机器人的动作，其实现原理和技术效果类似，在此不再赘述。
随着科学技术的进步，机器人已经走进越来越多人的生活，在人们的生活中扮演着重要的角色。目前，可通过主机实体键、手机应用程序(APP)、遥控器等对机器人进行操作。该些操作方法都存在弊端。为此，通过语音控制机器人的方式由于更加智能，受到用户广泛的欢迎。具有语音识别功能的机器人采集到语音信号后，识别语音信号并执行相关任务。
然而，目前的语音指令只能控制机器人开启、执行任务、停止、充电等。无法对机器人进行精确控制。例如，若用户想要机器人在特定区域内执行任务，则需要将机器人搬运到特定区域，然后语音控制机器人。再如，若某些区域为禁止区域，如卫生间，则机器人工作时用户必须关闭卫生间，才能防止机器人进入禁止区域。
又如，有时候用户想要机器人对多个区域执行任务，则需要将机器人挪动至其中一个区域，当机器人在该区域执行完任务后，再将机器人挪动至另一个区域。虽然有些机器人能够识别用户指示的工作区域，但是前提是用户发出的语音只能指示一个工作区域。若用户想要对多个区域执行任务，则机器人每次执行任务之后，用户都需要通过语音指示下一个工作区域，过程麻烦。
基于此，本申请实施例还提供一种自移动机器人语音控制方法、装置、设备及可读存储介质，通过一条语音信号向自移动机器人指示多个工作区域，使得自移动机器人能够依次对多个工作区域执行任务，精确度高、过程简单。
图12是本申请实施例提供的自移动机器人语音控制方法的流程图。本实施例的执行主体是自移动机器人，本实施例包括：
201、采集第一语音信号。
202、当所述第一语音信号与所述自移动机器人的唤醒指令匹配时，唤醒所述自移动机器人的语音控制功能。
自移动机器人上设置声音信号采集装置，声音信号采集装置例如为麦克风、麦克风阵列等。自移动机器人的语音控制功能未被唤醒时，声音信号采集装置不断的采集周围环境中的第一语音信号，对第一语音信号进行识别， 若第一语音信号与自移动机器人的唤醒指令匹配，则唤醒自移动机器人的语音控制功能；若第一语音信号与自移动机器人的唤醒指令不匹配，则保持语音控制功能处于等待唤醒状态。
当语音控制功能唤醒时，自移动机器人采集到第二语音信号后，将第二语音信号发送至云端的语音识别服务器，以使得语音识别服务器确定第二语音信号是否和控制指令匹配；当语音控制功能处于等待唤醒状态时，自移动机器人在本地识别采集到的第一语音信号是否与唤醒指令匹配。
203、在所述语音控制功能唤醒状态下采集第二语音信号。
自移动机器人的语音控制功能唤醒后，自移动机器人能够利用声音信号采集装置采集周围环境中的语音信号。例如，采集到用户发出的第二语音信号。
204、根据所述第二语音信号确定至少两个工作区域。
自移动机器人自身具有语音识别功能时，自移动机器人对第二语音信号进行识别，从而确定出至少两个工作区域。或者，自移动机器人将第二语音信号发送给语音识别服务器，由语音识别服务器确定出至少两个工作区域并指示给自移动机器人。例如，第二语音信号对应的文本内容为“清扫小明房间和书房”，则至少两个工作区域为小明房间和书房；再如，第二语音信号对应的文本内容为“清扫所有地毯区域”，则工作区域为以地毯为中心、包含地毯的多个区域。
图13是本申请实施例中语音识别的过程示意图。请参照图13，语音识别过程中，自移动机器人采集到第二语音信号，对第二语音信号进行降噪处理后，将降噪处理后的第二语音信号上传给语音识别服务器。语音识别服务器对第二语音信号进行语义识别得到语音指令，并将语音指令回传给自移动机器人。自移动机器人接收到语音指令后，执行语音指令指示的任务。任务可以是清扫、拖地、割草、净化空气等。
需要说明的是，虽然图13中是以自移动机器人采集第二语音信号为例进行说明。然而，本申请实施例并不限制，其他可行的实现方式中，用户还可以开启终端设备上的客户端然后发出第二语音信号，由终端设备采集第二语音信号，对第二语音信号进行降噪处理后，将降噪处理后的第二语音信号上传给语音识别服务器。
205、依次对所述至少两个工作区域执行所述第二语音信号指示的任务。
自移动机器人按照一定的顺序依次对各工作区域执行任务。例如，自移动机器人随机对各工作区域排序，得到随机队列，按照随机队列指示的顺序依次对各工作区域执行任务。
再如，若用户期望对多个工作区域执行清洁等任务，且用户期望优先对其中某个工作区域执行清洁任务，则可以按照优先级顺序发出第二语音信号。自移动机器人确定所述第二语音信号中所述至少两个工作区域中各工作区域出现的先后顺序。之后，根据所述先后顺序依次对所述至少两个工作区域执行所述第二语音信号指示的任务。
以自移动机器人为空气净化机器人为例，若用户想要净化书房和婴儿室的空气，且想要优先净化婴儿室的空气，则第二语音信号为“请净化婴儿室和书房”。自移动机器人根据第二语音信号确定出工作区域为婴儿室和书房，且婴儿室优先。此时，即使书房距离自移动机器人较近、婴儿室距离自移动机器人较远，自移动机器人也会先行进至婴儿室，完成婴儿室的空气净化后，再行进至书房，对书房进行空气净化。
采用该种方案，自移动机器人根据优先级依次对多个工作区域执行任务，极大程度上满足用户需求，更加人性化。
又如，工作区域比较多，且各个工作区域之间存在一定的距离，若自移动机器人按照随机顺序对各个工作区域执行任务，则耗能大、耗时长。为此，自移动机器人确定出多个工作区域后，确定自身与所述至少两个工作区域中各工作区域之间的距离。之后，按照距离由近至远的顺序对所述至少两个工作区域排序得到队列；并根据所述队列依次对所述至少两个工作区域执行所述第二语音信号指示的任务。
举例来说，第二语音信号为“清扫所有地毯区域”，则自移动机器人从环境地图中确定出所有包含地毯的区域，示例性的，请参照图14。图14是地毯区域的示意图。
请参照图15，图中有3个地毯区域，分别为地毯区域41、地毯区域42和地毯区域43。自移动机器人40确定出地毯区域41距其最近，接下来是地毯区域42，最后是地毯区域43。因此，清扫顺序为地毯区域41、地毯区域42和地毯区域43。
采用该种方案，自移动机器人按照距离由近及远的顺序对各个工作区域执行任务，能够最大限度的解约能耗并提高速度。
需要说明的是，虽然上述是以通过语音操作的形式操作自移动机器人对多个工作区域执行任务。然而，本申请实施例并不限制，其他可行的实现方式中，还可以通过语音操作的形式操作自移动机器人对单个工作区域执行任务。
本申请实施例提供的自移动机器人语音控制方法，自移动机器人采集到第二语音信号后，根据第二语音信号确定至少两个工作区域，并依次对各工作区域执行第二语音信号指示的任务。采用该种方案，通过一条语音信号就能够向自移动机器人指示多个工作区域，使得自移动机器人能够依次对多个工作区域执行任务，用户通过自然语言与自移动机器人交互以使得对自移动机器人的极简控制，精确度高、过程简单。
可选的，上述实施例中，自移动机器人依次对所述至少两个工作区域执行所述第二语音信号指示的任务的过程中，对所述至少两个工作区域中的一个工作区域执行完任务后，行进至下一个工作区域之前，停止执行任务。
自移动机器人确定出至少两个工作区域后，按照一定的顺序依次对各个工作区域执行任务。若相邻两个工作区域之间的距离较长，自移动机器人按照行驶路径从其中一个工作区域行进至另一个工作区域的过程中，可关闭工作模块。也就是说，自移动机器人在行驶路径上行进的过程中，不执行清洁、割草等任务。
再请参照图14，以第二语音信号为清洁地毯区域43、地毯区域42和地毯区域41。自移动机器人40从当前位置行进至地坛区域43。该过程中，自移动机器人40的工作模块处于关闭状态。当自移动机器人40对地毯区域43完成清洁之后，从地毯区域43行进至地毯区域42的过程中，以及当自移动机器人40对地毯区域42完成清洁之后，从地毯区域42行进至地毯区域41的过程中，在行驶路径上都关闭工作模块，行驶路径如图中虚线箭头所示。
采用该种方案，自移动机器人从一个工作区域移动至下一个工作区域的过程中无需开启工作模块，节约能量的同时提高行进速度。
可选的，上述实施例中，自移动机器人从当前位置移动到第一个工作区域之前，或者，从当前工作区域移动至下一个工作区域之前，还确定行驶路径的长度是否大于预设阈值。若行驶路径的长度大于预设长度，则关闭工作 模块并根据所述行驶路径向工作区域行进。若行驶路径的长度小于或等于预设长度，则在工作模块开启的状态下向工作区域行进。
示例性的，有时候两个工作区域之间的行驶路径比较短，比如，卧室和客厅之间行驶路径的长度几乎可以忽略。为避免频繁开关工作模块带来的损坏，当行驶路径较短时，无需关闭工作模块。
可选的，上述实施例中，自移动机器人根据所述第二语音信号确定至少两个工作区域时，首先，根据第二语音信号确定区域类别。之后，根据区域类别从环境地图对应的区域集合中确定出至少两个工作区域。
本申请实施例中，自移动机器人在完全未知环境中时会构建环境地图，或者，接收其他机器人发送的环境地图，之后，利用分区算法等将环境地图进行区域分割，从而得到多个工作区域，使得环境地图能够表征各个工作区域，如厨房、卫生间、卧室等。而且，环境地图也能够表征环境中不同物体在环境中的实际位置，使得自移动机器人能判断出各个工作区域内物体的摆放状态。
当用户想要对某种类别的工作区域进行清扫等，在第一语音指令中加入类别的信息。例如，“对所有卧室进行清扫”。自移动机器人识别该语音信号后仅对各个卧室进行清扫。
再如，“对客厅的家具进行清扫”，该语音信号指示的区域类别为“客厅”。因此，自移动机器人从环境地图中确定出客厅。进一步的，该语音信号还指示目标物体“家具”。因此，自移动机器人确定客厅内的家具，如沙发、茶几等。对于每个家具，以该家具为中心，确定一个包含家具的区域，对该区域进行清扫。
又如，“对全部卧室的床底进行清扫”，该语音信号指示的区域类别为“卧室”。因此，自移动机器人从环境地图中确定出卧室。进一步的，该语音信号还指示目标物体“床”。因此，自移动机器人继续确定各个卧室内的床的位置。对于每个床，以该床为中心，确定一个包含床的区域，对该区域进行清扫。
又如，“开始清扫沙发和餐桌区域”，自移动机器人的行为为：行走到沙发所在位置，以沙发中心为原点画一个大于沙发的矩形框，将该矩形框作为工作区域并清扫。之后，行进至餐桌所在区域，对餐桌中心为原点画一个大于餐桌的矩形框作为工作区域并清扫。
又如，“开始清扫沙发区域”，自移动机器人的行为为：行走到沙发所在位置，以沙发中心为原点画一个大于沙发的矩形框，将该矩形框作为工作区域并清扫。
采用该种方案，用户可通过语音控制自移动机器人仅对某种类别的工作区域执行清扫等任务，智能化高。进一步的，语音信号中还可以指示目标物体，使得自移动机器人对局部区域进行清扫等任务，进一步提高自移动机器人的智能化。
可选的，上述实施例中，自移动机器人能够根据所述环境地图、所述环境地图中物体的位置信息或所述环境地图中门的位置信息，将所述环境地图划分为多个工作区域以得到所述区域集合。之后，更新区域集合中各工作区域的标识，并向语音识别服务器发送更新信息，以使得语音识别服务器更新各工作区域的标识。
示例性的，自移动机器人上安装摄像头等拍摄装置。图15是家具识别过程示意图。请参照图15，自移动机器人构建环境地图或行进过程中不断的拍摄图像以进行图像采集。采集到图像后，对图像进行预处理，预处理包括增强对比度、无损放大、特征提取等中的一项或多项。之后，自移动机器人利用预先部署的训练模型对预处理后的图像进行AI识别以使得训练模型输出图像中家具的种类、位置坐标等识别结果，并将识别结果从(three-dimensional，3D)环境地图映射到二维(two-dimensional，2D)环境地图中保存并显示。其中，训练模型例如为以各种家具为样本训练出的AI模型。
图16是识别门的过程示意图。与上述图15的区别在于，训练模型是预先以各种门为样本训练出的AI模型，图像输入训练模型后，输出门的位置坐标等识别结果，并将识别结果从3D环境地图中映射到2D环境地图中保存并显示。
自移动机器人获取到2D环境地图后，融合2D环境地图中家具的识别结果、门的识别结果，利用分区算法对2D地图进行分区，从而将2D的环境地图划分为多个区域。之后，自移动机器人将分区结果发送给APP服务器和语音识别服务器，由APP服务器将分区结果发送给终端设备，终端设备上安装用于操控自移动机器人的APP。终端设备接收到分区结果后，显示分区结果。示例性的，请参照图17。
图17是自移动机器人和语音识别服务器同步过程示意图。请参照图17， 经过分区后，环境地图的区域集合包括区域1、区域2和区域3。用户可在客户端对各个区域的标识进行自定义编辑。例如，将上述的区域1、区域2和区域3各自的标识依次修改为小明的房间、客厅、厨房。之后，终端设备将更新信息发送给APP服务器，由APP服务器将更新信息发送给自移动机器人。自移动机器人接收到更新信息后，更新本地环境地图中各个工作区域的标识。
另外，自移动机器人更新各个工作区域的标识后，还向语音识别服务器同步更新工作区域的标识。自移动机器人接收到来自APP服务器的更新信息后，发现工作区域的标识发生变化。之后，自移动机器人更新本地的同时，将更新信息发送给语音识别服务器，使得语音识别服务器更新各工作区域的标识并保存。之后，用户就能够根据自定义命名与自移动机器人进行交互。
例如，用户说：“清扫小明的房间”，则自移动机器人的行为为：行走到自定义命名为“小明”的房间区域并清扫此区域。
再如，用户说：“清扫客厅和厨房”，自移动机器人的行为为：按照用户喊话的顺序先行走到自定义命名为“客厅”的区域，清扫该区域之后，再行走到自定义命名为“厨房”的区域并清扫。
采用该种方案，语音识别服务器上保存的各个工作区域的标识和自移动机器人上保存的对应工作区域的标识一致，提高语音识别服务器语音识别的准确性。
上述实施例中，当第二语音信号指示区域类别时，自移动机器人根据区域类别从预先构建的区域集合中确定出符合区域类别的工作区域。然而，本申请实施例并不限制，其他可行的实现方式中，当区域类别指示的区域是特定区域、且每次下发语音信号时特定区域的位置可能不一样时，自移动机器人还可以实时根据区域类别确定工作区域。比如，特定区域为家中有水渍的区域，显然，不同时间家中有水渍的地方不同。该种情况下，自移动机器人利用摄像头等采集图像，对图像进行识别从而确定出符合区域类别的至少两个工作区域。
例如，第二语音信号为“检查家中有水的地方并擦干”。自移动机器人采集到该语音信号并进行语义识别后，行进过程中不断拍摄图像并识别图像，若图像中存在有水的地方，则对有水的地方进行擦干。之后，继续行进并拍摄图像，每次识别到有水的地方则进行擦干。
再如，第二语音信号为“对厨房有油污的地方进行拖地”。自移动机器 人采集到该语音信号并进行语义识别后，在厨房内行进过程中不断拍摄图像并识别图像，每次识别到有油污的地方后进行拖地。
采用该种方案，实现对特定区域执行任务的目的。
可选的，上述实施例中，自移动机器人依次对所述至少两个工作区域执行所述第二语音信号指示的任务时，可根据区域类别确定作业方式，并根据作业方式一次对指示两个工作区域执行任务。
示例性的，用户发出第二语音信号的时候，可以不指示具体作业方式，而是由自移动机器人自主确定作业方式并执行。例如，用户说：“清洁厨房中的油污”。自移动机器人不断采集图像确定出油污区域后，确定作业方式为：加清洁液并提高拖地频率。之后，自移动机器人在油污区域喷洒清洁液并强力拖地。
再如，用户说：“清洁客厅内的水渍”。自移动机器人不断采集图像确定出水渍区域后，若水渍区域内的水比较多，则确定作业方式为：执行三次拖地。之后，自移动机器人在水渍区域拖地三次。若水渍区域内的水比较少，则确定作业方式为：执行一次拖地。之后，自移动机器人在水渍区域拖地一次。
采用该种方案，由自移动机器人自动确定出更合适的作业方式，实现提高任务执行效率的目的。
可选的，上述实施例中，假设自移动机器人采集到第二语音信号时位于初始区域，该初始区域不是第二语音信号中的任意一个工作区域。那么，自移动机器人从初始区域移动至工作区域之前，记录初始区域的任务执行情况。之后，依次对各个工作区域执行任务。执行完任务后，根据记录确定是否未对初始区域执行完任务。若自移动机器人未对初始区域执行完任务，则返回初始区域并执行任务。
采用该种方案，自移动机器人对第二语音信号指示的区域执行完任务后，返回初始区域继续执行任务，避免无法对初始区域完成任务。
可选的，上述实施例中，第二语音信号还可以包括任务参数等。例如，第二语音信号为：“采用强拖模式清扫小明的房间和书房，分别清扫10分钟”、“对油污区域拖地两次”等。
上述实施例中，为了防止用户没有交互需求的时候，自移动机器人不断识别语音信号，自移动机器人的语音控制功能通常处于静默状态。只有用户 发出特定的唤醒词之后，自移动机器人的语音控制功能才能被唤醒。语音控制功能处于静默状态时，自移动机器人可以是静止也可以正在工作。
自移动机器人工作过程中，由于电机的转动等产生噪声，该噪声很有可能干扰自移动机器人识别语音信号的准确率。本申请实施例能够避免该种弊端。以下将自移动机器人唤醒后的工作状态称之为第二工作状态，将自移动机器人唤醒前的工作状态称之为第一工作状态，自移动机器人在所述第二工作状态下产生的声音的音量小于在所述第一工作状态下产生的声音的音量。当自移动机器人在第一工作状态下采集到第二语音指令后，若第一语音信号与自移动机器人的唤醒指令匹配，则自移动机器人自动切换至第二工作状态，即自移动机器人通过降低输出功耗等方式切换为第二工作状态，并在第二工作状态下采集上述的第二语音信号。
示例性的，在自移动机器人上安装麦克风时，寻找噪音最低并且稳定的地方安装麦克风。而且，通过大量样本训练唤醒模型，使得自移动机器人在各种运行状态下的唤醒率得到提升。之后，当自移动机器人的语音控制功能在运行状态下被唤醒时，自移动机器人通过改变自身运行状态以降低自身产生的噪声的音量。之后，用户通过正常音量下发控制语音指令，自移动机器人接收并执行相应的任务。
例如，自移动机器人为扫地机器人，自移动机器人在第一工作状态下工作时，行进速度为0.2米/秒。用户发出第一语音信号，若第一语音信号和唤醒指令匹配，则自移动机器人切换至第二工作状态，行进速度为0.1米/秒，产生的噪声较低。之后，用户发出第二语音信号，自移动机器人采集第二语音信号并执行相关任务。第二语音信号的音量可以小于第一语音信号的音量。
另外，为了保证高噪声环境下能够唤醒自移动机器人，可预先通过多次训练和算法提高自移动机器人运行状态下的唤醒率。例如，自移动机器人在用户5米以内范围且处于第一工作状态时，正常人声唤醒率可达到85％。唤醒后，自移动机器人切换为第一工作状态，第一工作状态下的语音识别准确率几乎达到智能音箱的同等水平。
采用该种方案，自移动机器人被唤醒后，自动改变运行状态以降低自身产生的噪声，提高后续的语音识别准确率。
可选的，上述实施例中，当第一语音信号与所述自移动机器人的唤醒指令匹配时，自移动机器人确定第一语音信号的声源位置。之后，自移动 机器人根据声源位置控制所述自移动机器人从第一位姿切换为第二位姿，所述自移动机器人处于所述第二位姿时麦克风与所述声源位置的距离，小于所述自移动机器人处于所述第一位姿时麦克风与所述声源位置的距离，所述麦克风是设置在所述自移动机器人上的麦克风。
示例性的，智能语音技术已被广泛的应用于人机交互、智能操控、在线服务等领域，随着更多应用场景的扩展，智能语音技术已经成为人们信息获取和沟通最便捷，最有效的手段，智能语音技术包括语音识别技术和语音合成技术。声源定位是基于麦克风阵列来对声源进行定位的方法，实现方法可以分为定向波速形成和时间延迟估计。将智能语音技术、麦克风声源定位技术与自移动机器人结合可以设计出非常丰富的应用场景，比如下达语音指令给自移动机器人执行任务，与自移动机器人语音交互以获取相应指导，根据声源控制自移动机器人转向等。
在自移动机器人领域，通常是自移动机器人上设计麦克风阵列用于接收声源信息，用于声源定位，根据定位控制自移动机器人转向声源定位方向，增加交互的趣味性和下次语音识别的准确性。该类应用的缺点是麦克风阵列定位的误差较大，一般误差在±45°左右，导致误差的根本原因在于声源到达麦克风的时间差估计精度不够。由于误差的存在，最终自移动机器人转向声源的效果可能不准确，导致使用体验较差。
为此，本申请实施例中，当自移动机器人的语音控制功能被唤醒后，自移动机器人可调整位姿，使得自移动机器人上的麦克风靠近用户，从而提高语音采集的准确率。利用声源定位技术、语音识别技术和AI识别技术精确控制自移动机器人转向发声者，即转向用户。该过程中，自移动机器人通过麦克风阵列捕捉声源，信号转换之后识别为既定唤醒词。之后，确定声源相对于麦克风阵列的位置，进而确定声源相对于机体的位置，从而确定出大致的旋转角度，即定位到声源的大致位置。最后，自移动机器人再根据旋转角度旋转，旋转过程中结合AI识别以准确判断出声源的具体位置，进而控制自移动机器人停在面向用户的位置。
下面，对自移动机器人如何转向声源定位方向进行详细说明。
首先，确定第一位置，第一位置是声源相对于麦克风阵列中心的位置。
用户发出语音后，自移动机器人通过麦克风阵列拾取语音信号，并利用计算单元处理语音信号，得到语音识别结果。若语音识别结果与唤醒词 匹配，则确定第一位置；若语音识别结果与唤醒词不匹配，则保持等待唤醒状态。
图18A是确定声源相对于麦克风阵列中心的位置的示意图。请参照图18A，麦克风阵列包括6个麦克风，分别位于S1-S6，该6个麦克风均匀分布在半径为L1的圆周上，空间坐标系的原点O即为麦克风阵列的中心。声源发出声音后，声音到达各麦克风的时长不一样。因此，自移动机器人可以根据声音的传播速度、时间延迟、各麦克风的位置等确定出第一位置。其中，时间延迟是指不同麦克风接收声音的时长的差值。
需要说明的是，虽然此处是以6个麦克风为例，然而，本申请实施例并不限制。其他可行的实现方式中，可以设置更多或更少的麦克风。
其次，根据第一位置确定第二位置，根据第二位置确定旋转角度，其中，第二位置是声源相对于自移动机器人中心的位置。
通常情况下，麦克风阵列位于自移动机器人的机体的固定位置，确定出第一位置后，自移动机器人就能够根据麦克风阵列的位置和第一位置，确定出第二位置。
图18B是确麦克风阵列和自移动机器人机体的示意图。请参照图18B，机体的中心为大圆的中心，机体的中心和麦克风阵列的中心并不重合，但是两者的相对位置是已知的。因此，自移动机器人确定出第一位置后，就能够确定出第二位置。确定出第二位置后，就能够确定出旋转角度，旋转角度是指从第一位姿到第二位姿的过程中，自移动机器人的旋转角度。
第一位姿是唤醒前自移动机器人的位姿，第二位姿是自移动机器人的麦克风朝向用户的位姿，第二位姿也可以理解为自移动机器人正对着用户的位姿。自移动机器人正对着用户的位姿是指：自主移动设的摄像头朝向用户的位姿。
最后，根据旋转角度旋转。
旋转过程中，为了避免旋转过快影响AI识别效果、旋转过慢影响用户体验，所以，电子设备将所述旋转角度划分为第一角度和第二角度；在所述第一角度内以第一速度旋转，在所述第二角度内以第二速度旋转，所述第一速度大于所述第二速度。第二角度例如为α度。
旋转过程中，假设旋转角度为θ，先快速旋转θ-α度，再匀速旋转α度，α＜θ。匀速旋转过程中，利用摄像头不断拍摄图像并进行AI识别。 若识别到用户，则停止旋转；若未识别到用户，则旋转α度后停止旋转。其中，α和自移动机器人的统计误差有关，其可以是30度、60度等。本申请实施例并不限制。
以自移动机器人为扫地机器人为例，再请参照图18B，扫地机器人功能组件包括摄像头、麦克风阵列、激光测距传感器、红外接收传感器、边刷、驱动轮等。另外，扫地机器人还包括图中未示意出的沿边传感器、防跌落传感器、吸尘风机、运动电机、滚刷、计算存储单元、电池模块、wifi模块等。语音控制使用场景下，扫地机器人处于任意工作状态，使用者向扫地机发出语音唤醒指令，扫地机器人暂停当前工作，转向唤醒指令发出者，等待用户的下一步交互命令。
例如扫地机器人正在客厅清扫，用户发出唤醒词“小Q，小Q”,扫地机器人暂停清扫工作，转向用户，同时语音播报应答“我在”，等待用户进一步的指令，比如“请离开客厅，去别的房间打扫”，扫地机器人会语音播报应答“好的”，同时离开客厅进入卧室等其他房间继续打扫。
在这种语音交互体验中，就要求扫地机器人准确识别用户的唤醒指令，同时精确而快速的转向用户，等待下一步操控指令。如果无法准确定位到语音指令发出者的位置，这种交互场景将变的十分糟糕，第一种情况是定位不准确，机器人转向的别的方向，未准确面对语音操控者；第二种情况是定位到了语音操控者的方向，但是旋转过程缓慢，动作持续时间久，交互体验差。
为此，扫地机器人暂停清扫工作，转向用户的过程中，先确定声源相对于麦克风阵列中心的第一位置，再根据第一位置和麦克风阵列相对于机体的位置确定第二位置和旋转角度。之后，先快速旋转θ-α度，再匀速旋转α度，匀速旋转过程中，利用摄像头不断拍摄图像并进行AI识别。若识别到用户，则停止旋转；若未识别到用户，则旋转α度后停止旋转。
例如，图18B中，若用户位于扫地机器人的右边，则旋转角度θ＝180度。若α为60度，则自移动机器人先顺时针快速旋转120度，接下来匀速转动并采集图像，若旋转到170度时根据图像识别到用户，则停止转动。若未识别到用户，则匀速旋转60度后停止转动。
采用该种方案，通过结合声源定位、语音识别和AI识别，使得自移动机器人的转向动作更准确。而且，旋转过程中，根据旋转角度先快速旋 转、再匀速缓慢旋转动作更流畅、AI识别更准确。
上述实施例中，语音识别技术是一种基于语音特征参数的模式识别，语音识别服务器能够将输入的语音按照一定的模式进行分类，进而依据判断准则找出最佳匹配结果。原理框架图如图18C所示。
图18C是训练语音识别模型和识别语音的过程示意图。请参照图18C，训练过程中，输入的语音信号经过预处理后进行特征提取，利用提取出的特征进行模型训练，生成语音识别模型后保存语音识别模型。
训练好的语音识别模型部署在语音识别服务器上。用户发出的语音信号经过预处理后进行特征提取，语音识别服务器将提取出的特征输入至语音识别模型，以使得语音识别结果。
图19是本申请实施例提供的自移动机器人语音控制逻辑流程图。本实施例包括：
1901、自移动机器人处于第一工作状态，语音控制功能处于等待唤醒状态。
1902、自移动机器人采集到第一语音信号。
示例性的，用户发出第一语音信号，自移动机器人利用声音信号采集装置等采集第一语音信号。
1903、第一语音信号与自移动机器人的唤醒指令是否匹配，若第一语音信号和唤醒指令匹配，则说明成功唤醒自移动机器人并执行步骤1904，若第一语音信号和唤醒指令不匹配，则说明未能唤醒自移动机器人，自移动机器人执行步骤1911。
本步骤中，自移动机器人利用自身的语音识别功能确定第一语音信号和唤醒指令匹配是否匹配，或者，自移动机器人将第一语音信号发送至语音识别服务器，由语音识别服务器确定第一语音信号和唤醒指令匹配是否匹配。
1904、自移动机器人从第一工作状态切换至第二工作状态。
示例性的，相较于第一工作状态，第二工作状态下自移动机器人的功耗比较小、噪声比较小，例如，第一工作状态下，驱动轮正常滚动，其余发音部件正常运行；而第二工作状态下，驱动轮停止滚动，其余发音部件均降低运行功率。再如，第二工作状态下，驱动轮降速滚动，其余发音部件降级运行功率。
1905、自移动机器人确定预设时长内是否采集到第二语音信号，若自移 动机器人在预设时长内采集到第二语音信号则执行步骤1906；若自移动机器人在预设时长内未采集到第二语音信号，则执行步骤1912。
1906、自移动机器人判断是否成功解析第二语音信号。
示例性的，自移动机器人自身或语音是否服务器解析第二语音信号，若成功解析第二语音信号则执行步骤1907；若未能成功解析第二语音信号，则执行步骤1913。
1907、自移动机器人确定解析结果是否与控制指令匹配，若解析结果和控制指令匹配，则执行步骤1908；若解析结果和控制指令不匹配，则执行步骤1914。
示例性的，自移动机器人或语音识别服务器确定解析结果是否为机自身清洁、清扫、割草等任务。
1908、自移动机器人确定自身状态是否满足执行任务的要求，若自身状态满足执行任务的要求，则执行步骤1909；若自身状态不满足执行任务的要求，则执行步骤1915。
示例性的，自移动机器人确定自身的电量、尘盒剩余空间、水盒中剩余水量等是否满足任务要求。
1909、执行任务并语音反馈用户，之后执行步骤1910。
例如，自移动机器人开始行进并对用户说：“好的，即将清洁小明房间和书房”。
1910、结束本轮语音交互，语音控制功能进入等待唤醒状态。
1911、自移动机器人继续以第一工作状态工作，经过预设时长后，执行步骤1910。
1912、自移动机器人反馈指令超时，并恢复第一工作状态，之后，执行步骤1910。
示例性的，自移动机器人向用户发出：“语音交互超时，请重新唤醒”的语音反馈。同时，自移动机器人重新进入第一工作状态。之后，执行步骤1910。
1913、自移动机器人反馈未理解用户意图并恢复第一工作状态，之后，执行步骤1910。
示例性的，自移动机器人向用户发出：“未接收到正确指令，请重新唤醒”；或者“我没听清楚您说的是什么，请重新唤醒”语音反馈。同时，自 移动机器人重新进入第一工作状态。之后，执行步骤1910。
1914、自移动机器人继续在第二工作状态下工作，和用户进行语音交互、对答等。经过预设时长后，执行步骤1910。
示例性的，自移动机器人向用户发出：“这个任务太难，我无法执行，请换个描述方式”、“请问您是要清扫卧室的床底下么”，引导用户与其交互以弄清楚用户的意图。
1915、自移动机器人语音反馈无法执行任务，并继续以第二工作状态工作。
示例性的，自移动机器人对用户说：“我需要充电才能执行任务”、“我需要返回基站补水”、“请清理尘盒”等，并继续以第二工作状态工作。之后，经过预设时长后，执行步骤1910。
或者，自移动机器人对用户：“我要去充电，等我充电完成之后再去清扫卧室”，并自行移动至基站进行充电以维护自身状况。之后，自移动机器人再执行清扫等任务。
采用该种方案，自移动机器人在执行任务之前先判断自身状态，根据自身状态确定是否立即执行任务还是充电、补水之后再执行任务，能够避免执行任务过程中发生中断等。
上述实施例中，语音控制功能被唤醒后，经过预设时长后，语音控制功能重新进入等待唤醒状态。例如，语音控制功能被唤醒后，经过预设时长后未采集到第二语音信号，则自动进入等待唤醒状态。再如，执行完一次唤醒和指令下发循环后，自动进入等待唤醒状态。
采用该种方案，能够避免周围语音对自移动机器人的误触发。
可选的，上述实施例中，第二语音信号还可以指示任务禁区。自移动机器人从环境地图中确定出任务禁区，并从任务禁区以外的区域中确定出至少两个工作区域。
如果相对于工作区域，任务禁区比较少或更容易描述时，用户可以在第二语音信号中指出任务禁区。自移动机器人根据第二语音信号确定出任务禁区，进而将其他区域作为工作区域并执行任务。例如，用户说：“不清扫床底”，则自移动机器人对床底以外的区域进行清扫。
采用该种方案，降低用户下发语音指令的难度。
上述实施例中，语音信号主要用于工作区域的控制。然而，本申请实施 例并不限制。其他可行的实现方式中，还可以通过语音控制实现自移动机器人操控家中其他的电子设备、通过语音控制实现自移动机器人定点巡逻、监控和看护等功能、通过语音控制自移动机器人自动寻物等。下面，对该些场景分别进行说明。
首先，通过语音控制实现自移动机器人操控家中其他的电子设备的场景。示例性的，请参照图20。图20是本申请实施例提供的自移动机器人语音控制方法的另一个流程图。本实施例包括：
1001、自移动机器人处于第一工作状态，语音控制功能处于等待唤醒状态。
1002、用户通过第一语音信号唤醒语音控制功能，并发出第二语音信号。
示例性的，自移动机器人唤醒语音控制功能后还自动切换为第二工作状态，具体可参见上述描述，此处不再赘述。
1003、自移动机器人根据第二语音信号得到控制指令，该控制指令用于指示自移动机器人控制指定设备。
示例性的，自移动机器人自己解析第二语音信号，或者，由语音识别服务器解析以语音信号得到控制指令。该控制指令用于指示自动移动设备对指定区域内的指定设备进行控制。指定设备例如为空调、冰箱、窗帘等家电或家电。
1004、自移动机器人移动到指定区域完成对指定设备的控制。
例如，自移动机器人开机状态下，用户位于自移动机器人语音信号采集范围内，比如，用户和自移动机器人同时处于客体、自移动机器人距离用户5米等。用户发出的第二语音信号为“打开主卧空调，制冷25℃”。自移动机器人根据环境地图规划行驶路径进入主卧后，利用自身的硬件遥控模块打开主卧的空调，并将模式设置为制冷模式、将温度设置为25℃。
采用该种方案，通过语音控制结合自移动机器人的其他硬件、算法，实现增值功能，使得自移动机器人更加智能。
其次，通过语音控制实现自移动机器人定点巡逻的场景。示例性的，请参照图21。图21是本申请实施例提供的自移动机器人语音控制方法的又一个流程图。本实施例包括：
1101、自移动机器人处于第一工作状态，语音控制功能处于等待唤醒状态。
1102、用户通过第一语音信号唤醒语音控制功能，并发出第二语音信号。
示例性的，自移动机器人唤醒语音控制功能后还自动切换为第二工作状态，具体可参见上述描述，此处不再赘述。
1103、自移动机器人根据第二语音信号得到控制指令，该控制指令用于指示自移动机器人定点巡逻。
示例性的，自移动机器人自己解析第二语音信号，或者，由语音识别服务器解析以语音信号得到控制指令。该控制指令用于指示自动移动设备定点巡逻、监控或看护等。
1104、自移动机器人执行定点巡逻、监控或看护。
例如，用户发出的第二语音信号为“去老爸房间巡查”。自移动机器人根据环境地图规划行驶路径进入老爸房间后，行进至之前设定好的监护点，打开摄像头拍摄视频，并将视频回传至用户手机上的客户端，实现跨房间监护老人的功能。
采用该种方案，通过语音控制实现自移动机器人按照用户的意图进行家中定点巡逻、对指定区域进行监控、对特定房间进行看护等功能。
最后，通过语音控制自移动机器人自动寻物的场景。示例性的，请参照图22。图22是本申请实施例提供的自移动机器人语音控制方法的又一个流程图。本实施例包括：
1201、自移动机器人处于第一工作状态，语音控制功能处于等待唤醒状态。
1202、用户通过第一语音信号唤醒语音控制功能，并发出第二语音信号。
1203、自移动机器人根据第二语音信号得到控制指令，该控制指令用于指示自移动机器人寻找目标对象。
1204、自移动机器人确定环境地图中是否标记了目标对象的位置坐标，若环境地图中已标记目标对象，则执行步骤1205；若环境地铁中没有标记目标对象，则执行步骤1208。
示例性的，自移动机器人工作过程中将采集到的图像输入AI训练模型识从而得到物体的位置坐标、名称、种类等，并在环境地图中记录该些信息，用于后续的智能寻物。为了防止客户端界面凌乱，环境地图上可以不显示该些物体的位置坐标等。当用户发出指令寻找目标对象时，客户端显示目标对象在环境地图中的位置等。
1205、自移动机器人通过语音询问用户是否需要现在去寻找目标对象，若用户反馈现在寻找目标对象，则执行步骤1206；若用户反馈无需现在寻找目标对象，则执行步骤1207。
1206、引导用户到目标对象位置并在环境地图上显示目标对象的位置。
1207、在环境地图上显示目标对象的位置，并语音提示用户：已在客户端显示目标对象的位置。
搜索运行过程中发现的目标对象在环境地图中的位置。
1208、语音提示用户：未寻找到目标对象，请换一种描述方式。
例如，第一语音信号为：“请帮我找袜子”。自移动机器人识别第一语音信号后，确定本地是否已标记了袜子的位置坐标。若未标记袜子，则提示用户找不到袜子。若标记了袜子的位置坐标，则发出询问语音：“是否现在去寻找袜子”。如果接下来用户的答复是“可以”、“好的”等肯定答复，则自移动机器人行进以引导用户到达袜子所在位置，并在客户端环境地图的界面上显示袜子的位置坐标。如果用户的答复是“不需要”等否定答复，则自移动机器人只需指示客户端显示袜子的位置坐标即可。
采用该种方案，通过机器学习训练AI训练模型，利用AI训练模型识别特定物体，使得用户能够通过语音控制自移动机器人搜索一些特定物体，并在地图上标识出来或者带用户到达物体现场，实现智能寻物的目的的同时，扩展自移动机器人的用途，提高自移动机器人的智能性。
图23为本申请实施例提供的一种自移动机器人的结构示意图。如图23所示，该自移动机器人1200包括：
处理器1201和存储器1202；
所述存储器1202存储计算机指令；
所述处理器1201执行所述存储器1202存储的计算机指令，使得所述处理器1201执行如上自移动机器人实施的方法。
处理器1201的具体实现过程可参见上述方法实施例，其实现原理和技术效果类似，本实施例此处不再赘述。
可选地，该自移动机器人1200还包括通信部件1203。其中，处理器1201、存储器1202以及通信部件1203可以通过总线1204连接。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机指令，所述计算机指令被处理器执行时用于实现如上 自移动机器人实施的方法。
本申请实施例还提供一种计算机程序产品，该计算机程序产品包含计算机程序，计算机程序被处理器执行时实现如上自移动机器人实施的方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求书指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求书来限制。

Claims (29)

  1. 一种自移动机器人控制方法，其特征在于，包括：
    根据用户发出的语音信号确定声源方向；
    确定自移动机器人周围的移动对象；
    从所述移动对象中确定出位于所述声源方向的目标对象；
    根据所述目标对象确定工作区域；
    移动至所述工作区域并在所述工作区域内执行任务。
  2. 根据权利要求1所述的方法，其特征在于，所述确定自移动机器人周围的移动对象，包括：
    获取多幅即时定位与地图构建SLAM图和多幅直接飞行时间DTOF散点图，所述多幅SLAM图中的SLAM图和所述多幅DTOF散点图中的DTOF散点图一一对应；
    对于所述多幅DTOF散点图中的每一幅DTOF散点图，根据对应的SLAM图，从所述DTOF散点图中过滤掉表征静态对象的像素点以得到动态点集；
    根据相邻的多幅DTOF散点图的动态点集确定所述自移动机器人周围的移动对象。
  3. 根据权利要求2所述的方法，其特征在于，所述根据相邻的多幅DTOF散点图的动态点集确定所述自移动机器人周围的移动对象，包括：
    从第一DTOF散点图的第一动态点集中确定出第一子集；
    确定第二DTOF散点图的第二动态点集中是否存在第二子集，所述第一子集指示的第一位置与所述第二子集指示的第二位置之间的距离大于预设距离，且所述第一子集和所述第二子集中像素点数量的差值小于预设差值，所述第一DTOF散点图和所述第二DTOF散点图是所述多幅DTOF散点图中任意相邻的两幅DTOF散点图；
    若所述第二动态点集中存在所述第二子集，则确定所述第一子集和所述第二子集表征同一个对象且所述对象为移动对象。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述从所述移动对象中确定出位于所述声源方向的目标对象，包括：
    从所述移动对象中确定出脚部发生动作、且位于所述声源方向上的移 动对象，以得到所述目标对象。
  5. 根据权利要求1-3任一项所述的方法，其特征在于，所述根据所述目标对象确定工作区域，包括：
    移动至距所述目标对象预设距离的位置，若所述目标对象未发生位移，则根据所述目标对象的初始位置确定所述工作区域。
  6. 根据权利要求1-3任一项所述的方法，其特征在于，所述根据所述目标对象确定工作区域，包括：
    移动至距所述目标对象预设距离的位置后，若所述目标对象发生位移，则控制所述自移动机器人跟随所述目标对象移动；
    当所述目标对象停止移动时，根据所述目标对象停止移动时的位置确定所述工作区域。
  7. 根据权利要求6所述的方法，其特征在于，所述控制所述自移动机器人跟随所述目标对象移动，包括：
    确定相邻两幅DTOF散点图中是否均出现所述目标对象；
    若相邻两幅DTOF散点图中均出现所述目标对象，则确定相邻两幅DTOF散点图中目标对象的距离；
    根据所述距离调整速度以跟随所述目标对象移动。
  8. 根据权利要求6所述的方法，其特征在于，所述控制所述自移动机器人跟随所述目标对象移动，包括：
    利用人工智能AI相机捕捉目标对象的骨架图；
    当所述骨架图完整时，保持跟随状态以跟随所述目标对象移动；
    当所述骨架图不完整时，对所述自移动设备和所述目标对象之间的障碍物避障或越障直至所述AI相机捕捉到完整的骨架图像后，保持跟随状态以跟随所述目标对象移动。
  9. 根据权利要求1-3任一项所述的方法，其特征在于，所述根据用户发出的语音信号确定声源方向之前，还包括：
    唤醒所述自移动机器人；
    确定所述语音信号对应的控制指令用于控制所述自移动机器人即时根据所述目标对象确定工作区域。
  10. 根据权利要求1-3任一项所述的方法，其特征在于，还包括：
    确定是否跟丢所述目标对象；
    若跟丢所述目标对象，则根据所述目标对象最后一次出现的位置坐标确定寻找范围；
    在所述寻找范围内寻找所述目标对象；
    若未寻找到所述目标对象，则进入等待召唤状态。
  11. 根据权利要求1所述的方法，其特征在于，还包括：
    采集第一语音信号；
    当所述第一语音信号与所述自移动机器人的唤醒指令匹配时，唤醒所述自移动机器人的语音控制功能；
    在所述语音控制功能唤醒状态下采集第二语音信号；
    根据所述第二语音信号确定至少两个工作区域；
    依次对所述至少两个工作区域执行所述第二语音信号指示的任务。
  12. 根据权利要求11所述的方法，其特征在于，所述依次对所述至少两个工作区域执行所述第二语音信号指示的任务，包括：
    确定所述第二语音信号中所述至少两个工作区域中各工作区域出现的先后顺序；
    根据所述先后顺序依次对所述至少两个工作区域执行所述第二语音信号指示的任务。
  13. 根据权利要求11所述的方法，其特征在于，所述依次对所述至少两个工作区域执行所述第二语音信号指示的任务，包括：
    确定所述自移动机器人与所述至少两个工作区域中各工作区域之间的距离；
    按照距离由近至远的顺序对所述至少两个工作区域排序得到队列；
    根据所述队列依次对所述至少两个工作区域执行所述第二语音信号指示的任务。
  14. 根据权利要求11-13任一项所述的方法，其特征在于，所述依次对所述至少两个工作区域执行所述第二语音信号指示的任务，包括：
    对所述至少两个工作区域中的一个工作区域执行完任务后，行进至下一个工作区域之前，停止执行任务。
  15. 根据权利要求11-13任一项所述的方法，其特征在于，所述根据所述第二语音信号确定至少两个工作区域，包括：
    根据所述第二语音信号确定区域类别；
    根据所述区域类别从环境地图对应的区域集合中确定出所述至少两个工作区域。
  16. 根据权利要求15所述的方法，其特征在于，所述根据所述区域类别从环境地图中确定所述至少两个工作区域，包括：
    当所述区域类别指示目标物体时，以所述目标物体为中心，从所述环境地图中确定出包含所述目标物体的区域以得到所述至少两个工作区域。
  17. 根据权利要求15所述的方法，其特征在于，所述根据所述第二语音信号确定至少两个工作区域之前，还包括：
    根据所述环境地图、所述环境地图中物体的位置信息或所述环境地图中门的位置信息，将所述环境地图划分为多个工作区域以得到所述区域集合；
    更新所述区域集合中各工作区域的标识；
    向语音识别服务器发送更新信息，以使得所述语音识别服务器更新各工作区域的标识。
  18. 根据权利要求11-13任一项所述的方法，其特征在于，所述在所述语音控制功能唤醒状态下采集第二语音信号，包括：
    当所述第一语音信号与所述自移动机器人的唤醒指令匹配时，控制所述自移动机器人从第一工作状态切换为第二工作状态，所述自移动机器人在所述第二工作状态下产生的声音的音量小于在所述第一工作状态下产生的声音的音量，所述唤醒指令用于唤醒所述自移动机器人的语音控制功能；
    在所述第二工作状态下采集所述第二语音信号。
  19. 根据权利要求18所述的方法，其特征在于，还包括：
    当所述第一语音信号与所述自移动机器人的唤醒指令匹配时，确定所述第一语音信号的声源位置；
    根据所述声源位置控制所述自移动机器人从第一位姿切换为第二位姿，所述自移动机器人处于所述第二位姿时麦克风与所述声源位置的距离，小于所述自移动机器人处于所述第一位姿时麦克风与所述声源位置的距离，所述麦克风是设置在所述自移动机器人上的麦克风。
  20. 根据权利要求19所述的方法，其特征在于，所述根据所述声源位置控制所述自移动机器人从第一位姿切换为第二位姿，包括：
    根据所述声源位置确定旋转角度，所述旋转角度用于指示所述自移动 机器人从所述第一位姿切换为第二位姿时需要旋转的角度；
    将所述旋转角度划分为第一角度和第二角度；
    在所述第一角度内以第一速度旋转，在所述第二角度内以第二速度旋转，所述第一速度大于所述第二速度。
  21. 根据权利要求19所述的方法，其特征在于，所述当所述第一语音信号与自移动机器人的唤醒指令匹配时，根据所述声源位置控制所述自移动机器人从第一工作状态切换为第二工作状态之后，还包括：
    经过预设时长后控制所述语音控制功能进入等待唤醒状态。
  22. 根据权利要求11-13任一项所述的方法，其特征在于，所述依次对所述至少两个工作区域执行所述第二语音信号指示的任务之前，还包括：
    确定所述自移动机器人的自身状态是否满足执行任务的要求；
    若所述自身状态不满足执行任务的要求，则维护所述自移动机器人。
  23. 根据权利要求11-13任一项所述的方法，其特征在于，所述根据所述第二语音信号确定至少两个工作区域，包括：
    当所述第二语音信号指示任务禁区时，从环境地图中确定出所述任务禁区，从所述任务禁区以外的区域中确定出所述至少两个工作区域。
  24. 根据权利要求11-13任一项所述的方法，其特征在于，所述根据所述第二语音信号确定至少两个工作区域，包括：
    根据所述第二语音信号确定区域类别；
    采集图像；
    从所述图像中确定出所述区域类别对应的区域以得到所述至少两个工作区域。
  25. 根据权利要求24所述的方法，其特征在于，所述依次对所述至少两个工作区域执行所述第二语音信号指示的任务，包括：
    根据所述区域类别确定作业方式；
    根据所述作业方式依次对所述至少两个工作区域执行任务。
  26. 根据权利要求11-13任一项所述的方法，其特征在于，所述依次对所述至少两个工作区域执行所述第二语音信号指示的任务之后，还包括：
    确定是否对初始区域执行完任务，所述初始区域是所述自移动机器人采集所述第二语音信号时所处的区域；
    若未对所述初始区域执行完任务，则返回所述初始区域执行任务。
  27. 一种自移动机器人控制装置，其特征在于，包括：
    第一确定模块，用于根据用户发出的语音信号确定声源方向；
    第二确定模块，用于确定自移动机器人周围的移动对象；
    第三确定模块，用于从所述移动对象中确定出位于所述声源方向的目标对象；
    处理模块，用于根据所述目标对象确定工作区域；
    执行模块，用于移动至所述工作区域并在所述工作区域内执行任务。
  28. 一种自移动机器人，包括处理器、存储器及存储在所述存储器上并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时使得所述自移动机器人实现如权利要求1至26任一所述的方法。
  29. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至26任一所述的方法。
PCT/CN2022/109522 2021-08-17 2022-08-01 自移动机器人控制方法、装置、设备及可读存储介质 Ceased WO2023020269A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22857581.7A EP4390599A4 (en) 2021-08-17 2022-08-01 Self-moving robot control method and apparatus, device, and readable storage medium
US18/681,526 US20250127362A1 (en) 2021-08-17 2022-08-01 Autonomous mobile robot control method and apparatus, device and readable storage medium

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110940990.7A CN113793605B (zh) 2021-08-17 2021-08-17 自主移动设备语音控制方法、装置、设备及可读存储介质
CN202110940990.7 2021-08-17
CN202110959732.3 2021-08-20
CN202110959732.3A CN113787517B (zh) 2021-08-20 2021-08-20 自移动机器人控制方法、装置、设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2023020269A1 true WO2023020269A1 (zh) 2023-02-23

Family

ID=85240012

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/109522 Ceased WO2023020269A1 (zh) 2021-08-17 2022-08-01 自移动机器人控制方法、装置、设备及可读存储介质

Country Status (3)

Country Link
US (1) US20250127362A1 (zh)
EP (1) EP4390599A4 (zh)
WO (1) WO2023020269A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116339211A (zh) * 2023-04-11 2023-06-27 重庆海辉星科技有限公司 运用于酒店卫生数据采集的前端装置、方法及组件
WO2024244658A1 (zh) * 2023-05-31 2024-12-05 深圳市普渡科技有限公司 机器人、机器人跟随方法、装置和存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118102957A (zh) * 2021-10-29 2024-05-28 三星电子株式会社 机器人真空吸尘器监控陪伴动物及其控制方法
WO2024011518A1 (en) * 2022-07-14 2024-01-18 Abb Schweiz Ag Method for controlling industrial robot and industrial robot
TWI825896B (zh) * 2022-08-03 2023-12-11 優式機器人股份有限公司 環境整理控制方法
US20250189970A1 (en) * 2023-12-07 2025-06-12 Robert Bosch Gmbh Mobile Robot with Audio Perception System

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017030188A1 (ja) * 2015-08-19 2017-02-23 Cyberdyne株式会社 自律移動体および施設内管理運営システム
CN111142512A (zh) * 2019-11-22 2020-05-12 珠海格力电器股份有限公司 一种设备控制方法、跟随设备及存储介质
CN111640428A (zh) * 2020-05-29 2020-09-08 北京百度网讯科技有限公司 一种语音识别方法、装置、设备和介质
CN113787517A (zh) * 2021-08-20 2021-12-14 科沃斯机器人股份有限公司 自移动机器人控制方法、装置、设备及可读存储介质
CN113793605A (zh) * 2021-08-17 2021-12-14 科沃斯机器人股份有限公司 自主移动设备语音控制方法、装置、设备及可读存储介质
US20210396886A1 (en) * 2020-06-18 2021-12-23 Facebook Technologies, Llc Time of flight depth system including an illumination source with addressable illumination blocks

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130096539A (ko) * 2012-02-22 2013-08-30 한국전자통신연구원 자율 이동장치 및 그 제어 방법
KR102586010B1 (ko) * 2014-02-28 2023-10-11 삼성전자주식회사 청소 로봇 및 그에 포함되는 원격 제어기
US12099357B1 (en) * 2014-08-24 2024-09-24 AI Incorporated Method for robotic devices to authenticate users
DE102016102644A1 (de) * 2016-02-15 2017-08-17 RobArt GmbH Verfahren zur Steuerung eines autonomen mobilen Roboters
US12235659B2 (en) * 2016-02-29 2025-02-25 AI Incorporated Obstacle recognition method for autonomous robots
US11449061B2 (en) * 2016-02-29 2022-09-20 AI Incorporated Obstacle recognition method for autonomous robots
US11927965B2 (en) * 2016-02-29 2024-03-12 AI Incorporated Obstacle recognition method for autonomous robots
WO2018111920A1 (en) * 2016-12-12 2018-06-21 The Charles Stark Draper Laboratory, Inc. System and method for semantic simultaneous localization and mapping of static and dynamic objects
US11348269B1 (en) * 2017-07-27 2022-05-31 AI Incorporated Method and apparatus for combining data to construct a floor plan
JP2019171001A (ja) * 2017-09-29 2019-10-10 パナソニックＩｐマネジメント株式会社 自律移動掃除機、掃除方法、及びプログラム
DE102019103349B3 (de) * 2019-02-11 2020-06-18 Beckhoff Automation Gmbh Industrierobotersystem und Verfahren zur Steuerung eines Industrieroboters
US11266287B2 (en) * 2019-05-29 2022-03-08 Irobot Corporation Control of autonomous mobile robots
KR102790142B1 (ko) * 2019-06-18 2025-04-01 엘지전자 주식회사 커뮤니케이션 로봇
KR20190104263A (ko) * 2019-07-11 2019-09-09 엘지전자 주식회사 인공지능 기반의 음성 인식 서비스 제공 장치 및 방법
WO2021125510A1 (en) * 2019-12-20 2021-06-24 Samsung Electronics Co., Ltd. Method and device for navigating in dynamic environment
CN111714034B (zh) * 2020-05-18 2022-10-21 科沃斯机器人股份有限公司 一种自移动机器人的控制方法、系统及自移动机器人
US11768504B2 (en) * 2020-06-10 2023-09-26 AI Incorporated Light weight and real time slam for robots
EP3979029B1 (en) * 2020-09-30 2025-05-07 Carnegie Robotics, LLC Systems and methods for enabling navigation in environments with dynamic objects

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017030188A1 (ja) * 2015-08-19 2017-02-23 Cyberdyne株式会社 自律移動体および施設内管理運営システム
CN111142512A (zh) * 2019-11-22 2020-05-12 珠海格力电器股份有限公司 一种设备控制方法、跟随设备及存储介质
CN111640428A (zh) * 2020-05-29 2020-09-08 北京百度网讯科技有限公司 一种语音识别方法、装置、设备和介质
US20210396886A1 (en) * 2020-06-18 2021-12-23 Facebook Technologies, Llc Time of flight depth system including an illumination source with addressable illumination blocks
CN113793605A (zh) * 2021-08-17 2021-12-14 科沃斯机器人股份有限公司 自主移动设备语音控制方法、装置、设备及可读存储介质
CN113787517A (zh) * 2021-08-20 2021-12-14 科沃斯机器人股份有限公司 自移动机器人控制方法、装置、设备及可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4390599A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116339211A (zh) * 2023-04-11 2023-06-27 重庆海辉星科技有限公司 运用于酒店卫生数据采集的前端装置、方法及组件
CN116339211B (zh) * 2023-04-11 2023-08-15 重庆海辉星科技有限公司 运用于酒店卫生数据采集的前端装置、方法及组件
WO2024244658A1 (zh) * 2023-05-31 2024-12-05 深圳市普渡科技有限公司 机器人、机器人跟随方法、装置和存储介质

Also Published As

Publication number Publication date
US20250127362A1 (en) 2025-04-24
EP4390599A1 (en) 2024-06-26
EP4390599A4 (en) 2024-11-27

Similar Documents

Publication Publication Date Title
WO2023020269A1 (zh) 自移动机器人控制方法、装置、设备及可读存储介质
CN113787517B (zh) 自移动机器人控制方法、装置、设备及可读存储介质
CN113793605B (zh) 自主移动设备语音控制方法、装置、设备及可读存储介质
US11737635B2 (en) Moving robot and control method thereof
JP7395229B2 (ja) 状況認識のためのモバイル清掃ロボット人工知能
US11226633B2 (en) Mobile robot and method of controlling the same
CN112739244A (zh) 移动机器人清洁系统
CN109093633A (zh) 一种可分体式机器人及其控制方法
WO2019006974A1 (zh) 清洁设备的清洁操作执行方法、装置及可读存储介质
CN112805128B (zh) 人工智能移动机器人
CN208930273U (zh) 一种可分体式机器人
KR102290983B1 (ko) 인공지능 이동 로봇의 제어 방법
US11986959B2 (en) Information processing device, action decision method and program
Kallakuri et al. Probabilistic approach for building auditory maps with a mobile microphone array
KR102612822B1 (ko) 인공지능 이동 로봇의 제어 방법
KR102423572B1 (ko) 인공지능 이동 로봇의 제어 방법
US20250189970A1 (en) Mobile Robot with Audio Perception System
US20230138237A1 (en) Robotic vacuum cleaner for monitoring pet and method of controlling the same
US11925305B2 (en) Autonomous travel-type cleaner and control method
CN118948164A (zh) 基于脚势的清洁控制方法、装置、清洁机器人、介质及产品
KR20230134900A (ko) Uwb 및 라이다 포인트클라우드를 이용한 실내 자율 주행 로봇
CN121647570A (zh) 视障人士友好洗地机的控制方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22857581

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18681526

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022857581

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022857581

Country of ref document: EP

Effective date: 20240318

WWP Wipo information: published in national office

Ref document number: 18681526

Country of ref document: US