WO2022017485A1 - 机械臂控制方法和皮肤表面处理设备 - Google Patents

机械臂控制方法和皮肤表面处理设备 Download PDF

Info

Publication number
WO2022017485A1
WO2022017485A1 PCT/CN2021/108045 CN2021108045W WO2022017485A1 WO 2022017485 A1 WO2022017485 A1 WO 2022017485A1 CN 2021108045 W CN2021108045 W CN 2021108045W WO 2022017485 A1 WO2022017485 A1 WO 2022017485A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
robotic arm
interest
dimensional
execution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/108045
Other languages
English (en)
French (fr)
Inventor
连俊文
李志杰
多明戈·戈麦斯·多明戈斯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US17/628,564 priority Critical patent/US12090663B2/en
Priority to KR1020237000481A priority patent/KR20230051477A/ko
Priority to EP21844943.7A priority patent/EP4186457B1/en
Publication of WO2022017485A1 publication Critical patent/WO2022017485A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/32—Surgical robots operating autonomously
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00—Manipulators not otherwise provided for
    • B25J11/008—Manipulators for service tasks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B25J13/087—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices for sensing other physical parameters, e.g. electrical or chemical properties
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1602—Program controls characterised by the control system, structure, architecture
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00017—Electrical control of surgical instruments
    • A61B2017/00022—Sensing or detecting at the treatment site
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00017—Electrical control of surgical instruments
    • A61B2017/00022—Sensing or detecting at the treatment site
    • A61B2017/00084—Temperature
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00681—Aspects not otherwise provided for
    • A61B2017/00725—Calibration or performance testing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00743—Type of operation; Specification of treatment sites
    • A61B2017/00747—Dermatology
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/107—Visualisation of planned trajectories or target regions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2055—Optical tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2055—Optical tracking systems
    • A61B2034/2057—Details of tracking cameras
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2072—Reference field transducer attached to an instrument or patient
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B2090/364—Correlation of different images or relation of image positions in respect to the body
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
    • A61B2090/3983—Reference marker arrangements for use with image guided surgery

Definitions

  • the invention relates to the technical field of mechanical control, in particular to a mechanical arm control method and a skin surface treatment device which can be applied to skin surface treatment.
  • Skin cosmetology is a cosmetic method that does not involve the human body, including a modification method that is locally implemented on the skin, hair, etc., which can be seen by people, and is not for the purpose of treatment. Skin beauty can improve the status quo of unsatisfactory skin, improve skin health, and delay skin aging.
  • symptoms including, but not limited to, pigmentation, acne, pores, hair loss, wrinkles, and darkening can be improved by cosmetic skin treatments. It can restore the normal metabolic system of the skin.
  • Skin beauty comprehensively improves the skin from the four aspects of complexion, skin texture, skin age and skin health, and penetrates into different levels of the skin from the four dimensions of symptoms, metabolism, root causes, and long-term management to completely solve skin problems.
  • these beauty equipment and tools can be held by a robotic arm, that is, a robotic arm, and combined with various additional sensors to provide stable skin beauty services for non-treatment purposes.
  • a robotic arm that is, a robotic arm
  • various additional sensors to provide stable skin beauty services for non-treatment purposes.
  • it is to use the motion characteristics of the robotic arm to make it move along the geometry of the human skin surface. Therefore, the motion trajectory of the robotic arm needs to be designed, and it is necessary to ensure that the robotic arm will not touch sensitive areas on the human body surface or prohibit processing. Special areas (such as around the eyes or wound areas) for safety.
  • the data volume of the 3D point cloud will be very large. Specifically, depending on the resolution of the camera and data collection, only a scan of a human face may create hundreds of thousands of data points. Since the motion trajectory of the robotic arm is a point-to-point motion, it is possible to perform operations based on such a number of data points. Motion is also impractical, so 3D point clouds need to be optimized.
  • embodiments of the present application provide a method for controlling a robotic arm.
  • the robotic arm control method includes the following steps: dividing the surface to be processed of the processing object of the robotic arm into at least two areas of interest; obtaining a three-dimensional point cloud corresponding to each area of interest through a three-dimensional scanning device; optimizing each three-dimensional point cloud ; Combine the three-dimensional point clouds to obtain the execution point cloud; generate the movement path of the robot arm based on the execution point cloud; the robot arm processes the surface of the object according to the movement path.
  • Embodiments of the present application further disclose a skin surface treatment device, comprising: a three-dimensional scanning device, a robotic arm, and a controller; wherein the controller is used to divide the surface to be treated of a processing object of the robotic arm into at least two regions of interest; three-dimensional The scanning device is used to obtain the 3D point cloud corresponding to each area of interest; the controller is used to optimize each 3D point cloud, combine the 3D point clouds to obtain the execution point cloud, and generate the movement path of the robotic arm based on the execution point cloud; The arm is used to process the surface of the object according to the movement path.
  • Embodiments of the present application further disclose a computer-readable storage medium storing a computer program, wherein the computer program can implement the steps in the above method when executed.
  • Embodiments of the present application further disclose a skin surface treatment device, including a memory, a processor and a communication component, wherein the memory is used to store a program; the processor is coupled to the memory and used to execute the program stored in the memory to use To: divide the surface to be processed of the processing object of the robotic arm into at least two areas of interest; obtain the three-dimensional point cloud corresponding to each area of interest through the three-dimensional scanning device; optimize each three-dimensional point cloud; Obtain the execution point cloud; generate the movement path of the robot arm based on the execution point cloud; the robot arm processes the surface of the object according to the movement path.
  • the embodiment of the present application has lower requirements on the performance of the controller, which can reduce the manufacturing cost of the skin surface treatment device and improve its treatment efficiency.
  • FIG. 1 is a schematic diagram when two depth cameras are arranged on the same structural element according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of a depth camera set on a robotic arm according to an embodiment of the present application
  • FIG. 3 is a schematic flowchart of a method for controlling a robotic arm according to an embodiment of the present application
  • FIG. 4 is a schematic diagram of a three-dimensional point cloud of an embodiment of the present application before optimization and filtering;
  • FIG. 5 is a schematic diagram of a three-dimensional point cloud of an embodiment of the present application after being optimized and filtered;
  • FIG. 6 is a schematic flow chart of a method for controlling a robotic arm in a process of processing a surface to be processed according to an embodiment of the present application
  • FIG. 7 is a system block diagram of a skin surface treatment device according to an embodiment of the present application.
  • the first embodiment of the present application provides a skin surface treatment device, as shown in FIG. 1 or FIG. 2 , comprising: a three-dimensional scanning device 1, a robotic arm 2 and a controller 3; wherein, the controller 3 will The surface to be processed of the object to be processed by the robotic arm 2 is divided into at least two regions of interest; the three-dimensional scanning device 1 is a device for acquiring a three-dimensional point cloud of the object's three-dimensional surface.
  • the three-dimensional scanning device 1 is used to obtain a three-dimensional point cloud corresponding to each region of interest; the controller 3 is used to optimize each three-dimensional point cloud, combine the three-dimensional point clouds to obtain an execution point cloud, and
  • the movement path of the robot arm 2 is generated based on the execution point cloud; the robot arm 2 performs corresponding processing on the surface 4 of the object according to the movement path.
  • the controller 3 is a controller in a broad sense, which may be partially discretely provided outside the robotic arm 2 , or may be completely integrated within the robotic arm 2 .
  • it may include a lower control module 32 installed on the robotic arm 2 , and an upper computer 31 (such as a computer workstation, or other movable, non-movable terminal), when the controller 3 includes the lower control module 32 and the upper computer 31 at the same time, the upper computer 31 can be responsible for analyzing and processing the data acquired and collected by the three-dimensional scanning device 1 to obtain the execution action data of the robotic arm . Then, the upper computer 31 outputs the execution action data to the lower control module 32 , so that the operation logic of the robot arm 2 is realized by the lower control module 32 .
  • the lower control module 32 can often be a controller provided with the robotic arm 2 , which not only provides a control interface for the robotic arm 2 , but also is usually responsible for supplying power to the robotic arm 2 .
  • the three-dimensional scanning device 1 may comprise two depth cameras 11 statically mounted on the same structural element 12 .
  • two depth cameras 11 can be used and mounted on the structural element 12 .
  • the structural element 12 may be a support frame or some common kinds of clamps. The position of the structural element 12 can be marked by a ruler and can be adjusted to suit different sized treatment subjects. After the depth camera 11 is fixed, the coordinates of the depth camera 11 can be preset in the controller 3 to maintain calibration.
  • each depth camera 11 In the process of scanning the surface of the object, each depth camera 11 generates a three-dimensional point cloud.
  • one side surface of the object to be processed can be depicted by any one of the depth cameras 11 .
  • the three-dimensional point clouds of the left face and the right face can be drawn at the same time by using the depth cameras 11 arranged on the left and right sides of the human face.
  • the processing object is a human face
  • the controller 3 can optimize and merge these three-dimensional point clouds, so as to obtain the execution point cloud and then generate the movement path of the robot arm 2, and then drive the robot arm 2 to move along this movement path.
  • these depth cameras 11 are respectively corresponding to each region of interest, which can significantly improve the scanning efficiency.
  • the three-dimensional scanning device 1 may also include one or more depth cameras 11 mounted on the robotic arm 2 .
  • a depth camera 11 may be used and mounted on the robotic arm 2 .
  • the exact position of the robotic arm 2 itself can be known at any time according to the motion trajectory of the robotic arm 2 , and thus the exact position of the depth camera 11 can also be known. Therefore, by controlling the robotic arm 2 by means of the controller 3 , in the process of moving through the preset trajectory in front of the object to be processed, multiple three-dimensional point clouds can be obtained by means of a single depth camera 11 . In the controller 3, for each three-dimensional point cloud data, the coordinate data of the depth camera 11 when shooting the three-dimensional point cloud can be calibrated for subsequent processing.
  • the controller 3 can also optimize and merge these three-dimensional point clouds, so as to obtain the execution point cloud and then generate the moving path of the robotic arm 2, and then drive the robotic arm 2 to move along this moving path.
  • an embodiment of the present application also provides a method for controlling a robotic arm, as shown in FIG. 3 , including the following steps: dividing the surface to be processed of the processing object of the robotic arm 2 into at least two regions of interest; Scanning device 1, obtains a three-dimensional point cloud corresponding to each area of interest; optimizes each three-dimensional point cloud; merges each three-dimensional point cloud to obtain an execution point cloud; generates a moving path of the robotic arm 2 based on the execution point cloud; Move the path to process the surface of the object.
  • the controller 3 may analyze the surface 4 of the processing object through a preliminary scan to identify and set areas that are prohibited from being processed by the robotic arm 2 . That is to say, before the step of dividing the surface to be processed of the processing object of the robot arm into at least two regions of interest, the following step may also be included: pre-scanning the surface 4 of the processing object to mark the surface 4 of the processing object as the required area.
  • Treated and untreated surfaces are the surface to be treated, as the name implies, is the surface that needs to be treated.
  • An untreated surface is a surface that does not need to be treated. Untreated surfaces may include wounds on the skin, special organs such as eyes, ears, nipples, navels, etc., or areas of skin that are not intended to be treated in the program.
  • the pre-scanning can be achieved by means of the fast scanning mode, and the non-processed surface is discarded through the pre-scanning, which can reduce the time for subsequent fine scanning and processing of the region of interest, and significantly improve the operation efficiency of the robotic arm.
  • the surface to be treated and the non-treated surface can be automatically identified based on AI image recognition technology.
  • AI image recognition in order to ensure accuracy, the recognized areas can also be reviewed manually.
  • the depth camera After the depth camera obtains the 3D point cloud from the area of interest, the amount of 3D point cloud data obtained by the depth camera is very large and often cannot be used directly. Therefore, individual 3D point clouds can be optimized in a number of ways, reducing the amount of computation.
  • the steps of optimizing each 3D point cloud may include:
  • the 3D point cloud is filtered using a bilateral filter to reduce its noise.
  • Bilateral filter is a method of image denoising.
  • image denoising There are many methods of image denoising, such as median filter, Gaussian filter, Wiener filter and so on.
  • these noise reduction methods are easy to blur the edge details of the picture, and the protection effect of high frequency details is not obvious.
  • the bilateral filter can provide good edge protection, that is, it can protect the edge characteristics of the image while denoising.
  • the inventor of the present application found that, corresponding to the technical solution of constructing the moving path of the robot arm 2, the sharpness of the edge characteristics of the image is very important to provide the optimal moving path.
  • the kernel of the adopted bilateral filter can be expressed mathematically as follows: see the attachment for details.
  • Figures 4 and 5 illustrate the changes of the 3D point cloud before and after the bilateral filter is applied. It can be seen that after filtering by the bilateral filter, a sharp, clear, and less noisy 3D point cloud is obtained.
  • the step of merging the three-dimensional point clouds to obtain the execution point cloud may include:
  • registration refers to the process of finding the spatial transformation relationship between the two point sets given two sets of three-dimensional data points from different coordinate systems, so that the two point sets can be unified into the same coordinate system.
  • the iterative registration algorithm selected by the embodiments of the present application may also be referred to as an Iterative Closest Point (Iterative Closest Point, ICP) algorithm, which is a three-dimensional point cloud matching algorithm.
  • ICP Iterative Closest Point
  • This algorithm allows combining different 3D point clouds obtained from different depth cameras (or the same depth camera at different locations) into a single 3D point cloud, translating and rotating the 3D point cloud to match without distortion, until until the minimum error is obtained.
  • the data amount of the execution point cloud obtained by this algorithm is much smaller, so that it is easy to be processed and executed by the controller 3 .
  • the execution point cloud there is a single coordinate origin that represents the 3D information.
  • This coordinate origin can be chosen to be the same as any one of the depth cameras 11 . Since the position of the coordinate origin is known and associated with the coordinates of the robotic arm 2, an arbitrary movement path can be generated for the robotic arm 2 by adding compensation according to the execution point cloud.
  • the movement path based on the execution point cloud does not need to consider the calculation of the positional relationship between the three-dimensional point clouds acquired by the depth cameras 11 due to different coordinate origins, thus greatly reducing the amount of calculation.
  • the technical solution of the present application can simplify the calculation amount and improve the system performance.
  • the execution point cloud formed by merging the 3D point clouds may only be obtained after merging the 3D point clouds of one or two regions of interest.
  • the product can also be the final point cloud after merging all regions of interest in all regions of interest.
  • the robot arm 2 can execute all the merges uniformly, or as shown in Figure 6, it can be executed by the robot arm 2 while continuously merging 3D point clouds. Therefore, optionally, the movement path may be connected to the next area of interest after covering one area of interest, so that the robot arm 2 can smoothly complete the processing operation.
  • the three-dimensional point cloud obtained by the three-dimensional scanning device 1 is directly used to generate the motion trajectory of the robotic arm 2 .
  • the amount of data is too large, which places high demands on the working system of the robotic arm 2.
  • the embodiments of the present application rely on: 1. Analyze the surface 4 of the processing object through a preliminary pre-scan, identify and set the area that is prohibited from being processed by the robot arm, and ensure the processing of the robot arm Security; 2. Divide the surface 4 of the processing object into multiple areas of interest, so that the amount of data is divided; 3. Through filtering and optimization, useless 3D point cloud data is discarded and the data amount of a single 3D point cloud is reduced 4. By merging multiple 3D point clouds, an execution point cloud with a small amount of data and can be executed accurately is obtained.
  • the embodiment of the present application has lower requirements on the performance of the controller 3, which can reduce the manufacturing cost of the skin surface treatment device and improve its treatment efficiency. Since the traditional manual treatment is replaced by a robotic arm, the quality stability and treatment safety of the skin surface treatment operation are improved.
  • the second embodiment of the present application is a further improvement of the first embodiment.
  • the main improvement lies in that, in the second embodiment of the present application, the robotic arm 2 in the robotic arm control method processes the surface 4 of the object according to the movement path.
  • the steps, as shown in FIG. 1 and FIG. 2 further include: acquiring state data of the object surface through the sensor 7 ; adjusting the processing operation of the robotic arm 2 according to the state data.
  • the skin surface treatment device includes: a sensor 7 for acquiring state data of the object surface;
  • the controller 3 adjusts the processing operation of the robot arm 2 according to the state data.
  • the status data may include temperature or humidity.
  • the sensor 7 may be a device that can be used to detect the state of human skin.
  • the temperature sensor 7 can be used to measure the skin surface temperature
  • the humidity sensor 7 can be used to measure the humidity of the skin
  • the camera can be used to detect the texture of the skin
  • the spectral sensor 7 can be used to detect the spectrum or laser energy irradiated to the skin.
  • the disposition position of the sensor 7 can be consistent with the disposition position of the depth camera 11 , that is, when the depth camera 11 is disposed on the structural element 12 , the sensor 7 can also be disposed on the structural element 12 .
  • the sensor 7 can also be arranged on the robot arm 2 .
  • the sensor 7 can also be arranged on the robotic arm 2 to follow the movement of the robotic arm and give feedback of the status data of the current operating area in a more accurate and timely manner. Moreover, arranging the sensor 7 on the robot arm 2 can also prevent the sensor 7 from being blocked during the operation of the robot arm 2, thereby improving the data accuracy.
  • the temperature sensor 7 can detect the temperature of the operating area of the processing tool on the robot arm 2 , such as a laser, and send it to the controller 3 .
  • the controller 3 detects that the temperature rise reaches or exceeds the threshold, it can control the laser to reduce the output power, or control the laser to turn off, and even speed up the moving speed of the robotic arm 2 on the moving path, thereby shortening the temperature of the laser.
  • the dwell time on the skin surface that rises above a threshold is the threshold.
  • the humidity sensor 7 can detect the humidity of the operation area of the processing tool on the robot arm 2 and send it to the controller 3 .
  • the controller 3 detects that the humidity drop reaches or exceeds the threshold, it can control the laser to reduce the output power, control the high-pressure spray head to spray water, or control the atomizer to mist to replenish skin moisture, and so on.
  • the spectral sensor 7 can detect the laser energy received by the skin of the operating area of the treatment tool on the robotic arm 2 and send it to the controller 3 .
  • the controller 3 detects that the laser energy reaches or exceeds the threshold, it can control the laser to reduce the output power, or control the laser to turn off, and even speed up the moving speed of the robotic arm 2 on the moving path, thereby shortening the laser energy consumption.
  • the dwell time on the skin surface above the threshold is accumulated.
  • the temperature information fed back by the sensor 7 can be used to stabilize the temperature for a safety margin
  • the humidity information can be used to optimize and enhance the user experience
  • the laser energy can be used to precisely dose the laser energy output.
  • the step of acquiring the state data of the surface of the object through the sensor 7 may further include: when the state data exceeds the threshold limit, adjusting the movement path to temporarily avoid the part of the surface of the object that exceeds the threshold.
  • the third embodiment of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program can realize the steps or functions of the method in the first or second embodiment when executed.
  • embodiments of the present application also provide a skin surface treatment device, as shown in FIG. 7 , comprising a memory 51, a processor 52 and a communication component 53, wherein the memory 51 is used to store programs; the processor 52, It is coupled with the memory 51 and is used for executing the program stored in the memory 51, so as to: divide the surface 4 of the processing object of the robotic arm 2 into at least two areas of interest; obtain the corresponding area of interest through the three-dimensional scanning device 1 3D point cloud; optimize each 3D point cloud; combine each 3D point cloud to obtain execution point cloud; generate the movement path of the robot arm 2 based on the execution point cloud; the robot arm 2 processes according to the movement path the surface of the object.
  • the skin surface treatment device may further include: a communication component 53, a display 54, a power supply component 55, an audio component 56 and other components. Only some components are shown schematically in this application, which does not mean that the computing device only includes these components.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Robotics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Radiology & Medical Imaging (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)

Abstract

本申请涉及机械控制技术领域，公开了一种机械臂控制方法、皮肤表面处理设备和计算机可读存储介质，机械臂控制方法包括如下步骤：将机械臂的处理对象的需处理表面分为至少两个关注区域；通过三维扫描装置，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于所述执行点云生成所述机械臂的移动路径；所述机械臂根据所述移动路径，对所述对象的表面进行处理。本申请相对于现有技术而言具有更好的经济性，并能够提高处理效率，提升设备的安全性。

Description

机械臂控制方法和皮肤表面处理设备 技术领域
本发明涉及机械控制技术领域，特别涉及一种可应用于皮肤表面处理的机械臂控制方法和皮肤表面处理设备。
 背景技术
皮肤美容，是在不介入人体的美容方法，包括在皮肤、毛发等可为人们所视的部位局部实施的、非治疗目的的修饰方法。皮肤美容能够改善皮肤不理想的现状，提高皮肤健康度，延缓皮肤衰老。
具体来说，通过皮肤美容治疗能够改善包括且不限于如下症状：色斑、痤疮、毛孔、脱毛、皱纹和发黑。能让皮肤恢复正常的新陈代谢系统。
皮肤美容从肤色、肤质、肤龄、肤健四个方面全面地对皮肤进行改善提升，从症状、新陈代谢、根源、长远管理四个维度深入皮肤不同层次彻底解决皮肤问题。
在针对皮肤的处理操作中，会常用到多种工具。例如激光美容仪、IPL光子美容装置、射频仪器、超声紧肤仪器、脱毛仪、高压注水喷头等等。这些工具通常由经过训练的从业人员来操作，人工成本高昂。此外，过多且不同的人工介入无法保证和提供水平稳定且全程可控的服务品质，而且还有可能因为员工经验不足或人为粗心导致错误的处理引发医疗事故。
在对人体的表面进行处理的过程中，可以通过机器人手臂，也就是机械臂来握持这些美容设备和工具，并结合另外附设的各类传感器而提供稳定的非治疗目的的皮肤美容服务。具体而言，就是利用机械臂的运动特性，使之沿人类皮肤表面的几何形状移动，因此需要设计机械臂的运动轨迹，而且需要确保机械臂不会接触到人体表面的敏感区域或禁止处理的特殊区域（例如眼睛周边或伤口区域）以策安全。
为此，在现有技术中，往往尝试利用三维扫描装置，获取人体表面的三维点云，并根据这些三维点云生成机械臂的移动路径。
然而，在对整个皮肤表面部位进行处理的时候，三维点云的数据量将十分庞大。具体说来，取决于相机的分辨率和数据采集，仅针对人面部的扫描都可能会创建数十万个数据点， 由于机械臂的运动轨迹是点对点的运动的，根据如此数量的数据点进行运动也是不切实际的，因此需要对三维点云进行优化处理。
 技术问题
而，对于巨量的三维点云而言，无论是输入、存储、优化或是输出，都对控制器的性能提出了很高的要求。而如果控制器达不到要求，则很可能会导致数据溢出。
 技术解决方案
为了解决或部分解决上述技术问题，本申请的实施方式提供了一种机械臂控制方法。
其中，机械臂控制方法包括如下步骤：将机械臂的处理对象的需处理表面分为至少两个关注区域；通过三维扫描装置，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于执行点云生成机械臂的移动路径；机械臂根据移动路径，对对象的表面进行处理。
本申请实施方式还公开了一种皮肤表面处理设备，包括：三维扫描装置、机械臂和控制器；其中，控制器用于将机械臂的处理对象的需处理表面分为至少两个关注区域；三维扫描装置用于获取每个关注区域所对应的三维点云；控制器用于优化各个三维点云，将各三维点云合并以获得执行点云，并基于执行点云生成机械臂的移动路径；机械臂用于根据移动路径，对对象的表面进行处理。
本申请实施方式还公开了一种计算机可读存储介质，存储有计算机程序，其中，计算机程序被执行时能够实现上述方法中的步骤。
本申请实施方式还公开了一种皮肤表面处理设备，包括存储器、处理器及通信组件，其中，存储器，用于存储程序；处理器，与存储器耦合，用于执行存储器中存储的程序，以用于：将机械臂的处理对象的需处理表面分为至少两个关注区域;通过三维扫描装置，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于执行点云生成机械臂的移动路径；机械臂根据移动路径，对对象的表面进行处理。
 有益效果
相比于现有技术而言，本申请的实施方式借助于：
1、通过初步扫描来分析待处理对象的表面，以识别并设定禁止机械臂处理的区域，确保机械臂的处理安全性;2、将处理对象的表面分为多个关注区域，使得数据量得到了分割；3、通过过滤和优化，摒弃了无用的三维点云数据，减少了单个三维点云的数据量；4、通过对多个三维点云的合并，获得了数据量小、可以被精确执行的执行点云。
因此相比于现有技术而言，本申请的实施方式对控制器性能要求低，可以降低皮肤表面处理设备的生产制造成本，提高其处理效率。
 附图说明
为了更清楚地说明本申请实施方式或现有技术中的技术方案，下面将对实施方式或现有技术描述中所需要使用的附图作简单介绍。显而易见地，下面描述中的附图仅用于示意本申请的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图中未提及的技术特征、连接关系乃至方法步骤。
图1是本申请实施方式的将两个深度摄像头设置在同一结构元件上时的示意图；
图2是本申请实施方式的将一个深度摄像头设置在机械臂上时的示意图；
图3是本申请实施方式的一种机械臂控制方法的简要流程示意图；
图4是本申请实施方式的三维点云在优化过滤前的示意图；
图5是本申请实施方式的三维点云经优化过滤后的示意图；
图6是本申请实施方式的一种机械臂控制方法在针对需处理表面进行处理的过程中的流程示意图；
图7是本申请实施方式的一种皮肤表面处理设备的系统框图。
附图标记说明：1、三维扫描装置；11、深度摄像头；12、结构元件；2、机械臂；3、控制器； 31、上位机；32、下位控制模块；4、处理对象的表面；51、存储器；52、处理器；53、通信组件；54、显示器；55、电源组件；56、音频组件；6、表面处理工具；7、传感器。
 本发明的实施方式
下面结合说明书附图，对本发明进行进一步的详细说明。
实施方式一
如前文背景技术所分析的，现有技术尚缺乏一种具有更好的经济性和高性能的机械臂控制方法。
有鉴于此，本申请的第一实施方式提供了一种皮肤表面处理设备，参见图1或图2所示，包括：三维扫描装置1、机械臂2和控制器3；其中，控制器3将机械臂2的处理对象的需处理表面分为至少两个关注区域；三维扫描装置1是用于获取物体立体表面三维点云的装置。
在本申请的实施方式中，三维扫描装置1用于获取每个关注区域所对应的三维点云；控制器3用于优化各个三维点云，将各三维点云合并以获得执行点云，并基于执行点云生成机械臂2的移动路径；机械臂2根据移动路径针对对象的表面4进行相应的处理。
在本申请的实施方式中，机械臂2上可以安装有各种表面处理工具6，如激光器、IPL光子美容装置、射频仪器、超声紧肤仪器、脱毛仪、高压注水喷头等。这些表面处理工具6同样可以与控制器3电性连接，以便其与机械臂2的移动路径彼此配合动作。
可以理解地，控制器3为广义上的控制器，其可以部分分立设置于机械臂2之外，也可以被完全集成在机械臂2以内。典型地，如图1或图2中所示，其可以包括装设在机械臂2上的下位控制模块32，以及连接下位控制模块32的上位机31（如计算机工作站、或是其他可移动、不可移动的终端），当控制器3同时包括下位控制模块32和上位机31时，上位机31可以负责将三维扫描装置1所获取和收集的数据分析处理之后，得出机械臂的执行动作数据。然后，上位机31将执行动作数据输出到下位控制模块32之中，从而通过下位控制模块32实现机械臂2的操作逻辑。
另外，下位控制模块32往往可以是机械臂2自带的控制器，其不但提供针对机械臂2的控制接口，而且通常还可以负责为机械臂2供电。
可选地，三维扫描装置1可以包括静态地安装在同一结构元件12上的两个深度摄像头11。参见图1所示，在这一可选示例中，可以使用两个深度摄像头11，并将其安装在结构元件12上。其中，结构元件12可以是支撑架或一些常见种类的夹具。结构元件12的位置可以通过标尺标定，并可以被调整，以便适应不同体型的处理对象。在固定好深度摄像头11之后，可以在控制器3中预置深度摄像头11的坐标，从而保持校准。
在对物体的表面进行扫描的过程中，每个深度摄像头11都会生成一个三维点云。在此基础上，通过任意一个深度摄像头11都可以描绘出待处理对象的一侧表面。例如，以人脸作为待处理对象为例，通过设置于人脸左右两侧的深度摄像头11，可以在同一时间绘制出左脸和右脸的三维点云。进一步可选地，当处理对象为人脸时，关注区域可以为6至8个，对应地，深度摄像头11的数量也可以根据需求对应调整。
据此，控制器3可以针对这些三维点云进行优化和合并，从而获得执行点云并进而生成机械臂2的移动路径，而后驱动机械臂2沿着这一移动路径运动。
当然，在采用更多数量的深度摄像头11时，将这些深度摄像头11分别与各个关注区域对应，可以显著地提高扫描效率。
而可选地，三维扫描装置1也可以包括安装在机械臂2上的一个或多个深度摄像头11。
在一个可选示例中，参见图2所示，可以使用一个深度摄像头11，并将其安装在机械臂2上。
在控制器3中，可以根据机械臂2的运动轨迹随时获知机械臂2本身的确切位置，因此同样可以获知这一深度摄像头11的确切位置。因此借助于控制器3来控制机械臂2，在待处理对象前通过预设轨迹移动的过程中，即可借助单个深度摄像头11获得多个三维点云。在控制器3中，可以针对每个三维点云数据，标定深度摄像头11在拍摄该三维点云时的坐标数据，以便后续处理。
据此，控制器3同样可以针对这些三维点云进行优化和合并，从而获得执行点云并进而生成机械臂2的移动路径，而后驱动机械臂2沿着这一移动路径运动。
采用设置于机械臂2上的单个深度摄像头11时，由于无需设置多个深度摄像头11，因此可以显著地降低成本。
也就是说，本申请的实施方式还提供了一种机械臂控制方法，参见图3所示，包括如下步骤：将机械臂2的处理对象的需处理表面分为至少两个关注区域；通过三维扫描装置1，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于执行点云生成机械臂2的移动路径；机械臂2根据移动路径，对所述对象的表面进行处理。
可选地，控制器3可以通过初步扫描来分析处理对象的表面4，以识别并设定禁止机械臂2处理的区域。也就是说，在将机械臂的处理对象的需处理表面分为至少两个关注区域的步骤之前，还可以包括如下步骤：预扫描处理对象的表面4，以将处理对象的表面4标记为需处理表面和非处理表面。其中，需处理表面，顾名思义为需要被处理的表面。而非处理表面则是无需被处理的表面。非处理表面可以包括皮肤上的伤口、特殊器官如眼睛、耳朵、乳头、肚脐等等，或是在计划内并不打算处理的皮肤区域。
预扫描可以借助于快速扫描模式来实现，而通过预扫描摒弃了非处理表面，可以减少后续的关注区域的精细扫描和处理的时间，显著地提高机械臂的操作效率。优选地，可以基于AI图像识别技术来自动化地识别需处理表面和非处理表面。当然，在AI图像识别之后，为了确保准确性，还可以借助人工对识别的区域进行复核。
在深度摄像机从关注区域获取三维点云后，其所获取的三维点云数据量十分庞大，往往无法被直接使用。因此，可以通过多种方式优化各个三维点云，从而减少计算量。
具体可选地，参见图3、图6所示，优化各个三维点云的步骤，可以包括：
使用双边过滤器对三维点云进行过滤处理，以减少其噪声。
双边过滤器属于图像去噪的一种方法。图像去噪的方法很多，如中值滤波，高斯滤波，维纳滤波等等。但这些降噪方法容易模糊图片的边缘细节，对于高频细节的保护效果并不明显。相比较而言，双边滤波器可以提供很好的边缘保护，也就是可以在去噪的同时，保护图像的边缘特性。而，本申请的发明人发现，对应构造机械臂2的移动路径这一技术方案而言，图像的边缘特性的清晰程度对提供最优化的移动路径至关重要。
在本申请的实施方式中，所采用的双边过滤器的核可以用数学表示为：具体见附件。
图4和图5分别示意出了采用双边滤波器前后的三维点云变化，可以看到经过双边滤波器的过滤之后，获得了锐利、清晰、噪声少的三维点云。
而可选地，将各三维点云合并以获得执行点云的步骤，可以包括：
将第一个三维点云和第二个三维点云的最近点进行迭代配准，合并成新的三维点云，将所合并成的新的三维点云和下一个三维点云进行迭代配准，如此反复，直到所有的三维点云都合并成执行点云。
所谓配准，指的是给定两个来自不同坐标系的三维数据点集，找到两个点集空间的变换关系，使得两个点集能统一到同一坐标系统中的过程。
本申请实施方式所选用的迭代配准算法，又可称之为迭代最近点（Iterative Closest Point, ICP）算法，是一种三维点云匹配算法。这一算法允许将从不同的深度相机（或位于不同位置的同一深度相机）获得的不同三维点云合并到单个三维点云中，平移和旋转三维点云以使其匹配而不会变形，直到获得最小误差为止。
经过这一算法获得的执行点云，其数据量相比于三维扫描装置1所获得的初始三维点云而言要小得多，从而易于被控制器3处理和执行。
具体说来，在执行点云中，具有表示3D信息的单个坐标原点。这一坐标原点可以被选择为与任意一深度摄像头11相同。由于坐标原点的位置是已知的，并且与机械臂2的坐标相关联，因此可以依据执行点云，增加补偿来为机械臂2生成任意的移动路径。
基于执行点云的移动路径无需考虑各个深度摄像头11所获取的三维点云之间因不同的坐标原点而带来的位置关系的计算，因此大幅度地降低了计算量。
据此，本申请的技术方案相比于现有技术而言，能够简化计算量并提高系统性能。
另外值得一提的是，对本申请的实施方式而言，在上述的合并步骤中，将三维点云合并后形成的执行点云可以仅仅是合并了某一两个关注区域的三维点云之后的产物，也可以是将所有的关注区域的关注区域全部合并之后的最终点云。取决于数据量的多寡，可以如图1所示，在全部合并之后统一由机械臂2执行，也可以如图6所示，由机械臂2一边执行，一边持续合并三维点云。因此可选地，移动路径可以在每覆盖一个关注区域之后连接至下一个关注区域，以便机械臂2能够顺畅地完成处理操作。
在现有技术中，未经过良好的数据处理，直接利用三维扫描装置1所获取的三维点云来生成机械臂2的运动轨迹。其数据量过于庞大，对机械臂2的工作系统提出了很高的要求。
而相比于现有技术而言，本申请的实施方式借助于：1、通过初步的预扫描来分析处理对象的表面4，识别并设定禁止机械臂处理的区域，确保了机械臂的处理安全性；2、将处理对象的表面4分为多个关注区域，使得数据量得到了分割；3、通过过滤和优化，摒弃了无用的三维点云数据，减少了单个三维点云的数据量；4、通过对多个三维点云的合并，获得了数据量小、可以被精确执行的执行点云。
因此相比于现有技术而言，本申请的实施方式对控制器3性能要求低，可以降低皮肤表面处理设备的生产制造成本，提高其处理效率。由于借助了机械臂来替代传统的人工处理，因此提高了皮肤表面处理操作的品质稳定性和处理安全性。
实施方式二
本申请的第二实施方式是第一实施方式的进一步改进，主要改进之处在于，在本申请的第二实施方式中，机械臂控制方法中的机械臂2根据移动路径处理对象的表面4的步骤，参见图1、图2所示，还包括：通过传感器7获取对象表面的状态数据；根据状态数据调整机械臂2的处理操作。
与之对应地，皮肤表面处理设备则包括：传感器7，获取对象表面的状态数据；
控制器3根据所述状态数据调整所述机械臂2的处理操作。 
可选地，状态数据可以包括温度或湿度。
在本申请中，传感器7可以是可以用对人体皮肤的状态进行检测的设备。传感器7的种类有多种。例如温度传感器7，可用于测量皮肤表面温度；湿度传感器7，可用于测量皮肤的湿度；摄像头，可用于检测皮肤的纹理；光谱传感器7，可用于对照射至皮肤的光谱或激光能量进行检测。传感器7的设置位置可以和深度摄像头11的设置位置相一致，也就是说，当深度摄像头11被设置在结构元件12上时，传感器7也可以被设置在结构元件12上。而当深度摄像头11被设置在机械臂2上时，传感器7也可以被设置在机械臂2上。
当然，当多个深度摄像头11被设置在结构元件12上时，传感器7还可以被设置在机械臂2上，跟随机械臂运动并更准确及时地给予当前操作区域的状态数据反馈。而且，将传感器7设置在机械臂2上还能够防止机械臂2动作过程中可能对传感器7所产生的阻挡，提高了数据精度。
接下来，将以一些典型的传感器7为例，说明如何根据状态数据调整机械臂2的处理操作。
当传感器7包括温度传感器7时，温度传感器7可以检测机械臂2上的处理工具，如激光器的操作区域的温度，并发送给控制器3。
由于皮肤受到激光器的照射温度会上升。控制器3在检测到这一温度的上升达到或超过阈值时，可以控制激光器调低输出功率，也可以控制激光器关闭，甚至可以加快机械臂2在移动路径上的移动速度，从而缩短激光器在温度上升超过阈值的皮肤表面的停留时间。
当传感器7包括湿度传感器7时，湿度传感器7可以检测机械臂2上的处理工具的操作区域的湿度，并发送给控制器3。
由于皮肤湿度下降时，容易发生损伤。控制器3在检测到这一湿度的下降达到或超过阈值时，可以控制激光器调低输出功率，也可以控制高压喷头使其喷水，或控制雾化器起雾补充皮肤水分等等。
当传感器7包括光谱传感器7时，光谱传感器7可以检测机械臂2上的处理工具的操作区域的皮肤接收到的激光能量，并发送给控制器3。
由于皮肤短时间接收到过强的激光能量时，容易发生损伤。控制器3在检测到这一激光能量达到或超过阈值时，可以控制激光器调低输出功率，也可以控制激光器关闭，甚至可以加快机械臂2在移动路径上的移动速度，从而缩短激光器在激光能量累计超过阈值的皮肤表面的停留时间。
根据治疗类型和所使用的激光技术，传感器7所反馈的温度信息可用于稳定温度的安全裕度，湿度信息可用于优化和提升用户体验，激光能量则可以精确激光能量的输出剂量。
另外，可选地，通过传感器7获取对象表面的状态数据的步骤，还可以包括：在状态数据超出阈值限制时，调整移动路径以暂时避开对象的表面的超出阈值的部位。
实施方式三
本申请的第三实施方式提供了一种计算机可读存储介质，存储有计算机程序，其中，计算机程序被执行时能够实现第一或第二实施方式中的方法的步骤或功能。
借助计算机程序能够自动化地完成上述步骤，提高效率。
据此，本申请的实施方式还提供了一种皮肤表面处理设备，参见图7所示，包括存储器51、处理器52及通信组件53，其中，存储器51，用于存储程序；处理器52，与存储器51耦合，用于执行存储器51中存储的程序，以用于：将机械臂2的处理对象的表面4分为至少两个关注区域；通过三维扫描装置1，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于所述执行点云生成所述机械臂2的移动路径；所述机械臂2根据所述移动路径处理所述对象的表面。
进一步，皮肤表面处理设备还可以包括：通信组件53、显示器54、电源组件55、音频组件56等其它组件。本申请中仅示意性给出部分组件，并不意味着计算设备只包括这些。
最后应说明的是，本领域的普通技术人员可以理解，为了使读者更好地理解本申请，本申请的实施方式提出了许多技术细节。但是，即使没有这些技术细节和基于上述各实施方式的种种变化和修改，也可以基本实现本申请各权利要求所要求保护的技术方案。因此，在实际应用中，可以在形式上和细节上对上述实施方式作各种改变，而不偏离本申请的精神和范围。

Claims (10)

  1. 一种机械臂控制方法，其特征在于，包括如下步骤：将机械臂的处理对象的需处理表面分为至少两个关注区域；通过三维扫描装置，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于所述执行点云生成所述机械臂的移动路径；所述机械臂根据所述移动路径，对所述对象的表面进行处理。
  2. 根据权利要求1所述的机械臂控制方法，其特征在于，所述优化各个三维点云的步骤，包括：使用双边过滤器对所述三维点云进行过滤处理，以减少其噪声。
  3. 根据权利要求1所述的机械臂控制方法，其特征在于，所述将各三维点云合并以获得执行点云的步骤，包括：将第一个三维点云和第二个三维点云的最近点进行迭代配准，合并成新的三维点云，将所合并成的新的三维点云和下一个三维点云进行迭代配准，如此反复，直到所有的三维点云都合并成执行点云。
  4. 根据权利要求1所述的机械臂控制方法，其特征在于，所述处理对象为人脸，所述关注区域为6至8个；所述移动路径在每覆盖一个关注区域之后连接至下一个关注区域。
  5. 根据权利要求1所述的机械臂控制方法，其特征在于，所述机械臂根据所述移动路径处理所述对象的表面的步骤，包括：通过传感器获取对象表面的状态数据；根据所述状态数据调整所述机械臂的处理操作；所述状态数据包括温度或湿度；所述通过传感器获取对象表面的状态数据的步骤，包括：在所述状态数据超出阈值限制时，调整所述移动路径以暂时避开对象的表面的超出阈值的部位。
  6. 根据权利要求1所述的机械臂控制方法，其特征在于，在所述将机械臂的处理对象的需处理表面分为至少两个关注区域的步骤之前，还包括如下步骤：预扫描处理对象的表面，以将处理对象的表面标记为需处理表面和非处理表面。
  7. 一种皮肤表面处理设备，其特征在于，包括：三维扫描装置、机械臂和控制器；其中，所述控制器用于将所述机械臂的处理对象的需处理表面分为至少两个关注区域；所述三维扫描装置用于获取每个关注区域所对应的三维点云；所述控制器用于优化各个三维点云，将各三维点云合并以获得执行点云，并基于所述执行点云生成所述机械臂的移动路径；所述机械臂用于根据所述移动路径，对所述对象的表面进行处理。
  8. 根据权利要求7所述的皮肤表面处理设备，其特征在于，所述三维扫描装置包括安装在同一结构元件上的两个深度摄像头；或者，所述三维扫描装置包括安装在所述机械臂上的一个深度摄像头。
  9. 根据权利要求7所述的皮肤表面处理设备，其特征在于，还包括：传感器，获取对象表面的状态数据；所述控制器根据所述状态数据调整所述机械臂的处理操作。
     
  10. 一种皮肤表面处理设备，包括存储器、处理器及通信组件，其中，存储器，用于存储程序；处理器，与存储器耦合，用于执行存储器中存储的程序，以用于:将机械臂的处理对象的需处理表面分为至少两个关注区域；通过三维扫描装置，获取每个关注区域所对应的三维点云；优化各个三维点云；将各三维点云合并以获得执行点云；基于所述执行点云生成所述机械臂的移动路径；所述机械臂根据所述移动路径，对所述对象的表面进行处理。
     
PCT/CN2021/108045 2020-07-23 2021-07-23 机械臂控制方法和皮肤表面处理设备 Ceased WO2022017485A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/628,564 US12090663B2 (en) 2020-07-23 2021-07-23 Robot arm control method and skin surface treatment apparatus
KR1020237000481A KR20230051477A (ko) 2020-07-23 2021-07-23 매니퓰레이터 제어방법과 피부 표면 처리설비
EP21844943.7A EP4186457B1 (en) 2020-07-23 2021-07-23 Robotic arm control method and skin surface treatment device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010714728.6A CN113967070A (zh) 2020-07-23 2020-07-23 机械臂控制方法和皮肤表面处理设备
CN202010714728.6 2020-07-23

Publications (1)

Publication Number Publication Date
WO2022017485A1 true WO2022017485A1 (zh) 2022-01-27

Family

ID=79585148

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/108045 Ceased WO2022017485A1 (zh) 2020-07-23 2021-07-23 机械臂控制方法和皮肤表面处理设备

Country Status (5)

Country Link
US (1) US12090663B2 (zh)
EP (1) EP4186457B1 (zh)
KR (1) KR20230051477A (zh)
CN (1) CN113967070A (zh)
WO (1) WO2022017485A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116276987A (zh) * 2023-03-01 2023-06-23 安费诺飞凤(安吉)通信部品有限公司 一种机械臂上下料作业路径的规划方法
US12524017B2 (en) 2022-06-09 2026-01-13 Wing Aviation Llc UAV autoloader systems and methods

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12370681B2 (en) * 2020-07-31 2025-07-29 GrayMatter Robotics Inc. System and method for autonomously scanning and processing a workpiece
CN114842079B (zh) * 2022-04-23 2023-09-19 四川大学 盾构隧道内对预制中隔墙进行位姿测量的设备及方法
CN116251298B (zh) * 2022-09-08 2024-09-27 深圳市宗匠科技有限公司 美容仪的控制方法、美容仪及计算机可读存储介质
CN116972973B (zh) * 2023-07-17 2024-08-13 中国科学院上海光学精密机械研究所 发光物体的空间光谱全自动测量装置及方法
EP4691404A1 (en) * 2024-08-06 2026-02-11 Kyniska Robotics Method for obtaining a 3d intraoperative image with a depth camera

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109493375A (zh) * 2018-10-24 2019-03-19 深圳市易尚展示股份有限公司 三维点云的数据匹配及合并方法、装置、可读介质
CN110037918A (zh) * 2019-05-24 2019-07-23 熊江华 一种全自动艾灸机械臂的艾灸系统
US20190247122A1 (en) * 2018-02-14 2019-08-15 Epica International, Inc. Method for determination of surgical procedure access
CN110382048A (zh) * 2017-03-06 2019-10-25 Oh & Lee医疗机器人公司 用于激光治疗的区域划分方法、利用其的激光治疗方法以及装置
CN111223048A (zh) * 2019-12-27 2020-06-02 中铭谷智能机器人(广东)有限公司 一种3d视觉点云数据拼接的方法及其系统

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9715761B2 (en) * 2013-07-08 2017-07-25 Vangogh Imaging, Inc. Real-time 3D computer vision processing engine for object recognition, reconstruction, and analysis
CN106971147A (zh) * 2017-03-06 2017-07-21 武汉嫦娥医学抗衰机器人股份有限公司 一种基于人脸区域分割的中医面诊系统及面诊方法
US10909650B2 (en) * 2017-06-23 2021-02-02 Cloud 9 Perception, LP System and method for sensing and computing of perceptual data in industrial environments
US10060857B1 (en) * 2017-11-16 2018-08-28 General Electric Company Robotic feature mapping and motion control
WO2019155473A1 (en) * 2018-02-08 2019-08-15 Skyline Robotics Ltd. Device and method for use in cleaning a facade
CN108466265B (zh) * 2018-03-12 2020-08-07 珠海市万瑙特健康科技有限公司 机械手臂路径规划与作业方法、装置以及计算机设备
US10614579B1 (en) * 2018-10-10 2020-04-07 The Boeing Company Three dimensional model generation using heterogeneous 2D and 3D sensor fusion
CN109202912B (zh) * 2018-11-15 2020-09-11 太原理工大学 一种基于单目深度传感器和机械臂配准目标轮廓点云的方法
US20200194117A1 (en) * 2018-12-13 2020-06-18 University Of Maryland, College Park Systems, methods, and media for remote trauma assessment
KR20210107072A (ko) * 2018-12-21 2021-08-31 알2 테크놀로지스, 인크. 자동화된 피부학적 냉동 분무 처리 계획 시스템
CN110110672A (zh) * 2019-05-10 2019-08-09 广东工业大学 一种人脸表情识别方法、装置及设备
US20210339399A1 (en) * 2020-04-29 2021-11-04 Cobalt Robotics Inc. Mobile robot for elevator interactions

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110382048A (zh) * 2017-03-06 2019-10-25 Oh & Lee医疗机器人公司 用于激光治疗的区域划分方法、利用其的激光治疗方法以及装置
US20190247122A1 (en) * 2018-02-14 2019-08-15 Epica International, Inc. Method for determination of surgical procedure access
CN109493375A (zh) * 2018-10-24 2019-03-19 深圳市易尚展示股份有限公司 三维点云的数据匹配及合并方法、装置、可读介质
CN110037918A (zh) * 2019-05-24 2019-07-23 熊江华 一种全自动艾灸机械臂的艾灸系统
CN111223048A (zh) * 2019-12-27 2020-06-02 中铭谷智能机器人(广东)有限公司 一种3d视觉点云数据拼接的方法及其系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4186457A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12524017B2 (en) 2022-06-09 2026-01-13 Wing Aviation Llc UAV autoloader systems and methods
CN116276987A (zh) * 2023-03-01 2023-06-23 安费诺飞凤(安吉)通信部品有限公司 一种机械臂上下料作业路径的规划方法

Also Published As

Publication number Publication date
EP4186457A1 (en) 2023-05-31
US12090663B2 (en) 2024-09-17
KR20230051477A (ko) 2023-04-18
EP4186457A4 (en) 2024-08-14
CN113967070A (zh) 2022-01-25
EP4186457B1 (en) 2026-03-11
EP4186457C0 (en) 2026-03-11
US20230356394A1 (en) 2023-11-09

Similar Documents

Publication Publication Date Title
WO2022017485A1 (zh) 机械臂控制方法和皮肤表面处理设备
CN112784773B (zh) 图像处理方法及装置、存储介质、终端
CN117636446B (zh) 脸部穴位定位方法、针灸方法、针灸机器人及存储介质
CN101299267A (zh) 一种人脸图像处理方法及装置
CN107730465B (zh) 一种图像中人脸美颜方法及装置
US20140369554A1 (en) Face beautification system and method of use thereof
CN110785269A (zh) 用于管理机器人和相关治疗机器人的运动的设备
CN113344837B (zh) 人脸图像处理方法及装置、计算机可读存储介质、终端
US10467822B2 (en) Reducing collision-based defects in motion-stylization of video content depicting closely spaced features
WO2024156287A1 (zh) 一种人像视频皱纹祛除方法、装置以及设备
CN102103259A (zh) 一种避免投影仪强光照射装置和方法
CN110335194B (zh) 一种人脸变老图像处理方法
CN113506604B (zh) 一种按摩轨迹调整方法
CN113221958B (zh) 按摩轨迹与按摩区域匹配方法、装置、系统及存储介质
US11893826B2 (en) Determining position of a personal care device relative to body surface
CN115937931A (zh) 人脸姿态判断方法及换脸方法
CN111915478B (zh) 一种基于保边模糊的美颜方法、装置、设备及计算机可读存储介质
HK40086266A (zh) 机械臂控制方法和皮肤表面处理设备
CN114145756B (zh) 协作机器人控制方法、装置及计算机可读存储介质
CN118781628A (zh) 一种身份识别系统及激光理疗装置
CN117153364A (zh) 柔性美容仪控制方法、存储介质及电子设备
WO2022258013A1 (zh) 图像处理方法、装置、电子设备及可读存储介质
KR102074977B1 (ko) 전자 장치 및 그의 제어 방법
Meng et al. Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculpting
CN106667496A (zh) 一种面部数据测量方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21844943

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2021844943

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021844943

Country of ref document: EP

Effective date: 20230223

WWG Wipo information: grant in national office

Ref document number: 2021844943

Country of ref document: EP