WO2020189143A1 - Dispositif de commande et procédé de commande - Google Patents

Dispositif de commande et procédé de commande Download PDF

Info

Publication number
WO2020189143A1
WO2020189143A1 PCT/JP2020/006260 JP2020006260W WO2020189143A1 WO 2020189143 A1 WO2020189143 A1 WO 2020189143A1 JP 2020006260 W JP2020006260 W JP 2020006260W WO 2020189143 A1 WO2020189143 A1 WO 2020189143A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
robot
operator
unit
selection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/006260
Other languages
English (en)
Japanese (ja)
Inventor
チャリス ラサンタ フェルナンド
高志 岩佐
山中　宏治
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telexistence Inc
Original Assignee
Telexistence Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telexistence Inc filed Critical Telexistence Inc
Publication of WO2020189143A1 publication Critical patent/WO2020189143A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/02—Hand grip control means

Definitions

  • the present invention relates to a control device and a control method for controlling a robot.
  • Patent Document 1 discloses a control device for assisting an operator who is not proficient in operating a robot so that the robot can be appropriately operated.
  • the robot By assisting the operation of the robot by the control device, the robot can be operated appropriately even when the operator performs an inappropriate operation.
  • the operator may feel uncomfortable when the robot performs an operation different from the operation content performed by the robot.
  • the work efficiency may be lowered because the robot performs an operation different from the operation content performed by the operator.
  • the selection unit selects the selection assistance process based on, for example, the operation skill of the operator.
  • the selection unit may select the selection assisting process based on the content of the operation indicated by the operation information.
  • the selection unit may select the selection assisting process based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other.
  • the information acquisition unit further acquires detection information indicating a state around the robot detected by a sensor provided in the robot, and the selection unit selects the selection assist process based on the detection information. May be good.
  • the selection unit may change the selection assist process when the detection information indicates that the robot is not operating normally.
  • the information acquisition unit displays information indicating candidates for the plurality of auxiliary processes on an information terminal accessible to the operator or the administrator of the operator, and then indicates the candidates selected in the information terminal. Even if the selection candidate information is acquired and the selection unit selects the one or more auxiliary processes designated by the operator or the administrator of the operator as the selection auxiliary process based on the selection candidate information. Good.
  • the information acquisition unit displays an auxiliary screen for selecting the degree of assistance as information indicating the plurality of auxiliary processing candidates on the information terminal, and then indicates the degree of assistance input in the information terminal.
  • the level information may be acquired, and the selection unit may determine the content of the selection auxiliary processing based on the auxiliary level information.
  • the auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information, and the selection unit uses the correction information as the control information in the robot control unit as the selection auxiliary processing. You may select at least one of the correction process for controlling the robot and the instruction process for displaying the information for assisting the operation on the operation device used by the operator for the operation.
  • the auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information, and the selection unit causes the robot control unit to control the robot by using the operation information as the control information. You may switch between the non-correction mode and the correction mode in which the robot control unit controls the robot by using the correction information as the control information as the selection assist process.
  • the auxiliary processing unit may create the correction information by substituting the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator.
  • the selection unit inputs the operation information acquired by the information acquisition unit into the machine learning model that outputs one or more auxiliary processes when the operation information is input, while the operator performs a predetermined operation.
  • One or more auxiliary processes output by the machine learning model may be selected.
  • the selection unit is executed as the predetermined operation before the operator performs a given operation, and the operation acquired by the information acquisition unit while performing a test operation corresponding to the content of the operation.
  • One or more auxiliary processes may be selected by inputting information into the machine learning model.
  • the selection unit may display the type of the selection assist processing on the operation device used by the operator for the operation.
  • the robot control unit may change the operating speed of the robot when the selection unit selects a new one or more auxiliary processes.
  • the control method of the second aspect of the present invention includes a step of acquiring operation information indicating the content of the operation by the operator who operates the robot, which is executed by the computer, and a plurality of assists for assisting the operation of the operator.
  • a step of selecting one or more auxiliary processes from the processing candidates, a step of assisting the operator by executing the selected auxiliary process, which is the one or more auxiliary processes, and the selection auxiliary process are executed.
  • it has a step of controlling the robot by using the control information corresponding to the type of the selection assisting process created based on the operation information.
  • the operator U operates the robot 2 by operating the operation device 3. Specifically, when the operator U operates the operation device 3, the operation device 3 transmits the operation information indicating the operated content to the control device 1 via the network N.
  • the control device 1 is a computer that remotely controls the robot 2 by transmitting control information based on the operation information to the robot 2.
  • the operation device 3 is a device for the operator U to remotely control the robot 2.
  • the operation device 3 has, for example, an operation tool worn by the operator U and a cockpit including a communication unit for notifying the control device 1 of the state of the operation tool of the operator U.
  • the operation device 3 identifies the operation of the operator U with the operation tool attached, and transmits the operation information indicating the specified operation to the control device 1.
  • the cockpit identifies the operation of the operator U based on the signal received from the sensor of the operation tool worn by the operator U.
  • the cockpit may irradiate the operator U with infrared light and specify the operation of the operator U based on the reflected light of the irradiated infrared light.
  • the cockpit may identify the operation of the operator U by analyzing an image of the operator U taken by the camera.
  • the operating device 3 may notify the operating tool worn by the operator U of the captured image and the perceptual information received from the robot 2 via the network N.
  • the captured image is an image generated by the imaging device provided in the robot 2 taking a picture of the surroundings of the robot 2, and the display of the goggles included in the operation tool worn by the operator U or the eyes of the operator U. It is displayed on contact lenses worn on.
  • the perceptual information is information indicating a state detected by a sensor provided in the robot 2, and the operating tool worn by the operator U on the hand, foot, arm, etc. changes the temperature or vibrates based on the perceptual information. To do. With these configurations, the operator U can experience the environment around the robot 2 at a place away from the robot 2 as if he / she is in the robot 2.
  • the control device 1 By assisting the operation of the operator U in various forms, the control device 1 enables the operator U to appropriately operate the robot 2 regardless of the operation skill of the operator U.
  • the control device 1 operates in any of (1) normal mode, (2) correction mode, (3) instruction mode, and (4) automatic mode, for example.
  • the control device 1 may operate in combination with any of these plurality of modes. For example, the control device 1 operates in the normal mode for some of the series of operations performed by the operator U, and operates in the correction mode for some of the other operations.
  • FIG. 1 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the normal mode.
  • the control device 1 transmits the operation information received from the operation device 3 to the robot 2 as control information without substantially changing it. Specifically, when the operation device 3 transmits the operation information to the control device 1 ((1) in FIG. 1) and the control device 1 receives the operation information ((2) in FIG. 1), the control device 1 receives the operation information. The received operation information is transmitted to the robot 2 as the control information for operating the robot 2 ((3) in FIG. 1). The robot 2 operates based on the operation information received from the control device 1 ((4) in FIG. 1). Since the robot 2 operates as operated by the operator U, the normal mode is suitable when the operator U has sufficient operation skills.
  • FIG. 2 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the correction mode.
  • the control device 1 creates correction information by correcting the operation information received from the operation device 3, and transmits the correction information to the robot 2 as control information.
  • the details of the correction information will be described later, but the correction information is information for changing the operation information to an appropriate content when the operation information is not appropriate.
  • the control device 1 corrects the operation information. Correction information is created ((2) in FIG. 2).
  • the control device 1 transmits the correction information to the robot 2 ((3) in FIG. 2), and the robot 2 operates based on the correction information ((4) in FIG. 2).
  • the correction mode is effective when the operator U cannot perform an accurate operation.
  • FIG. 3 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the instruction mode.
  • the control device 1 transmits auxiliary information including at least one of an image or a text indicating an operation method to be performed by the operator U to the operation device 3 ((1) in FIG. 3).
  • the operating device 3 presents the received auxiliary information to the operator U ((2) in FIG. 3).
  • the control device 1 presents the auxiliary information to the operator U, for example, by displaying the auxiliary information on the display of the operation device 3 worn by the operator U or the information terminal visible to the operator U.
  • the operation device 3 transmits the operation information indicating the contents operated by the operator U to the control device 1 with reference to the auxiliary information ((3) in FIG. 3).
  • the control device 1 receives the operation information ((4) in FIG. 3)
  • the control device 1 transmits the received operation information to the robot 2 as control information for operating the robot 2 ((5) in FIG. 3). )).
  • the robot 2 operates based on the operation information received from the control device 1 ((6) in FIG. 3).
  • the control device 1 may transmit the correction information obtained by correcting the operation information to the robot 2 as control information.
  • the instruction mode is effective when the operator U is not familiar with the operation method.
  • FIG. 4 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the automatic mode.
  • the control device 1 In the automatic mode, the control device 1 automatically creates control information for operating the robot 2 based on a predetermined scenario, and transmits the created control information to the robot 2.
  • the scenario is information in which the elapsed time from the reference time and the operation content of the robot 2 are associated with each other.
  • the reference time is, for example, the time when the robot 2 starts operating, but is not limited to this, and may be another time such as the time when the user who operates the robot 2 performs a predetermined operation.
  • the control device 1 acquires a scenario corresponding to the work performed by the operator U, for example, based on an instruction from the operator U or the administrator who manages the operator U ((in FIG. 4). 1)).
  • the control device 1 may acquire the scenario from another computer, for example, or may acquire the scenario from the storage medium in the control device 1.
  • the control device 1 creates control information based on the scenario ((2) in FIG. 4).
  • the robot 2 operates based on the control information ((3) in FIG. 4).
  • the automatic mode is effective when the skill of the operator U is low and the pattern for operating the robot 2 is determined.
  • FIG. 5 is a diagram showing a functional configuration of the control device 1.
  • the control device 1 has a communication unit 11, a storage unit 12, and a control unit 13.
  • the control unit 13 includes an information acquisition unit 131, a selection unit 132, an auxiliary processing unit 133, and a robot control unit 134.
  • the communication unit 11 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a LAN (Local Area Network) controller.
  • the communication unit 11 inputs the robot state information received from the robot 2 and the operation information received from the operation device 3 to the information acquisition unit 131.
  • the storage unit 12 includes a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk.
  • the storage unit 12 stores a program executed by the control unit 13.
  • the storage unit 12 stores auxiliary information used in the auxiliary mode.
  • the storage unit 12 may store the reference control information used in the correction mode. Details of the reference control information will be described later.
  • the control unit 13 includes, for example, a CPU (Central Processing Unit), and functions as an information acquisition unit 131, a selection unit 132, an auxiliary processing unit 133, and a robot control unit 134 by executing a program stored in the storage unit 12. .. Details of the information acquisition unit 131, the selection unit 132, the auxiliary processing unit 133, and the robot control unit 134 will be described later.
  • a CPU Central Processing Unit
  • FIG. 6 is a diagram showing a functional configuration of the robot 2.
  • the robot 2 has a communication unit 21, a mechanism unit 22, a sensor unit 23, a storage unit 24, and a control unit 25.
  • the control unit 25 includes an instruction receiving unit 251, a mechanism control unit 252, and a robot state transmitting unit 253.
  • the communication unit 21 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a wireless communication controller.
  • the communication unit 21 receives the control information transmitted from the operation device 3 via the network N. Further, the communication unit 21 transmits the robot state information indicating the state of the robot 2 to the control device 1 via the network N.
  • the mechanical unit 22 includes actuators for each part of the robot 2.
  • the mechanism unit 22 includes a mechanism member that moves under the control of the control unit 25, such as an actuator provided at a joint of each part such as a finger, a hand, a foot, a neck, and a head.
  • the sensor unit 23 is provided in each unit of the robot 2 and includes a device for detecting various states.
  • the sensor unit 23 is, for example, a camera provided in the eyes of the robot 2, a microphone provided in the temporal region of the robot 2, or a temperature sensor and a pressure sensor provided in the fingers of the robot 2.
  • the sensor unit 23 notifies the robot state transmission unit 253 of a signal indicating the state detected by each sensor.
  • the storage unit 24 includes a storage medium such as a ROM, a RAM, and a hard disk.
  • the storage unit 24 stores a program executed by the control unit 25.
  • the control unit 25 has, for example, a CPU, and functions as an instruction receiving unit 251, a mechanism control unit 252, and a robot state transmitting unit 253 by executing a program stored in the storage unit 24.
  • the instruction receiving unit 251 receives the control information transmitted by the control device 1 via the communication unit 21.
  • the instruction receiving unit 251 notifies the mechanism control unit 252 of the received control information.
  • the mechanism control unit 252 controls each actuator included in the mechanism unit 22 based on the control information notified from the instruction receiving unit 251.
  • the mechanism control unit 252 acquires control information including an instruction to move the hand to the right, for example, the mechanism control unit 252 moves the hand of the robot 2 to the right by controlling the actuator provided in the hand.
  • the mechanism control unit 252 may detect the position of each actuator after moving the mechanism unit 22 and notify the robot state transmission unit 253 of the detected position.
  • the robot state transmission unit 253 generates robot state information indicating the state of the robot 2 based on the signal detected by the sensor unit 23 and the state of the mechanism unit 22 notified from the mechanism control unit 252.
  • the robot state transmission unit 253 transmits the generated robot state information to the control device 1 via the communication unit 21.
  • FIG. 7 is a diagram showing a functional configuration of the operating device 3.
  • the operation device 3 includes an operation unit 31, a detection unit 32, a communication unit 33, a display unit 34, a storage unit 35, and a control unit 36.
  • the control unit 36 includes a state specifying unit 361, an instruction transmitting unit 362, a state transmitting unit 363, and a display control unit 364.
  • the operation unit 31 is a device for the operator U to operate the robot 2, and includes, for example, a member worn by the operator U and a button or a joystick operated by the operator U.
  • the operation unit 31 notifies the state specifying unit 361 of the operation content performed by the detected operator U.
  • the detection unit 32 includes a sensor for detecting the state of the operator U.
  • the detection unit 32 detects, for example, a camera that detects the position of the pupil of the operator U, a humidity sensor that detects the sweating state of the operator U, a pulse sensor that detects the heart rate of the operator U, and a blood pressure of the operator U. It includes a blood pressure sensor, a breathing sensor that detects the respiratory rate or exhaled component of the operator U, or an acceleration sensor that detects the movement of each part of the operator U.
  • the detection unit 32 notifies the state identification unit 361 of the detected state of the operator U.
  • the communication unit 33 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a wireless communication controller.
  • the communication unit 33 transmits the control information for the robot 2 generated by the control unit 36 to the robot 2 based on the operation content of the operator U detected by the operation unit 31 via the network N. Further, the communication unit 33 transmits the operator status information indicating the status of the operator U to the control device 1 via the network N.
  • the control unit 36 has, for example, a CPU, and functions as a state identification unit 361, an instruction transmission unit 362, a state transmission unit 363, and a display control unit 364 by executing a program stored in the storage unit 35.
  • the instruction transmitting unit 362 generates operation information for operating the robot 2 based on the operation content notified from the state specifying unit 361.
  • the instruction transmission unit 362 transmits the generated operation information to the control device 1 via the communication unit 33.
  • the state transmission unit 363 generates operator state information indicating the state of the operator U based on the state of the operator U notified from the state identification unit 361.
  • the state transmission unit 363 transmits the generated robot state information to the control device 1 via the communication unit 33.
  • the display control unit 364 creates display data to be displayed on the display unit 34.
  • the display control unit 364 causes the display unit 34 to display, for example, an image captured by the robot 2. Further, the display control unit 364 causes the display unit 34 to display the auxiliary information transmitted from the control device 1 for the operator U to visually recognize.
  • the information acquisition unit 131 acquires operation information indicating the content of the operation by the operator U who operates the robot 2 from the operation device 3.
  • the information acquisition unit 131 may acquire operation information via another device capable of communicating with the operation device 3.
  • the operation information is information including numerical values corresponding to the amount of movement of each part of the robot 2, the direction of movement, or the speed of movement.
  • the operation information is the movement of the operator U related to the operation of the robot 2, such as the amount of movement of each part of the operator U such as a hand, finger, foot, or head, the direction in which each part is moved, or the speed of movement. It is information indicating.
  • the operation information may be a numerical value or text input to the operation device 3 or another device by the operator U.
  • the information acquisition unit 131 inputs the acquired operation information to the selection unit 132 and the auxiliary processing unit 133.
  • the information acquisition unit 131 may further acquire detection information indicating a state around the robot detected by a sensor provided in the robot 2.
  • the sensors are, for example, an image sensor, a temperature sensor, a pressure sensor, and an acceleration sensor.
  • the information acquisition unit 131 inputs the acquired detection information to the selection unit 132 and the auxiliary processing unit 133.
  • the information acquisition unit 131 may acquire various types of information from other devices.
  • the information acquisition unit 131 acquires skill information indicating the operation skill of the operator U from, for example, a computer used by the administrator of the operator U.
  • the operation skill is used to select the mode to be executed by the selection unit 132.
  • the selection unit 132 has one or more auxiliary processes (hereinafter, may be referred to as selection auxiliary processes) from a plurality of auxiliary process candidates for assisting the operation of the operator U as the auxiliary processes to be executed by the auxiliary processing unit 133.
  • Selection auxiliary processes are, for example, a correction process executed in the above-mentioned correction mode, a guidance process executed in the guidance mode, and an automatic process executed in the automatic mode.
  • the selection unit 132 selects, for example, at least one of a correction process for causing the robot control unit 134 to control the robot 2 based on the correction information and a guidance process for displaying auxiliary information for assisting the operation on the operation device 3. ..
  • the selection unit 132 switches between a non-correction mode in which the robot control unit 134 uses the operation information as control information to control the robot and a correction mode in which the robot control unit 134 uses the correction information as control information to control the robot. May be good.
  • the selection unit 132 may display the selected auxiliary processing type on the operation device 3.
  • the operator U can operate the robot 2 while grasping the contents of the auxiliary processing.
  • the operator U can recognize that the cause is the assistance, so that the operation can be performed. It is possible to prevent the person U from feeling uncomfortable.
  • the selection unit 132 notifies the auxiliary processing unit 133 of the selection processing information indicating the selected auxiliary processing. For example, when the correction process is selected, the selection unit 132 notifies the auxiliary processing unit 133 of "01" as the selection processing information, and when the guidance processing is selected, the selection unit 132 notifies the auxiliary processing unit 133 of "10" as the selection processing information. Is notified, and when automatic processing is selected, "11" is notified to the auxiliary processing unit 133 as selection processing information. When the selection unit 132 chooses not to execute the auxiliary processing, the selection unit 132 may notify the auxiliary processing unit 133 of "00" indicating that the normal processing has been selected as the selection processing information. Further details of the operation of the selection unit 132 will be described later.
  • the auxiliary processing unit 133 assists the operator U by executing the auxiliary processing selected by the selection unit 132.
  • the auxiliary processing unit 133 corrects the operation information input from the information acquisition unit 131 to create correction information used as control information for operating the robot 2.
  • the auxiliary processing unit 133 creates correction information by replacing the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator U.
  • the reference control information is, for example, operation information acquired when a person with high operation skill performs an operation.
  • the auxiliary processing unit 133 estimates the content of the operation intended by the operator U based on, for example, the operation information, and associates the content of the estimated operation with the content of the reference control information stored in the storage unit 12. Correct the operation information so that the contents of the operation information come closer.
  • the operation information indicates the operation content of grasping with two fingers.
  • the shape of the object photographed by the robot 2 is spherical, and the reference control information corresponding to the case of grasping the spherical object indicates the operation content of grasping with three fingers.
  • the auxiliary processing unit 133 corrects the operation information based on the reference control information corresponding to the case of grasping the spherical object, and creates the correction information corresponding to the operation of grasping with three fingers.
  • the auxiliary processing unit 133 displays an image for instruction to assist the operator U to be displayed on the operation device 3, a voice for instruction, and a part of the operation device 3 for instruction.
  • Create guidance information that includes at least one of the vibration information to vibrate.
  • the auxiliary processing unit 133 may select one or more images, sounds, and vibration information from a plurality of images, sounds, and vibration information for instruction prepared in advance.
  • the auxiliary processing unit 133 displays an image for instruction to be displayed on the operation device 3 based on at least one of the work content performed by the operator U, the scenario used by the operator U, or the operation information received from the operation device 3. , Voice, create vibration information.
  • the image for instruction is, for example, an image showing a position where each part such as a finger, an arm, a foot, and a head should be moved.
  • the auxiliary processing unit 133 is an image of a color (for example, a color different from the article) indicating a position where the finger should be brought into contact with the image of the article acquired from the robot 2.
  • the instructional voice is a voice that guides the operation method, such as "a little more right" or "the index finger is out of alignment”.
  • the vibration information for guidance is information for giving vibration to the operation device 3 worn by the operator U when, for example, the operator U makes a mistake in the operation.
  • the auxiliary processing unit 133 is for teaching to indicate the position where the finger should be touched, provided that the object to be grasped has a shape that is difficult to grasp when the operation skill of the operator U is high. Create an image.
  • the auxiliary processing unit 133 may determine the content of the instruction image based on the operation information transmitted from the operation device 3.
  • the auxiliary processing unit 133 indicates, for example, an operation to be performed next when it is determined that the movement of the operator U has stopped based on the operation information, or when it is determined that the movement of the operator U has become excessively slow.
  • Create a teaching image The auxiliary processing unit 133 guides the robot 2 to change the operation content when it is determined that the operation content indicated by the operation information is inappropriate based on the operation information and the detection information transmitted from the robot 2. You may create a teaching image for this.
  • the auxiliary processing unit 133 refers to a scenario showing a predetermined operation content and creates control information based on the information included in the scenario.
  • the auxiliary processing unit 133 may specify the state of the robot 2 based on the detection information transmitted from the robot 2, and create control information based on the specified state of the robot 2 and the operation content indicated by the scenario.
  • the auxiliary processing unit 133 determines the reward amount for the work of the operator U based on the content of the selected auxiliary processing, and outputs the determined reward amount to an external device (for example, the information terminal of the operator U). You may.
  • the auxiliary processing unit 133 determines that the smaller the degree of assistance, the higher the reward amount. By determining the reward amount in this way, the auxiliary processing unit 133 can motivate the operator U so that he / she can work without receiving assistance.
  • the robot control unit 134 controls the robot 2 using the control information corresponding to the type of selection assist processing created based on the operation information.
  • the robot control unit 134 transmits the control information generated based on the operation information and notified from the auxiliary processing unit 133 to the robot control unit 134 to the robot 2 while the auxiliary processing unit 133 is executing the selection auxiliary processing. This controls the robot 2.
  • the robot control unit 134 may change the operating speed of the robot 2 when the selection unit 132 selects one or more new auxiliary processes. That is, the robot control unit 134 may change the operating speed of the robot 2 when the content of the auxiliary processing is switched. For example, the robot control unit 134 temporarily stops the operation of the robot 2 at the time of switching the contents of the auxiliary processing. At this time, the robot control unit 134 may make the robot 2 emit a voice saying "switch to automatic operation" to notify people around the robot 2 that the mode is switched. Further, the robot control unit 134 may display the operation device 3 that the mode is switched. By operating the robot control unit 134 in this way, people around the robot 2 can grasp that the operation of the robot 2 changes, so that it is possible to avoid a danger such as a collision with the robot 2. ..
  • the robot control unit 134 may gradually slow down the speed of movement before pausing when switching modes. By operating the robot control unit 134 in this way, it is possible to prevent the robot 2 from dropping an article. Further, in order to prevent the operator U who is visually recognizing the surroundings of the robot 2 through the camera provided on the head of the robot 2 from feeling unwell, even when switching the mode, contrary to the operation of the operator U. The movement of the head may not be paused.
  • the robot control unit 134 may transmit control information to the robot 2 including information for causing the robot 2 to display a display corresponding to the content to be operated by the robot 2. For example, when the right arm of the robot 2 is moved to the right, the robot control unit 134 transmits control information including an instruction to light a light emitting device provided on the right arm of the robot 2. By operating the robot control unit 134 in this way, people around the robot 2 can grasp how the robot 2 moves, so that safety is improved.
  • the robot control unit 134 may transmit control information to the robot 2 including information for causing the robot 2 to display a display indicating which mode it is operating in. For example, the robot control unit 134 lights a blue lamp during operation in the normal mode, lights a red lamp during operation in the automatic mode, and lights a green lamp during operation in the instruction mode. The yellow lamp lights up during operation in the correction mode. By operating the robot control unit 134 in this way, people around the robot 2 can predict what kind of movement the robot 2 can make, so that safety is improved.
  • the robot control unit 134 may control the robot 2 so as to change the angle of view of the image captured by the robot 2 depending on the mode. For example, in the automatic mode, the robot control unit 134 captures an image on the back side of the robot 2, and in the manual mode, it captures a wider range of images than in the automatic mode. This improves operability in the manual mode.
  • the robot control unit 134 before the auxiliary processing unit 133 executes the new auxiliary processing.
  • the robot 2 may be operated so that the posture of the robot 2 matches the posture of the operator U. That is, if the posture of the operator U and the posture of the robot 2 do not match at the timing when the mode is switched, the robot control unit 134 matches the posture of the robot 2 with the posture of the operator U before changing the mode. The robot 2 may be operated so as to do so.
  • the robot control unit 134 causes the operation device 3 to display information for instructing the operator U to stand by in a specific posture when returning to the manual mode after operating in the automatic mode.
  • the robot control unit 134 causes the operation device 3 to display information permitting the restart of the operation.
  • selection operation by selection unit 132 Subsequently, the details of the operation in which the selection unit 132 selects the mode will be described.
  • the selection unit 132 may select a mode by combining various factors described below.
  • the selection unit 132 selects an auxiliary process to be executed, for example, based on the operation skill of the operator U.
  • the selection unit 132 may select an auxiliary process to be executed using the operation skill indicated by the skill information acquired by the information acquisition unit 131, and identifies the operation skill based on the operation information acquired by the information acquisition unit 131.
  • Auxiliary processing may be selected using the specified operation skill.
  • the selection unit 132 may use the operation skill stored in the storage unit 12 in association with the operator U.
  • the selection unit 132 selects the normal mode or the instruction mode when the operator U having a high operation skill operates the robot 2, and corrects when the operator U having a low operation skill operates the robot 2. Select mode or automatic mode.
  • the selection unit 132 may select an auxiliary process to be executed based on the content of the operation indicated by the operation information.
  • the selection unit 132 switches between the non-correction mode and the correction mode, for example, based on the content of the operation indicated by the operation information.
  • the selection unit 132 selects, for example, the non-correction mode for the movement of moving the arm, and selects the correction mode for the movement of grasping an object.
  • the selection unit 132 may select the non-correction mode for the movement of moving the arm in a wide place, and may select the correction mode for the movement of moving the arm in a place where there is an obstacle in the vicinity.
  • the selection unit 132 may select the automatic mode when fine movement is required.
  • the selection unit 132 may specify the operation skill of the operator U for each operation content and select an auxiliary process to be executed based on the operation skill of the operator U for each operation content. For example, when the operator U has a high skill in carrying a box but a low skill in grasping a spherical object, the selection unit 132 selects a normal mode in the operation of carrying the box and has a spherical shape. The correction mode may be selected in the operation of grasping an object.
  • the selection unit 132 may select an auxiliary process to be executed based on the difficulty level determined for each operation content.
  • the selection unit 132 executes auxiliary processing corresponding to the difficulty level of the operation by referring to a table previously stored in the storage unit 12 showing the relationship between the content of the operation and the difficulty level of the operation.
  • the selection unit 132 selects a normal mode while causing the robot 2 to perform an operation of moving an arm to a predetermined position, and then performs an operation of grasping an object having a complicated shape after moving the arm to a predetermined position.
  • the instruction mode or the correction mode is selected while the robot is executed.
  • the selection unit 132 may select the instruction mode, the correction mode, or the automatic mode while moving the arm.
  • the selection unit 132 may select an auxiliary process to be executed based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other.
  • the operator U operates the robot 2 based on the scenario, the operator U needs to perform various types of operations in order to execute various operations included in one scenario.
  • the selection unit 132 switches the operation mode based on the type of each work included in the scenario, the difficulty level of each work, and the like. For example, the selection unit 132 selects a non-correction mode for some operations among a plurality of operations indicated by a plurality of operation information acquired by the information acquisition unit 131, and a correction mode for some other operations. Select. By operating the selection unit 132 in this way, when the scenario includes various operations, the operator U can appropriately perform a series of operations included in the scenario.
  • the information acquisition unit 131 displays auxiliary level information indicating the degree of assistance input in the information terminal after displaying a screen for selecting the degree of assistance on the information terminal as information indicating a plurality of candidates for auxiliary processing. You may get it.
  • the degree of assistance is represented by, for example, the ratio of the time during which the operator U receives the correction process or the instruction process in the scheduled work time.
  • the selection unit 132 inputs the operation information acquired by the information acquisition unit 131 while the operator U performs a predetermined operation into the machine learning model that outputs one or more auxiliary processes when the operation information is input, and the machine learning model.
  • One or more auxiliary processes output by may be selected as auxiliary processes to be executed.
  • the machine learning model is a model learned by using as teacher data data in which a large number of operation information and the contents of auxiliary processing selected for each operation information are associated with each other, for example, the storage unit 12 or the outside. It is stored in the computer.
  • the teacher data is, for example, data in which the operation information acquired when a large number of people have operated the robot 2 in the past and the content of the auxiliary processing selected by the operator U at the time of the operation are associated with each other.
  • the teacher data includes positive teacher data including a combination of operation information and auxiliary processing contents when a good operation result is obtained, and operation information and auxiliary processing contents when a bad operation result is obtained. It may include negative teacher data including a combination with.
  • the selection unit 132 may have a learning processing function for training the machine learning model.
  • the selection unit 132 causes the machine learning model to learn by inputting the operation information and the content of the auxiliary process into the machine learning model each time the content of the auxiliary process is determined.
  • the selection unit 132 transmits the teacher data in which the operation information and the content of the auxiliary processing are associated with the external computer to the external computer via the communication unit 11, thereby transmitting the machine. Let the learning model train.
  • the selection unit 132 may select the operation mode based on the communication quality of the network N.
  • the communication quality of the network N is poor, it is possible that the operation information indicating the content of the operation performed by the operator U may not be normally transmitted to the control device 1. Therefore, the selection unit 132 may select the automatic mode when the quality of communication between the operation device 3 and the control device 1 deteriorates to a level at which the operation of the robot 2 is hindered.
  • the selection unit 132 may change the operation mode based on the communication quality even when the operation mode is set by the operator U. In this case, the selection unit 132 may display to the operation device 3 that the mode has been changed to the automatic mode due to poor communication quality.
  • Control device 2 Robot 3 Operation device 11 Communication unit 12 Storage unit 13 Control unit 21 Communication unit 22 Mechanism unit 23 Sensor unit 24 Storage unit 25 Control unit 31 Operation unit 32 Detection unit 33 Communication unit 34 Display unit 35 Storage unit 36 Control unit 131 Information acquisition unit 132 Selection unit 133 Auxiliary processing unit 134 Robot control unit 251 Instruction reception unit 252 Mechanism control unit 253 Robot status transmission unit 361 State identification unit 362 Instruction transmission unit 363 Status transmission unit 364 Display control unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)

Abstract

Dispositif de commande (1) comprenant : une section d'acquisition d'informations (131) destinée à acquérir des informations d'opération indiquant le contenu d'une opération réalisée par un opérateur (U) actionnant un robot (2); une section de sélection (132) permettant de sélectionner un ou plusieurs processus d'aide parmi une pluralité de processus d'aide candidats pour assister l'opération réalisée par l'opérateur; une section de traitement d'aide (133) destinée à aider l'opérateur (U) en exécutant un processus d'aide sélectionné qui correspond au(x) processus d'aide sélectionné(s) par la section de sélection (132); et une section de commande de robot (134) servant à commander le robot (2) à l'aide d'informations correspondant à un type du processus d'aide sélectionné créé sur la base des informations d'opération tandis que la section de traitement d'aide (133) exécute le processus d'aide sélectionné.
PCT/JP2020/006260 2019-03-20 2020-02-18 Dispositif de commande et procédé de commande Ceased WO2020189143A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-052443 2019-03-20
JP2019052443 2019-03-20

Publications (1)

Publication Number Publication Date
WO2020189143A1 true WO2020189143A1 (fr) 2020-09-24

Family

ID=72520169

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/006260 Ceased WO2020189143A1 (fr) 2019-03-20 2020-02-18 Dispositif de commande et procédé de commande

Country Status (1)

Country Link
WO (1) WO2020189143A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022196011A1 (fr) * 2021-03-15 2022-09-22 オムロン株式会社 Instrument pouvant être porté sur soi
CN115225682A (zh) * 2021-04-15 2022-10-21 本田技研工业株式会社 管理服务器、远程操作系统、远程操作方法以及存储介质
JP2024033189A (ja) * 2022-08-30 2024-03-13 本田技研工業株式会社 遠隔操作補助システム、遠隔操作補助方法、およびプログラム

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018096902A1 (fr) * 2016-11-22 2018-05-31 パナソニックＩｐマネジメント株式会社 Système de capture et son procédé de commande
WO2018212226A1 (fr) * 2017-05-17 2018-11-22 Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社 Dispositif de commande, procédé de commande de robot et système de commande de robot

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018096902A1 (fr) * 2016-11-22 2018-05-31 パナソニックＩｐマネジメント株式会社 Système de capture et son procédé de commande
WO2018212226A1 (fr) * 2017-05-17 2018-11-22 Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社 Dispositif de commande, procédé de commande de robot et système de commande de robot

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022196011A1 (fr) * 2021-03-15 2022-09-22 オムロン株式会社 Instrument pouvant être porté sur soi
JP2022141177A (ja) * 2021-03-15 2022-09-29 オムロン株式会社 ウェアラブル機器
CN115225682A (zh) * 2021-04-15 2022-10-21 本田技研工业株式会社 管理服务器、远程操作系统、远程操作方法以及存储介质
CN115225682B (zh) * 2021-04-15 2024-01-19 本田技研工业株式会社 管理服务器、远程操作系统、远程操作方法以及存储介质
US12583120B2 (en) 2021-04-15 2026-03-24 Honda Motor Co., Ltd. Management server, remote operation system, remote operation method, and storage medium
JP2024033189A (ja) * 2022-08-30 2024-03-13 本田技研工業株式会社 遠隔操作補助システム、遠隔操作補助方法、およびプログラム
JP7809033B2 (ja) 2022-08-30 2026-01-30 本田技研工業株式会社 遠隔操作補助システム、遠隔操作補助方法、およびプログラム

Similar Documents

Publication Publication Date Title
EP3975909B1 (fr) Systèmes et procédés de commande de mode de fonctionnement pour système chirurgical assisté par ordinateur
KR20230037147A (ko) 비접촉 응시 기반 헤딩 정보를 이용한 사용자 명령 인식 장치 및 이를 이용한 방법
WO2020189143A1 (fr) Dispositif de commande et procédé de commande
WO2017033367A1 (fr) Système de robot commandé à distance
CN121116061A (zh) 用于通过眼睛反馈实现通信的系统和方法
JP6978453B2 (ja) 管理装置、管理方法及び管理システム
CN107798733A (zh) 显示系统、便携信息设备、可佩戴型终端和信息显示方法
JP2019184904A (ja) 操作訓練システム
KR20130101395A (ko) 실감 인터랙션을 이용한 인지재활 시스템 및 방법
US20180316911A1 (en) Information processing apparatus
US20190355281A1 (en) Learning support system and recording medium
JP2018045059A (ja) 教育支援装置
US10747308B2 (en) Line-of-sight operation apparatus, method, and medical device
JP2020019127A (ja) 協調動作支援装置
US20200179077A1 (en) Control device, control method, and surgical system
JP6834620B2 (ja) 情報表示システム
US12242256B2 (en) Systems and methods for providing time-based user experiences for an industrial automation system
US11829126B2 (en) Systems and methods for providing context-based data for an industrial automation system based on a virtual coordinate system
WO2010142455A2 (fr) Procédé pour déterminer la position d'un objet dans une image, l'expression du visage d'une personne et procédé pour commander un dispositif d'entrée en fonction de la détection de l'expression ou du regard
CN109558004B (zh) 一种人体辅助机器人的控制方法及装置
US11651528B2 (en) Systems and methods for providing context-based data for an industrial automation system
JP2019164216A (ja) 指示位置伝達システムおよび指示位置伝達方法
WO2020082181A1 (fr) Téléguidage précis d'êtres humains
JP2019202354A (ja) ロボット制御装置、ロボット制御方法及びロボット制御プログラム
US12064180B2 (en) Display apparatus, display method, and display program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20773492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20773492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP