US20140100850A1 - Method and apparatus for performing preset operation mode using voice recognition - Google Patents

Method and apparatus for performing preset operation mode using voice recognition Download PDF

Info

Publication number
US20140100850A1
US20140100850A1 US13/954,432 US201313954432A US2014100850A1 US 20140100850 A1 US20140100850 A1 US 20140100850A1 US 201313954432 A US201313954432 A US 201313954432A US 2014100850 A1 US2014100850 A1 US 2014100850A1
Authority
US
United States
Prior art keywords
text
voice
preset operation
input
displayed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/954,432
Other languages
English (en)
Inventor
Sung-Joon Won
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Assigned to SAMSUNG ELECTRONICS CO., LTD. reassignment SAMSUNG ELECTRONICS CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: WON, SUNG-JOON
Publication of US20140100850A1 publication Critical patent/US20140100850A1/en
Priority to US16/247,987 priority Critical patent/US10825456B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233—Character input methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0486—Drag-and-drop
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/221—Announcement of recognition results
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Definitions

  • the present invention relates generally to voice recognition technology used for a terminal, and more particularly, to a method and an apparatus for performing a preset operation mode using voice recognition which recognizes an input voice command to perform a preset operation mode of a terminal.
  • terminals are implemented as multimedia devices having complex functions such as photographing pictures or videos, reproducing music or video files, playing games, receiving broadcasting, and executing applications.
  • terminals such as smart phones and tablet PCs are provided with touch screens to execute various functions of the terminals by inputting text through a touch input, scrolling, and dragging.
  • Korean Patent Publication No. 2009-0020265 discloses a function of modifying a message to which a voice signal is applied, such as inputting, moving, deleting, modifying, and searching for a message through voice recognition.
  • Korean Patent Publication No. 2010-0064875 discloses a function of converting a user's voice to a text through voice recognition and then displaying the text, and a function of executing a text modifying operation by selecting a user's voice, a touch, or a key input from displayed command lists through a touch of a part to be modified.
  • voice recognition through which operations are performed is utilized.
  • Advanced voice recognition technologies are still being researched, but there is still no perfect voice recognition technology which can precisely recognize a voice. Accordingly, when a preset main operation is executed by applying voice recognition, the operation is occasionally not performed due to an incorrectly recognized voice, and users are inconvenienced by these errors. That is, when an error is generated in the voice recognition corresponding to the main operation, many more control steps may be required, and accordingly, it takes a lot of time to correctly perform the operation. Therefore, when the main operation is to be executed by applying the voice recognition technology, it may be somewhat difficult to conveniently and widely apply voice recognition.
  • the present invention has been made to address at least the problems and/or disadvantages described above, and to provide at least the advantages described below.
  • an aspect of the present invention is to provide a method and an apparatus for performing a preset operation mode by using voice recognition, which can reduce inconvenience due to an error of the voice recognition which can be generated when a main action is to be executed by applying the voice recognition.
  • a method of performing a preset operation by using voice recognition includes performing the preset operation of a preset operation mode according to a key input or a touch input in the preset operation mode; and recognizing an input voice during performance of the preset operation of the preset operation mode and assisting the performance of the preset operation according to the recognized voice.
  • an apparatus for performing a preset operation by using voice recognition including an input/output module including at least one button and a physical or virtual keypad configured to receive a control input of a user, and a microphone receiving an input voice from the user; a touch screen configured to receive a control input of the user and display an execution image, an operation state, and a menu state of an application program; and a controller configured to control the input/output module and the touch screen, the controller including a voice recognition module for recognizing the voice input by the user through the microphone of the input/output module, the controller further configured to perform the preset operation according to a key input or a touch input from the touch screen, and apply a recognized voice of the user received from the voice recognition module to assist performance of the preset operation.
  • an input/output module including at least one button and a physical or virtual keypad configured to receive a control input of a user, and a microphone receiving an input voice from the user
  • a touch screen configured to receive a control input of the user and display an execution image, an
  • FIG. 1 is a block diagram of a mobile device according to the present invention.
  • FIG. 2 is a flowchart illustrating a process of applying performance of a preset operation mode by using voice recognition according to an embodiment of the present invention
  • FIG. 3 is a flowchart illustrating a process of modifying a text and assisting a preset operation by using voice recognition according to a first embodiment of the present invention
  • FIGS. 4A to 4C illustrate an example of assisting a text modification by using voice recognition according to the first embodiment of the present invention
  • FIGS. 5A to 5C illustrate an example of assisting a preset operation by using voice recognition according to a second embodiment of the present invention
  • FIGS. 6A to 6C illustrate an example of assisting a preset operation by using voice recognition according a third embodiment of the present invention
  • FIG. 7 is a flowchart illustrating a process of assisting a preset operation and a text modification by using the voice recognition according to the second embodiment of the present invention.
  • FIG. 8 is a flowchart illustrating a process of assisting a text modification by using the voice recognition according to the third embodiment of the present invention.
  • FIG. 9 is a flowchart illustrating a process of assisting an input operation in a list menu by using voice recognition according to a fourth embodiment of the present invention.
  • FIGS. 10A to 10C illustrate an example of assisting the input operation in the list menu by using the voice recognition according to the fourth embodiment of the present invention
  • FIGS. 11A to 11C illustrate an example of assisting an input operation in a list menu by using voice recognition according to a fifth embodiment of the present invention
  • FIGS. 12A to 12C illustrate an example of assisting an input operation in a list menu by using voice recognition according to a sixth embodiment of the present invention
  • FIG. 13 is a flowchart illustrating a process of assisting a home screen editing by using the voice recognition according to the fifth embodiment of the present invention.
  • FIGS. 14A to 14E illustrate an example of assisting a home screen editing by using voice recognition according to the seventh embodiment of the present invention.
  • FIG. 1 a block diagram of a mobile device according to an embodiment of the present invention.
  • a device 100 includes a display unit 190 and a display controller 195 . Further, the device 100 may include a controller 110 , a mobile communication module 120 , a sub communication module 130 , a multimedia module 140 , a camera module 150 , a GPS module 155 , an input/output module 160 , a sensor module 170 , a storage unit 175 , and power supplier 180 .
  • the sub communication module 130 includes at least one of a wireless LAN module 131 and a Near Field Communication (NFC) module 132
  • the multimedia module 140 includes at least one of a broadcasting communication module 141 , an audio reproduction module 142 , and video reproduction module 143 .
  • NFC Near Field Communication
  • the camera module 150 includes at least one of a first camera 151 and a second camera 152 .
  • the controller 110 controls the mobile communication module 120 , the sub communication module 130 , the multimedia module 140 , the camera module 150 , the GPS module 155 , the input/output module 160 , the sensor module 170 , the storage unit 175 , the power supplier 180 , the touch screen 190 , and the touch screen controller 195 .
  • the controller 110 includes a voice recognition module 111 which recognizes a voice input from a microphone 162 of the input/output module 160 .
  • the controller 110 receives a control of the user from the input/output module 160 or the touch screen 190 to perform a preset operation, and assists performance of the preset operation by receiving a voice of the user from the voice recognition module 111 and applying the recognized voice.
  • the control of the user from the input/output module 160 can be received by the controller 110 through keypad 166 .
  • the mobile communication module 120 connects the mobile communication device 100 with an external device by using one or a plurality of antennas (not shown) according to a control of the controller 110 .
  • the mobile communication module 120 transmits/receives a radio signal for voice call, a video call, a Short Message Service (SMS), or a Multimedia Message Service (MMS) with a mobile phone (not shown), a smart phone (not shown), a tablet PC, or another device (not shown) which has a phone number input to the device 100 .
  • SMS Short Message Service
  • MMS Multimedia Message Service
  • the wireless LAN module 131 of the sub communication module 130 can be Internet-connected according to a control of the controller 110 in a place where a wireless Access Point (AP) (not shown) is installed.
  • the wireless LAN module 131 supports a wireless LAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers.
  • the NFC module 132 can wirelessly perform near field communication between the portable terminal 100 and an image forming apparatus (not shown) according to a control of the controller 110 .
  • the device 100 includes at least one of the mobile communication module 120 , the wireless LAN module 131 , and the NFC module 132 .
  • the device 100 includes a combination of the mobile communication module 120 , the wireless LAN module 131 , and the NFC module 132 according to the capability.
  • the multimedia module 140 includes the audio reproduction module 142 and the video reproduction module 143 , and may or may not include the broadcasting communication module 141 . Further, the audio reproduction module 142 or the video reproduction module 143 of the multimedia module 140 may be included in the controller 110 .
  • the input/output module 160 includes a plurality of buttons 161 , a microphone 162 , and a keypad 166 .
  • the button 161 may be formed on a housing of the mobile device 100 .
  • the microphone 162 receives a voice or a sound to generate an electrical signal according to a control of the controller 110 .
  • the keypad 166 receives a key input from the user for the control of the device 100 .
  • the keypad 166 includes a physical keypad (not shown) formed in the device 100 or a virtual keypad (not shown) displayed on the display unit 190 .
  • the physical keypad (not shown) formed in the device 100 may be excluded according to capability or a structure of the device 100 .
  • the storage unit 175 can store a signal or data input/output according to operations of the mobile communication module 120 , the sub communication module 130 , the multimedia module 140 , the camera module 150 , the GPS module 155 , the input/output module 160 , the sensor module 170 , and the display unit 190 .
  • the storage unit 175 can store a control program and applications for controlling the device 100 or the controller 110 .
  • the term “storage unit” includes the storage unit 175 , a ROM 112 and a RAM 113 within the controller 110 , or a memory card (not shown) (for example, an SD card and a memory stick) installed in the device 100 .
  • the storage unit may include a nonvolatile memory, a volatile memory, a Hard Disk Drive (HDD), or a Solid State Drive (SSD).
  • HDD Hard Disk Drive
  • SSD Solid State Drive
  • the touch screen 190 receives a control of the user and displays an execution image, an operation state, and a menu state of an application program.
  • the touch screen 190 provides the user with a user interface corresponding to various services (for example, phone communication, data transmission, broadcasting, taking a picture, etc.).
  • the touch screen 190 transmits an analog signal corresponding to at least one touch input to the user interface to the touch screen controller 195 .
  • the touch screen 190 receives at least one touch through a body part of the user (for example, fingers including a thumb) or a touchable input device. Further, the touch screen 190 receives successive actions of one touch among at least one touch.
  • the touch screen 190 transmits the analog signal corresponding to successive actions of the input touch to the touch screen controller 195 .
  • the touch screen 190 may be implemented, for example, in a resistive type, a capacitive type, an infrared type, or an acoustic wave type.
  • the touch screen controller 195 converts the analog signal received from the touch screen 190 to a digital signal (for example, X and Y coordinates) and then transmits the digital signal to the controller 110 .
  • the controller 110 controls the touch screen 190 by using the digital signal received from the touch screen controller 195 .
  • the controller 110 can enable a shortcut icon (not shown) displayed on the touch screen 190 to be selected or executed in response to the touch.
  • the touch screen controller 195 may be included in the controller 110 .
  • the touch screen 190 may include at least two touch screen panels which detect a touch or an approach of a user's body part or a touchable input device in order to simultaneously receive inputs of the user's body part and the touchable input device.
  • the at least two touch screen panels provide different output values to the touch screen controller 195 , and the touch screen controller 195 differently recognizes the values input from the at least two touch screen panels to determine whether the input from the touch screen is an input by the user's body part or an input by the touchable input device.
  • a preset operation is selected from a plurality of preset operation modes of the device 100 through a user's control input using one of the button 161 , the keypad 166 , and the microphone 162 of the input/output module 160 , or the touch screen 190 , and then the selected preset operation mode is executed in step 200 .
  • a key input or touch input by the user for executing the preset operation of the preset operation mode is performed.
  • step a preset operation according to the touch input in step 202 is performed.
  • the preset operation mode includes various operation modes executed by a user's control input, such as, for example, a text writing mode, a list menu execution mode, and a home screen execution mode. Further, the preset operation includes a text input, a touch input, and scrolling and dragging applying the touch input.
  • touch means an operation in which the user contacts a particular area of the touch screen by using a body part or a touchable input device and then removing the body part or touchable input device which contacts the particular area, or a flick action in which the user contacts a particular area of the touch screen by using a body part or a touchable input device and then removes the body part or touchable input device in a movement direction on the terminal.
  • FIG. 3 is a flowchart illustrating a process of modifying a text and assisting a preset operation by using voice recognition according to the first embodiment of the present invention. An operation of simultaneously inputting a text and a voice in the text writing mode and assisting a modification of the input text by using the input voice will be described.
  • the text writing mode is executed through a user's control input in step 300 , and a voice recognition function of the voice recognition module 111 is activated in step 302 .
  • the voice recognition function of the voice recognition module 111 may be automatically activated simultaneously with the execution of the text writing mode or may be activated by a user's selection.
  • the input text is displayed on a preset text display window (i), as seen in FIG. 4A . If not, the whether there is a voice input is checked in step 308 .
  • the remaining steps correspond to the second process of FIG. 2 of recognizing the voice input during the execution of the preset operation of the preset operation mode of the first process of FIG. 2 to assist performance of the operation of the first process according to the recognized voice.
  • the recognized voice is converted to the text in step 314 , and the voice converted text is displayed as seen in FIG. 4A on a preset voice assist window (j), in step 315 .
  • the voice assist window (j) is set to be distinguished from the text display window (i), and is located and displayed adjacent to the text display window (i) at an upper, lower, left or right side.
  • step 319 The user then selects whether to end the text writing mode in step 319 .
  • the text writing mode does not end and the process returns to step 304 .
  • the user selects to end the text writing mode the text writing mode ends.
  • step 312 if the input text is identical to the recognized voice, that is, there is no error, steps 320 to 324 are performed, which describe performance of a preset operation command. That is, as a result of the determination where the input text has the error in step 312 , when the displayed text has no error and a recognized voice is a preset operation command, steps 320 to 324 describe the performance of the preset operation command by applying the recognized voice.
  • step 320 it is determined whether the recognized voice is the preset operation command in step 320 .
  • the recognized voice is the preset operation command
  • the preset operation command is performed by applying the recognized voice in step 322 .
  • a result of the operation is output in step 324 .
  • performance of the operation command should not interrupt further inputting of the text and displaying the input text. That is, the text input can be performed simultaneously with the voice input and recognition of the input voice.
  • the text is input while the preset operation command of the input and recognized voice is performed, the text can be displayed.
  • the preset operation command is not applied when the voice recognition is applied.
  • step 324 the user selects whether to end the text writing mode in step 319 .
  • the user also selects whether to end the text writing mode in step 319 if the recognized voice has no similarity to the displayed text in step 320 .
  • the text writing mode does not end and the process returns to step 304 .
  • the text writing mode ends.
  • FIGS. 4A to 4C illustrate an example of assisting the text writing for modification of the text by using voice recognition according to the first embodiment of the present invention.
  • the text is input through the physical or virtual keypad 166 .
  • a voice having the same contents as the text is input through the microphone 162 by the user reading the text aloud.
  • the input text is displayed on the text display window (i), and the input voice is converted to text and displayed on the voice assist window (j).
  • the text displayed on the voice assist window (j) corresponds to an image generated by recognizing the input voice through the voice recognition, converting the voice to text, and then displaying the text on the voice assist window (j).
  • the text input while the voice is input by the user may be preset to be highlighted or the input text may be preset to be distinguished from the voice of the user through an underline or highlight.
  • the preset function may be a preset item that allows the user to select an application which changes the contents of the text display window (i) to the contents of the voice assist window (j), the preset function may be set to be displayed on the touch screen 190 as a popup window, or the preset function may be a preset voice command.
  • FIG. 4C illustrates an image generated by changing the contents of the text display window (i) to the contents of the voice assist window (j).
  • FIGS. 5A to 5C illustrate an example of assisting a preset operation using voice recognition according to a second embodiment of the present invention. An operation of assisting performance of the preset operation during a text input by using the voice recognition will be described in detail with reference to FIG. 5 .
  • FIG. 5A illustrates an image on the terminal including the text display window (i) and the voice assist window (j), prior to the input of the voice.
  • FIG. 5C illustrates an example of a displayed result image generated by executing a map showing a user's position which is an output result of the input text displayed on the text display window (i) after the preset operation command is completely executed.
  • FIGS. 6A to 6C illustrate an example of assisting a preset operation by using voice recognition according to a third embodiment of the present invention. An operation of assisting the preset operation by using the voice recognition while the text is input will be described in detail with reference to FIG. 6 .
  • the input voice is recognized using voice recognition, and the operation command is performed according to the recognized voice of “Move a cursor behind ‘store’”.
  • FIG. 6B illustrates an image generated by moving the position of a cursor according to the operation command of the recognized voice of “Move a cursor behind ‘store’”.
  • FIG. 6C illustrates an image showing a result generated by inputting the text “right now” in a position where the cursor is located according to the recognized operation command of “Input ‘right now’”.
  • the preset operation command in the text writing mode may be set to operate functions preset to the terminal by the input voice, such as reproduction of music or a video, searching the Internet, and an execution of a particular application. Further, a result of the operation may be directly displayed on the text window or may appear as a popup window.
  • FIG. 7 is a flowchart illustrating a process of assisting a preset operation and a text modification by using voice recognition according to the second embodiment of the present invention. An operation of simultaneously inputting a voice and a text in the text writing mode, performing a preset operation by using the input voice and assisting a modification of the input text by using the input voice will be described.
  • steps 400 to 406 are steps for performing a preset operation according to a key input or a touch input in the preset operation mode of the first process of FIG. 2 , which describe an operation of executing the text writing mode and displaying the input text on a preset text display window.
  • the text writing mode is executed through a user control input in step 400 , and a voice recognition function of the voice recognition module 111 is activated in step 402 .
  • the voice recognition function of the voice recognition module 111 may be automatically activated simultaneously with the execution of the text writing mode or may be activated by a user's selection.
  • the input text is displayed on a preset text display window (i) in step 406 . If not, voice input is determined in step 408 .
  • the remaining steps are processes corresponding to the second process of FIG. 2 of recognizing the voice input during the execution of the preset operation of the preset operation mode as described in FIG. 2 to assist performance of the operation of the first process according to the recognized voice.
  • a voice is input through the microphone 162 in step 408 .
  • the voice input at this time can be made in all cases such as where the text is not input, the text is being input, or the text is input and then displayed in the text writing mode.
  • all these cases, where the text is not input, the text is being input, or the text is input and then displayed in the text writing mode, will be described with reference to the case where the text is being input.
  • the activated voice recognition module 111 recognizes the input voice in step 410 . If there is no voice input, the user determines whether to end the text writing mode at step 418 .
  • Steps 412 to 416 describe performance of the preset operation command.
  • the preset operation command is performed by applying the recognized voice.
  • the recognized voice is the preset operation command in step 412 .
  • the preset operation command is performed by applying the recognized voice in step 414 .
  • a result of the performance of the operation is output in step 416 .
  • the performance of the operation command should not interrupt the inputting of the text and the displaying of the input text. That is, the text input can be performed simultaneously with the voice input and recognition of the input voice. Further, when the text is input while the preset operation command of the input and recognized voice is performed, the text can be displayed.
  • the user selects whether to end the text writing mode in step 418 .
  • the process returns to step 404 .
  • the text writing mode ends.
  • Steps 420 to 429 describe the operation of assisting the modification of the text. That is, in step 412 , if it is determined that the recognized voice is not the preset operation command, the possibility of an error in the input text is analyzed by comparing the displayed text with the recognized voice in step 420 . When the displayed text is not identical to the recognized voice, it is determined that the displayed text has an error, and thus the recognized voice is converted to the text and a modification of the displayed text is assisted.
  • step 422 it is determined whether the input text has an error in step 422 . That is, when the displayed text is not identical to the recognized voice through the comparison between the displayed text and the recognized voice, it is determined that the displayed text has an error.
  • the recognized voice is converted to text in step 424 , and the voice converted text is displayed on the preset voice assist window (j) in step 426 .
  • the user identifies the contents of the text display window (i) and contents of the voice assist window (j) and determines whether to change the contents of the text display window (i) to the contents of the voice assist window (j) in step 428 .
  • the user When the user desires to change the contents of the text display window (i) to the contents of the voice assist window (j), the user converts the text displayed on the text display window (i) into the voice converted text displayed on the voice assist window (j) by applying a preset function to change the contents of the text display window (i) into the contents of the voice assist window (j) through a user's control input in step 429 , which may be done by any of buttons 161 , keypad 166 , or a voice command input through microphone 162 .
  • the preset function for changing the contents of the text display window (i) into the contents of the voice assist window (j) may be set as an item in the text writing mode, a popup window on the touch screen 190 , or an input of a preset voice command.
  • the preset operation command is not applied when the voice recognition is applied.
  • step 429 the user selects whether to end the text writing mode in step 418 .
  • the process returns to step 404 .
  • the text writing mode ends.
  • step 422 if there is an error in the text, the user selects whether to end the text writing mode in step 418 . Also, in step 428 , if the user decides not to change the contents of text display window (i) to the contents of voice assist window (j), the user selects whether to end the text writing mode in step 418 .
  • FIG. 8 is a flowchart illustrating a process of assisting a text modification by using voice recognition according to a third embodiment of the present invention. An operation of simultaneously inputting a text and a voice in the text writing mode and assisting an error modification of the input text by using the input voice will be described.
  • FIG. 8 The operation of FIG. 8 is identical to the operation of assisting the modification of the input text by using the input voice in the text writing mode of FIG. 3 . Accordingly, the operation of assisting the modification of the text by using voice recognition will briefly described.
  • the text writing mode is executed through a user control input in step 500 , and a voice recognition function of the voice recognition module 111 is activated in step 502 . Thereafter, it is determined whether the text is input through the physical or virtual keypad 166 according to a key input or a touch input in step 504 . When the text is input, the input text is displayed on a preset text display window (i) in step 506 . If not, voice input is determined in step 508 .
  • step 508 it is determined whether a voice is input through the microphone 162 in step 508 .
  • the activated voice recognition module 111 recognizes the input voice in step 510 . If there is no voice input, the user determines whether to end the text writing mode in step 520 .
  • the possibility of the error of the input text is analyzed by comparing the displayed text with the recognized voice in step 511 , and it is determined whether the input text has an error in step 512 . That is, when the displayed text is not identical to the recognized voice through the comparison between the display text and the recognized voice, it is determined that the displayed text has an error. When the displayed text is identical to the recognized voice, it is determined that the displayed text has no error, and the user determines whether to end the text writing mode in step 520 .
  • the recognized voice is converted to text in step 514 , and the voice converted text is displayed on a preset voice assist window (j) in step 515 .
  • the user identifies contents of the text display window (i) and contents of the voice assist window (j) and determines whether to change the contents of the text display window (i) to the contents of the voice assist window (j) in step 517 .
  • the user When the user desires to change the contents of the text display window (i) to the contents of the voice assist window (j), the user converts the text displayed on the text display window (i) into the voice converted text displayed on the voice assist window (j) by applying a preset function to change the contents of the text display window (i) to the contents of the voice assist window (j) through a user's control input in step 519 , as described above.
  • the user selects whether to end the text writing mode in step 520 .
  • the text writing mode does not end and the process returns to step 504 .
  • the user selects to end the text writing mode the text writing mode ends.
  • FIG. 9 is a flowchart illustrating a process of assisting an input operation in a list menu by using voice recognition according to a fourth embodiment of the present invention. An operation of simultaneously inputting a preset operation command and a voice in a screen displaying the list and assisting an execution of the preset operation command by using the input voice will be described.
  • steps 600 to 603 are steps for performing the preset operation of the operation mode according to a key input or a touch input in the preset operation mode in the first process of FIG. 2 .
  • the list menu is executed through a user control input in step 600 , and lists of the executed list menu are displayed in step 601 .
  • the voice recognition function of the voice recognition module 111 may be automatically activated or may be activated by a user's selection, using buttons 161 , keypad 166 or microphone 162 .
  • an operation of the touch input is executed in step 603 .
  • the touch input at this time is a scrolling touch input, which corresponds to an input of a flick operation in which the user contacts a particular area of displayed lists by using a body part or a touchable input device and then removes the body part or the touchable input device from the particular area in a movement direction.
  • the displayed lists may be scrolled in an up, down, left, or right direction according to a scrolling direction.
  • the remaining steps correspond to the second process of FIG. 2 of recognizing the voice input during the execution of the preset operation of the operation mode of the first process of FIG. 2 in which an operation of assisting a touch operation input in a list menu is described.
  • step 604 it is determined whether a voice is input through the microphone 162 . If not, the user determines whether to end the list menu mode in step 609 .
  • the input voice at this time can be made in all cases, such as where the touch is not input, the touch is being input, and a preset operation command is being performed during the touch input when the lists of the list menu are displayed.
  • all these cases where the touch is not input, the touch is being input, or the preset operation command is being performed during the touch input when the lists of the list menu are displayed will be described with reference to the case where the touch operation is being performed.
  • the activated voice recognition module 111 recognizes the input voice in step 605 .
  • the recognized voice is the preset operation command in step 606 . If not, the user determines whether to end the list menu mode in step 609 . A recognized voice which has no similarity with the preset operation command is not applied.
  • the recognized voice is the preset operation command
  • the preset operation command of the recognized voice is performed during performance of the touch operation in step 607 , and a result of the execution of the operation command is output in step 608 .
  • the preset operation command may be a command set to automatically perform the scrolling operation from the display list to a desired position in the preset up, down, left, or right direction.
  • the command set to automatically perform the scrolling operation from the list to the desired position may include a command set to automatically perform a scrolling operation to a position of one or more words, a character string, or a phrase, a position of a part of the all lists, and a position of a language for each country.
  • step 609 it is determined whether to end the list menu through a user's selection in step 609 .
  • the list menu does not end and the process returns to step 602 .
  • the list menu ends through the user's selection a screen displaying the list menu ends.
  • FIGS. 10 to 12 are examples of assisting an input operation in the list menu by using voice recognition according to fourth, fifth, and sixth embodiments of the present invention, respectively. Performance of the input operation in the list menu by using voice recognition will be described in detail with reference to FIGS. 10 to 12 .
  • FIG. 10A , FIG. 11A , and FIG. 12A are images of screens where the list menu is executed and then lists are displayed.
  • FIG. 10B , FIG. 11B , and FIG. 12B are images of screens displaying particular images when the displayed lists are scrolled.
  • FIG. 10C illustrates an image of a screen where a scrolling operation is performed on the list to a part where items in the list beginning with the letter J begins when a voice of the preset command of “Until J” is input.
  • FIG. 11C illustrates an image of a screen where a scrolling operation is performed to a part where a center of the entire list begins when a voice of the preset command of “Until half the list” is input.
  • FIG. 12C is an image of a screen where a scrolling operation is performed to a part of the entire list where items in the list exist in the Korean language begins when a voice of the preset command of “Until Korean starts” is input.
  • FIG. 13 is a flowchart of assisting a home screen editing by using voice recognition according to a fifth embodiment of the present invention. An operation of simultaneously inputting a preset operation command and a voice in an executed home screen and assisting an execution of the preset operation command by using the input voice is described.
  • steps 700 to 703 are steps for performing the preset operation of the operation mode according to a key input or a touch input in the preset operation mode in the first process of FIG. 2 .
  • a home screen is executed through a user control input in step 700 , and a page of the executed home screen is displayed in step 701 .
  • the home screen is preset to include preset home screen pages including preset items, wherein one or more home screen pages may be preset.
  • the voice recognition function of the voice recognition module 111 may be automatically activated or may be activated by a user's selection using buttons 161 , keypad 166 or microphone 162 .
  • an operation of the touch input is performed in step 703 . If not, the process ends.
  • the touch input at this time is a dragging touch input, which corresponds to an input of a flick operation in which the user contacts one or more particular items in a displayed page of the home screen by using a body part or a touchable input device and then removes the body part or the touchable input device from the contacted particular items in a movement direction.
  • the items can be moved from one page of the home screen to another page of the home screen through the dragging in an up, down, left, or right direction according to a dragging direction.
  • the remaining steps correspond to the second process of FIG. 2 of recognizing the voice input during the execution of the preset operation of the operation mode of the first process of FIG. 2 .
  • a voice is input through the microphone 162 in step 704 .
  • the voice input through the microphone 162 can be made in all cases where a touch is not input, a touch is being input, and a preset operation command is being performed during the touch input when the home screen is displayed.
  • the touch is being input, or the preset operation command is being performed during the touch input when the home screen is displayed, will be described with reference to the case where the touch operation is being performed.
  • the activated voice recognition module 111 recognizes the input voice in step 705 . If there is no voice input, the process ends.
  • step 706 it is determined whether the recognized voice is the preset operation command in step 706 . At this time, a recognized voice which has no similarity with the preset operation command is not applied. If the recognized voice is not the preset operation command, the process ends.
  • a determination of whether the recognized voice is the preset operation command in step 706 will be described in detail.
  • “dragging the item” means “placing the item”.
  • “dragging the item” means “returning the item to an original place”.
  • the recognized voice in a dragged state is the preset operation command.
  • the preset operation command of the recognized voice is performed during performance of the touch operation in step 707 , and a result of performance of the operation command is output in step 708 .
  • the preset operation command may be a preset operation command which moves a preset item of a page of the preset home screen to another page. Further, the preset operation command may be a preset command which generates a new page of the home screen.
  • the operations from step 702 may be performed again when the user desires to continuously perform the operations, and the home screen editing ends when the user does not perform the touch input in step 709 .
  • FIGS. 14A to 14E illustrate an example of assisting the home screen editing by using voice recognition according to the seventh embodiment of the present invention.
  • FIG. 14A illustrates an image of a screen where the home screen is executed and then displayed
  • FIG. 14B illustrates an image of an editing mode screen of the home screen when a touch is input and a touch state is maintained. It is assumed that the home screen of FIG. 14A and FIG. 14B is page 1.
  • FIG. 14C illustrates an image where the item on the home screen is dragged to another page of the home screen through the touch input by the user. It is assumed that the home screen of FIG. 14C is page 3 and there is no space to place a new item in the screen of page 3. As illustrated in FIG. 14C , when there is no space to place the new item in the page of the home screen to which the item is dragged, a message informing the user that there is no space is displayed on the screen or the user is informed of the fact that there is no space through a voice, through which the user can identify that there is no space.
  • the user inputs the preset operation command through the voice in a dragging operation state, and the terminal performs the preset operation command by recognizing the input voice. For example, the user can move all items in page 3 to another page of the home screen through a voice command of “Move all other items to the next page”.
  • FIG. 14D illustrates an image in a touch and dragging state where the preset operation command is performed, and all items in page 3 are moved to another page of the home screen.
  • FIG. 14E illustrates an image in a touch and dragging state where the touch ends, an icon is placed in a position where the touch ends, and an editing mode ends.
  • the preset operation command in the home screen may be set to perform functions such as deletion and copy of the item or set to move only some items selected by the user when the user desires to move items.
  • the embodiments of the present invention can be implemented in software, hardware, or a combination thereof. Any such software may be stored as described above, for example, in a volatile or non-volatile storage device such as a ROM, a memory such as a RAM, a memory chip, a memory device, or a memory IC, or a recordable optical or magnetic medium such as a CD, a DVD, a magnetic disk, or a magnetic tape, regardless of its ability to be erased or its ability to be re-recorded.
  • the memory included in the mobile terminal is one example of machine-readable devices suitable for storing a program including instructions that are executed by a processor device to thereby implement embodiments of the present invention. Therefore, embodiments of the present invention provide a program including codes for implementing a system or method claimed in any claim of the accompanying claims and a machine-readable device for storing such a program.
  • this program may be electronically conveyed through any medium such as a communication signal transferred via a wired or wireless connection, and embodiments of the present invention appropriately include equivalents thereto.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)
US13/954,432 2012-10-08 2013-07-30 Method and apparatus for performing preset operation mode using voice recognition Abandoned US20140100850A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/247,987 US10825456B2 (en) 2012-10-08 2019-01-15 Method and apparatus for performing preset operation mode using voice recognition

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2012-0111402 2012-10-08
KR1020120111402A KR102009423B1 (ko) 2012-10-08 2012-10-08 음성 인식을 이용한 미리 설정된 동작 모드의 수행 방법 및 장치

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/247,987 Continuation US10825456B2 (en) 2012-10-08 2019-01-15 Method and apparatus for performing preset operation mode using voice recognition

Publications (1)

Publication Number Publication Date
US20140100850A1 true US20140100850A1 (en) 2014-04-10

Family

ID=49303887

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/954,432 Abandoned US20140100850A1 (en) 2012-10-08 2013-07-30 Method and apparatus for performing preset operation mode using voice recognition
US16/247,987 Expired - Fee Related US10825456B2 (en) 2012-10-08 2019-01-15 Method and apparatus for performing preset operation mode using voice recognition

Family Applications After (1)

Application Number Title Priority Date Filing Date
US16/247,987 Expired - Fee Related US10825456B2 (en) 2012-10-08 2019-01-15 Method and apparatus for performing preset operation mode using voice recognition

Country Status (8)

Country Link
US (2) US20140100850A1 (de)
EP (1) EP2717259B1 (de)
JP (1) JP6347587B2 (de)
KR (1) KR102009423B1 (de)
CN (1) CN103716454A (de)
AU (1) AU2013237690B2 (de)
BR (1) BR102013025963A2 (de)
RU (1) RU2013144921A (de)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140309996A1 (en) * 2013-04-10 2014-10-16 Via Technologies, Inc. Voice control method and mobile terminal apparatus
US20160283453A1 (en) * 2015-03-26 2016-09-29 Lenovo (Singapore) Pte. Ltd. Text correction using a second input
US20170024405A1 (en) * 2015-07-24 2017-01-26 Samsung Electronics Co., Ltd. Method for automatically generating dynamic index for content displayed on electronic device
US10572104B2 (en) 2014-11-24 2020-02-25 Samsung Electronics Co., Ltd Electronic device for executing a plurality of applications and method for controlling the electronic device
WO2020167006A1 (en) * 2019-02-14 2020-08-20 Samsung Electronics Co., Ltd. Method of providing speech recognition service and electronic device for same
US10983673B2 (en) * 2018-05-22 2021-04-20 Konica Minolta, Inc. Operation screen display device, image processing apparatus, and recording medium
US11289114B2 (en) 2016-12-02 2022-03-29 Yamaha Corporation Content reproducer, sound collector, content reproduction system, and method of controlling content reproducer

Families Citing this family (79)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US10255566B2 (en) 2011-06-03 2019-04-09 Apple Inc. Generating and processing task items that represent tasks to perform
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
DE112014000709B4 (de) 2013-02-07 2021-12-30 Apple Inc. Verfahren und vorrichtung zum betrieb eines sprachtriggers für einen digitalen assistenten
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
KR101959188B1 (ko) 2013-06-09 2019-07-02 애플 인크. 디지털 어시스턴트의 둘 이상의 인스턴스들에 걸친 대화 지속성을 가능하게 하기 위한 디바이스, 방법 및 그래픽 사용자 인터페이스
KR101749009B1 (ko) 2013-08-06 2017-06-19 애플 인크. 원격 디바이스로부터의 활동에 기초한 스마트 응답의 자동 활성화
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
WO2015184186A1 (en) 2014-05-30 2015-12-03 Apple Inc. Multi-command single utterance input method
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
CN104252285B (zh) * 2014-06-03 2018-04-27 联想(北京)有限公司 一种信息处理方法及电子设备
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
CN105573534A (zh) * 2014-10-09 2016-05-11 中兴通讯股份有限公司 操作对象的处理方法及装置
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US10146355B2 (en) 2015-03-26 2018-12-04 Lenovo (Singapore) Pte. Ltd. Human interface device input fusion
KR101632534B1 (ko) * 2015-04-06 2016-06-21 주식회사 카카오 오타 처리 방법 및 그 방법을 수행하는 사용자 단말
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10200824B2 (en) 2015-05-27 2019-02-05 Apple Inc. Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10740384B2 (en) 2015-09-08 2020-08-11 Apple Inc. Intelligent automated assistant for media search and playback
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10956666B2 (en) 2015-11-09 2021-03-23 Apple Inc. Unconventional virtual assistant interactions
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US12223282B2 (en) 2016-06-09 2025-02-11 Apple Inc. Intelligent automated assistant in a home environment
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
US12197817B2 (en) 2016-06-11 2025-01-14 Apple Inc. Intelligent device arbitration and control
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
CN106940595B (zh) * 2017-03-16 2019-10-11 北京云知声信息技术有限公司 一种信息编辑方法及装置
JP6862952B2 (ja) * 2017-03-16 2021-04-21 株式会社リコー 情報処理システム、情報処理装置、情報処理プログラム及び情報処理方法
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK180048B1 (en) 2017-05-11 2020-02-04 Apple Inc. MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. LOW-LATENCY INTELLIGENT AUTOMATED ASSISTANT
DK201770411A1 (en) 2017-05-15 2018-12-20 Apple Inc. Multi-modal interfaces
US10303715B2 (en) 2017-05-16 2019-05-28 Apple Inc. Intelligent automated assistant for media exploration
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
CN108509175B (zh) * 2018-03-30 2021-10-22 联想(北京)有限公司 一种语音交互方法及电子设备
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. VIRTUAL ASSISTANT OPERATION IN MULTI-DEVICE ENVIRONMENTS
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
DK179822B1 (da) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
KR102136463B1 (ko) * 2018-07-27 2020-07-21 (주)휴맥스 스마트 디바이스 및 그 제어 방법
CN109361814A (zh) * 2018-09-25 2019-02-19 联想(北京)有限公司 一种控制方法及电子设备
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. User activity shortcut suggestions
DK201970510A1 (en) 2019-05-31 2021-02-11 Apple Inc Voice identification in digital assistant systems
US11227599B2 (en) 2019-06-01 2022-01-18 Apple Inc. Methods and user interfaces for voice-based control of electronic devices
US11838459B2 (en) 2019-06-07 2023-12-05 Canon Kabushiki Kaisha Information processing system, information processing apparatus, and information processing method
US20210090558A1 (en) * 2019-09-24 2021-03-25 Audio Analytic Ltd Controlling a user interface
US11061543B1 (en) 2020-05-11 2021-07-13 Apple Inc. Providing relevant data items based on context
US12301635B2 (en) 2020-05-11 2025-05-13 Apple Inc. Digital assistant hardware abstraction
US11183193B1 (en) 2020-05-11 2021-11-23 Apple Inc. Digital assistant hardware abstraction
US11755276B2 (en) 2020-05-12 2023-09-12 Apple Inc. Reducing description length based on confidence
US11490204B2 (en) 2020-07-20 2022-11-01 Apple Inc. Multi-device audio adjustment coordination
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones
KR20230102506A (ko) 2021-12-30 2023-07-07 삼성전자주식회사 전자 장치 및 이의 제어 방법
KR102793798B1 (ko) * 2022-08-23 2025-04-11 국방과학연구소 정보를 출력하는 전자 장치의 동작 방법 및 이를 지원하는 전자 장치
CN116312514B (zh) * 2023-02-22 2025-10-28 海信视像科技股份有限公司 显示设备和语音控制方法

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030117365A1 (en) * 2001-12-13 2003-06-26 Koninklijke Philips Electronics N.V. UI with graphics-assisted voice control system
US20030157968A1 (en) * 2002-02-18 2003-08-21 Robert Boman Personalized agent for portable devices and cellular phone
US6725194B1 (en) * 1999-07-08 2004-04-20 Koninklijke Philips Electronics N.V. Speech recognition device with text comparing means
US20050159948A1 (en) * 2001-09-05 2005-07-21 Voice Signal Technologies, Inc. Combined speech and handwriting recognition
US20060106614A1 (en) * 2004-11-16 2006-05-18 Microsoft Corporation Centralized method and system for clarifying voice commands
US20070100635A1 (en) * 2005-10-28 2007-05-03 Microsoft Corporation Combined speech and alternate input modality to a mobile device
US7380203B2 (en) * 2002-05-14 2008-05-27 Microsoft Corporation Natural input recognition tool
US20080228496A1 (en) * 2007-03-15 2008-09-18 Microsoft Corporation Speech-centric multimodal user interface design in mobile technology
US20090214117A1 (en) * 2008-02-26 2009-08-27 Microsoft Corporation Handwriting symbol recognition accuracy using speech input
US20090253463A1 (en) * 2008-04-08 2009-10-08 Jong-Ho Shin Mobile terminal and menu control method thereof
US20090326938A1 (en) * 2008-05-28 2009-12-31 Nokia Corporation Multiword text correction
US20100009720A1 (en) * 2008-07-08 2010-01-14 Sun-Hwa Cha Mobile terminal and text input method thereof
US20110115702A1 (en) * 2008-07-08 2011-05-19 David Seaberg Process for Providing and Editing Instructions, Data, Data Structures, and Algorithms in a Computer System
US20120075184A1 (en) * 2010-09-25 2012-03-29 Sriganesh Madhvanath Silent speech based command to a computing device
US20130035941A1 (en) * 2011-08-05 2013-02-07 Samsung Electronics Co., Ltd. Method for controlling electronic apparatus based on voice recognition and motion recognition, and electronic apparatus applying the same

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1867068A (zh) * 1998-07-14 2006-11-22 联合视频制品公司 交互式电视节目导视系统及其方法
US6545669B1 (en) * 1999-03-26 2003-04-08 Husam Kinawi Object-drag continuity between discontinuous touch-screens
US20030061053A1 (en) 2001-09-27 2003-03-27 Payne Michael J. Method and apparatus for processing inputs into a computing device
JP2003195939A (ja) * 2001-12-26 2003-07-11 Toshiba Corp プラント監視制御システム
CN1864204A (zh) * 2002-09-06 2006-11-15 语音信号技术有限公司 用来完成语音识别的方法、系统和程序
GB2433002A (en) * 2003-09-25 2007-06-06 Canon Europa Nv Processing of Text Data involving an Ambiguous Keyboard and Method thereof.
US7574356B2 (en) * 2004-07-19 2009-08-11 At&T Intellectual Property Ii, L.P. System and method for spelling recognition using speech and non-speech input
NZ589382A (en) * 2005-06-16 2012-03-30 Keyless Systems Ltd Data Entry System
WO2008064137A2 (en) * 2006-11-17 2008-05-29 Rao Ashwin P Predictive speech-to-text input
KR20090020265A (ko) * 2007-08-23 2009-02-26 삼성전자주식회사 휴대 단말기 및 그 메시지 작성 방법
US7877700B2 (en) 2007-11-20 2011-01-25 International Business Machines Corporation Adding accessibility to drag-and-drop web content
KR100988397B1 (ko) 2008-06-09 2010-10-19 엘지전자 주식회사 이동 단말기 및 그의 텍스트 수정방법
KR101513635B1 (ko) 2008-12-05 2015-04-22 엘지전자 주식회사 단말기 및 그 제어 방법
KR101613838B1 (ko) * 2009-05-19 2016-05-02 삼성전자주식회사 휴대 단말기의 홈 스크린 지원 방법 및 이를 지원하는 휴대 단말기
US8661369B2 (en) * 2010-06-17 2014-02-25 Lg Electronics Inc. Mobile terminal and method of controlling the same
US8359020B2 (en) 2010-08-06 2013-01-22 Google Inc. Automatically monitoring for voice input based on context
KR101718027B1 (ko) * 2010-09-09 2017-03-20 엘지전자 주식회사 이동 단말기 및 그의 메모 관리방법
JP6024675B2 (ja) * 2014-01-17 2016-11-16 株式会社デンソー 音声認識端末装置、音声認識システム、音声認識方法
US9606716B2 (en) 2014-10-24 2017-03-28 Google Inc. Drag-and-drop on a mobile device

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6725194B1 (en) * 1999-07-08 2004-04-20 Koninklijke Philips Electronics N.V. Speech recognition device with text comparing means
US20050159948A1 (en) * 2001-09-05 2005-07-21 Voice Signal Technologies, Inc. Combined speech and handwriting recognition
US20030117365A1 (en) * 2001-12-13 2003-06-26 Koninklijke Philips Electronics N.V. UI with graphics-assisted voice control system
US20030157968A1 (en) * 2002-02-18 2003-08-21 Robert Boman Personalized agent for portable devices and cellular phone
US7380203B2 (en) * 2002-05-14 2008-05-27 Microsoft Corporation Natural input recognition tool
US20060106614A1 (en) * 2004-11-16 2006-05-18 Microsoft Corporation Centralized method and system for clarifying voice commands
US20070100635A1 (en) * 2005-10-28 2007-05-03 Microsoft Corporation Combined speech and alternate input modality to a mobile device
US20080228496A1 (en) * 2007-03-15 2008-09-18 Microsoft Corporation Speech-centric multimodal user interface design in mobile technology
US20090214117A1 (en) * 2008-02-26 2009-08-27 Microsoft Corporation Handwriting symbol recognition accuracy using speech input
US20090253463A1 (en) * 2008-04-08 2009-10-08 Jong-Ho Shin Mobile terminal and menu control method thereof
US20090326938A1 (en) * 2008-05-28 2009-12-31 Nokia Corporation Multiword text correction
US20100009720A1 (en) * 2008-07-08 2010-01-14 Sun-Hwa Cha Mobile terminal and text input method thereof
US20110115702A1 (en) * 2008-07-08 2011-05-19 David Seaberg Process for Providing and Editing Instructions, Data, Data Structures, and Algorithms in a Computer System
US20120075184A1 (en) * 2010-09-25 2012-03-29 Sriganesh Madhvanath Silent speech based command to a computing device
US20130035941A1 (en) * 2011-08-05 2013-02-07 Samsung Electronics Co., Ltd. Method for controlling electronic apparatus based on voice recognition and motion recognition, and electronic apparatus applying the same

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140309996A1 (en) * 2013-04-10 2014-10-16 Via Technologies, Inc. Voice control method and mobile terminal apparatus
US10572104B2 (en) 2014-11-24 2020-02-25 Samsung Electronics Co., Ltd Electronic device for executing a plurality of applications and method for controlling the electronic device
US20160283453A1 (en) * 2015-03-26 2016-09-29 Lenovo (Singapore) Pte. Ltd. Text correction using a second input
US10726197B2 (en) * 2015-03-26 2020-07-28 Lenovo (Singapore) Pte. Ltd. Text correction using a second input
US20170024405A1 (en) * 2015-07-24 2017-01-26 Samsung Electronics Co., Ltd. Method for automatically generating dynamic index for content displayed on electronic device
US11289114B2 (en) 2016-12-02 2022-03-29 Yamaha Corporation Content reproducer, sound collector, content reproduction system, and method of controlling content reproducer
US10983673B2 (en) * 2018-05-22 2021-04-20 Konica Minolta, Inc. Operation screen display device, image processing apparatus, and recording medium
WO2020167006A1 (en) * 2019-02-14 2020-08-20 Samsung Electronics Co., Ltd. Method of providing speech recognition service and electronic device for same
US11474780B2 (en) 2019-02-14 2022-10-18 Samsung Electronics Co., Ltd. Method of providing speech recognition service and electronic device for same

Also Published As

Publication number Publication date
KR102009423B1 (ko) 2019-08-09
KR20140045181A (ko) 2014-04-16
US20190147879A1 (en) 2019-05-16
RU2013144921A (ru) 2015-04-20
US10825456B2 (en) 2020-11-03
BR102013025963A2 (pt) 2018-06-26
AU2013237690B2 (en) 2018-07-05
JP6347587B2 (ja) 2018-06-27
EP2717259A2 (de) 2014-04-09
JP2014078007A (ja) 2014-05-01
CN103716454A (zh) 2014-04-09
AU2013237690A1 (en) 2014-04-24
EP2717259A3 (de) 2014-04-30
EP2717259B1 (de) 2016-09-14

Similar Documents

Publication Publication Date Title
US10825456B2 (en) Method and apparatus for performing preset operation mode using voice recognition
EP2701053B1 (de) Verfahren zur Steuerung einer Funktionsausführung in einem mobilen Endgerät durch Erkennen der Schreibgestik und Vorrichtung zur Durchführung davon
US10185456B2 (en) Display device and control method thereof
KR101761190B1 (ko) 휴대 단말기의 사용자 인터페이스 제공 방법 및 장치
RU2658176C2 (ru) Устройство отображения для выполнения множества приложений и способ управления им
US20140365950A1 (en) Portable terminal and user interface method in portable terminal
CN110069203B (zh) 电子设备和操作电子设备的方法
KR20230014873A (ko) 터치 감응형 이차 디스플레이에서 사용자 인터페이스 제어부들을 동적으로 제공하기 위한 시스템들, 디바이스들, 및 방법들
KR20140143555A (ko) 휴대 장치의 잠금 화면 상에서 빠른 어플리케이션 실행 방법 및 이를 위한 휴대 장치
KR20120020853A (ko) 이동 단말기 및 그 제어방법
KR20140128208A (ko) 사용자 단말 장치 및 그 제어 방법
CN104111794A (zh) 用于提供与设备状态相应的改变的快捷图标的方法和设备
JP2015049901A (ja) フィールドの属性に応じてコンテンツを提供する電子装置及び方法
JP2013130979A (ja) 情報処理装置、情報処理装置の制御方法、情報処理装置制御プログラムおよび該プログラムを記録したコンピュータ読み取り可能な記録媒体
EP2955616A1 (de) Elektronische vorrichtung und bearbeitungsverfahren für symbole in einer elektronischen vorrichtung
KR20160090709A (ko) 이동단말기 및 그 제어방법
US10871898B2 (en) Display apparatus for providing preview UI and method of controlling display apparatus
US20150185988A1 (en) Method, apparatus and recording medium for guiding text editing position
CN114489424A (zh) 桌面组件的控制方法及装置
US9886167B2 (en) Display apparatus and control method thereof
KR102496603B1 (ko) 어플리케이션 실행 화면의 위치 선정 방법
KR20200086653A (ko) 컨텐츠의 스크롤 시 대표 정보를 표시하는 전자 장치 및 그 제어 방법
CN118264739A (zh) 提示信息显示方法、功能服务唤醒方法及相关装置

Legal Events

Date Code Title Description
AS Assignment

Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WON, SUNG-JOON;REEL/FRAME:031164/0153

Effective date: 20130625

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION