CN102737101B - 用于自然用户界面系统的组合式激活 - Google Patents

用于自然用户界面系统的组合式激活 Download PDF

Info

Publication number
CN102737101B
CN102737101B CN201210091176.3A CN201210091176A CN102737101B CN 102737101 B CN102737101 B CN 102737101B CN 201210091176 A CN201210091176 A CN 201210091176A CN 102737101 B CN102737101 B CN 102737101B
Authority
CN
China
Prior art keywords
user
signals
visual display
gesture
context
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201210091176.3A
Other languages
English (en)
Other versions
CN102737101A (zh
Inventor
L·P·赫克
M·金达昆塔
D·米特比
L·施蒂费尔曼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/077,303 external-priority patent/US9858343B2/en
Priority claimed from US13/077,396 external-priority patent/US9842168B2/en
Priority claimed from US13/077,455 external-priority patent/US9244984B2/en
Priority claimed from US13/076,862 external-priority patent/US9760566B2/en
Priority claimed from US13/077,233 external-priority patent/US20120253789A1/en
Priority claimed from US13/077,368 external-priority patent/US9298287B2/en
Priority claimed from US13/077,431 external-priority patent/US10642934B2/en
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Publication of CN102737101A publication Critical patent/CN102737101A/zh
Application granted granted Critical
Publication of CN102737101B publication Critical patent/CN102737101B/zh
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/9032—Query formulation
    • G06F16/90332—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • General Health & Medical Sciences (AREA)
  • Machine Translation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • User Interface Of Digital Computer (AREA)
  • Information Transfer Between Computers (AREA)
  • Telephonic Communication Services (AREA)
  • Stored Programmes (AREA)

Abstract

本发明涉及用于自然用户界面系统的组合式激活。可提供用户交互激活。可对从用户接收的多个信号进行评估以确定该多个信号是否与视觉显示相关联。如果是的话，该多个信号可被翻译成代理动作，并且可检索与视觉显示相关联的上下文。可根据所检索的上下文来执行代理动作，并且可向用户显示与所执行的代理动作相关联的结果。

Description

用于自然用户界面系统的组合式激活
技术领域
本发明涉及用户交互系统，更具体地，涉及用于自然用户界面系统的组合式激活。
背景技术
自然用户界面系统的组合式激活可提供多模式自然用户界面激活系统，该系统可使用多种模式来激活或操作应用。在一些情形中，自然用户界面系统注重于单一模式的激活或操作。例如，用户通过语音命令或通过敲击屏幕来激活应用。然而，常规系统中的单一模式的命令激活可以是高度敏感的或容易出现各种类型的不准确，诸如无意的激活。
发明内容
提供本概述以便以简化形式介绍将在以下详细描述中进一步描述的一些概念。此发明内容既不旨在标识所要求保护的主题的关键特征或必要特征。本发明内容也不旨在用于限制所要求保护的主题的范围。
可提供用户交互激活。可对从用户接收的多个信号进行评估以确定该多个信号是否与视觉显示相关联。如果是的话，该多个信号可被翻译成代理动作，并且可检索与视觉显示相关联的上下文。可根据所检索的上下文来执行代理动作，并且可向用户显示与所执行的代理动作相关联的结果。
以上概括描述和以下详细描述两者都提供了示例，并且只是说明性的。因此，以上概括描述和以下详细描述不应当被认为是限制性的。此外，除了本文中所阐述的那些特征或变体以外，还可以提供其他特征或变体。例如，实施例可涉及具体实施方式中所描述的各种特征组合和子组合。
附图说明
合并在本公开中并构成其一部分的附图示出本发明的实施例。在附图中：
图1是操作环境的框图；
图2是一种用于提供用户交互激活的方法的流程图；以及
图3是包括计算设备的系统的框图。
具体实施方式
以下详细描述参考各个附图。只要可能，就在附图和以下描述中使用相同的附图标记来指示相同或相似的元件。尽管可能描述了本发明的实施例，但修改、改编、以及其他实现是可能的。例如，可对附图中所示的元件进行置换、添加、或修改，并且可通过对所公开的方法置换、重新排序、或添加阶段来修改本文中所描述的方法。因此，以下详细描述并不限制本发明。相反，本发明的正确范围由所附权利要求书定义。
口述对话系统(SDS)使得人们能够用他们的声音与计算机进行交互。驱动该SDS的主要组件可以包括对话管理器：该组件管理与用户的基于对话的会话。对话管理器可通过多个输入源的组合来确定用户的意图，这多个输入源诸如语音识别和自然语言理解组件输出、来自先前对话轮次的上下文、用户上下文、和/或从知识库(例如搜索引擎)返回的结果。在确定意图后，对话管理器可采取动作，诸如向用户显示最终结果和/或继续与用户的对话以满足他们的意图。
图1是操作环境100的框图，操作环境100包括用户105、服务器107、网络120、以及用户设备130。服务器107可包括口述对话系统(SDS)110、个人助理程序112、和/或搜索代理118。SDS 110可用于经由网络120接收用户词组、查询、动作、和/或动作请求。网络120可包括专有网络(例如，企业内联网)、蜂窝网络和/或诸如因特网等公共网络。操作环境100还可包括多个数据源150(A)-(C)。用户设备130可用于提供显示的图像132，诸如与照片、视频、和/或游戏相关联的图像。用户设备130可被耦合到相机135，相机135可用于记录用户105以及捕捉用户105所作的动作和/或手势。用户设备130还可进一步用来捕捉用户105诸如通过话筒137口述的单词，和/或捕捉来自用户105的诸如通过键盘和/或鼠标(未绘出)的其它输入。根据本发明的其它实施例，相机135可以包括能够检测用户105的移动的任何运动检测设备。例如，相机135可以包括微软运动捕捉设备，它包括多个相机和多个话筒。
图2是阐述根据本发明的一实施例的用于提供用户查询的个性化的方法200中所涉及的各概略阶段的流程图。方法200可使用计算设备300来实现，这将在下面参考图3予以更详细描述。在下文中将更详细地描述实现方法200的各阶段的方式。方法200可开始于起始框205并继续至阶段210，在那里，计算设备300可接收来自用户的多个信号。例如，SDS110可接收口述查询并由相机135标识用户105所执行的第一手势。例如，用户可挥手并说一个命令，如“你好，xbox”。
随后，方法200可前进到阶段220，在那里，计算设备300可确定该信号是否针对该系统。例如，用户105指向屏幕可包括一个激活手势，而用户105从相机135前走过可不包括激活手势。根据本发明的各实施例，用户105可将任何手势定义为相关联的手势。如果所标识的手势和/或语音信号被标识为不针对SDS 110，则方法200可在阶段270结束。
如果信号针对该系统，则方法200可前进到阶段230，在那里，计算设备300可检索与视觉显示相关联的上下文。例如，元数据可与视频流相关联，视频流提供诸如标题、演员、描述、评级等信息。对于另一示例，可从数据源150(A)-(C)中的一个检索上下文。例如，数据源150(A)可包括电影信息网站。
方法200可随后前进到步骤240，在那里，计算设备300可将接收到的信号翻译成代理动作。例如，相机135可捕捉用户105的指向手势，指向手势可用于指示视觉显示的子集。例如，在电影视频的当前帧中有三名演员，则相机可标识用户105正指向三名演员中的哪一名。指示可被用于创建与诸如“那个演员是谁？”之类的语音查询相关联的代理动作。因此，代理动作可能够选择性地标识三名演员中用户所指示的那一个。
方法200随后可前进至阶段250，在那里，计算设备300可根据所检索到的上下文和所接收到的信号执行代理动作。例如，SDS 110可从数据源150(A)检索所显示的电影中的所有演员的列表，将结果缩小到在信号被接收时所显示的三名演员，并且根据用户105指向哪个演员来标识具体的演员。
方法200接着可前进到阶段260，在那里，计算设备300可向用户显示与所执行的查询相关联的结果。例如，可在用户设备130上显示一个字幕来提供查询的结果。随后，方法200可在阶段270结束。
根据本发明的一实施例可包括用于提供用户交互激活的系统。该系统可包括存储器存储和耦合到该存储器存储的处理单元。处理单元可用于接收来自用户的查询、检索与视觉显示相关联的上下文、根据所检索到的上下文执行查询、以及向用户显示与所执行的查询相关联的结果。视觉显示可包括例如静态图像、视频、和/或游戏图像。可用于根据所检索的上下文来检索查询可包括处理单元可用于根据所检索的上下文将多个结果缩小到所述多个结果的子集。处理单元还可进一步用于接收来自用户的手势、根据该手势(例如指向手势)更新所检索的上下文、以及根据所更新的上下文执行查询。可用于根据指向手势来更新所检索的上下文可包括处理单元可用于标识指向手势所指示的视觉显示的元素。
根据本发明的另一实施例可包括用于提供用户交互激活的系统。该系统可包括存储器存储和耦合到该存储器存储的处理单元。处理单元可用于接收包括自然语音词组(例如，口述词组)的请求、检索与视觉显示相关联的上下文、标识用户所作出的手势、根据所检索的上下文和所标识的手势来执行与请求所关联的动作、以及向用户提供与所执行的动作相关联的结果。根据本方面的各实施例，自然语言词组可包括口述和/或会话语法而不是特别格式化的查询。例如，“那个建筑物是什么”可包括自然语言词组以及可与电影“盗梦空间”的视觉显示相关联。诸如可被提供给搜索引擎之类的可用来对比的格式化查询可包括“domain:imdb.com title：Inception time：1:32‘identify building’coordinates：132，425”。视觉显示可包括与用户相关联的记录设备所捕捉的图像。例如，用户可用相机拍摄一张数码照片并查看图像。用户的手势可包括激活手势。例如，用户105可直接指向相机135以指示用户105将要作出查询和/或动作。
根据本发明的又一实施例可包括用于提供用户交互激活的系统。该系统可包括存储器存储和耦合到该存储器存储的处理单元。处理单元可用于接收来自用户的多个同时发生的信号，其中至少一个第一信号包括经由至少一个麦克风接收的语音信号并且至少一个第二信号包括经由至少一个相机接收的手势，并且处理单元可用于确定所述多个信号是否针对该系统。响应于确定所述多个信号针对该系统，处理单元可用于：接收来自用户的查询；检索与视觉显示相关联的上下文；标识经由相机从用户接收的第二手势；将所述多个信号翻译成与视觉显示相关联的至少一个代理动作，其中手势包括可用于选择视觉显示的子集的指向手势；根据所检索的上下文和所标识的第二手势来执行查询代理动作；以及向用户显示与所执行的查询代理动作相关联的结果。
图3是包括计算设备300的系统的框图。根据本发明的一个实施例，上述存储器存储和处理单元可在诸如图3的计算设备300之类的计算设备中实现。可使用硬件、软件或固件的任何合适的组合来实现存储器存储和处理单元。例如，存储器存储和处理单元可用计算设备300或结合计算设备300的其他计算设备318中的任一个来实现。根据本发明的实施例，上述系统、设备和处理器是示例，而其他系统、设备和处理器可包括上述存储器存储和处理单元。此外，计算设备300可包括如上所述的操作环境100。系统100可在其他环境中操作，并且不限于计算设备300。
参考图3，根据本发明的一实施例的系统可包括计算设备，诸如计算设备300。在基本配置中，计算设备300可包括至少一个处理单元302和系统存储器304。取决于计算设备的配置和类型，系统存储器304可包括，但不限于，易失性存储器(例如，随机存取存储器(RAM))、非易失性存储器(例如，只读存储器(ROM))、闪存、或任何组合。系统存储器304可以包括操作系统305、一个或多个编程模块306，且可以包括个人助理程序112。例如，操作系统305可适用于控制计算设备300的操作。此外，本发明的实施例可结合图形库、其他操作系统、或任何其他应用程序来实践，并且不限于任何特定应用或系统。该基本配置在图3中由虚线308内的那些组件示出。
计算设备300可具有附加特征或功能。例如，计算设备300还可包括附加数据存储设备(可移动和/或不可移动)，诸如例如，磁盘、光盘、或磁带。这些附加存储在图3中由可移动存储309和不可移动存储310示出。计算机存储介质可包括以用于存储诸如计算机可读指令、数据结构、程序模块、或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。系统存储器304、可移动存储309和不可移动存储3 10都是计算机存储介质(即，存储器存储)的示例。计算机存储介质可包括，但不限于，RAM、ROM、电可擦除只读存储器(EEPROM)、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光存储、磁带盒、磁带、磁盘存储或其他磁性存储设备、或者可用于存储信息且可由计算设备300访问的任何其他介质。任何此类计算机存储介质可以是设备300的一部分。计算设备300还可以具有输入设备312，如键盘、鼠标、笔、声音输入设备、触摸输入设备等。还可包括诸如显示器、扬声器、打印机等输出设备314。上述设备是示例，并且可使用其他设备。
计算设备300还可包含可允许设备300诸如通过分布式计算环境中的网络(例如，内联网或因特网)来与其他计算设备318进行通信的通信连接316。通信连接316是通信介质的一个示例。通信介质通常由诸如载波或其他传输机制之类的已调制数据信号中的计算机可读指令、数据结构、程序模块、或其他数据来体现，并且包括任何信息传送介质。术语“已调制数据信号”可以描述以对该信号中的信息进行编码的方式设定或者改变其一个或多个特征的信号。作为示例而非限制，通信介质包括诸如有线网络或直接线连接等有线介质，以及诸如声学、射频(RF)、红外线和其他无线介质等无线介质。如此处所使用的术语“计算机可读介质”可包括存储介质和通信介质两者。
如上所述，可在系统存储器304中存储包括操作系统305在内的多个程序模块和数据文件。当在处理单元302上执行时，编程模块306(例如，个人助理程序112)可执行各过程，包括例如，如上所述的方法200的各阶段中的一个或多个。上述过程是一个示例，且处理单元302可执行其他过程。根据本发明的实施例可使用的其他编程模块可包括电子邮件和联系人应用程序、文字处理应用程序、电子表格应用程序、数据库应用程序、幻灯片演示应用程序、绘图或计算机辅助应用程序等。
一般而言，根据本发明的实施例，程序模块可包括可执行特定任务或可实现特定抽象数据类型的例程、程序、组件、数据结构和其他类型的结构。此外，本发明的实施例可用其他计算机系统配置来实践，包括手持式设备、多处理器系统、基于微处理器的系统或可编程消费电子产品、小型机、大型计算机等。本发明的实施例还可在其中任务由通过通信网络链接的远程处理设备执行的分布式计算环境中实践。在分布式计算环境中，程序模块可位于本地和远程存储器存储设备两者中。
此外，本发明的实施例可在包括分立电子元件的电路、包含逻辑门的封装或集成电子芯片、利用微处理器的电路、或在包含电子元件或微处理器的单个芯片上实践。本发明的实施例还可使用能够执行诸如例如，AND(与)、OR(或)和NOT(非)的逻辑运算的其他技术来实践，包括但不限于，机械、光学、流体和量子技术。另外，本发明的实施例可在通用计算机或任何其他电路或系统中实践。
例如，本发明的实施例可被实现为计算机过程(方法)、计算系统、或诸如计算机程序产品或计算机可读介质之类的制品。计算机程序产品可以是计算机系统可读并对用于执行计算机过程的指令的计算机程序编码的计算机存储介质。计算机程序产品还可以是计算系统可读并对用于执行计算机过程的指令的计算机程序编码的载体上的传播信号。因此，本发明可以硬件和/或软件(包括固件、常驻软件、微码等)来体现。换言之，本发明的实施例可采用其上包含有供指令执行系统使用或结合其使用的计算机可使用或计算机可读程序代码的计算机可使用或计算机可读存储介质上的计算机程序产品的形式。计算机可使用或计算机可读介质可以是可包含、存储、通信、传播、或传输程序以供指令执行系统、装置或设备使用或结合其使用的任何介质。
计算机可使用或计算机可读介质例如可以是、但不限于电、磁、光、电磁、红外、或半导体系统、装置、设备或传播介质。更具体的计算机可读介质示例(非穷尽列表)，计算机可读介质可包括以下：具有一条或多条导线的电连接、便携式计算机盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或闪存)、光纤、以及便携式压缩盘只读存储器(CD-ROM)。注意，计算机可使用或计算机可读介质甚至可以是其上打印有程序的纸张或另一合适的介质，因为程序可经由例如对纸张或其他介质的光学扫描而电子地捕获，随后如有必要被编译、解释、或以其他合适的方式处理，并且随后存储在计算机存储器中。
以上参考例如根据本发明的实施例的方法、系统和计算机程序产品的框图和/或操作示图描述了本发明的实施例。框中所注明的各功能/动作可按不同于任何流程图所示的次序出现。例如，取决于所涉及的功能/动作，连续示出的两个框实际上可基本同时执行，或者这些框有时可按相反的次序执行。
尽管已描述了本发明的特定实施例，但也可能存在其他实施例。此外，虽然本发明的实施例被描述为与存储在存储器和其他存储介质中的数据相关联，但是数据还可被存储在其他类型的计算机可读介质上或从其读取，诸如辅助存储设备(像硬盘、软盘、或CD-ROM)、来自因特网的载波、或其他形式的RAM或ROM。此外，所公开的方法的各步骤可以任何方式修改，包括通过对各步骤重新排序和/或插入或删除步骤，而不背离本发明。
包括此处所包括的代码中的版权在内的所有权利都归属于申请人并且是本申请人的财产。本申请人保持并保留此处所包括的代码中的所有权利，并且授予仅关于所授权专利的再现且未出于其他目的再现该材料的许可。
尽管本说明书包括示例，但本发明的范围由所附权利要求书来指示。此外，尽管用对结构特征和/或方法动作专用的语言描述了本说明书，但权利要求书并不限于以上所描述的特征或动作。相反，以上所描述的特定特征和动作是作为本发明的实施例的示例来公开的。

Claims (9)

1.一种用于提供用户(105)交互激活的方法(200)，所述方法(200)包括：
接收(210)来自用户(105)的多个信号；
确定(220)所述多个信号是否与视觉显示相关联，其中，所述视觉显示包括静态图像、视频和/或游戏图像；
响应于确定(220)所述多个信号与视觉显示相关联：
将所述多个信号翻译(240)成代理动作，其中所述多个信号包括通过照相机标识的激活手势，其中所述翻译(240)包括利用由所述激活手势标识的所述视觉显示的子集的指示来创建所述代理动作；
响应于所标识的指向所述视觉显示的激活手势，检索与所述视觉显示相关联的上下文，
根据所检索的上下文和接收的信号执行(250)所述代理动作，以及
向用户(105)显示(260)与所执行的代理动作相关联的结果。
2.如权利要求1所述的方法(200)，其特征在于，所述多个信号还包括：用户通过话筒口述的单词或通过键盘或鼠标的输入。
3.如权利要求1所述的方法(200)，其特征在于，根据所检索的上下文执行(250)代理动作包括根据所检索的上下文将多个结果缩小为所述多个结果的子集。
4.如权利要求3所述的方法(200)，其特征在于，还包括向用户(105)显示(260)所述多个结果的子集。
5.如权利要求1所述的方法(200)，其特征在于，还包括：
接收(210)来自用户(105)的手势，其中所述手势包括所述多个信号中的至少之一；
根据所述手势更新(240)所检索的上下文；以及
根据所更新的上下文执行(250)所述代理动作。
6.如权利要求1所述的方法(200)，其中：
所接收的多个信号包括语音信号。
7.如权利要求6所述的方法，其特征在于，所述手势和所述语音信号是同时从用户(105)接收的。
8.如权利要求6所述的方法，其特征在于，与所述视觉显示相关联的上下文是从与所述视觉显示相关联的多个元数据中检索的。
9.一种用于提供用户交互激活的系统，所述系统包括：
用于接收来自用户的多个信号的装置；
用于确定所述多个信号是否与视觉显示相关联的装置，其中，所述视觉显示包括静态图像、视频和/或游戏图像；
用于响应于确定所述多个信号与视觉显示相关联：
将所述多个信号翻译成代理动作的装置，其中所述多个信号包括通过照相机标识的激活手势，其中该翻译(240)包括利用由所述激活手势标识的所述视觉显示的子集的指示来创建所述代理动作；
响应于所标识的指向所述视觉显示的激活手势，检索与所述视觉显示相关联的上下文的装置；
根据所检索的上下文和接收的信号执行所述代理动作的装置；以及
向用户显示与所执行的代理动作相关联的结果的装置。
CN201210091176.3A 2011-03-31 2012-03-30 用于自然用户界面系统的组合式激活 Expired - Fee Related CN102737101B (zh)

Applications Claiming Priority (14)

Application Number Priority Date Filing Date Title
US13/077,396 US9842168B2 (en) 2011-03-31 2011-03-31 Task driven user intents
US13/077,455 US9244984B2 (en) 2011-03-31 2011-03-31 Location based conversational understanding
US13/076,862 2011-03-31
US13/077,396 2011-03-31
US13/076,862 US9760566B2 (en) 2011-03-31 2011-03-31 Augmented conversational understanding agent to identify conversation context between two humans and taking an agent action thereof
US13/077,233 US20120253789A1 (en) 2011-03-31 2011-03-31 Conversational Dialog Learning and Correction
US13/077,368 2011-03-31
US13/077,431 2011-03-31
US13/077,455 2011-03-31
US13/077,368 US9298287B2 (en) 2011-03-31 2011-03-31 Combined activation for natural user interface systems
US13/077,303 US9858343B2 (en) 2011-03-31 2011-03-31 Personalization of queries, conversations, and searches
US13/077,431 US10642934B2 (en) 2011-03-31 2011-03-31 Augmented conversational understanding architecture
US13/077,233 2011-03-31
US13/077,303 2011-03-31

Publications (2)

Publication Number Publication Date
CN102737101A CN102737101A (zh) 2012-10-17
CN102737101B true CN102737101B (zh) 2018-09-04

Family

ID=46931884

Family Applications (8)

Application Number Title Priority Date Filing Date
CN201610801496.1A Active CN106383866B (zh) 2011-03-31 2012-03-29 基于位置的会话理解
CN201210087420.9A Active CN102737096B (zh) 2011-03-31 2012-03-29 基于位置的会话理解
CN201210091176.3A Expired - Fee Related CN102737101B (zh) 2011-03-31 2012-03-30 用于自然用户界面系统的组合式激活
CN201210090349.XA Expired - Fee Related CN102737099B (zh) 2011-03-31 2012-03-30 对查询、会话和搜索的个性化
CN201210090634.1A Active CN102750311B (zh) 2011-03-31 2012-03-30 扩充的对话理解体系结构
CN201210093414.4A Active CN102737104B (zh) 2011-03-31 2012-03-31 任务驱动的用户意图
CN201210101485.4A Expired - Fee Related CN102750271B (zh) 2011-03-31 2012-03-31 谈话式对话学习和纠正
CN201210092263.0A Active CN102750270B (zh) 2011-03-31 2012-03-31 扩充的对话理解代理

Family Applications Before (2)

Application Number Title Priority Date Filing Date
CN201610801496.1A Active CN106383866B (zh) 2011-03-31 2012-03-29 基于位置的会话理解
CN201210087420.9A Active CN102737096B (zh) 2011-03-31 2012-03-29 基于位置的会话理解

Family Applications After (5)

Application Number Title Priority Date Filing Date
CN201210090349.XA Expired - Fee Related CN102737099B (zh) 2011-03-31 2012-03-30 对查询、会话和搜索的个性化
CN201210090634.1A Active CN102750311B (zh) 2011-03-31 2012-03-30 扩充的对话理解体系结构
CN201210093414.4A Active CN102737104B (zh) 2011-03-31 2012-03-31 任务驱动的用户意图
CN201210101485.4A Expired - Fee Related CN102750271B (zh) 2011-03-31 2012-03-31 谈话式对话学习和纠正
CN201210092263.0A Active CN102750270B (zh) 2011-03-31 2012-03-31 扩充的对话理解代理

Country Status (5)

Country Link
EP (6) EP2691870A4 (zh)
JP (4) JP6105552B2 (zh)
KR (3) KR20140014200A (zh)
CN (8) CN106383866B (zh)
WO (7) WO2012135226A1 (zh)

Families Citing this family (215)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US10002189B2 (en) 2007-12-20 2018-06-19 Apple Inc. Method and apparatus for searching using an active ontology
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US10255566B2 (en) 2011-06-03 2019-04-09 Apple Inc. Generating and processing task items that represent tasks to perform
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US10032127B2 (en) 2011-02-18 2018-07-24 Nuance Communications, Inc. Methods and apparatus for determining a clinician's intent to order an item
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US9760566B2 (en) 2011-03-31 2017-09-12 Microsoft Technology Licensing, Llc Augmented conversational understanding agent to identify conversation context between two humans and taking an agent action thereof
US10642934B2 (en) 2011-03-31 2020-05-05 Microsoft Technology Licensing, Llc Augmented conversational understanding architecture
US9842168B2 (en) 2011-03-31 2017-12-12 Microsoft Technology Licensing, Llc Task driven user intents
US9064006B2 (en) 2012-08-23 2015-06-23 Microsoft Technology Licensing, Llc Translating natural language utterances to keyword search queries
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
CN104704797B (zh) 2012-08-10 2018-08-10 纽昂斯通讯公司 用于电子设备的虚拟代理通信
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
DE112014000709B4 (de) 2013-02-07 2021-12-30 Apple Inc. Verfahren und vorrichtung zum betrieb eines sprachtriggers für einen digitalen assistenten
EP2946322A1 (en) * 2013-03-01 2015-11-25 Nuance Communications, Inc. Methods and apparatus for determining a clinician's intent to order an item
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US9436287B2 (en) * 2013-03-15 2016-09-06 Qualcomm Incorporated Systems and methods for switching processing modes using gestures
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
WO2014197334A2 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
KR101959188B1 (ko) 2013-06-09 2019-07-02 애플 인크. 디지털 어시스턴트의 둘 이상의 인스턴스들에 걸친 대화 지속성을 가능하게 하기 위한 디바이스, 방법 및 그래픽 사용자 인터페이스
US9728184B2 (en) 2013-06-18 2017-08-08 Microsoft Technology Licensing, Llc Restructuring deep neural network acoustic models
US9589565B2 (en) 2013-06-21 2017-03-07 Microsoft Technology Licensing, Llc Environmentally aware dialog policies and response generation
US9311298B2 (en) 2013-06-21 2016-04-12 Microsoft Technology Licensing, Llc Building conversational understanding systems using a toolset
KR101749009B1 (ko) 2013-08-06 2017-06-19 애플 인크. 원격 디바이스로부터의 활동에 기초한 스마트 응답의 자동 활성화
US10296160B2 (en) 2013-12-06 2019-05-21 Apple Inc. Method for extracting salient dialog usage from live data
US20150170053A1 (en) * 2013-12-13 2015-06-18 Microsoft Corporation Personalized machine learning models
CN104714954A (zh) * 2013-12-13 2015-06-17 中国电信股份有限公司 基于上下文理解的信息搜索方法和系统
US10534623B2 (en) 2013-12-16 2020-01-14 Nuance Communications, Inc. Systems and methods for providing a virtual assistant
US10015770B2 (en) 2014-03-24 2018-07-03 International Business Machines Corporation Social proximity networks for mobile phones
US9529794B2 (en) 2014-03-27 2016-12-27 Microsoft Technology Licensing, Llc Flexible schema for language model customization
US20150278370A1 (en) * 2014-04-01 2015-10-01 Microsoft Corporation Task completion for natural language input
US10111099B2 (en) 2014-05-12 2018-10-23 Microsoft Technology Licensing, Llc Distributing content in managed wireless distribution networks
US9874914B2 (en) 2014-05-19 2018-01-23 Microsoft Technology Licensing, Llc Power management contracts for accessory devices
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
WO2015184186A1 (en) 2014-05-30 2015-12-03 Apple Inc. Multi-command single utterance input method
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9355640B2 (en) * 2014-06-04 2016-05-31 Google Inc. Invoking action responsive to co-presence determination
US9717006B2 (en) 2014-06-23 2017-07-25 Microsoft Technology Licensing, Llc Device quarantine in a wireless network
JP6275569B2 (ja) * 2014-06-27 2018-02-07 株式会社東芝 対話装置、方法およびプログラム
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9798708B1 (en) 2014-07-11 2017-10-24 Google Inc. Annotating relevant content in a screen capture image
US10146409B2 (en) * 2014-08-29 2018-12-04 Microsoft Technology Licensing, Llc Computerized dynamic splitting of interaction across multiple content
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
KR102188268B1 (ko) * 2014-10-08 2020-12-08 엘지전자 주식회사 이동단말기 및 그 제어방법
EP3210096B1 (en) * 2014-10-21 2019-05-15 Robert Bosch GmbH Method and system for automation of response selection and composition in dialog systems
KR102329333B1 (ko) 2014-11-12 2021-11-23 삼성전자주식회사 질의를 처리하는 장치 및 방법
US9836452B2 (en) * 2014-12-30 2017-12-05 Microsoft Technology Licensing, Llc Discriminating ambiguous expressions to enhance user experience
US10713005B2 (en) 2015-01-05 2020-07-14 Google Llc Multimodal state circulation
US10572810B2 (en) 2015-01-07 2020-02-25 Microsoft Technology Licensing, Llc Managing user interaction for input understanding determinations
WO2016129767A1 (ko) * 2015-02-13 2016-08-18 주식회사 팔락성 온라인 사이트 링크방법
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10200824B2 (en) 2015-05-27 2019-02-05 Apple Inc. Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device
US10083688B2 (en) * 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US9792281B2 (en) * 2015-06-15 2017-10-17 Microsoft Technology Licensing, Llc Contextual language generation by leveraging language understanding
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10249297B2 (en) 2015-07-13 2019-04-02 Microsoft Technology Licensing, Llc Propagating conversational alternatives using delayed hypothesis binding
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10740384B2 (en) 2015-09-08 2020-08-11 Apple Inc. Intelligent automated assistant for media search and playback
KR20170033722A (ko) * 2015-09-17 2017-03-27 삼성전자주식회사 사용자의 발화 처리 장치 및 방법과, 음성 대화 관리 장치
US10262654B2 (en) * 2015-09-24 2019-04-16 Microsoft Technology Licensing, Llc Detecting actionable items in a conversation among participants
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10970646B2 (en) * 2015-10-01 2021-04-06 Google Llc Action suggestions for user-selected content
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10956666B2 (en) 2015-11-09 2021-03-23 Apple Inc. Unconventional virtual assistant interactions
KR102393928B1 (ko) 2015-11-10 2022-05-04 삼성전자주식회사 응답 메시지를 추천하는 사용자 단말 장치 및 그 방법
CN108351890B (zh) * 2015-11-24 2022-04-12 三星电子株式会社 电子装置及其操作方法
KR102502569B1 (ko) 2015-12-02 2023-02-23 삼성전자주식회사 시스템 리소스 관리를 위한 방법 및 장치
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US9905248B2 (en) 2016-02-29 2018-02-27 International Business Machines Corporation Inferring user intentions based on user conversation data and spatio-temporal data
US9978396B2 (en) 2016-03-16 2018-05-22 International Business Machines Corporation Graphical display of phone conversations
US10587708B2 (en) 2016-03-28 2020-03-10 Microsoft Technology Licensing, Llc Multi-modal conversational intercom
US11487512B2 (en) 2016-03-29 2022-11-01 Microsoft Technology Licensing, Llc Generating a services application
US10158593B2 (en) * 2016-04-08 2018-12-18 Microsoft Technology Licensing, Llc Proactive intelligent personal assistant
US10945129B2 (en) * 2016-04-29 2021-03-09 Microsoft Technology Licensing, Llc Facilitating interaction among digital personal assistants
US10409876B2 (en) * 2016-05-26 2019-09-10 Microsoft Technology Licensing, Llc. Intelligent capture, storage, and retrieval of information for task completion
WO2017210613A1 (en) * 2016-06-03 2017-12-07 Maluuba Inc. Natural language generation in a spoken dialogue system
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US10282218B2 (en) * 2016-06-07 2019-05-07 Google Llc Nondeterministic task initiation by a personal assistant module
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179588B1 (en) 2016-06-09 2019-02-22 Apple Inc. INTELLIGENT AUTOMATED ASSISTANT IN A HOME ENVIRONMENT
US12223282B2 (en) 2016-06-09 2025-02-11 Apple Inc. Intelligent automated assistant in a home environment
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
DK201670540A1 (en) * 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
US12197817B2 (en) 2016-06-11 2025-01-14 Apple Inc. Intelligent device arbitration and control
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
US10216269B2 (en) * 2016-06-21 2019-02-26 GM Global Technology Operations LLC Apparatus and method for determining intent of user based on gaze information
CA3033724A1 (en) * 2016-08-23 2018-03-01 Illumina, Inc. Semantic distance systems and methods for determining related ontological data
US10474753B2 (en) 2016-09-07 2019-11-12 Apple Inc. Language identification using recurrent neural networks
US10446137B2 (en) * 2016-09-07 2019-10-15 Microsoft Technology Licensing, Llc Ambiguity resolving conversational understanding system
US10503767B2 (en) * 2016-09-13 2019-12-10 Microsoft Technology Licensing, Llc Computerized natural language query intent dispatching
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US9940390B1 (en) * 2016-09-27 2018-04-10 Microsoft Technology Licensing, Llc Control system using scoped search and conversational interface
CN107885744B (zh) * 2016-09-29 2023-01-03 微软技术许可有限责任公司 对话式的数据分析
US10535005B1 (en) 2016-10-26 2020-01-14 Google Llc Providing contextual actions for mobile onscreen content
JP6697373B2 (ja) 2016-12-06 2020-05-20 カシオ計算機株式会社 文生成装置、文生成方法及びプログラム
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
EP3552114B1 (en) * 2017-02-08 2026-04-01 Microsoft Technology Licensing, LLC Natural language content generator
US10643601B2 (en) * 2017-02-09 2020-05-05 Semantic Machines, Inc. Detection mechanism for automated dialog systems
CN116991971A (zh) * 2017-02-23 2023-11-03 微软技术许可有限责任公司 可扩展对话系统
WO2018156978A1 (en) 2017-02-23 2018-08-30 Semantic Machines, Inc. Expandable dialogue system
US10798027B2 (en) * 2017-03-05 2020-10-06 Microsoft Technology Licensing, Llc Personalized communications using semantic memory
US10636418B2 (en) 2017-03-22 2020-04-28 Google Llc Proactive incorporation of unsolicited content into human-to-computer dialogs
US9865260B1 (en) 2017-05-03 2018-01-09 Google Llc Proactive incorporation of unsolicited content into human-to-computer dialogs
US10237209B2 (en) * 2017-05-08 2019-03-19 Google Llc Initializing a conversation with an automated agent via selectable graphical element
US10417266B2 (en) 2017-05-09 2019-09-17 Apple Inc. Context-aware ranking of intelligent response suggestions
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. USER INTERFACE FOR CORRECTING RECOGNITION ERRORS
US10395654B2 (en) 2017-05-11 2019-08-27 Apple Inc. Text normalization based on a data-driven learning network
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
DK180048B1 (en) 2017-05-11 2020-02-04 Apple Inc. MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. LOW-LATENCY INTELLIGENT AUTOMATED ASSISTANT
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
US11301477B2 (en) 2017-05-12 2022-04-12 Apple Inc. Feedback analysis of a digital assistant
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK201770411A1 (en) 2017-05-15 2018-12-20 Apple Inc. Multi-modal interfaces
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US10403278B2 (en) 2017-05-16 2019-09-03 Apple Inc. Methods and systems for phonetic matching in digital assistant services
US10303715B2 (en) 2017-05-16 2019-05-28 Apple Inc. Intelligent automated assistant for media exploration
US10311144B2 (en) 2017-05-16 2019-06-04 Apple Inc. Emoji word sense disambiguation
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES
US10664533B2 (en) * 2017-05-24 2020-05-26 Lenovo (Singapore) Pte. Ltd. Systems and methods to determine response cue for digital assistant based on context
US10679192B2 (en) * 2017-05-25 2020-06-09 Microsoft Technology Licensing, Llc Assigning tasks and monitoring task performance based on context extracted from a shared contextual graph
US10657328B2 (en) 2017-06-02 2020-05-19 Apple Inc. Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling
US10742435B2 (en) * 2017-06-29 2020-08-11 Google Llc Proactive provision of new content to group chat participants
US11132499B2 (en) 2017-08-28 2021-09-28 Microsoft Technology Licensing, Llc Robust expandable dialogue system
US10445429B2 (en) 2017-09-21 2019-10-15 Apple Inc. Natural language understanding using vocabularies with compressed serialized tries
US10755051B2 (en) 2017-09-29 2020-08-25 Apple Inc. Rule-based natural language processing
US10546023B2 (en) 2017-10-03 2020-01-28 Google Llc Providing command bundle suggestions for an automated assistant
US10636424B2 (en) 2017-11-30 2020-04-28 Apple Inc. Multi-turn canned dialog
CN110019718B (zh) * 2017-12-15 2021-04-09 上海智臻智能网络科技股份有限公司 修改多轮问答系统的方法、终端设备以及存储介质
US11341422B2 (en) 2017-12-15 2022-05-24 SHANGHAI XIAOl ROBOT TECHNOLOGY CO., LTD. Multi-round questioning and answering methods, methods for generating a multi-round questioning and answering system, and methods for modifying the system
US10733982B2 (en) 2018-01-08 2020-08-04 Apple Inc. Multi-directional dialog
US10839160B2 (en) * 2018-01-19 2020-11-17 International Business Machines Corporation Ontology-based automatic bootstrapping of state-based dialog systems
US10733375B2 (en) 2018-01-31 2020-08-04 Apple Inc. Knowledge-based framework for improving natural language understanding
US10789959B2 (en) 2018-03-02 2020-09-29 Apple Inc. Training speaker recognition models for digital assistants
US10592604B2 (en) 2018-03-12 2020-03-17 Apple Inc. Inverse text normalization for automatic speech recognition
KR102635811B1 (ko) * 2018-03-19 2024-02-13 삼성전자 주식회사 사운드 데이터를 처리하는 시스템 및 시스템의 제어 방법
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
US10909331B2 (en) 2018-03-30 2021-02-02 Apple Inc. Implicit identification of translation payload with neural machine translation
US10685075B2 (en) * 2018-04-11 2020-06-16 Motorola Solutions, Inc. System and method for tailoring an electronic digital assistant query as a function of captured multi-party voice dialog and an electronically stored multi-party voice-interaction template
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
US10984780B2 (en) 2018-05-21 2021-04-20 Apple Inc. Global semantic word embeddings using bi-directional recurrent neural networks
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
US11386266B2 (en) 2018-06-01 2022-07-12 Apple Inc. Text correction
DK179822B1 (da) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. VIRTUAL ASSISTANT OPERATION IN MULTI-DEVICE ENVIRONMENTS
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US10504518B1 (en) 2018-06-03 2019-12-10 Apple Inc. Accelerated task performance
EP3803632A4 (en) * 2018-06-04 2022-03-02 Disruptel, Inc. Systems and methods for operating an output device
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
CN111428721A (zh) * 2019-01-10 2020-07-17 北京字节跳动网络技术有限公司 词语释义的确定方法、装置、设备及存储介质
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. User activity shortcut suggestions
DK201970510A1 (en) 2019-05-31 2021-02-11 Apple Inc Voice identification in digital assistant systems
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
US11227599B2 (en) 2019-06-01 2022-01-18 Apple Inc. Methods and user interfaces for voice-based control of electronic devices
WO2021021012A1 (en) * 2019-07-29 2021-02-04 Ai Robotics Limited Stickering method and system for linking contextual text elements to actions
US11488406B2 (en) 2019-09-25 2022-11-01 Apple Inc. Text detection using global geometry estimators
WO2021173611A1 (en) * 2020-02-25 2021-09-02 Liveperson, Inc. Intent analysis for call center response generation
US12301635B2 (en) 2020-05-11 2025-05-13 Apple Inc. Digital assistant hardware abstraction
US11183193B1 (en) 2020-05-11 2021-11-23 Apple Inc. Digital assistant hardware abstraction
US11061543B1 (en) 2020-05-11 2021-07-13 Apple Inc. Providing relevant data items based on context
US11755276B2 (en) 2020-05-12 2023-09-12 Apple Inc. Reducing description length based on confidence
US11490204B2 (en) 2020-07-20 2022-11-01 Apple Inc. Multi-device audio adjustment coordination
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones
US11783827B2 (en) 2020-11-06 2023-10-10 Apple Inc. Determining suggested subsequent user actions during digital assistant interaction
EP4174848A1 (en) * 2021-10-29 2023-05-03 Televic Rail NV Improved speech to text method and system
CN116644810B (zh) * 2023-05-06 2024-04-05 国网冀北电力有限公司信息通信分公司 一种基于知识图谱实现的电网故障风险处置方法及装置

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1845052A (zh) * 2006-01-12 2006-10-11 广东威创日新电子有限公司 一种用于交互式输入设备的智能识别编码方法
CN1963752A (zh) * 2006-11-28 2007-05-16 李博航 基于自然语言的电子设备人机交互操作界面技术

Family Cites Families (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5265014A (en) * 1990-04-10 1993-11-23 Hewlett-Packard Company Multi-modal user interface
US5748974A (en) * 1994-12-13 1998-05-05 International Business Machines Corporation Multimodal natural language interface for cross-application tasks
US5970446A (en) * 1997-11-25 1999-10-19 At&T Corp Selective noise/channel/coding models and recognizers for automatic speech recognition
WO2000011571A1 (en) * 1998-08-24 2000-03-02 Bcl Computers, Inc. Adaptive natural language interface
US6499013B1 (en) * 1998-09-09 2002-12-24 One Voice Technologies, Inc. Interactive user interface using speech recognition and natural language processing
US6332120B1 (en) * 1999-04-20 2001-12-18 Solana Technology Development Corporation Broadcast speech recognition system for keyword monitoring
JP3530109B2 (ja) * 1999-05-31 2004-05-24 日本電信電話株式会社 大規模情報データベースに対する音声対話型情報検索方法、装置および記録媒体
EP1236096A1 (en) * 1999-06-01 2002-09-04 Geoffrey M. Jacquez Help system for a computer related application
US6598039B1 (en) * 1999-06-08 2003-07-22 Albert-Inc. S.A. Natural language interface for searching database
JP3765202B2 (ja) * 1999-07-09 2006-04-12 日産自動車株式会社 対話型情報検索装置、コンピュータを用いた対話型情報検索方法及び対話型情報検索処理を行うプログラムを記録したコンピュータ読取り可能な媒体
JP2001125896A (ja) * 1999-10-26 2001-05-11 Victor Co Of Japan Ltd 自然言語対話システム
US7050977B1 (en) * 1999-11-12 2006-05-23 Phoenix Solutions, Inc. Speech-enabled server for internet website and method
JP2002024285A (ja) * 2000-06-30 2002-01-25 Sanyo Electric Co Ltd ユーザ支援方法およびユーザ支援装置
JP2002082748A (ja) * 2000-09-06 2002-03-22 Sanyo Electric Co Ltd ユーザ支援装置
US7197120B2 (en) * 2000-12-22 2007-03-27 Openwave Systems Inc. Method and system for facilitating mediated communication
GB2372864B (en) * 2001-02-28 2005-09-07 Vox Generation Ltd Spoken language interface
JP2003115951A (ja) * 2001-10-09 2003-04-18 Casio Comput Co Ltd 話題情報提供システムおよび話題情報提供方法
US7224981B2 (en) * 2002-06-20 2007-05-29 Intel Corporation Speech recognition of mobile devices
US7693720B2 (en) * 2002-07-15 2010-04-06 Voicebox Technologies, Inc. Mobile systems and methods for responding to natural language speech utterance
EP1411443A1 (en) * 2002-10-18 2004-04-21 Hewlett Packard Company, a Delaware Corporation Context filter
JP2004212641A (ja) * 2002-12-27 2004-07-29 Toshiba Corp 音声入力システム及び音声入力システムを備えた端末装置
JP2004328181A (ja) * 2003-04-23 2004-11-18 Sharp Corp 電話機及び電話網システム
JP4441782B2 (ja) * 2003-05-14 2010-03-31 日本電信電話株式会社 情報提示方法及び情報提示装置
EP1625516A1 (en) * 2003-05-16 2006-02-15 NTT DoCoMo, Inc. Personalized service selection
JP2005043461A (ja) * 2003-07-23 2005-02-17 Canon Inc 音声認識方法及び音声認識装置
KR20050032649A (ko) * 2003-10-02 2005-04-08 (주)이즈메이커 인공생명을 학습시키는 방법 및 시스템
US7747601B2 (en) * 2006-08-14 2010-06-29 Inquira, Inc. Method and apparatus for identifying and classifying query intent
US7720674B2 (en) * 2004-06-29 2010-05-18 Sap Ag Systems and methods for processing natural language queries
JP4434972B2 (ja) * 2005-01-21 2010-03-17 日本電気株式会社 情報提供システム、情報提供方法及びそのプログラム
EP1686495B1 (en) * 2005-01-31 2011-05-18 Ontoprise GmbH Mapping web services to ontologies
GB0502259D0 (en) * 2005-02-03 2005-03-09 British Telecomm Document searching tool and method
CN101120341A (zh) * 2005-02-06 2008-02-06 凌圭特股份有限公司 以自然语言进行移动式信息访问的方法和设备
US7409344B2 (en) * 2005-03-08 2008-08-05 Sap Aktiengesellschaft XML based architecture for controlling user interfaces with contextual voice commands
US20060206333A1 (en) * 2005-03-08 2006-09-14 Microsoft Corporation Speaker-dependent dialog adaptation
WO2006108061A2 (en) * 2005-04-05 2006-10-12 The Board Of Trustees Of Leland Stanford Junior University Methods, software, and systems for knowledge base coordination
US7991607B2 (en) * 2005-06-27 2011-08-02 Microsoft Corporation Translation and capture architecture for output of conversational utterances
US7640160B2 (en) * 2005-08-05 2009-12-29 Voicebox Technologies, Inc. Systems and methods for responding to natural language speech utterance
US7620549B2 (en) * 2005-08-10 2009-11-17 Voicebox Technologies, Inc. System and method of supporting adaptive misrecognition in conversational speech
US7822699B2 (en) * 2005-11-30 2010-10-26 Microsoft Corporation Adaptive semantic reasoning engine
US7627466B2 (en) * 2005-11-09 2009-12-01 Microsoft Corporation Natural language interface for driving adaptive scenarios
US20070136222A1 (en) * 2005-12-09 2007-06-14 Microsoft Corporation Question and answer architecture for reasoning and clarifying intentions, goals, and needs from contextual clues and content
US20070143410A1 (en) * 2005-12-16 2007-06-21 International Business Machines Corporation System and method for defining and translating chat abbreviations
US8209407B2 (en) * 2006-02-10 2012-06-26 The United States Of America, As Represented By The Secretary Of The Navy System and method for web service discovery and access
RU2442213C2 (ru) * 2006-06-13 2012-02-10 Майкрософт Корпорейшн Панель управления поисковым механизмом
US20080005068A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Context-based search, retrieval, and awareness
US8204739B2 (en) * 2008-04-15 2012-06-19 Mobile Technologies, Llc System and methods for maintaining speech-to-speech translation in the field
WO2008067676A1 (en) * 2006-12-08 2008-06-12 Medhat Moussa Architecture, system and method for artificial neural network implementation
US20080172359A1 (en) * 2007-01-11 2008-07-17 Motorola, Inc. Method and apparatus for providing contextual support to a monitored communication
US20080172659A1 (en) 2007-01-17 2008-07-17 Microsoft Corporation Harmonizing a test file and test configuration in a revision control system
US20080201434A1 (en) * 2007-02-16 2008-08-21 Microsoft Corporation Context-Sensitive Searches and Functionality for Instant Messaging Applications
US20090076917A1 (en) * 2007-08-22 2009-03-19 Victor Roditis Jablokov Facilitating presentation of ads relating to words of a message
US7720856B2 (en) * 2007-04-09 2010-05-18 Sap Ag Cross-language searching
US8762143B2 (en) * 2007-05-29 2014-06-24 At&T Intellectual Property Ii, L.P. Method and apparatus for identifying acoustic background environments based on time and speed to enhance automatic speech recognition
US7788276B2 (en) * 2007-08-22 2010-08-31 Yahoo! Inc. Predictive stemming for web search with statistical machine translation models
MX2010002350A (es) * 2007-08-31 2010-07-30 Microsoft Corp Identificacion de relaciones semanticas dentro de lenguaje reportado.
US8165886B1 (en) * 2007-10-04 2012-04-24 Great Northern Research LLC Speech interface system and method for control and interaction with applications on a computing system
US8504621B2 (en) * 2007-10-26 2013-08-06 Microsoft Corporation Facilitating a decision-making process
JP2009116733A (ja) * 2007-11-08 2009-05-28 Nec Corp アプリケーション検索システム、アプリケーション検索方法、モニタ端末、検索サーバおよびプログラム
JP5158635B2 (ja) * 2008-02-28 2013-03-06 インターナショナル・ビジネス・マシーンズ・コーポレーション パーソナル・サービス支援のための方法、システム、および装置
US20090234655A1 (en) * 2008-03-13 2009-09-17 Jason Kwon Mobile electronic device with active speech recognition
CN101499277B (zh) * 2008-07-25 2011-05-04 中国科学院计算技术研究所 一种服务智能导航方法和系统
US8874443B2 (en) * 2008-08-27 2014-10-28 Robert Bosch Gmbh System and method for generating natural language phrases from user utterances in dialog systems
JP2010128665A (ja) * 2008-11-26 2010-06-10 Kyocera Corp 情報端末及び会話補助プログラム
JP2010145262A (ja) * 2008-12-19 2010-07-01 Pioneer Electronic Corp ナビゲーション装置
US8326637B2 (en) * 2009-02-20 2012-12-04 Voicebox Technologies, Inc. System and method for processing multi-modal device interactions in a natural language voice services environment
JP2010230918A (ja) * 2009-03-26 2010-10-14 Fujitsu Ten Ltd 検索装置
US8700665B2 (en) * 2009-04-27 2014-04-15 Avaya Inc. Intelligent conference call information agents
US20100281435A1 (en) * 2009-04-30 2010-11-04 At&T Intellectual Property I, L.P. System and method for multimodal interaction using robust gesture processing
KR101622111B1 (ko) * 2009-12-11 2016-05-18 삼성전자 주식회사 대화 시스템 및 그의 대화 방법
KR101007336B1 (ko) * 2010-06-25 2011-01-13 한국과학기술정보연구원 온톨로지 기반 개인화 서비스 시스템 및 방법
US20120253789A1 (en) * 2011-03-31 2012-10-04 Microsoft Corporation Conversational Dialog Learning and Correction

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1845052A (zh) * 2006-01-12 2006-10-11 广东威创日新电子有限公司 一种用于交互式输入设备的智能识别编码方法
CN1963752A (zh) * 2006-11-28 2007-05-16 李博航 基于自然语言的电子设备人机交互操作界面技术

Also Published As

Publication number Publication date
EP2691870A4 (en) 2015-05-20
WO2012135791A2 (en) 2012-10-04
CN102737101A (zh) 2012-10-17
CN106383866B (zh) 2020-05-05
WO2012135218A3 (en) 2013-01-03
CN102750311A (zh) 2012-10-24
EP2691949A2 (en) 2014-02-05
JP2017123187A (ja) 2017-07-13
KR20140025361A (ko) 2014-03-04
KR20140025362A (ko) 2014-03-04
EP2691877A2 (en) 2014-02-05
CN102737099A (zh) 2012-10-17
CN102750270B (zh) 2017-06-09
JP6105552B2 (ja) 2017-03-29
CN102737104A (zh) 2012-10-17
WO2012135210A2 (en) 2012-10-04
KR101963915B1 (ko) 2019-03-29
CN102737104B (zh) 2017-05-24
JP2014512046A (ja) 2014-05-19
CN102750270A (zh) 2012-10-24
WO2012135229A3 (en) 2012-12-27
CN102750311B (zh) 2018-07-20
WO2012135783A2 (en) 2012-10-04
EP2691870A2 (en) 2014-02-05
JP6087899B2 (ja) 2017-03-01
EP2691876A2 (en) 2014-02-05
EP2691885A1 (en) 2014-02-05
EP2691949A4 (en) 2015-06-10
EP2691877A4 (en) 2015-06-24
EP2691876A4 (en) 2015-06-10
JP6305588B2 (ja) 2018-04-04
KR101922744B1 (ko) 2018-11-27
EP2691875A2 (en) 2014-02-05
EP2691885A4 (en) 2015-09-30
WO2012135783A3 (en) 2012-12-27
JP2014515853A (ja) 2014-07-03
CN102737096A (zh) 2012-10-17
CN102737096B (zh) 2017-08-25
JP2014509757A (ja) 2014-04-21
CN102737099B (zh) 2017-12-19
CN102750271A (zh) 2012-10-24
KR20140014200A (ko) 2014-02-05
WO2012135229A2 (en) 2012-10-04
CN106383866A (zh) 2017-02-08
EP2691875A4 (en) 2015-06-10
WO2012135791A3 (en) 2013-01-10
WO2012135218A2 (en) 2012-10-04
WO2012135157A2 (en) 2012-10-04
WO2012135210A3 (en) 2012-12-27
CN102750271B (zh) 2017-10-17
WO2012135157A3 (en) 2013-01-10
WO2012135226A1 (en) 2012-10-04

Similar Documents

Publication Publication Date Title
CN102737101B (zh) 用于自然用户界面系统的组合式激活
US9298287B2 (en) Combined activation for natural user interface systems
US10866785B2 (en) Equal access to speech and touch input
US9299342B2 (en) User query history expansion for improving language model adaptation
US10572602B2 (en) Building conversational understanding systems using a toolset
US9412363B2 (en) Model based approach for on-screen item selection and disambiguation
US9292492B2 (en) Scaling statistical language understanding systems across domains and intents
US9886958B2 (en) Language and domain independent model based approach for on-screen item selection
CN102356372B (zh) 双模块便携式设备
US9875237B2 (en) Using human perception in building language understanding models
JP6204982B2 (ja) 自然動作入力を使用する文脈的クエリ調整
US20240202582A1 (en) Multi-stage machine learning model chaining
CN105474207A (zh) 用于搜索多媒体内容的用户界面方法和设备
US20130218836A1 (en) Deep Linking From Task List Based on Intent
US20220091864A1 (en) Page guiding methods, apparatuses, and electronic devices
CN119317911A (zh) 包括与来自视频的实体相关的描述性内容的实体卡
WO2024137122A1 (en) Multi-stage machine learning model chaining

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
ASS Succession or assignment of patent right

Owner name: MICROSOFT TECHNOLOGY LICENSING LLC

Free format text: FORMER OWNER: MICROSOFT CORP.

Effective date: 20150724

C41 Transfer of patent application or patent right or utility model
TA01 Transfer of patent application right

Effective date of registration: 20150724

Address after: Washington State

Applicant after: MICROSOFT TECHNOLOGY LICENSING, LLC

Address before: Washington State

Applicant before: Microsoft Corp.

GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20180904

CF01 Termination of patent right due to non-payment of annual fee