Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/50—Network services
  • H04L67/56—Provisioning of proxy services
  • H04L67/565—Conversion or adaptation of application format or content
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F40/00—Handling natural language data
  • G06F40/40—Processing or translation of natural language
  • G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
  • H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
  • H04L51/06—Message adaptation to terminal or network requirements
  • H04L51/063—Content adaptation, e.g. replacement of unsuitable content
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/10—Protocols in which an application is distributed across nodes in the network
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M3/00—Automatic or semi-automatic exchanges
  • H04M3/42—Systems providing special services or facilities to subscribers
  • H04M3/42382—Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L13/00—Speech synthesis; Text to speech systems
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/26—Speech to text systems
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M2201/00—Electronic components, circuits, software, systems or apparatus used in telephone systems
  • H04M2201/39—Electronic components, circuits, software, systems or apparatus used in telephone systems using speech synthesis
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M2201/00—Electronic components, circuits, software, systems or apparatus used in telephone systems
  • H04M2201/40—Electronic components, circuits, software, systems or apparatus used in telephone systems using speech recognition
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M2242/00—Special services or facilities
  • H04M2242/12—Language recognition, selection or translation arrangements

Definitions

• the present invention relates generally to the field of machine or computer based translation systems and methods, and more particularly to systems and methods for providing application services, cloud based, on text-based and voice-based messaging and translation.
• This provides a multilingual system with the ability to translate and convert text-to-voice text in all directions between all languages.
• translation is intended to mean a conversion of the meaning of an expression or word in one language to the same meaning in another language.
• Text-based & voice-based messaging is a method of communication whereby textual character strings and voice recordings can be sent and received as translated messages. Messages can be exchanged between people in order to communicate information.
• Various types and configurations of computer based translation and/or conversion systems and methods have been known in the art.
• One such example for translation is described in international application WO 2015/145259, published October 1 , 2015, the contents of which is hereby incorporated by reference in its entirety.
• WO 2015/145259 a computer system is employed for effecting translations.
• a translation system is used. Further, U.S.
• the invention provides for a system and method of authorizing and registering a message, whether voice or text, from a first client for translation and transmission to a second client, with a server, translating the message and providing the translated message to the second client.
• a computer based system and method of the invention includes sending a message in a source language using a transmission protocol from a first client device to a second client device.
• the source language message is transmitted using a transmission protocol to a translation server for translating the message from the source language to a destination language by either the first or second client device.
• the message, in its destination language form, is displayed and stored.
• the translation server is either on-line or off-line.
• Text-to-voice and voice-to-text converters are used for conversion of original text messages to voice, or of original voice to text, for
• FIGS. 1(a) and 1(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a text-to-text communication sequence between two devices with an embedded offline translation system (translation server) used in an example embodiment of the present invention;
• translation server embedded offline translation system
• FIGS. 2(a) and 2(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a text-to-text communication sequence between two devices with a cloud -based translation system used in an example embodiment of the present invention
• FIGS. 3(a) and 3(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a text-to-voice communication sequence between two devices with an embedded offline translation system, offline voice-to-text converter and offline text-to-voice converter used in an example embodiment of the present invention
• FIGS. 4(a) and 4(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a text-to-voice communication sequence between two devices with an embedded offline translation system, cloud-based voice-to-text converter and cloud-based text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 5(a) and 5(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a text-to-voice communication sequence between two devices with an embedded cloud-based translation system, offline voice-to-text converter and offline text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 6(a) and 6(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a text-to-voice communication sequence between two devices with an embedded cloud-based translation system, cloud-based voice-to- text converter and cloud-based text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 7(a) and 7(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a voice-to-text communication sequence between two devices with an embedded offline translation system, offline voice-to-text converter and offline text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 8(a) and 8(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a voice-to-text communication sequence between two devices with an embedded offline translation system, cloud-based voice-to-text converter and cloud-based text-to-voice converter used in an example embodiment of the present invention.
• FIGS. 9(a) and 9(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a voice-to-text communication sequence between two devices with an embedded cloud-based translation system, offline voice-to-text converter and offline text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 10(a) and 10(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for a voice-to-text communication sequence between two devices with an embedded cloud-based translation system, cloud-based voice-to- text converter and cloud-based text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 11(a) and 11(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for voice-to-voice communication sequence between two devices with an embedded offline translation system, offline voice-to-text converter and offline text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 12(a) and 12(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for voice-to-voice communication sequence between two devices with an embedded offline translation system, cloud-based voice-to-text converter and cloud-based text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 13(a) and 13(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for voice-to-voice communication sequence between two devices with an embedded cloud-based translation system, offline voice-to-text converter and offline text-to-voice converter used in an example embodiment of the present invention;
• FIGS. 14(a) and 14(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for voice-to-voice communication sequence between two devices with an embedded cloud-based translation system, cloud-based voice-to- text converter and cloud-based text-to-voice converter used in an example embodiment of the present invention.
• FIGS. 15(a) and 15(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for authorization;
• FIGS. 16(a) and 16(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for registration;
• FIGS. 17(a) and 17(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for text message translation; and
• FIGS. 18(a) and 18(b) are respectively: a schematic diagram of system elements; and a flow diagram illustrating process steps, for sending of voice messages.
• FIGS. 1(a) and 1 (b) through FIGS. 14(a) and 14(b) illustrate both system elements and process steps of various communication sequences as embodiments of the present invention.
• FIG 1(a) there is shown system elements for communication between a first client 1, sending a message, and a second client 2, receiving the message, through a translation system (translation server) 11.
• translation server translation server
• FIG 1(b) illustrates the process of a text to text message
• client device 1 sends a message in the original source language to client device 2, which receives the message in the original source language at step 13.
• client 2 then transmits the text message to a translation system off-line at step 14 and the text is then transmitted back to client 2 for display in a destination language at step 15.
• FIGS. 2(a) and 2(b) illustrate a similar text to text sequence translation and communication of a message from client 1 to client 2 with an online translation system.
• the same system elements as in FIG 1(a) are illustrated.
• the process here, however, is different.
• the client 1 sends the text message in the original source language to client 2 which receives it at step 22 in the original source language.
• client 2 transmits the text message in its original source language to the translation system 1 1 where it is translated at step 23 into a destination language.
• the translated message in its destination language is transmitted to client 2 and displayed at step 23 in the destination language.
• FIGS 3(a) and 3(b) illustrate a sequence of text to voice transmission of a message with an off-line translation system and an off-line voice recognition system.
• system elements for client 1 a voice recognition system 31 , a translation system 32, and client 2.
• Voice recognition system 31 includes voice-to-text converter 31 (a) and text-to-text voice converter 31(b).
• the process here involves client 1 sending a text message at step 33, in its original source language, to client 2 which receives the message at step 34 in its original source language.
• Client 2 then transmits the text message to an off-line transmission system 32 at step 35 for translation.
• the text message, now in its destination language after translation is then converted to a voice message in the destination language by text-to-voice converter 31(b) at step 36.
• the client receives both the text message and the voice message in the destination language.
• FIGS 4(a) and 4(b) another text to voice transmission sequence from client 1 to client 2 is illustrated.
• the translation system is off-line, but the voice recognition system is online.
• Client 1 sends the text message in the original source language at step 41 to client 2.
• Client 2 receives the text message in the original source language at step 42 and transmits the message to the translation system 32, which translates the text message into the destination language and then transmits the text message in the destination language to voice recognition system 31 where, at step 44 it is converted to a voice message in the destination language by text-to-voice converter 31(b).
• the text and voice messages in the destination language are then transmitted to client 2 who receives both the text message and the voice message in the destination language at step 45.
• FIGS 5(a) and 5(b) A further text to voice transmission sequence is illustrated in FIGS 5(a) and 5(b).
• the message is sent in the source language from the client 1 at step 51 to client 2, who receives it in the original source language at step 52.
• the message is then sent to an online translation system 52 where it is translated to the destination language at step 53.
• the destination language text is then forwarded to the voice recognition system 31, which is off-line, where it is converted at step 54 to a voice message in the destination language by the text-to- voice converter 31(b). From here, it is transmitted to client 2 which receives both the text and voice messages in the destination language at step 55.
• a further text to voice translation sequence is illustrated in 6(a) and (b) where the text message, in its original source language, is transmitted at step 61 to client 2 who receives the message in its original source language at step 62.
• Client 2 transmits the text message in its original source language to the translation system 62, which is an online system, for translation at step 63 into the destination language.
• the text message now in its destination language is transmitted back to client 2 which receives it in its destination language at step 64, and then transmits the text message in the destination language to online voice recognition system 31 where it is converted to a voice message in the destination language at step 65 by text-to-voice converter 31 (b).
• the text and voice messages in the destination language are then transmitted to the client 2 who receives it at step 66.
• FIGS 7(a) and 7(b) illustrate an off-line translation sequence with an off-line voice recognition system for transmitting a voice message from client 1 into a text message for delivery to client 2.
• Client 1 at step 73, records a voice message in the source language which is then transmitted for conversion to a text message by the voice-to-text converter 31(a) of voice recognition system 31 at step 74.
• the text message form is then transmitted to client 2 which receives it at step 75.
• the text message still in its source language, is then translated by translation system 72, which is off-line, and the translated text message is then displayed at step 77 at client 2.
• FIGS. 8(a) and 8(b) show client 1 recording a voice message in its source language at step 83 and which transmits the recorded message to online voice recognition system 31 where the voice message is converted by the voice-to-text converter 31 (a) to a text message in the source language at step 84.
• the converted text message is transmitted to client 1 who receives it at step 85 in its original source language.
• the converted text message is transmitted to client 2 which receives it in its original source language at step 86, and who thereupon transmits it for translation to the off-line translation system 82 where, at step 87, the converted text message is translated to a text destination language.
• the text destination language is then transmitted to client 2 at step 88.
• FIGS 9(a) and 9(b) illustrate a sequence where client 1 records a voice message in a source language and transmits it to off-line voice recognition system 31 where at step 94 it is converted, by voice-to-text converter 31 (a), to a text message in the source language.
• the converted text message is then transmitted to client 2 where it is received at step 95 and then transmitted to an online translation system 92 where the text message is translated to the destination language at step 96.
• the translated message in text form is then transmitted to client 2 where the translated text message is displayed in the destination language at step 97.
• FIGS 10(a) and 10(b) illustrate yet another voice to text translation sequence.
• client 1 records a voice message in the source language and transmits it to online voice recognition system 31 where at step 104 the recorded voice message in the source language is converted by converter 3 l(a)to a text message in the same source language.
• the text message is then transmitted to client 1 who receives it at step 105 and transmits it to client 2 which receives it at step 106.
• the text message in the source language is then transmitted to the online translation system 102 where at step 107 the text message is translated into the destination language.
• the translated text message is transmitted to client 2 where at step 108 it is displayed in the destination language.
• FIG 1 Yet another transmission sequence for a message from one client device to another, this time from voice to voice, is illustrated in FIG 1 1.
• Client 1 records a voice message in a source language at step 1 13 and transmits that message to voice recognition system 31 where it is converted by converter 31(a) to a text message still in the source language at step 1 14.
• the converted message, now in text form, is transmitted to client 2, which receives it at step 115.
• the text message in the source language is then transmitted to the off-line translation system where it is translated into the destination language in text form at step 1 16.
• Step 1 17 is where the text message, now in the destination language, is converted by text-to-voice converter 31(b) of voice recognition system 31 into a voice message and transmits both the text message and the voice message in the destination language to client 2 who receives it at step 118.
• FIG 12(a) and 12(b) illustrate a voice to voice transmission sequence using on off-line translation system and an online voice recognition system.
• client 1 records a voice message in the source language at step 123 and then transmits it to the converter 31 (a) of voice recognition system 31 where, at step 124, it is converted to a text message in the source language.
• the text message in the source language is then transmitted from the voice recognition system 121 back to client 1 who receives it at step 125.
• the text message in the source language is then transmitted to client 2 which receives it at step 126 and then transmits it to the translation system 122 for translation into the destination language at step 127.
• the translated message in text form is then transmitted back to the voice recognition system 31 where it is converted by converter 3 l(b)to a voice message, but this time in the destination language at step 128.
• the translated messages, both in text and voice form, in the destination language are then transmitted to client 2 where they are received at step 129.
• FIGS 13(a) and 13(b) illustrate a voice to voice communication sequence between the devices client 1 and client 2 with an embedded cloud-based online translation system and an off-line voice to text converter and an off-line text to voice converter.
• client at step 133 records a voice message in the source language and transmits the voice message to voice recognition system 31 where it is converted by converter 31(a) into a text message in the source language at step 134.
• the converted text message is then transmitted to client 2 which receives the text message in the source language at step 135.
• Client 2 then transmits the text message to the embedded cloud-based translation system 132 where it is translated into the destination language at step 136.
• the translated text message is then transmitted to the text- to- voice converter 31(b) at voice recognition system 31 where at step 137 the text message is converted to a voice message in the destination language and then transmitted in both text and voice format, in the destination language, to client 2 which receives it at step 138.
• FIGS 14(a) and 14(b) represent a voice to voice communication sequence between client 1 and client 2 using an embedded cloud based online translation system and a cloud based voice recognition system with both a voice-to-text converter and a text-to-voice converter.
• a voice message is recorded at client 1 in a source language at step 143 and transmitted to online voice recognition system 31 where the voice-to-text converter 31(a) converts the voice message to a text message in the source language at step 144.
• the text message is then transmitted to client 1 which receives it at step 145.
• the text message is then retransmitted to client 2 which receives it step 146.
• the text message is transmitted to translation system 142 where at step 147 it is translated into the destination language in text format.
• the text translation is transmitted to client 2 which receives it at step 148 in the destination language and then transmits it to voice recognition system 31 where it is converted by the text-to-voice converter 31(b) into a voice message in the destination language at step 149.
• Client 2 receives the translated and converted text and voice messages in the destination language at step 150.
• Authorization is completed by sending in http or https request containing a user's phone number and a unique device ID (smart phone) to a server.
• FIGS 15(a) and 15(b) illustrate authorization.
• a client device such as client 1 is connected to a server 3.
• An application at the client device obtains a device ID and a phone number of the user at step 151 and transmits them to the server 3 via http/https.
• the server matches the device ID and the phone number and sends back an access token or key which is required for access to the server, and the token or key is stored locally at client 1 , step 153.
• Received messages are locally stored on the client device for subsequent di splay. If the user enables a translation of messages for a displayed dialog, then all incoming messages are translated into the language specified by the user.
• text and voice based messaging is a communication technique where textual character strings and voice recordings can be sent and received as translated messages.
• Sending a voice message is illustrated at FIGS. 18(a) and (b).
• client 1 is connected to a server 181 which is connected to a voice recognition system 31 which in turn is connected to a translation server 183 connected to client 2.
• the voice message is recorded at the client 1 device, step 184, and then transmitted to server 181 where it is filed and stored, step 185.
• An object ID is then sent to the second client device, client 2, at step 186.
• the received message is then displayed in a dialog at client 2, step 187, and the original message can be played back.
• the user can then choose to translate the message in a context menu.
• client 2 transmits the voice message to voice-to-text converter 31(a) of voice recognition system 31 where it is converted to a text message in the original source language at step 188.
• the recognized text message is locally stored as text for future translation at step 189.
• the message can then be translated by translation server 103 at step 190 and can then be locally stored by client 2 for later display.
• the user can use text-to-voice converter 31(b) of voice recognition system 31 and play back the translated voice message, step 191.
• a text message can be sent to client 2, via an Extensible Messaging and Presence Protocol (XMPP) (a communications protocol for message-oriented middleware based on XML (Extensible Markup Language) which enables the near-real-time exchange of structured yet extensible data between any two or more network entities.
• XMPP Extensible Messaging and Presence Protocol
• the message can be locally stored for later display at 173.
• the message can then be translated by translation server 171, step 174 and then at step 175 displayed according to the translation display setting.
• Parameters for translations such as source language, target language and text, are provided to the translation service/server.
• Translated text is also stored locally on the client device for future viewing.
• the user can playback the synthesized (by the device or third party applications) translated voice message.
• the client device transmits a Short Message System (an SMS) with a confirmation code to the provided phone number at step 162 and at step 163 the user confirms registration by entering a confirmation code which is sent back to the service via an http or https request.
• SMS Short Message System
• the server matches the device ID, the phone number and the confirmation code.
• the user submits to the client a name and photo where at 166 the server records the user's data to a database.
• the application begins an authorization process at step 167.

Landscapes

• Engineering & Computer Science (AREA)
• Computer Networks & Wireless Communication (AREA)
• Signal Processing (AREA)
• Theoretical Computer Science (AREA)
• Computational Linguistics (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Artificial Intelligence (AREA)
• General Health & Medical Sciences (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Health & Medical Sciences (AREA)
• Machine Translation (AREA)
• Information Transfer Between Computers (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=59294798

Family Applications (1)

Country Status (2)

Families Citing this family (4)

Family Cites Families (3)

• 2015
  • 2015-10-08 CN CN201580055053.6A patent/CN107111613A/zh active Pending
  • 2015-10-08 EP EP15849745.3A patent/EP3204870A4/de not_active Withdrawn

Also Published As

Similar Documents

Legal Events