WO2017151615A1 - System for connecting a mobile device and a common display - Google Patents

System for connecting a mobile device and a common display Download PDF

Info

Publication number
WO2017151615A1
WO2017151615A1 PCT/US2017/019951 US2017019951W WO2017151615A1 WO 2017151615 A1 WO2017151615 A1 WO 2017151615A1 US 2017019951 W US2017019951 W US 2017019951W WO 2017151615 A1 WO2017151615 A1 WO 2017151615A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
mobile device
common
computer
common display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2017/019951
Other languages
French (fr)
Inventor
Michael R. Feldman
James E. Morris
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
T1V Inc
Original Assignee
T1V Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by T1V Inc filed Critical T1V Inc
Priority to EP25191101.2A priority Critical patent/EP4617849A3/en
Priority to EP17760610.0A priority patent/EP3423934A4/en
Publication of WO2017151615A1 publication Critical patent/WO2017151615A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1446—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display display composed of modules, e.g. video walls
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0464—Positioning
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2352/00—Parallel handling of streams of display data
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/04—Display device controller operating with a plurality of display units
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/02—Networking aspects
    • G09G2370/022—Centralised management of display operation, e.g. in a server instead of locally
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/04—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
    • G09G2370/042—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller for monitor identification
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/06—Consumer Electronics Control, i.e. control of another device by a display or vice versa
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/16—Use of wireless transmission of display information

Definitions

  • One or more embodiments are directed to a method of connecting a mobile device to a common display having associated with a display computer controlling the common display, including, for each mobile device downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display; and sending a data stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device.
  • Rl first resolution
  • R2 second resolution
  • R3 third resolution
  • An entirety of the mobile display may be selected to be sent to the common display and the region may be a window with an R2 less than Rl .
  • An entirety of the mobile display may be selected to be sent to the common display and the region may be a window with Rl .
  • the region may be a tray region outside a display region of the common display where windows are displayed.
  • An entirety of the mobile display may be selected to be sent to the common display and the region is a window, the common display is a touch screen display and the user may select only a portion of the window to be displayed by performing a touch tap or gesture on the common display.
  • An entirety of the mobile display may be selected to be sent to the common display and the region is may be a window on a touchscreen display, separate from the common display, the user may select only a portion of the window on the touchscreen display to be displayed on the common display.
  • Selecting what is to be displayed on the common display may include selecting only a portion of a screen on the mobile device.
  • the window may be an editable window can be moved, resized, and marked on.
  • the modified file may be transmitted to the corresponding mobile device.
  • the method may include automatically displaying the selection in the window.
  • the method may include displaying a device icon in a tray region outside a display
  • the method may include displaying the selection in the window when the device icon is moved toward the display region on the common display.
  • the display computer may transmit the file represented in the window to the corresponding device.
  • the method may include receiving a warning signal from the display computer when the device is more than a predetermined distance from the display computer to which it is connected.
  • Selecting may include selecting from a list of display computers on the mobile device.
  • Selecting the display computer may include inputting a machine identifier of the display computer.
  • Selecting may include automatically connecting to the common display closest to the device.
  • One or more embodiments are directed a method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device: downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, only a portion of the mobile device to be displayed on the common display; and sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device, the window for the portion of the mobile device occupying a display region of the common display.
  • the method may include connecting another mobile device to the common display and selecting, on the other mobile device, only a portion of the mobile device to be displayed on the common display, wherein windows for the mobile device and the other mobile device occupy the display region of the common display.
  • the method may include determining, by the display computer, a size of the window and a resolution associated therewith, sending the resolution to the device, and down sampling the selection in accordance with the resolution and only transmitting the down sampled resolution to the display computer.
  • One or more embodiments are directed a method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device: downloading a display application onto the mobile device, detecting a signal associated with display computers indicative of a location of the display computers, selecting the display computer for the common display in the display application based on a closest display computer to the mobile device, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display, and sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device.
  • the method may include automatically determining, selecting, and connecting to the common display closest to the device.
  • the signal may be from a beacon associated with the display computer.
  • Selecting may include showing a list of the display computers in the mobile device with the closest display computer being listed first.
  • One or more embodiments are directed a method of connecting a mobile device to a common display having a discernible machine identifier associated with a display computer controlling the common display, including downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display, sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device, and sending a warning signal from the display computer to the mobile device when the mobile device is more than a predetermined distance from the display computer to which it is connected.
  • FIG. 1 illustrate a block diagram of a display system in accordance with an embodiment
  • FIGS. 2 A and 2B illustrate a flowchart in accordance with an embodiment
  • FIG. 3 illustrates an application selection page on a mobile device
  • FIG. 4 illustrates a block diagram of a display system including a beacon in accordance with an embodiment
  • FIGS. 5 to 9 illustrate block diagrams of a display system interacting with mobile
  • FIG. 1 illustrates a block diagram of a display system 100 interacting with one or more mobile devices 200a, 200b, and so forth.
  • the display system 100 includes a Common Display 1 10, a Display Computer 120, and a wireless router 130 serving as a wireless access point (WAP), all interconnected.
  • the Common Display 1 10 may be an LCD display, LED display, or other monitor that is capable of having an electronic video signal as an input and converting the input to a visual image.
  • the Common Display 1 10 may include a display region 1 12 and a tray region 1 14, e.g., below the display region.
  • the Common Display 1 10 a touch sensor 1 16, e.g., overlaying an entirety of the Common Display 1 10, that it is sensitive to touch inputs including taps and gestures.
  • Information regarding a Machine Identifier 122 of the Display Computer 120 and the digital information to be displayed on the Common Display 1 10 may be sent from the Display Computer 120 to the Common Display 1 10.
  • This digital information can be within windows or Mobile Device Windows (MDWs), e.g.. editable windows, or on the entire screen of display region 1 12 of the Common Display 1 10.
  • MDWs Mobile Device Windows
  • MDI mobile device icons
  • the tray region 1 14 may be a region on which the MDWs cannot be zoomed and pinched, annotated, and so forth, but may be dragged, tapped or tossed onto the display region 1 12, e.g., to open an MDW corresponding to the MDI, and/or to received MDWs from the display region 1 12 to transmit that MDW to the mobile device corresponding to the MDI..
  • Digital information from Mobile Device 1 (200a) and Mobile Device2 (200b) is streamed from these Mobile Devices to the Display Computer 120 through the network.
  • Digital information is streamed from the mobile devices through the WAP 130 to the Display Computer 120.
  • This streaming data may then be displayed in the Mobile Device Windows (MDW) (see FIGS. 4 to 9) on the Common Display 1 10.
  • MDW Mobile Device Windows
  • the data streamed from the Mobile Device is a video signal that is the same as the signal displayed on the Mobile Device.
  • This streaming data will be referred to as a Mobile Devices Data Stream (MDDS).
  • MDDS is typically displayed within a MDW (see associated screens in FIGs. 6 to 9) in the display region 1 12 on the Common Display 1 10.
  • an identifier associated with the Mobile Device e.g, the name of the MDDS, a name associated with the corresponding Mobile Device, the Mobile Device's owner, and so forth.
  • the mobile device 200a is running a Mac® operating system or an iphone® operation system (iOS)
  • iOS an iphone® operation system
  • the user can use Airplay® to locate the Display Computer 120 that is configured as an Airplay® receiver.
  • Airplay® is a video mirroring capability built in to Mac® OS and iOS. Using Airplay® to locate Airplay® receivers, the user can then view any devices on the network that are configured to receive Airplay® inputs, typically just Apple TV®s.
  • the display computer 120 may include and run software, firmware, and/or hardware that acts as a video mirroring receiver. Then, when a user gets on the wireless network and searches for Airplay® receivers, the name of the display computer 120 will appear as an option to stream video within Airplay®. Once connected, the entire contents of the mobile device 200a may be streamed to the display computer 120.
  • Airplay® employs a Bonjour® network protocol.
  • a user of a mobile device may download a display application 210a, 210b thereon to assist in connecting the to the display computer 120.
  • a display application 210a, 210b may be written for each common operating system and is herein referred to as an AirConnect 1 M App.
  • AirConnect I M App Instructions for downloading the AirConnect I M App may be readily viewable, e.g., on or adjacent the common display 1 10, or a region to be scanned, e.g., a bar code, quick response (QR) code, and so forth, using a mobile device QR, so that once scanned, the AirConnectTM App could be downloaded.
  • QR quick response
  • a mobile device QR so that once scanned, the AirConnectTM App could be downloaded.
  • a user can launch the AirConnectTM App and then enter the Machine Identifier 122 associated common display 1 10.
  • the Machine Identifier 122 may by an IP address or other alphanumeric code associated with the Display Computer 120.
  • the Machine Identifier 122 may be simply displayed on the Common Display 1 10, in which case the user of the AirConnectTM App may simply enter the Machine Identifier 122 when prompted by the AirConnectTM App on their Mobile Device. Alternatively, the Machine Identifier 122 may be automatically transferred to the Mobile Device either by displaying a QR code on the Common Display 1 10 or by transmitting through bluetooth or wireless communication.
  • a low energy bluetooth beacon (CD-Beacon) 140 (see FIG. 4), may be placed near the Common Display 1 10.
  • This CD-Beacon 140 may transmit an identification number, e.g., a unique universal identification number (UUID).
  • UUID unique universal identification number
  • the AirConnectTM App can detect this UUID and send this information to a cloud server.
  • the cloud server can store a lookup table with each CD-Beacon and the associated Machine Identifier 122 for each Display Computer 120.
  • Machine Identifier 122 to connect the Mobile Device directly to the display computer 120.
  • FIGS. 2A and 2B is a flowchart of an operation of connecting Mobile Device(s) to the common display 1 10 in accordance with an embodiment.
  • the AirConnect 1 M App is on the MD in operation 20.
  • the user launches the AirConnect 1 M App in operation 30 on the MD. If not, then the user may be instructed, e.g., by a notice posted on or near the common display 1 10, to download the
  • the AirConnect App may display a list of locally stored names and/or recently used connections on the mobile device.
  • the Common Display 1 10 may also display the name or Machine Identifier 122 of the Display Computer 120 that it is connected to and/or the name of the Common Display 1 10 or Display Computer 120 may be posted near the Common Display 1 10. If the name associated with the Common Display 1 10 is not listed in the
  • the user may type a Machine Identifier 122 that may be continuously displayed on or near the common display 1 10 to add the computer display 120 to the list of FIG. 3 in operation 45.
  • the Machine Identifier 122 may be, e.g., name of the computer (host name), a UUID, and so forth.
  • the CD-Beacon 140 method described above may be used to add the computer to the list of computers in operation 45.
  • the AirConnectTM App may capture UUIDs for any beacons within range of the Mobile Device. All of these UUIDs may be transmitted to an AirConnectTM Server and then the AirConnectTM Server may determine if any of these UUIDs are for CD-Beacons. If so, the corresponding Machine Identifiers can then be sent to the AirConnectTM App on the Mobile Device.
  • FIG. 3 An example of a screen 250 that may appear when using the AirConnectTM App is shown in FIG. 3. Here, a user would be prompted to select which display to be connected with. These options may include a name of a room in which the common display 1 10 is located, a nickname for the common display that is visually apparent, the machine identifier of the common display that is visually apparent, and so forth.
  • the screen 250 for selection may look the same regardless of the operating system of the mobile device running the AirConnect I M App.
  • the user of the MD may select a computer in operation 50, and then the AirConnect App can establish a connection between the MD device and the Display Computer, using the Machine Identifier associated with the name of the display computer in the list.. Once the connection is established, the display computer 120 may tell the MD what resolution to stream in operation 60, up to a maximum resolution of the MD, and the desired content of the MD may be streamed to the selected computer as discussed below in detail with reference to FIG. 2B.
  • the Machine Information 122 regarding the selected computer may be saved on the mobile device 200a, 200b in operation 55. For example, if the user then leaves the room and disconnects and then returns, all of the initialization noted above may not need to be repeated.
  • the Machine Identifiers 122 are assigned dynamically, there is a good chance that the display computer 120 will have a new Machine Identifier 122 at such a later time. If a computer name or generic name was used, then once the user starts up the AirConnect I M App on the mobile device, a list of computer names or generic names can be displayed in the AirConnect 1 M App and the user can chose the one to connect to. These names can be either the computers running a common application that are on the same network that is associated with the wireless network that the mobile device is on, or they can be the ones that have been used previously by this mobile device. Further discussion of the common application that allows more than one device to be displayed on the Common Display 1 10 may be found in U.S. Patent Application No. 14/540,946, which is hereby incorporated by reference in its entirety for all purposes. If the Machine Identifier 122 does not change very often, the AirConnect rM App may remember the Machine Identifier 122 in a list.
  • the AirConnect rM App on the mobile device may not be able to determine the machine identifier of the display computer 120 to send the information thereto.
  • the user can retype in the new Machine Identifier, or a list of names and Machine Identifies can be stored on a server and the Air Connect App can look up the new Machine Identifier on a the server.
  • Another way to solve this problem is to associate a beacon 140 with each display
  • AirAirConnectTM Connect Server either in the Cloud (see FIG. 1 ) or at a predetermined network location .
  • the AirConnect I M App on the mobile device can report information on nearby beacons to the AirConnect 1 M Server including the UUID for each beacon and the signal strength of each beacon. Based on this information, the AirConnect' M server can determine which display computer 120 is closest to the mobile device 200a.
  • the AirConnect I M App on the display computer 120 can also report information on
  • the AirConnect server may then automatically connect the mobile device 200a, 200b with the appropriate display computer 120 or can transmit to the AirConnect 1 M App generic names and/or computer names of all display computers 120 within range and also it can transmit the Machine Identifiers 122 for each of these.
  • the AirConnect 1 M Server may maintain a list of all display computer 120 names and their associated Machine Identifiers 122 that can be
  • the list can also include generic names, if updated by a user.
  • the AirConnect 1 M App on the mobile device may list these computer names or generic names on the screen 250 of the mobile device as shown in FIG. 3 and prompt the user to select the display computer to which it wants to connect.
  • GPS information or other Bluetooth protocols can be used to send location information to the cloud server from the application on the display computer 120.
  • CD-Beacons 140 may be
  • the AirConnect IM App upon initial activation can receive a list of CD-Beacons nearby.
  • the AirConnectTM App may then be in sleep mode and be programmed to wake-up when the mobile device is within a pre-determined range of a CD-Beacon 140, either any CD- Beacon 140, or any one of a list of specific CD-Beacons 140.
  • the AirConnectTM App may present a list of potential Common Displays 1 10 to the user to connect to or may automatically connect the Mobile Device to the Common Display 1 10 with the CD-Beacon 140 that is closest to the Mobile Device.
  • a confirmation screen of the connection may be displayed on the Common Display 1 10, and the user would tap or perform a gesture near the Common Display 1 10 to confirm the connection to the mobile device.
  • the gesture may be recognized by an appropriately placed camera associated with the display computer 120.
  • the mobile device(s) 220a and the common display 1 10 may interact in numerous manners, as discussed below with reference to FIG. 2B.
  • an icon of this mobile device (Mobile Device Icon or MDI) 220a. 220b, etc., may be shown on the Common Display 1 10, e.g., in the tray region 1 14, and the MD may stream data to the display computer 120, in operation 65, after receiving a resolution request from the display computer 120 in operation 60.
  • the MDI could be either a graphic image or a thumbnail of a small live video stream of the MDDS.
  • the display computer 120 may tell the mobile device MD the resolution of the data stream needed for the MDI, e.g., 160x90, in operation 60.
  • the user of the mobile device MD may stream an entirety of the display on the MD (see FIGS. 4 to 7) or may select only a portion of the display on the mobile display MD (see FIG. 8).
  • the data stream from the MD may be shown in a tray region 1 14, e.g., an area of the common display 1 10 designated for indicating which mobile devices MDs are connected thereto. As shown in FIGS. 4 to 8, the tray region 1 14 may be in a bottom portion of the common display 1 10.
  • MDIs can be dragged by a user by a touch gesture on the Common Display touch sensor 1 16, e.g., a drag from the tray region 1 14 to the display region 1 12.
  • Alternative method of dragging the MDIs 220a, 220b would include gesture recognition, voice recognition, or other user interface, as discussed below with reference to FIG. 9.
  • the MDIs in the tray region 1 14 could be automatically displayed on the display region 1 12.
  • the MDDS may occupy a MDW to match the aspect ratio of the MDDS. This MDW may occupy as much of the display region 1 12 as possible (See FIGS. 4 and 5). The MDW may take up the entire screen if the MDDS is the same aspect ratio as the display region 1 12 of the Common Display 1 10.
  • each MDW may have a same size and together occupy approximately half of the screen (see FIG. 7). Similarly, when there are four MDWs, each MDW may occupy one quarter of the display region 1 12 (see FIG. 6).
  • the Canvas is a virtual region that expands to greater than the physical area of the display region of the Common Display, e.g., any number of times the physical area up to infinite.
  • the use of the Canvas allows additional files to be accessible and can be saved, but off the display region. Further discussion of the canvas may be found in U.S. Patent Application No. 14/540,946, which is hereby incorporated by reference in its entirety for all purposes.
  • Gestures, such as pan, zoom and pinch gestures can be made to move and resize the scale of the Canvas, allowing the full canvas to be display at once or only a small section thereof.
  • the system can be configured so that the MDW can be moved and manipulated.
  • touch gestures such as pan, zoom and pinch may be used to move and/or resize the MDWs. Snapshots of the MDWs can be taken from the Common Display, with the use of touch gestures. Annotations could be performed on the MDWs. Further details may be found, e.g., in U.S. Patent Application No. 14/540,946.
  • the MDW when the first MD is connected, the MDW may be automatically placed onto the center of the Canvas and the Canvas may be automatically zoomed in such that the MDW fills most of the display region 1 12. If MVS is set to 1 , then when a second MDDS is connected, the second MDW may be automatically placed onto an empty section of the canvas (typically next to the first MDW) and the Canvas may be automatically panned and zoomed to view only the second MDW. To see both MDWs side by side, the user can pinch, zoom and pan the Canvas and/or the MDW. As in the AutoMode, the MDW may be sized to the same aspect ratio of the MDDS (or the MD screen).
  • the MVS is set to 4
  • 4 MDWs at a time may be automatically displayed at one time, with each occupying approximately 1 ⁇ 4 of the viewable Canvas on the display region 1 12.
  • the oldest MDDS connection is not dropped, but the Canvas is just moved so that the oldest MDW is out of the viewable area of the Canvas on the display region 1 12.
  • a screen of the mobile device 220a may be displayed in an
  • the MDWa in the display 120 may fill up the display region 1 16.
  • the MDWa to MDWd showing contents of each of these devices can also fill the screen, as shown in FIG. 6.
  • FIG. 7 when a different number of mobile devices or devices having different aspect ratios are to have MDWs displayed on the common display 1 10, and these are to be as large as possible, then there will be a lot of dead space above and below the mobile device contents, as shown in FIG. 7.
  • contents of multiple mobile devices 200a, 200b may be displayed in MDWa, MDWb on the display region 1 10 in a manner that fills up the entire display region 112, as shown in FIG. 8.
  • User 1 could send half of the screen on the mobile device 200a and User 2 could send half of the screen 200b to the common display 1 10, such that the ratio of the video signal needed for each user would then be, e.g., 8:9, rather than 16:9.
  • These signals would be sent to the display computer 120, which can then combine these two signals on the display region 1 12 to fill the display region 1 12. Implementations to realize the partial screen display are discussed in detail below.
  • the MDDS from multiple devices may be displayed in individual MDWs on the display
  • each MDW may be moved, expanded and or resized, for example through the use of touch or other gestures.
  • different resolutions may be used within the MDW.
  • the MDW covers the entirety of the common display 120, a full resolution, e.g., 1920x1080 resolution may be used.
  • the stream resolution may be reduced, e.g., 960x540 pixels, and if the user zooms the window smaller the stream resolution may be further reduced, e.g., 480x270 pixels.
  • the display computer 1 12 may instruct the mobile device 200a. 200b to down sample the information to be transmitted to the common display 1 10, in operation 75.
  • This down sampling (Dynamic Resolution Scaling (DRS)) may increase the speed of the system and reduce network congestion.
  • DRS Dynamic Resolution Scaling
  • the Display Computer 120 may instruct the MD to only send the resolution and frame rate (e.g., 160x90 pixels @ 1 frame per 2 seconds) needed to render the MDI in the tray 1 14, since this will be the only visible representation of the MDDS.
  • the Display Computer 120 may determine, in operation 70, a first resolution
  • Rl of the data stream sent by the MD to the Display Computer 120 and whether a region, e.g., the tray region 1 14 or an MDW in the display region 1 12, in which the data stream is to be presented and a second resolution (R2) associated with that region is different than the first resolution Rl .
  • Display Computer 120 may at first expand the content to fill the MDW, even though sufficient resolution cannot be displayed. At the same time, the Display Computer 120 may request, in operation 75, the MD through the AirConnect rM App to transmit the MDDS at a higher resolution. The net result will be that as the MDW is expanded it will first appear slightly blurry as it expands and then, as the higher resolution MDDS is received, it will fill in the MDW to show the MDDS at the resolution needed to match the MDW. Alternatively, the Display Computer 120 may request a resolution larger than needed for the MDW, but not as large as the full resolution of the display region 1 12, e.g., to smooth the transition to a larger image.
  • FRM Full Resolution Mode
  • a MD when a MD first connects to the display computer, it sends its MDDS at the full resolution of the MD.
  • FRM is described, e.g., in U.S. Patent Application No. 14/540,946, in which the full resolution of the MDDS is stored on the display computer and down sampled to the size needed for the MDW. Then as the MDW is expanded the downsampling is reduced. While the FRM may allow resolution can be quickly changed as the window is expanded, FRM may slow down the system, when multiple MDDS are simultaneously being transmitted.
  • Partial screen display may be implemented using a touchscreen. Each user could send their full contents of their mobile devices to the display computer 120.
  • the display computer 120 could send both signals to the common display 1 10. Then, if the common display 1 10 is a touchscreen, a user could use the touchscreen to select a portion of each mobile device signal to be displayed on the common display 1 10. For example, the user could select with the use of a window drawn on the common display 1 10, the left half of each of the two signals displayed. Once selected the unselected portions (e.g. the right half) of each signal would not be displayed. At this point the display computer 120 could expand the contents of the selected regions to fill the common display 1 10 or the user could expand the contents therein through the use of gestures.
  • Yet another option is to select the portion of their mobile device to be sent to the common display 1 10 using the mobile device. Only this selected portion would then be sent to the display computer 120. Selection on the mobile device may be realized by either framing a window on the mobile device or selecting a third party application running on the mobile device and sending only the contents of this third party application to the display computer 120, reducing the information transmitted to the display computer 120.
  • Other options to select a portion of the mobile device include: set options, e.g. send left half or right half of screen, 1/3 or other fractions, and have dividers placed on screen so that users can move documents in to fill up frames displayed. This may also increase the speed of the system.
  • Another option is to do the same if a touch sensor is associated with the common display
  • a smaller touchscreen display 150 e.g., in table, having a similar aspect ratio as the common display 1 10 and a touch sensor, may be used to control the common display 1 10, as shown in FIG. 9.
  • the connected mobile device contents could be displayed on this small touch screen.
  • five mobile devices 200a to 200e are connected wirelessly to the display computer 120. All of these devices could then be displayed on the small touch screen 150, e.g., each in a window with a 16x9 ratio, to be referred to as a "user proto-window.”
  • a user could select individual user proto-windows to be displayed on the common display 1 10 and may then select portions of each of these user proto-windows.
  • a special outline or graphic could be added to the MDW for a MD device that would denote that items can be dragged to the MDW, or in other words that FTM is activated for the corresponding MD.
  • an image or a group of pdfs or jpegs on the display region of the common display 1 10 could then be dragged to the Mobile Device Icon or the MDW to send these files to the mobile device 220a.
  • the Display Computer would send the file for the image to the MD through the AirConnect App.
  • the AirConnect App may display a question on the MD asking the user if the file transfer is to be allowed.
  • files may also be dragged from an MDW or an MDI to the Canvas.
  • a File Structure Window may be superimposed on or near the MDW on the Common Display, displaying icon representations of each file and folders that the AirConnect rM App has access to.
  • the user of the MD may give the AirConnectTM App access to their entire hard drive or to only files that they place in certain folders.
  • a user may drag the file icon from the FSW on to the Common Display. This will then the cause the Display Computer to signal the MD through the AirConnect App for the AirConnect App to send a copy of the file to the Display Computer.
  • the Display Computer can then display the file on the Common Display. For example, if the file is a pdf, then the icon of the pdf would be displayed in the FSW. Once dragged to the Common Display and the pdf is received by the Display Computer, the Display Computer could then display the the pdf with the first page of the pdf initially visible. By performing touch gestures on this first page, the other pages can then be displayed, since the Display Computer has the entire file.
  • a user can go to a touch screen, select a file from a MDW, drag it off of the MDW on to the Common Display, then edit the file on the Common Display (e.g., annotate) and then drag it back to the MDW to send the edited file back to the MD.
  • Whether the full mobile device MD window is still in view on the Common Display 1 10 may be determined in operation 80. If not, the display computer 120 may request that the mobile device MD just stream the thumbnail data in operation 85. If yes, then the streaming may proceed unaltered.
  • Such changes in the display region 1 16 of the common display 1 10 may be determined any time a change is made on the image of the associated mobile device or may only be determined when a user indicates the image is to be updated on the mobile device, e.g., drags the changed image back to the drags the image back to the tray region 1 14.
  • the display computer 120 may stream the changed data back to the corresponding mobile device.
  • a warning may be sent to the mobile device alerting the mobile device that it is about to be disconnected in operation 92. Such a warning may be audible, visible, and/or tactile. If a user selects to stay connected in operation 94, the connection may be maintained. If the user selects to disconnect in operation 94, the session may end in operation 96 and the mobile device is disconnected and disappears from the display portion 1 16 and the tray 1 14.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • User Interface Of Digital Computer (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Telephone Function (AREA)

Abstract

A method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display, and sending a data stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device.

Description

SYSTEM FOR CONNECTING A MOBILE DEVICE AND A COMMON DISPLAY
SUMMARY
[0001] One or more embodiments are directed to a method of connecting a mobile device to a common display having associated with a display computer controlling the common display, including, for each mobile device downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display; and sending a data stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device. The display computer is to determine a first resolution (Rl) of the data stream sent by the mobile device to the display computer, determine a region in which the data stream is to be presented and a second resolution (R2) associated therewith, and when R2 is less than Rl , then sending a third resolution (R3) to the device, where Rl > R3 >= R2, and the display application on the mobile device is to down sample the selection in accordance with the resolution, R3, and only transmitting the down sampled resolution (R3) to the display computer.
[0002] An entirety of the mobile display may be selected to be sent to the common display and the region may be a window with an R2 less than Rl . An entirety of the mobile display may be selected to be sent to the common display and the region may be a window with Rl . The region may be a tray region outside a display region of the common display where windows are displayed.
[0003] An entirety of the mobile display may be selected to be sent to the common display and the region is a window, the common display is a touch screen display and the user may select only a portion of the window to be displayed by performing a touch tap or gesture on the common display.
[0004] An entirety of the mobile display may be selected to be sent to the common display and the region is may be a window on a touchscreen display, separate from the common display, the user may select only a portion of the window on the touchscreen display to be displayed on the common display.
[0005] Selecting what is to be displayed on the common display may include selecting only a portion of a screen on the mobile device.
[0006] The window may be an editable window can be moved, resized, and marked on. [0007] When a user modifies a file on the common display and then performs a predetermined action on the common display involving the selection of one of the mobile devices, the modified file may be transmitted to the corresponding mobile device.
[0008] The method may include automatically displaying the selection in the window.
[0009] The method may include displaying a device icon in a tray region outside a display
region of the common display where windows are displayed.
[0010] The method may include displaying the selection in the window when the device icon is moved toward the display region on the common display.
[001 1 ] When the window is moved toward the device icon, the display computer may transmit the file represented in the window to the corresponding device.
[0012] The method may include receiving a warning signal from the display computer when the device is more than a predetermined distance from the display computer to which it is connected.
[0013] Selecting may include selecting from a list of display computers on the mobile device.
[0014] Selecting the display computer may include inputting a machine identifier of the display computer.
[0015] Selecting may include automatically connecting to the common display closest to the device.
[0016] One or more embodiments are directed a method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device: downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, only a portion of the mobile device to be displayed on the common display; and sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device, the window for the portion of the mobile device occupying a display region of the common display.
[0017] The method may include connecting another mobile device to the common display and selecting, on the other mobile device, only a portion of the mobile device to be displayed on the common display, wherein windows for the mobile device and the other mobile device occupy the display region of the common display. [0018] The method may include determining, by the display computer, a size of the window and a resolution associated therewith, sending the resolution to the device, and down sampling the selection in accordance with the resolution and only transmitting the down sampled resolution to the display computer.
[0019] One or more embodiments are directed a method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device: downloading a display application onto the mobile device, detecting a signal associated with display computers indicative of a location of the display computers, selecting the display computer for the common display in the display application based on a closest display computer to the mobile device, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display, and sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device.
[0020] The method may include automatically determining, selecting, and connecting to the common display closest to the device.
[0021 ] The signal may be from a beacon associated with the display computer.
[0022] Selecting may include showing a list of the display computers in the mobile device with the closest display computer being listed first.
[0023] One or more embodiments are directed a method of connecting a mobile device to a common display having a discernible machine identifier associated with a display computer controlling the common display, including downloading a display application onto the mobile device, selecting the display computer for the common display in the display application, connecting the mobile device to the display computer, selecting, on the mobile device, what is to be displayed on the common display, sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device, and sending a warning signal from the display computer to the mobile device when the mobile device is more than a predetermined distance from the display computer to which it is connected.
BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Features will become apparent to those of skill in the art by describing in detail
exemplary embodiments with reference to the attached drawings in which: [0025] FIG. 1 illustrate a block diagram of a display system in accordance with an embodiment;
[0026] FIGS. 2 A and 2B illustrate a flowchart in accordance with an embodiment;
[0027] FIG. 3 illustrates an application selection page on a mobile device;
[0028] FIG. 4 illustrates a block diagram of a display system including a beacon in accordance with an embodiment; and
[0029] FIGS. 5 to 9 illustrate block diagrams of a display system interacting with mobile
device(s) in accordance with embodiments.
DETAILED DESCRIPTION
[0030] Example embodiments will now be described more fully hereinafter with reference to the accompanying drawings; however, they may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey exemplary implementations to those skilled in the art.
[0031] FIG. 1 illustrates a block diagram of a display system 100 interacting with one or more mobile devices 200a, 200b, and so forth. The display system 100 includes a Common Display 1 10, a Display Computer 120, and a wireless router 130 serving as a wireless access point (WAP), all interconnected. The Common Display 1 10 may be an LCD display, LED display, or other monitor that is capable of having an electronic video signal as an input and converting the input to a visual image.
[0032] The Common Display 1 10 may include a display region 1 12 and a tray region 1 14, e.g., below the display region. The Common Display 1 10 a touch sensor 1 16, e.g., overlaying an entirety of the Common Display 1 10, that it is sensitive to touch inputs including taps and gestures.
[0033] Information regarding a Machine Identifier 122 of the Display Computer 120 and the digital information to be displayed on the Common Display 1 10 may be sent from the Display Computer 120 to the Common Display 1 10. This digital information can be within windows or Mobile Device Windows (MDWs), e.g.. editable windows, or on the entire screen of display region 1 12 of the Common Display 1 10. In addition, there may be windows displaying contents from mobile devices (MD) or other appropriate mobile device icons (MDI) 220a, 220b, e.g., a thumbnail of what is displayed on the mobile device, in the tray region 1 14 on the Common Display 1 10, e.g., at a lower region thereof. The tray region 1 14 may be a region on which the MDWs cannot be zoomed and pinched, annotated, and so forth, but may be dragged, tapped or tossed onto the display region 1 12, e.g., to open an MDW corresponding to the MDI, and/or to received MDWs from the display region 1 12 to transmit that MDW to the mobile device corresponding to the MDI..
[0034] Digital information from Mobile Device 1 (200a) and Mobile Device2 (200b) is streamed from these Mobile Devices to the Display Computer 120 through the network. In FIG. 1 , digital information is streamed from the mobile devices through the WAP 130 to the Display Computer 120. This streaming data may then be displayed in the Mobile Device Windows (MDW) (see FIGS. 4 to 9) on the Common Display 1 10.
[0035] Typically, the data streamed from the Mobile Device (200a and/or 200b) is a video signal that is the same as the signal displayed on the Mobile Device. This streaming data will be referred to as a Mobile Devices Data Stream (MDDS). A MDDS is typically displayed within a MDW (see associated screens in FIGs. 6 to 9) in the display region 1 12 on the Common Display 1 10. Within or adjacent this MDQ, there may also be displayed an identifier associated with the Mobile Device, e.g, the name of the MDDS, a name associated with the corresponding Mobile Device, the Mobile Device's owner, and so forth.
[0036] If the mobile device 200a is running a Mac® operating system or an iphone® operation system (iOS), then the user can use Airplay® to locate the Display Computer 120 that is configured as an Airplay® receiver. Airplay® is a video mirroring capability built in to Mac® OS and iOS. Using Airplay® to locate Airplay® receivers, the user can then view any devices on the network that are configured to receive Airplay® inputs, typically just Apple TV®s.
[0037] However, in accordance with an embodiment, the display computer 120 may include and run software, firmware, and/or hardware that acts as a video mirroring receiver. Then, when a user gets on the wireless network and searches for Airplay® receivers, the name of the display computer 120 will appear as an option to stream video within Airplay®. Once connected, the entire contents of the mobile device 200a may be streamed to the display computer 120.
[0038] However, Airplay® employs a Bonjour® network protocol. Some companies block
Bonjour® protocol. Further, Bonjour® broadcasts will not travel across sub-networks. Finally, if the mobile device is using an operating system other than Mac® or iOS, Airplay® is not an option. [0039] Therefore, in accordance with an embodiment, a user of a mobile device may download a display application 210a, 210b thereon to assist in connecting the to the display computer 120. Such an application may be written for each common operating system and is herein referred to as an AirConnect1 M App. Instructions for downloading the AirConnectI M App may be readily viewable, e.g., on or adjacent the common display 1 10, or a region to be scanned, e.g., a bar code, quick response (QR) code, and so forth, using a mobile device QR, so that once scanned, the AirConnect™ App could be downloaded. Once the AirConnect1 M App is downloaded, then a user can launch the AirConnect™ App and then enter the Machine Identifier 122 associated common display 1 10. The Machine Identifier 122 may by an IP address or other alphanumeric code associated with the Display Computer 120. The Machine Identifier 122 may be simply displayed on the Common Display 1 10, in which case the user of the AirConnect™ App may simply enter the Machine Identifier 122 when prompted by the AirConnect™ App on their Mobile Device. Alternatively, the Machine Identifier 122 may be automatically transferred to the Mobile Device either by displaying a QR code on the Common Display 1 10 or by transmitting through bluetooth or wireless communication.
[0040] For example, a low energy bluetooth beacon (CD-Beacon) 140 (see FIG. 4), may be placed near the Common Display 1 10. This CD-Beacon 140 may transmit an identification number, e.g., a unique universal identification number (UUID). The AirConnect™ App can detect this UUID and send this information to a cloud server. The cloud server can store a lookup table with each CD-Beacon and the associated Machine Identifier 122 for each Display Computer 120.
[0041 ] Once the Machine Identifier 122 is entered, the AirConnect I M App may then use the
Machine Identifier 122 to connect the Mobile Device directly to the display computer 120.
[0042] Details of this are illustrated in FIGS. 2A and 2B, which is a flowchart of an operation of connecting Mobile Device(s) to the common display 1 10 in accordance with an embodiment.
[0043] In operation 10, a user walks within range of the display system 100 with a mobile device
(MD), connected to a wireless network. If the AirConnect1 M App is on the MD in operation 20. the user launches the AirConnect1 M App in operation 30 on the MD. If not, then the user may be instructed, e.g., by a notice posted on or near the common display 1 10, to download the
AirConnect1 M App on the MD in operation 25, and then to proceed to launch the AirConnect I M App in operation 30. [0044] Once launched, the AirConnect App may display a list of locally stored names and/or recently used connections on the mobile device. The Common Display 1 10 may also display the name or Machine Identifier 122 of the Display Computer 120 that it is connected to and/or the name of the Common Display 1 10 or Display Computer 120 may be posted near the Common Display 1 10. If the name associated with the Common Display 1 10 is not listed in the
AirConnect I M App, the user may type a Machine Identifier 122 that may be continuously displayed on or near the common display 1 10 to add the computer display 120 to the list of FIG. 3 in operation 45. The Machine Identifier 122 may be, e.g., name of the computer (host name), a UUID, and so forth.
[0045] Additionally or alternatively, the CD-Beacon 140 method described above may be used to add the computer to the list of computers in operation 45. The AirConnect™ App may capture UUIDs for any beacons within range of the Mobile Device. All of these UUIDs may be transmitted to an AirConnect™ Server and then the AirConnect™ Server may determine if any of these UUIDs are for CD-Beacons. If so, the corresponding Machine Identifiers can then be sent to the AirConnect™ App on the Mobile Device.
[0046] An example of a screen 250 that may appear when using the AirConnect™ App is shown in FIG. 3. Here, a user would be prompted to select which display to be connected with. These options may include a name of a room in which the common display 1 10 is located, a nickname for the common display that is visually apparent, the machine identifier of the common display that is visually apparent, and so forth. The screen 250 for selection may look the same regardless of the operating system of the mobile device running the AirConnect I M App.
[0047] Once the display computer 120 associated with the Common Display 1 10 is in the list displayed by the AirConnect™ App as shown in FIG. 3, the user of the MD may select a computer in operation 50, and then the AirConnect App can establish a connection between the MD device and the Display Computer, using the Machine Identifier associated with the name of the display computer in the list.. Once the connection is established, the display computer 120 may tell the MD what resolution to stream in operation 60, up to a maximum resolution of the MD, and the desired content of the MD may be streamed to the selected computer as discussed below in detail with reference to FIG. 2B. [0048] The Machine Information 122 regarding the selected computer may be saved on the mobile device 200a, 200b in operation 55. For example, if the user then leaves the room and disconnects and then returns, all of the initialization noted above may not need to be repeated.
[0049] If the Machine Identifiers 122 are assigned dynamically, there is a good chance that the display computer 120 will have a new Machine Identifier 122 at such a later time. If a computer name or generic name was used, then once the user starts up the AirConnectI M App on the mobile device, a list of computer names or generic names can be displayed in the AirConnect1 M App and the user can chose the one to connect to. These names can be either the computers running a common application that are on the same network that is associated with the wireless network that the mobile device is on, or they can be the ones that have been used previously by this mobile device. Further discussion of the common application that allows more than one device to be displayed on the Common Display 1 10 may be found in U.S. Patent Application No. 14/540,946, which is hereby incorporated by reference in its entirety for all purposes. If the Machine Identifier 122 does not change very often, the AirConnectrM App may remember the Machine Identifier 122 in a list.
[0050] However, when using generic names and/or computer names, if the Machine Identifier has changed, the AirConnectrM App on the mobile device may not be able to determine the machine identifier of the display computer 120 to send the information thereto. In this case there are several ways to solve this problem. The user can retype in the new Machine Identifier, or a list of names and Machine Identifies can be stored on a server and the Air Connect App can look up the new Machine Identifier on a the server.
[0051 ] Another way to solve this problem is to associate a beacon 140 with each display
computer 120, as illustrated in FIG. 4. Then when a person walks into a room and starts the AirConnect1 M App on a mobile device 200a, 200b, a request can be made to a server
(AirAirConnect™ Connect Server) either in the Cloud (see FIG. 1 ) or at a predetermined network location . When this request is made, the AirConnectI M App on the mobile device can report information on nearby beacons to the AirConnect1 M Server including the UUID for each beacon and the signal strength of each beacon. Based on this information, the AirConnect' M server can determine which display computer 120 is closest to the mobile device 200a.
[0052] The AirConnect I M App on the display computer 120 can also report information on
nearby beacon(s) 140. [0053] The AirConnect server may then automatically connect the mobile device 200a, 200b with the appropriate display computer 120 or can transmit to the AirConnect 1 M App generic names and/or computer names of all display computers 120 within range and also it can transmit the Machine Identifiers 122 for each of these. The AirConnect1 M Server may maintain a list of all display computer 120 names and their associated Machine Identifiers 122 that can be
automatically updated. The list can also include generic names, if updated by a user.
[0054] Once the AirConnect 1 M App on the mobile device 200a, 200b has the names of the
display computers within range, the AirConnect1 M App on the mobile device may list these computer names or generic names on the screen 250 of the mobile device as shown in FIG. 3 and prompt the user to select the display computer to which it wants to connect. Alternatively, GPS information or other Bluetooth protocols can be used to send location information to the cloud server from the application on the display computer 120.
[0055] Another option is to have a set-up procedure, whereby CD-Beacons 140 may be
registered with associated Common Displays 1 10 and display computers 120 and approximate GPS locations. The AirConnect IM App, upon initial activation can receive a list of CD-Beacons nearby. The AirConnect™ App may then be in sleep mode and be programmed to wake-up when the mobile device is within a pre-determined range of a CD-Beacon 140, either any CD- Beacon 140, or any one of a list of specific CD-Beacons 140. Upon wake-up, the AirConnect™ App may present a list of potential Common Displays 1 10 to the user to connect to or may automatically connect the Mobile Device to the Common Display 1 10 with the CD-Beacon 140 that is closest to the Mobile Device.
[0056] Alternatively or additionally, a confirmation screen of the connection may be displayed on the Common Display 1 10, and the user would tap or perform a gesture near the Common Display 1 10 to confirm the connection to the mobile device. The gesture may be recognized by an appropriately placed camera associated with the display computer 120.
[0057] Once connected, the mobile device(s) 220a and the common display 1 10 may interact in numerous manners, as discussed below with reference to FIG. 2B.
[0058] As shown in FIGS. 4 to 9, once a mobile device (MD) is connected to the display
computer 120, an icon of this mobile device (Mobile Device Icon or MDI) 220a. 220b, etc., may be shown on the Common Display 1 10, e.g., in the tray region 1 14, and the MD may stream data to the display computer 120, in operation 65, after receiving a resolution request from the display computer 120 in operation 60. The MDI could be either a graphic image or a thumbnail of a small live video stream of the MDDS. The display computer 120 may tell the mobile device MD the resolution of the data stream needed for the MDI, e.g., 160x90, in operation 60. Further, the user of the mobile device MD may stream an entirety of the display on the MD (see FIGS. 4 to 7) or may select only a portion of the display on the mobile display MD (see FIG. 8). Depending on an operational mode of the common display 1 10, the data stream from the MD may be shown in a tray region 1 14, e.g., an area of the common display 1 10 designated for indicating which mobile devices MDs are connected thereto. As shown in FIGS. 4 to 8, the tray region 1 14 may be in a bottom portion of the common display 1 10.
] In one operational mode (Standard Mode), to move Mobile Device data streams (MDDS) from the tray region 1 14 to the display region 1 12 of the Common Display the MDIs can be dragged by a user by a touch gesture on the Common Display touch sensor 1 16, e.g., a drag from the tray region 1 14 to the display region 1 12. Alternative method of dragging the MDIs 220a, 220b, would include gesture recognition, voice recognition, or other user interface, as discussed below with reference to FIG. 9.
] In an alternative operational mode (AutoMode) the MDIs in the tray region 1 14 could be automatically displayed on the display region 1 12. When the first MD is connected in AutoMode, the MDDS may occupy a MDW to match the aspect ratio of the MDDS. This MDW may occupy as much of the display region 1 12 as possible (See FIGS. 4 and 5). The MDW may take up the entire screen if the MDDS is the same aspect ratio as the display region 1 12 of the Common Display 1 10. When a second MDDS is connected, each MDW may have a same size and together occupy approximately half of the screen (see FIG. 7). Similarly, when there are four MDWs, each MDW may occupy one quarter of the display region 1 12 (see FIG. 6). When more than a specific number (Maximum Viewable Streams (MVS)) of MDDS windows are displayed, then when subsequent new MDDS connections are made, the oldest MDDS connection is dropped.] In another operational mode (AutoCanvasMode) there is a virtual canvas (Canvas) that exists within the software on the Display Computer 120. that may be much larger than the physical display region 1 12 on the Common Display 1 10. Any section of the Canvas can be viewed on the display region 1 12. Suppose that at first the entire Canvas is viewed on the display region 1 12. Then by zooming into regions, a portion of the Canvas can be viewed. By pinching, zooming and panning with gestures on the touch sensor 1 16, a user can zoom into various regions of the Canvas to be shown on the display region 1 12.
[0062] The Canvas is a virtual region that expands to greater than the physical area of the display region of the Common Display, e.g., any number of times the physical area up to infinite. The use of the Canvas allows additional files to be accessible and can be saved, but off the display region. Further discussion of the canvas may be found in U.S. Patent Application No. 14/540,946, which is hereby incorporated by reference in its entirety for all purposes. Gestures, such as pan, zoom and pinch gestures can be made to move and resize the scale of the Canvas, allowing the full canvas to be display at once or only a small section thereof. The system can be configured so that the MDW can be moved and manipulated. For example, if the Common Display 1 10 includes the touch sensor 1 16, then touch gestures such as pan, zoom and pinch may be used to move and/or resize the MDWs. Snapshots of the MDWs can be taken from the Common Display, with the use of touch gestures. Annotations could be performed on the MDWs. Further details may be found, e.g., in U.S. Patent Application No. 14/540,946.
[0063] In the Standard Mode, if a Canvas is used, and a user drags an MDI in to the display region 1 12 of the Common Display 1 10, then at this point, the user can zoom the MDDS window or can zoom the Canvas to make the MDDS window to fill most of the display region 1 12. Then if the user pans the Canvas, so that the MDDS window is off-screen, then the MDDS will not be visible on the display region 1 12.
[0064] In AutoCanvasMode, when the first MD is connected, the MDW may be automatically placed onto the center of the Canvas and the Canvas may be automatically zoomed in such that the MDW fills most of the display region 1 12. If MVS is set to 1 , then when a second MDDS is connected, the second MDW may be automatically placed onto an empty section of the canvas (typically next to the first MDW) and the Canvas may be automatically panned and zoomed to view only the second MDW. To see both MDWs side by side, the user can pinch, zoom and pan the Canvas and/or the MDW. As in the AutoMode, the MDW may be sized to the same aspect ratio of the MDDS (or the MD screen). Alternatively, if the MVS is set to 4, then 4 MDWs at a time may be automatically displayed at one time, with each occupying approximately ¼ of the viewable Canvas on the display region 1 12. When more than the MVS number of MDWs are displayed, then when subsequent new MDDS connections are made, the oldest MDDS connection is not dropped, but the Canvas is just moved so that the oldest MDW is out of the viewable area of the Canvas on the display region 1 12.
[0065] As shown in FIGS. 4 and 5, a screen of the mobile device 220a may be displayed in an
MDWa in the display 120. When the common display 1 10 and the mobile device each have a similar aspect ratio, the MDWa displaying the screen of the mobile device 200a may fill up the display region 1 16.
[0066] When four mobile devices 200a to 200d having similar aspect ratios as the display region
1 12 are connected, then the MDWa to MDWd showing contents of each of these devices can also fill the screen, as shown in FIG. 6. However, when a different number of mobile devices or devices having different aspect ratios are to have MDWs displayed on the common display 1 10, and these are to be as large as possible, then there will be a lot of dead space above and below the mobile device contents, as shown in FIG. 7.
[0067] However, by only sending partial screens of each mobile device 200a, 200b, contents of multiple mobile devices 200a, 200b may be displayed in MDWa, MDWb on the display region 1 10 in a manner that fills up the entire display region 112, as shown in FIG. 8. For example, suppose there are two mobile devices 200a, 200b, each operated by different user. User 1 could send half of the screen on the mobile device 200a and User 2 could send half of the screen 200b to the common display 1 10, such that the ratio of the video signal needed for each user would then be, e.g., 8:9, rather than 16:9. These signals would be sent to the display computer 120, which can then combine these two signals on the display region 1 12 to fill the display region 1 12. Implementations to realize the partial screen display are discussed in detail below.
[0068] The MDDS from multiple devices may be displayed in individual MDWs on the display
1 12. When they are editable windows, each MDW may be moved, expanded and or resized, for example through the use of touch or other gestures. As the size of an MDW on the common display 1 10 changes, different resolutions may be used within the MDW. When the MDW covers the entirety of the common display 120, a full resolution, e.g., 1920x1080 resolution may be used. When the MDW covers only ¼ of the screen, as shown in FIG. 6, the stream resolution may be reduced, e.g., 960x540 pixels, and if the user zooms the window smaller the stream resolution may be further reduced, e.g., 480x270 pixels. Thus, in accordance with a size of the MDW or reduced resolution on the common display 1 10 determined in operation 70, the display computer 1 12 may instruct the mobile device 200a. 200b to down sample the information to be transmitted to the common display 1 10, in operation 75. This down sampling (Dynamic Resolution Scaling (DRS)) may increase the speed of the system and reduce network congestion. Also in the case that a MDDS window is moved off of the viewable Canvas, the Display Computer 120 may instruct the MD to only send the resolution and frame rate (e.g., 160x90 pixels @ 1 frame per 2 seconds) needed to render the MDI in the tray 1 14, since this will be the only visible representation of the MDDS.
[0069] For example, the Display Computer 120 may determine, in operation 70, a first resolution
(Rl) of the data stream sent by the MD to the Display Computer 120 and whether a region, e.g., the tray region 1 14 or an MDW in the display region 1 12, in which the data stream is to be presented and a second resolution (R2) associated with that region is different than the first resolution Rl . When R2 is less than Rl , the Display Computerl20 may request, in operation 75, a third resolution (R3) to the device, where Rl > R3 >= R2, and the AirConnect1 M App on the mobile device is to down sample the selection in accordance with the resolution, R3, and only transmitting the down sampled resolution (R3) to the Display Computer 120
[0070] Additionally, in DRS mode when the MDW is expanded on the touchscreen 1 16, the
Display Computer 120 may at first expand the content to fill the MDW, even though sufficient resolution cannot be displayed. At the same time, the Display Computer 120 may request, in operation 75, the MD through the AirConnectrM App to transmit the MDDS at a higher resolution. The net result will be that as the MDW is expanded it will first appear slightly blurry as it expands and then, as the higher resolution MDDS is received, it will fill in the MDW to show the MDDS at the resolution needed to match the MDW. Alternatively, the Display Computer 120 may request a resolution larger than needed for the MDW, but not as large as the full resolution of the display region 1 12, e.g., to smooth the transition to a larger image.
[0071 ] In contrast, in Full Resolution Mode (FRM), when a MD first connects to the display computer, it sends its MDDS at the full resolution of the MD. FRM is described, e.g., in U.S. Patent Application No. 14/540,946, in which the full resolution of the MDDS is stored on the display computer and down sampled to the size needed for the MDW. Then as the MDW is expanded the downsampling is reduced. While the FRM may allow resolution can be quickly changed as the window is expanded, FRM may slow down the system, when multiple MDDS are simultaneously being transmitted. [0072] Partial screen display may be implemented using a touchscreen. Each user could send their full contents of their mobile devices to the display computer 120. The display computer 120 could send both signals to the common display 1 10. Then, if the common display 1 10 is a touchscreen, a user could use the touchscreen to select a portion of each mobile device signal to be displayed on the common display 1 10. For example, the user could select with the use of a window drawn on the common display 1 10, the left half of each of the two signals displayed. Once selected the unselected portions (e.g. the right half) of each signal would not be displayed. At this point the display computer 120 could expand the contents of the selected regions to fill the common display 1 10 or the user could expand the contents therein through the use of gestures.
[0073] Yet another option is to select the portion of their mobile device to be sent to the common display 1 10 using the mobile device. Only this selected portion would then be sent to the display computer 120. Selection on the mobile device may be realized by either framing a window on the mobile device or selecting a third party application running on the mobile device and sending only the contents of this third party application to the display computer 120, reducing the information transmitted to the display computer 120. Other options to select a portion of the mobile device include: set options, e.g. send left half or right half of screen, 1/3 or other fractions, and have dividers placed on screen so that users can move documents in to fill up frames displayed. This may also increase the speed of the system.
[0074] Another option is to do the same if a touch sensor is associated with the common display
1 10, but does not overlay the common display 1 10. For example, a smaller touchscreen display 150, e.g., in table, having a similar aspect ratio as the common display 1 10 and a touch sensor, may be used to control the common display 1 10, as shown in FIG. 9. The connected mobile device contents could be displayed on this small touch screen. Suppose five mobile devices 200a to 200e are connected wirelessly to the display computer 120. All of these devices could then be displayed on the small touch screen 150, e.g., each in a window with a 16x9 ratio, to be referred to as a "user proto-window." A user could select individual user proto-windows to be displayed on the common display 1 10 and may then select portions of each of these user proto-windows. These user proto-windows could be rearranged on the small touchscreen display 150 by moving them around, expanding and contracting them to fill the small touchscreen display 150, including selecting and removing portions of each and arrange them in a way to fill the small touchscreen display 150 and then mirror the entire touchscreen display 150 on to the common display 1 10. [0075] Actions performed on the display region 1 16 (or the touchscreen display 150) may be transmitted back to the corresponding mobile device. For example, when a user connects with the AirConnect1 M App, a special MDI may appear in the primary display region, that denotes that items can be dragged to this MDI (File Transfer Mode (FTM)). Alternatively, a special outline or graphic could be added to the MDW for a MD device that would denote that items can be dragged to the MDW, or in other words that FTM is activated for the corresponding MD. For example, an image or a group of pdfs or jpegs on the display region of the common display 1 10 could then be dragged to the Mobile Device Icon or the MDW to send these files to the mobile device 220a. For example, if the MDW has FTM activated and an image is dragged to the MDW, then the Display Computer would send the file for the image to the MD through the AirConnect App. The AirConnect App may display a question on the MD asking the user if the file transfer is to be allowed.
[0076] File Transfer from MD to Common Display - Initiated from Common Display
[0077] Similarly, files may also be dragged from an MDW or an MDI to the Canvas. For
example, if FTM is activated on an MDW or an MDI, then by performing a gesture or tapping or selecting a graphic on the MDW, a File Structure Window (FSW) may be superimposed on or near the MDW on the Common Display, displaying icon representations of each file and folders that the AirConnect rM App has access to. The user of the MD may give the AirConnect™ App access to their entire hard drive or to only files that they place in certain folders. Once the FSW is displayed on the Common Display, then a user may drag the file icon from the FSW on to the Common Display. This will then the cause the Display Computer to signal the MD through the AirConnect App for the AirConnect App to send a copy of the file to the Display Computer. Once received, the Display Computer can then display the file on the Common Display. For example, if the file is a pdf, then the icon of the pdf would be displayed in the FSW. Once dragged to the Common Display and the pdf is received by the Display Computer, the Display Computer could then display the the pdf with the first page of the pdf initially visible. By performing touch gestures on this first page, the other pages can then be displayed, since the Display Computer has the entire file.
[0078] Note that with this FTM, a user can go to a touch screen, select a file from a MDW, drag it off of the MDW on to the Common Display, then edit the file on the Common Display (e.g., annotate) and then drag it back to the MDW to send the edited file back to the MD. [0079] Whether the full mobile device MD window is still in view on the Common Display 1 10 may be determined in operation 80. If not, the display computer 120 may request that the mobile device MD just stream the thumbnail data in operation 85. If yes, then the streaming may proceed unaltered.
[0080] Such changes in the display region 1 16 of the common display 1 10 may be determined any time a change is made on the image of the associated mobile device or may only be determined when a user indicates the image is to be updated on the mobile device, e.g., drags the changed image back to the drags the image back to the tray region 1 14. Once a change is determined in operation 80, the display computer 120 may stream the changed data back to the corresponding mobile device.
[0081 ] If operation 90 determined a mobile device is moved further than a predetermined
distance from the common display 1 10 and is still connected thereto, a warning may be sent to the mobile device alerting the mobile device that it is about to be disconnected in operation 92. Such a warning may be audible, visible, and/or tactile. If a user selects to stay connected in operation 94, the connection may be maintained. If the user selects to disconnect in operation 94, the session may end in operation 96 and the mobile device is disconnected and disappears from the display portion 1 16 and the tray 1 14.
[0082] Example embodiments have been disclosed herein, and although specific terms are
employed, they are used and are to be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, as would be apparent to one of ordinary skill in the art as of the filing of the present application, features, characteristics, and/or elements described in connection with a particular embodiment may be used singly or in combination with features, characteristics, and/or elements described in connection with other embodiments unless otherwise specifically indicated. Accordingly, it will be understood by those of skill in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as set forth in the following claims.

Claims

WHAT IS CLAIMED IS:
1. A method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device:
downloading a display application onto the mobile device;
selecting the display computer for the common display in the display application; connecting the mobile device to the display computer;
selecting, on the mobile device, what is to be displayed on the common display; and sending a data stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device, wherein the display computer is to:
determine a first resolution (Rl) of the data stream sent by the mobile device to the display computer,
determine a region in which the data stream is to be presented and a second resolution (R2) associated therewith, and
when R2 is less than Rl , then sending a third resolution (R3) to the device, where Rl > R3 >= R2, and
the display application on the mobile device is to
down sample the selection in accordance with the resolution, R3, and only transmitting the down sampled resolution (R3) to the display computer.
2. The method as claimed in claim 1 , wherein an entirety of the mobile display is selected to be sent to the common display and the region is a window with an R2 less than Rl .
3. The method as claimed in claim 2, wherein an entirety of the mobile display is selected to be sent to the common display and the region is a window with Rl .
4. The method as claimed in claim 1. wherein the region is a tray region outside a display region of the common display where windows are displayed.
5. The method as claimed in claim 1 , wherein an entirety of the mobile display is selected to be sent to the common display and the region is a window, the common display is a touch screen display and the user selecting only a portion of the window to be displayed by performing a touch tap or gesture on the common display.
6. The method as claimed in claim 1 , wherein an entirety of the mobile display is selected to be sent to the common display and the region is a window on a touchscreen display, separate from the common display, the user selecting only a portion of the window on the touchscreen display to be displayed on the common display.
7. The method as claimed in claim 1 , wherein selecting what is to be displayed on the common display includes selecting only a portion of a screen on the mobile device.
8. The method as claimed in claim 1 , wherein the window is an editable window can be moved, resized, and marked on.
9. The method as claimed in claim 1 , wherein, when a user modifies a file on the common display and then performs a predetermined action on the common display involving the selection of one of the mobile devices, the modified file is transmitted to the corresponding mobile device.
10. The method as claimed in claim 1 , further comprising automatically displaying the selection in the window.
1 1. The method as claimed in claim 1 , further comprising displaying a device icon in a tray region outside a display region of the common display where windows are displayed.
12. The method as claimed in claim 1 1 , further comprising displaying the selection in the window when the device icon is moved toward the display region on the common display.
13. The method as claimed in claim 1 1 , wherein, when the window is moved toward the device icon, the display computer transmits the file represented in the window to the corresponding device.
14. The method as claimed in claim 1 , further comprising:
receiving a warning signal from the display computer when the device is more than a predetermined distance from the display computer to which it is connected.
15. The method as claimed in claim 1 , wherein selecting includes selecting from a list of display computers on the mobile device.
16. The method as claimed in claim 1 , wherein selecting the display computer includes inputting a machine identifier of the display computer.
17. The method as claimed in claim 1 , wherein selecting includes automatically connecting to the common display closest to the device.
18. A method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device:
downloading a display application onto the mobile device;
selecting the display computer for the common display in the display application;
connecting the mobile device to the display computer;
selecting, on the mobile device, only a portion of the mobile device to be displayed on the common display; and
sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device, the window for the portion of the mobile device occupying a display region of the common display.
19. The method as claimed in claim 18, further comprising:
connecting another mobile device to the common display; and selecting, on the other mobile device, only a portion of the mobile device to be displayed on the common display, wherein windows for the mobile device and the other mobile device occupy the display region of the common display.
20. The method as claimed in claim 18, further comprising:
determining, by the display computer, a size of the window and a resolution associated therewith;
sending the resolution to the device; and
down sampling the selection in accordance with the resolution and only transmitting the down sampled resolution to the display computer.
21. A method of connecting a mobile device to a common display associated with a display computer controlling the common display, including, for each mobile device:
downloading a display application onto the mobile device;
detecting a signal associated with display computers indicative of a location of the display computers;
selecting the display computer for the common display in the display application based on a closest display computer to the mobile device;
connecting the mobile device to the display computer;
selecting, on the mobile device, what is to be displayed on the common display; and sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device.
22. The method as claimed in claim 21 , further comprising automatically
determining, selecting, and connecting to the common display closest to the device.
23. The method as claimed in claim 21 , wherein the signal is from a beacon associated with the display computer.
24. The method as claimed in claim 21 , wherein selecting includes showing a list of the display computers in the mobile device with the closest display computer being listed first.
25. A method of connecting a mobile device to a common display having a discernible machine identifier associated with a display computer controlling the common display, including:
downloading a display application onto the mobile device;
selecting the display computer for the common display in the display application; connecting the mobile device to the display computer;
selecting, on the mobile device, what is to be displayed on the common display; sending a live continuous video stream of the selection to the display computer to be displayed in a window on the common display corresponding to the mobile device; and
sending a warning signal from the display computer to the mobile device when the mobile device is more than a predetermined distance from the display computer to which it is connected.
PCT/US2017/019951 2016-02-29 2017-02-28 System for connecting a mobile device and a common display Ceased WO2017151615A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP25191101.2A EP4617849A3 (en) 2016-02-29 2017-02-28 System for connecting a mobile device and a common display
EP17760610.0A EP3423934A4 (en) 2016-02-29 2017-02-28 SYSTEM FOR CONNECTING A MOBILE DEVICE AND A COMMON DISPLAY DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/056,787 2016-02-29
US15/056,787 US10616633B2 (en) 2016-02-29 2016-02-29 System for connecting a mobile device and a common display

Publications (1)

Publication Number Publication Date
WO2017151615A1 true WO2017151615A1 (en) 2017-09-08

Family

ID=59678621

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/019951 Ceased WO2017151615A1 (en) 2016-02-29 2017-02-28 System for connecting a mobile device and a common display

Country Status (3)

Country Link
US (2) US10616633B2 (en)
EP (2) EP3423934A4 (en)
WO (1) WO2017151615A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ES2725461T3 (en) 2014-02-14 2019-09-24 Pluto Inc Procedures and systems to generate and provide program and content guides
US20240314378A1 (en) * 2018-05-09 2024-09-19 Pluto Inc. Methods and systems for generating and providing program guides and content
EP3803559A4 (en) 2018-06-05 2022-04-27 T1V, Inc. VIDEO CONFERENCE DURING REAL-TIME COLLABORATION ON A VIRTUAL SCREEN
KR102689698B1 (en) * 2019-01-03 2024-07-31 삼성전자주식회사 Display apparatus and controlling method thereof
CN110677727B (en) * 2019-09-10 2022-05-10 北京奇艺世纪科技有限公司 Audio and video playing method and device, electronic equipment and storage medium
US11404028B2 (en) 2019-12-16 2022-08-02 Microsoft Technology Licensing, Llc Sub-display notification handling
US11093046B2 (en) * 2019-12-16 2021-08-17 Microsoft Technology Licensing, Llc Sub-display designation for remote content source device
US11487423B2 (en) 2019-12-16 2022-11-01 Microsoft Technology Licensing, Llc Sub-display input areas and hidden inputs
US11042222B1 (en) 2019-12-16 2021-06-22 Microsoft Technology Licensing, Llc Sub-display designation and sharing
US20260072635A1 (en) * 2024-09-11 2026-03-12 Jpmorgan Chase Bank, N.A. Systems and methods for hybrid cross-reality collaboration around an interactive common display

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050066089A1 (en) 2003-09-22 2005-03-24 Jeyhan Karaoguz Consumption based source throttling
US20120086815A1 (en) * 2010-10-06 2012-04-12 Microsoft Corporation Rapidly initializing and dynamically adjusting media streams
US20120178368A1 (en) 2011-01-07 2012-07-12 Microsoft Corporation Wireless Communication Techniques
US20140359493A1 (en) * 2013-05-30 2014-12-04 Samsung Electronics Co., Ltd. Method, storage medium, and electronic device for mirroring screen data
US20150002369A1 (en) 2013-06-28 2015-01-01 Kabushiki Kaisha Toshiba Information processing apparatus, and information processing method
US20150052193A1 (en) * 2013-08-15 2015-02-19 Comcast Cable Communications, LLC. Media Fling System
US20150200985A1 (en) 2013-11-13 2015-07-16 T1visions, Inc. Simultaneous input system for web browsers and other applications

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030222915A1 (en) * 2002-05-30 2003-12-04 International Business Machines Corporation Data processor controlled display system with drag and drop movement of displayed items from source to destination screen positions and interactive modification of dragged items during the movement
US7242389B1 (en) * 2003-10-07 2007-07-10 Microsoft Corporation System and method for a large format collaborative display for sharing information
US7614075B2 (en) * 2004-08-13 2009-11-03 Microsoft Corporation Dynamically generating video streams for user interfaces
US20090234983A1 (en) * 2008-03-17 2009-09-17 Golden Signals, Inc. Methods and apparatus for sharing a computer display screen
US8850052B2 (en) * 2008-09-30 2014-09-30 Apple Inc. System and method for simplified resource sharing
US8875183B2 (en) * 2008-10-31 2014-10-28 International Business Machines Corporation Intelligent TV mosaic for IPTV
US8789131B2 (en) * 2010-05-14 2014-07-22 Lg Electronics Inc. Electronic device and method of sharing contents thereof with other devices
US20120108215A1 (en) * 2010-10-29 2012-05-03 Nader Kameli Remote notification device
KR101276846B1 (en) * 2011-04-22 2013-06-18 엘지전자 주식회사 Method and apparatus for streaming control of media data
US11611595B2 (en) 2011-05-06 2023-03-21 David H. Sitrick Systems and methodologies providing collaboration among a plurality of computing appliances, utilizing a plurality of areas of memory to store user input as associated with an associated computing appliance providing the input
US20130103446A1 (en) * 2011-10-20 2013-04-25 Microsoft Corporation Information sharing democratization for co-located group meetings
CN103379433B (en) * 2012-04-27 2017-02-01 华为终端有限公司 Method and device for giving alarm through distance measurement
US20150095822A1 (en) 2012-07-02 2015-04-02 eScoreMusic, Inc. Systems and methods for music display, collaboration, annotation, composition, and editing
US9049470B2 (en) * 2012-07-31 2015-06-02 Google Technology Holdings LLC Display aware transcoder source selection system
US8918822B2 (en) 2012-09-26 2014-12-23 Apple Inc. Displaying a synchronized instance of content on a portable electronic device
US9253229B1 (en) * 2013-01-07 2016-02-02 Cox Communications, Inc. Correlating video quality with video playback view
EP3657824A3 (en) 2013-02-26 2020-09-16 Mersive Technologies, Inc. System and method for multi-user control and media streaming to a shared display
WO2015171912A1 (en) 2014-05-08 2015-11-12 Mersive Technologies, Inc. System and method for display device discovery
WO2015179688A1 (en) 2014-05-21 2015-11-26 Mersive Technologies, Inc. Intelligent shared display infrastructure and associated methods
US9485540B2 (en) 2014-06-03 2016-11-01 Disney Enterprises, Inc. System and method for multi-device video image display and modification

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050066089A1 (en) 2003-09-22 2005-03-24 Jeyhan Karaoguz Consumption based source throttling
US20120086815A1 (en) * 2010-10-06 2012-04-12 Microsoft Corporation Rapidly initializing and dynamically adjusting media streams
US20120178368A1 (en) 2011-01-07 2012-07-12 Microsoft Corporation Wireless Communication Techniques
US20140359493A1 (en) * 2013-05-30 2014-12-04 Samsung Electronics Co., Ltd. Method, storage medium, and electronic device for mirroring screen data
US20150002369A1 (en) 2013-06-28 2015-01-01 Kabushiki Kaisha Toshiba Information processing apparatus, and information processing method
US20150052193A1 (en) * 2013-08-15 2015-02-19 Comcast Cable Communications, LLC. Media Fling System
US20150200985A1 (en) 2013-11-13 2015-07-16 T1visions, Inc. Simultaneous input system for web browsers and other applications

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3423934A4

Also Published As

Publication number Publication date
EP4617849A3 (en) 2026-02-25
US10616633B2 (en) 2020-04-07
US10931996B2 (en) 2021-02-23
EP3423934A4 (en) 2019-08-28
US20170251238A1 (en) 2017-08-31
EP4617849A2 (en) 2025-09-17
EP3423934A1 (en) 2019-01-09
US20200204851A1 (en) 2020-06-25

Similar Documents

Publication Publication Date Title
US10931996B2 (en) System for connecting a mobile device and a common display
US12124761B2 (en) Multi-group collaboration system and associated methods
US11729355B2 (en) Video conferencing during real time collaboration on a virtual canvas
EP2571263B1 (en) Conference system, event management server, and program
US9916066B2 (en) Object transfer method using gesture-based computing device
US20120162351A1 (en) Multimedia, multiuser system and associated methods
CN112416223A (en) Display method, electronic device and readable storage medium
JP5555555B2 (en) In-vehicle device that cooperates with a portable device and realizes an input operation possible for the portable device
JP2008210212A (en) Item selection device, item selection method, and program
CN105874417A (en) Cross-interface data transfer method and terminal
AU2017280957A1 (en) Application icon customization
JP5921404B2 (en) Information display device, information display method, and program
CN103218386A (en) Data display method and mobile device adapted thereto
JP6031750B2 (en) Display control apparatus, image display system, display control method, and program
JP2010011307A (en) Camera information display unit and camera information display method
US20140380161A1 (en) Information processing apparatus, information processing method, and program
US8948798B2 (en) Transmission control device, reception control device, and content transceiving system
RU2641245C2 (en) Method and device (versions) for transferring files
US9596335B2 (en) Communication terminal and communication method
CN109313529B (en) Carousel between documents and pictures
CN106997234A (en) Virtual real mode processing method, equipment and system
US10291883B1 (en) System, method, and computer program for receiving device instructions from one user to be overlaid on an image or video of the device for another user
CN104903833A (en) Client device, monitoring system, method and computer program for displaying images on a screen
KR101528271B1 (en) How to send messages from your mobile device
JP2011022862A (en) Display control unit, display control system, and computer program

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2017760610

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2017760610

Country of ref document: EP

Effective date: 20181001

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17760610

Country of ref document: EP

Kind code of ref document: A1

WWW Wipo information: withdrawn in national office

Ref document number: 2017760610

Country of ref document: EP