JP2017194922A - Mirroring system and operation detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mirroring system and an operation detection method capable of increasing operation detection accuracy. SOLUTION: In one embodiment, a first communication device and a second communication device are included, and a second communication unit transmits information of an image to be displayed on a second touch screen display to the first communication unit. , The first control unit displays the image to be displayed on the second touch screen display received by the first communication unit on the first touch screen display, and the second control unit determines that the contact operation is continuous. The operation included in the first operation information is determined based on the reference delay time, the display of the second touch screen display is controlled based on the operation of the determination result, and the image information is displayed on the second touch screen display. The delay time is set based on the delay time of the first operation information acquired at the start of the process to be displayed. [Selection diagram] Fig. 1

Description

  The present application relates to a mirroring system and an operation detection method for sharing a screen between two portable communication devices.

  As a mirroring technology for sharing a screen among a plurality of mobile communication devices, there is a mirroring system capable of inputting operations in both directions (see Patent Document 1).

Special table 2014-509476 gazette

  When an operation is detected in both directions, screen control based on the input operation may be performed by each mobile communication device. In this case, one operation may be determined as a different operation in each mobile communication device, which causes a problem as a mirroring system. There is a need for a mirroring system and operation detection method that can increase the accuracy of operation detection.

  A mirroring system according to one aspect outputs a first touch screen display, a first control unit that controls display of the first touch screen display, and first operation information detected by the first touch screen display. A first communication device including a first communication unit, a second communication unit that transmits and receives data to and from the first communication unit, a second touch screen display that displays an image, and the first operation information, A second control unit that controls display on the second touch screen display, and the second communication unit displays the image on the second touch screen display on the first communication unit. The first control unit displays the information on the second touch screen display received by the first communication unit. An image is displayed on the first touch screen display, and the second control unit determines an operation included in the first operation information based on a delay time serving as a reference for determining that the operation is a continuous contact operation. Based on the operation of the determination result, the display of the second touch screen display is controlled, and the delay time of the first operation information acquired at the start of the process of displaying the image information on the second touch screen display Based on this, the delay time is set.

  An operation detection method according to one aspect outputs a first touch screen display, a first control unit that controls display of the first touch screen display, and first operation information detected by the first touch screen display. A first communication device including a first communication unit, a second communication unit that transmits and receives data to and from the first communication unit, a second touch screen display that displays an image, and the first operation information. A second communication device comprising: a second control unit configured to control display of the second touch screen display; and a method for detecting operation of a mirroring system, wherein the second control unit is configured to display the second touch screen display. Based on the delay time of the first operation information acquired at the start of the process of displaying the image information on the Setting a delay time serving as a reference to determine that, and transmitting information on an image to be displayed on the second touch screen display, and displaying an image to be displayed on the second touch screen display on the first touch screen display And a step in which the second control unit determines an operation included in the first operation information based on the delay time, and controls display on the second touch screen display based on an operation of the determination result. And including.

FIG. 1 is a diagram showing a schematic configuration of a mirroring system. FIG. 2 is a block diagram of the mobile communication device. FIG. 3 is a block diagram of the other portable communication device. FIG. 4 is a sequence diagram showing the operation of the mirroring system. FIG. 5 is a diagram illustrating an example of an image displayed on a screen of a mobile communication device that detects an operation. FIG. 6 is a flowchart illustrating an example of processing of the mobile communication device that detects an operation. FIG. 7 is a flowchart illustrating an example of processing of the mobile communication device that has acquired the operation information through communication. FIG. 8 is a flowchart illustrating an example of processing of the mobile communication device that has acquired operation information through communication. FIG. 9 is a flowchart illustrating an example of processing of the mobile communication device that has acquired operation information through communication.

  An embodiment for carrying out a mirroring system and an operation detection method according to the present application will be described in detail with reference to the drawings.

(Embodiment)
A mirroring system according to an embodiment will be described with reference to FIGS. 1 to 3. FIG. 1 is a diagram showing a schematic configuration of a mirroring system. FIG. 2 is a block diagram of the mobile communication device. FIG. 3 is a block diagram of the other portable communication device.

  The mirroring system 100 includes a first mobile communication device 20 and a second mobile communication device 30. The first mobile communication device 20 and the second mobile communication device 30 transmit and receive screen information (data) by communication 42 using the communication unit, and display the same image on at least a part of each display unit. It has a function. The mirroring function uses a technology called Miracast (registered trademark) or AirPlay (registered trademark), for example. Miracast is a wireless transmission technology for images between devices, formulated by Wi-Fi Alliance. In the present embodiment, the first mobile communication device 20 and the second mobile communication device 30 will be described as realizing a mirroring function using Miracast. The mirroring system 100 of this embodiment can operate a screen shared by both the first mobile communication device 20 and the second mobile communication device 30. In the present embodiment, the operation input to the first mobile communication device 20 is transmitted to the second mobile communication device 30, and the second mobile communication device 30 executes processing based on the operation input to the first mobile communication device 20. This will be described as a case.

  In addition, the first mobile communication device 20 and the second mobile communication device 30 of the mirroring system 100 perform other communication in parallel with the communication 42 executed between the first mobile communication device 20 and the second mobile communication device 30. Communication with the device 40 can be performed. Here, the communications 42 and 44 of the present embodiment perform short-range wireless communication.

  Next, the configuration of the first mobile communication device 20 and the second mobile communication device 30 will be described with reference to FIGS. 2 and 3. In the following description, the same code | symbol may be attached | subjected to the same component. Furthermore, duplicate descriptions may be omitted.

  FIG. 2 is a block diagram of the first mobile communication device 20. As shown in FIG. 2, the first mobile communication device 20 includes a touch panel 2, an operation unit 3, a communication unit 6 a, a communication unit 6 b, a receiver 7, a microphone 8, a storage unit 9, and a control unit 10. And a speaker 11.

  The touch panel 2 includes a display unit 2A and a touch sensor 2B. In the example of FIG. 2, the display unit 2A and the touch sensor 2B are substantially rectangular, but the shapes of the display unit 2A and the touch sensor 2B are not limited to this. Each of the display unit 2A and the touch sensor 2B can take any shape such as a polygon or a circle. In the example of FIG. 3, the display unit 2 </ b> A and the touch sensor 2 </ b> B are arranged in an overlapping manner, but the arrangement of the display unit 2 </ b> A and the touch sensor 2 </ b> B is not limited to this. For example, the display unit 2A and the touch sensor 2B may be arranged side by side or may be arranged apart from each other. In the example of FIG. 3, the long side of the display unit 2A is along the long side of the touch sensor 2B, and the short side of the display unit 2A is along the short side of the touch sensor 2B. The method of overlapping is not limited to this. When the display unit 2A and the touch sensor 2B are arranged so as to overlap each other, for example, one or more sides of the display unit 2A may not be along any side of the touch sensor 2B.

  The display unit 2A includes a liquid crystal display (LCD: Liquid Crystal Display), an organic EL display (OELD: Organic Electro-Luminescence Display), or an inorganic EL display (IELD: Inorganic Electro-Luminescence Display). . The display unit 2A displays characters, images, symbols, graphics, and the like based on the image information from the control unit 10.

  The touch sensor 2B detects an operation on the image displayed on the display unit 2A. The touch sensor 2B detects contact of the finger F, a pen, a stylus pen, or the like with the touch sensor 2B. The touch sensor 2B can detect a position where a plurality of fingers F, pens, stylus pens, or the like are in contact with the touch sensor 2B. The detection method of the touch sensor 2B may be any method such as a capacitance method, a resistive film method, a surface acoustic wave method (or an ultrasonic method), an infrared method, an electromagnetic induction method, and a load detection method.

  The touch sensor 2B detects and touches at least one of the touch on the display unit 2A, the position where the touch is detected, the change in the position where the touch is detected, the interval where the touch is detected, and the number of times the touch is detected. The sensor contact information is output to the control unit 10.

  In the present embodiment, the touch sensor 2B can be applied to the touch sensor 2B by, for example, “touch”, “long touch”, “release”, “swipe”, “tap”, “double tap”, “long tap”, “drag”. ”,“ Flick ”,“ pinch-in ”, and“ pinch-out ”may be input as operations.

  The touch sensor 2 </ b> B is controlled by the control unit 10 to start and stop detection of contact. More specifically, in the touch sensor 2B, when the first mobile communication device 20 is normally used, the control unit 10 enables the touch sensor 2B to detect contact. Specifically, in the touch sensor 2B, when the first mobile communication device 20 is in normal use, the control unit 10 starts detecting contact with the touch sensor 2B. At this time, the touch sensor 2 </ b> B detects contact with the touch sensor 2 </ b> B and outputs first contact information to the control unit 10.

  The operation unit 3 includes one or more devices for receiving user operations. Devices for accepting user operations include, for example, keys, buttons, and the like. The operation unit 3 inputs a signal corresponding to the accepted operation to the control unit 10.

  The communication unit 6a is a communication unit that performs communication using the first communication method. In the present embodiment, the first communication method is a communication method such as 2G, 3G, or 4G for performing communication via the public line network 90. Communication standards for performing communication via the public line network 90 include, for example, LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (registered trademark). ) (Global System for Mobile Communications), PHS (Personal Handy-phone System), and the like. The first communication method may be a communication method for performing data communication via the public network 90, such as WiMAX (Worldwide Interoperability for Microwave Access). When the first communication method is a communication method for performing data communication, a call is realized by combining a data transmission technology and a technology for transmitting voice such as VoIP (Voice over Internet Protocol) as data. The The communication unit 6a may support a plurality of communication methods for performing communication via the public line network 90.

  The communication unit (first communication unit) 6b is a communication unit that performs communication using the second communication method. In the present embodiment, the second communication method is a short-range wireless communication method. Short-range wireless communication systems include WiFi (registered trademark), IEEE 802.11 (IEEE is an abbreviation for The Institute of Electrical and Electronics Engineers, Inc.), Bluetooth (registered trademark), ZigBe (registered trademark), ZigBe (registered trademark), and ZigBe (registered trademark). (Digital Enhanced Cordless Communications), Z-Wave, Wi-Sun (Wireless Smart Utility Network) and the like. The communication unit 6b may support a plurality of short-range wireless communication methods.

  The receiver 7 and the speaker 11 are sound output units. The receiver 7 and the speaker 11 output the sound signal input from the control unit 10 as sound. The receiver 7 is used, for example, to output the other party's voice during a call. The speaker 11 is used for outputting a ring tone and music, for example. One of the receiver 7 and the speaker 11 may also function as the other. The microphone 8 is a sound input unit. The microphone 8 converts a user's voice or the like into a sound signal and inputs the sound signal to the control unit 10.

  The storage unit 9 stores programs and data. The storage unit 9 is also used as a work area for temporarily storing the processing result of the control unit 10. The storage unit 9 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage unit 9 may include a plurality of types of storage media. The storage unit 9 may include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk and a storage medium reader. The storage unit 9 may include a storage device that is used as a temporary storage area such as a RAM (Random Access Memory).

  The storage unit 9 stores, for example, a communication control program 9a and connection related data 9b. The communication control program 9a provides a function of executing communication using a public wireless communication network via the communication unit 6a and communication using short-range wireless communication via the communication unit 6b. The connection related data 9b includes information related to connection with other portable communication devices.

  The communication control program 9a provides a function for establishing communication by short-range wireless communication and a mirroring function using short-range wireless communication. The communication control program 9a of the present embodiment provides a function of adjusting settings related to detection of operation information transmitted from one mobile communication device to the other mobile communication device during mirroring. Specifically, the communication control program 9a determines whether the operation information received when the operation information is received is continuous operation information, that is, whether the operation information is continuous with the previously received operation information. A function of adjusting a delay time that is a reference of a determination time is provided.

  The connection related data 9b includes information for establishing short-range wireless communication, information on a communication device that permits mirroring, and information on a reference condition for setting a delay time. The reference information for setting the delay time is a reference for determining the delay time based on the interval information of the operation information, for example, whether the longest interval among the intervals of the plurality of operation information is used or an average is used. is there. Further, as reference information for setting the delay time, information on timing for adjusting the delay time is also stored.

  The control unit 10 is an arithmetic processing device. The arithmetic processing unit includes, for example, a CPU (Central Processing Unit), an SoC (System-on-a-Chip), an MCU (Micro Control Unit), an FPGA (Field-Programmable Gate Array), and a coprocessor. It is not limited. The control unit 10 controls the operation of the first mobile communication device 20 to realize various functions.

  Specifically, the control unit 10 executes instructions included in the program stored in the storage unit 9 while referring to the data stored in the storage unit 9 as necessary. And the control part 10 controls a function part according to data and a command, and implement | achieves various functions by it. The functional unit includes, for example, the touch panel 2, the operation unit, the communication units 6a and 6b, the receiver 7, and the speaker 11, but is not limited thereto. The control unit 10 may change the control according to the detection result of the detection unit. Although a detection part contains the operation part 3, the communication parts 6a and 6b, and the microphone 8, for example, it is not limited to these. The control unit 10 executes the operation of the first mobile communication device 20 in the operation detection method by executing the communication control program 9a.

  The functional configuration of the first mobile communication device 20 shown in FIG. 2 is an example, and may be changed as appropriate within a range that does not impair the gist of the present invention.

  FIG. 3 is a block diagram of the other second mobile communication device 30. As illustrated in FIG. 3, the second mobile communication device 30 includes a touch panel 2, an operation unit 3, a communication unit 6 a, a communication unit (second communication unit) 6 b, a receiver 7, a microphone 8, and a storage unit. 9, a control unit 10, and a speaker 11.

  The storage unit 9 stores, for example, a communication control program 9e and connection related data 9f. The communication control program 9e is a program similar to the communication control program 9a. The connection related data 9 f has the same configuration as the connection related data 9 b of the first mobile communication device 20.

  The control unit 10 executes the operation of the second portable communication device 30 in the communication control method shown in FIGS. 1 and 2 by executing the communication control program 9e.

  Next, the operation of the mirroring system 100 will be described with reference to FIGS. FIG. 4 is a sequence diagram showing the operation of the mirroring system. FIG. 5 is a diagram illustrating an example of an image displayed on a screen of a mobile communication device that detects an operation. The processing shown in FIG. 4 is performed by transmitting and receiving data (information) between the communication unit 6b of the first mobile communication device 20 and the communication unit 6b of the second mobile communication device 30 using short-range wireless communication, and sharing the screen. That is, an example of processing for performing mirroring is shown. The process shown in FIG. 4 requests mirroring from the first mobile communication device 20 to the second mobile communication device 20, and transmits operation information from the first mobile communication device 20 to the second mobile communication device 30. And the process in the case of performing display control of a screen with the 2nd portable communication apparatus 30 is shown.

  First, the mirroring system 100 transmits a connection request for requesting establishment of short-range wireless communication from the first mobile communication device 20 to the second mobile communication device 30 (step S12). When the second mobile communication device 30 receives the connection request, the second mobile communication device 30 outputs a connection response including information necessary for establishing communication to the first mobile communication device 20 (step S14). When the first mobile communication device 20 receives the connection response from the second mobile communication device 30, the first mobile communication device 20 performs connection response including information necessary for establishing communication and completes the connection (step S16). Thereby, the first portable communication device 20 and the second portable communication device 30 establish a short-distance wireless communication line using the communication unit (first communication unit, second communication unit) 6b, and transmit and receive data. It becomes possible.

  Next, the first mobile communication device 20 transmits a mirroring request to the second mobile communication device 30 (step S18). The mirroring request is information for requesting screen synchronization. When the second mobile communication device 30 receives the mirroring request, the second mobile communication device 30 transmits a mirroring permission to the first mobile communication device 20. Mirroring permission is information that permits screen synchronization.

  Next, the first mobile communication device 20 transmits a delay time adjustment start request to the second mobile communication device 30 (step S22). When receiving the delay time adjustment start request, the second mobile communication device 30 transmits a delay time adjustment start request to the first mobile communication device 20 (step S24). As a result, the mirroring system 100 starts the delay time adjustment process. Specifically, the first mobile communication device 20 that transmits the operation information displays a delay time synchronization processing input screen (step S26). Specifically, as shown in FIG. 5, the first mobile communication device 20 displays a screen for inputting an operation necessary for detecting the delay time. The screen shown in FIG. 5 includes a mark 60 as an operation start point, a mark 62 as an end point, a line 63 that connects the start point and the end point, and corresponds to a trajectory for moving the contact point, and “swipe from A to B”. "Please" message 64 is displayed. The image to be displayed on the screen is not limited to this, and any image that allows the user to input an operation may be used. The second mobile communication device 30 receiving the operation information displays a delay time synchronization processing screen (step S28). Specifically, the second mobile communication device 30 displays a screen indicating that processing necessary for mirroring with the first mobile communication device 20 is being executed.

  The mirroring system 100 detects an operation with the touch sensor 2B of the first mobile communication device 20 in a state where the screen is displayed on the respective display units 2A of the first mobile communication device 20 and the second mobile communication device 30, The coordinate information of the operation detected by communication using the communication unit 6b is transmitted to the second portable communication device 30 (steps S30 and S32). Although shown as two steps in FIG. 4, the detected operation information is transmitted from the first mobile communication device 20 to the second mobile communication device 30 a plurality of times.

  The second mobile communication device 30 sets the delay time after receiving the operation information transmitted from the first mobile communication device 20 a plurality of times (step S34). Specifically, the reception interval of the operation information transmitted a plurality of times is detected, and the delay time is determined based on the detected interval. Note that the delay time may be determined based on the average of the reception intervals or may be determined based on the longest interval. The delay time is longer than the interval for detecting the operation.

  When the delay time is set, the second mobile communication device 30 transmits delay time adjustment completion information to the first mobile communication device 20 (step S36). When transmitting the delay time adjustment completion information, the mirroring system 100 executes mirroring between the first mobile communication device 20 and the second mobile communication device 30 (step S38). Specifically, the same screen (image) is displayed on at least a part of each display unit 2 </ b> A of the first mobile communication device 20 and the second mobile communication device 30. In addition, when the mirroring system 100 detects an operation with the touch sensor 2B of the first mobile communication device 20, the mirroring system 100 transmits the operation information to the second mobile communication device 30 through the communication unit 6b. The second mobile communication device 30 determines whether or not the received operation information is a continuous operation in consideration of the delay time, executes processing based on the determined result, and controls the screen displayed on the display unit 2A. .

  The mirroring system 100 of this embodiment can set the delay time based on the state of the short-range wireless communication using the communication unit 6b by executing the process of adjusting the delay time at the start of mirroring. Accordingly, it is possible to suppress a continuous operation from being determined as a non-continuous operation in a short-distance wireless communication state in which the operation information transmission interval is long. For example, when the communication unit 6b is executing processing with multi-channel concurrency that performs short-range wireless communication with a plurality of communication devices, compared with a case where communication is performed with one communication device with single-channel concurrency. The operation information reception interval becomes longer. The mirroring system 100 according to the present embodiment performs processing with multi-channel concurrency by executing processing for adjusting the delay time at the start of mirroring, and even when the operation information reception interval becomes long, the delay time adjustment is performed. By calculating a reception interval by processing and setting a delay time based on the calculated interval, a continuous operation can be detected as a continuous operation.

  Here, the mirroring system 100 can change the delay time even when the mirroring process is executed. Hereinafter, an example of processing will be described with reference to FIGS. First, the operation of the mobile communication device in which the user inputs an operation will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of processing of the mobile communication device that detects an operation. Hereinafter, the case where the first mobile communication device 20 detects an operation will be described. The process illustrated in FIG. 6 can be realized by the control unit 10 of the first mobile communication device 20 executing the operation of each unit. The first mobile communication device 20 detects an operation (step S52). Specifically, the operation by the user is detected by the touch sensor 2B. When the first mobile communication device 20 detects an operation, the first mobile communication device 20 transmits the detected operation to the communication device mirroring the detected operation, in this embodiment, the second mobile communication device 30 (step S54). The first mobile communication device 20 performs short-range wireless communication with the communication unit 6 of the second mobile communication device 30 by the communication unit 6 b and transmits operation information to the second mobile communication device 30.

  Next, processing executed by the mobile communication device that has acquired the operation information through communication will be described with reference to FIGS. Hereinafter, a case where the second mobile communication device 30 detects operation information transmitted from the communication unit 6b of the first mobile communication device 20 will be described. The processing illustrated in FIGS. 7 to 9 can be realized by the control unit 10 of the second mobile communication device 30 executing the operation of each unit. FIG. 7 is a flowchart illustrating an example of processing of the mobile communication device that has acquired the operation information through communication. In the present embodiment, a case will be described where the second mobile communication device 30 receives operation information from the first mobile communication device 20 by short-range wireless communication of the communication unit 6b. The second mobile communication device 30 detects an operation (step S62). Specifically, the second mobile communication device 30 receives the operation information transmitted from the first mobile communication device 20 by the communication unit 6b.

  The second mobile communication device 30 determines whether the detected operation is an incomplete operation (step S64). The incomplete operation determines whether the operation is associated with the process as an operation input to the function (application) that displays the screen on the display unit 2A. The operation with which the process is associated is an operation that can be executed by a function of displaying a screen on the display unit 2A when input. The second mobile communication device 30 determines that the detected operation is an incomplete operation when the detected operation is not an operation associated with the process.

  If the second mobile communication device 30 determines that the operation is not incomplete (No in step S64), the process ends. The second mobile communication device 30 executes processing based on the input operation as application processing. If the second mobile communication device 30 determines that the operation is incomplete (Yes in step S64), the second mobile communication device 30 changes the delay time to delay time + α (step S66), and ends the process. Here, α is a positive number. That is, when the second mobile communication device 30 detects an incomplete operation, the second mobile communication device 30 changes the delay time to a longer time.

  When the second mobile communication device 30 detects an incomplete operation during the mirroring process, the second mobile communication device 30 increases the delay time, thereby ensuring the continuous operation included in the operation information transmitted from the other mobile communication device during the mirroring. Therefore, the operation detection accuracy can be further increased.

  FIG. 8 is a flowchart illustrating an example of processing of the mobile communication device that has acquired operation information through communication. The second mobile communication device 30 detects an operation (step S72). Specifically, the second mobile communication device 30 receives the operation information transmitted from the first mobile communication device 20 by the communication unit 6b. The second mobile communication device 30 determines whether the detected operation is an incomplete operation (step S74). If the second mobile communication device 30 determines that the operation is not incomplete (No in step S74), the process proceeds to step S78.

  If it is determined that the operation is incomplete (Yes in step S74), the second mobile communication device 30 determines whether there is an operation that can be determined as a continuous operation within a predetermined time (step S76). Specifically, after detecting the operation information that becomes an incomplete operation and the delay time has elapsed, the operation information transmitted from the first mobile communication device 20 is received within a predetermined time set in advance, and the reception is received. It is determined whether the performed operation information is an operation that can be determined as a continuous operation linked to an incomplete operation.

  If it is determined that there is an operation that can be determined as a continuous operation within a predetermined time (Yes in step S76), the second mobile communication device 30 proceeds to step S78. If the second mobile communication device 30 determines No in step S74 or Yes in step S76, the second mobile communication device 30 executes processing based on the detected operation (step S78). If it is determined that there is no operation that can be determined as a continuous operation within a predetermined time (No in step S76), the second mobile communication device 30 executes the process as an incomplete operation (step S79). The operations set for the incomplete operation include a process for invalidating the incomplete operation and a process that does not correspond to a function process based on the incomplete operation, for example, in the case of a swipe process, the operation is not moved. Execute the pointer movement.

  When the second mobile communication device 30 detects an incomplete operation, the second mobile communication device 30 complements the operation once determined as an incomplete operation as a continuous operation by detecting whether a process corresponding to the continuous process is detected for a predetermined time. can do. Thereby, it is not necessary for the user to re-input the operation, and the operability can be improved.

  FIG. 9 is a flowchart illustrating an example of processing of the mobile communication device that has acquired operation information through communication. The second mobile communication device 30 detects continuous operation (step S82). Specifically, the second mobile communication device 30 receives a plurality of operation information transmitted from the first mobile communication device 20 by the communication unit 6b. The second mobile communication device 30 determines that the continuous operation has been detected when the reception interval of the received operation information is within the delay time and the operation can be determined that a plurality of pieces of operation information are continuous.

  The second mobile communication device 30 determines whether or not the acquired operation information acquisition interval <delay time−β (step S84). That is, the second mobile communication device 30 determines whether the reception interval of the operation information transmitted from the first mobile communication device 20 has a margin of a certain time β or more with respect to the delay time.

  If the second mobile communication device 30 determines that the acquired operation information acquisition interval <delay time−β (Yes in step S84), the second mobile communication device 30 sets the delay time to be delay time−β (step S86). If the second mobile communication device 30 determines that the detected operation information acquisition interval <delay time−β is not satisfied (No in step S84), this process ends.

  When the operation information acquisition interval has a certain margin with respect to the delay time during the mirroring process, the second mobile communication device 30 can acquire the operation more efficiently by shortening the delay time. In addition, by executing when there is a certain margin with respect to the delay time, processing is performed while suppressing a decrease in detection accuracy of continuous operations included in operation information transmitted from other mobile communication devices during mirroring. It can be executed efficiently. Further, the delay time becomes long, and it is possible to suppress detection of a non-continuous operation as a continuous operation. For example, detection of a double tap as a long tap can be suppressed.

  Although the mirroring system 100 of this embodiment performed the process based on operation in each of two portable communication apparatuses, it is not limited to this. The mirroring system 100 according to the present embodiment may be operated by each of two portable communication devices, and one of the two portable communication devices may perform processing based on the operation (display processing of a screen or the like). Also in this case, the above-described effect can be obtained by executing the processing of the present embodiment for detecting the operation information transmitted from the mobile communication device without performing the processing based on the operation. Moreover, the mirroring system 100 of this embodiment may detect operation with one portable communication apparatus, and may perform the process based on operation with the other portable communication apparatus. Also in this case, the above effect can be obtained. Although the mirroring system 100 of this embodiment communicated with two portable communication devices, it may communicate with three or more portable communication devices. Further, although the present embodiment has been described as the case where short-range wireless communication is used, the delay time similarly varies depending on the communication situation when public wireless communication is used. When the mirroring system 100 uses both short-range wireless communication and public wireless communication, and also uses public wireless communication, the mirroring system 100 executes this processing at the start of mirroring in the same manner. An effect can be obtained.

  The characterizing embodiments have been described in order to fully and clearly disclose the technology according to the appended claims. However, the appended claims should not be limited to the above-described embodiments, but all modifications and alternatives that can be created by those skilled in the art within the scope of the basic matters shown in this specification. Should be embodied by a simple configuration.

2 touch panel 3 operation unit 6a, 6b communication unit 7 receiver 8 microphone 9 storage unit 9a, 9e communication control program 9b, 9f connection related data 10 control unit 11 speaker 20 first mobile communication device 30 second mobile communication device 40 communication device 100 Mirroring system

Claims (6)

A first touch screen display; a first control unit that controls display of the first touch screen display; and a first communication unit that outputs first operation information detected by the first touch screen display. Communication equipment,
A second communication unit that transmits and receives data to and from the first communication unit; a second touch screen display that displays an image; and a second control that controls display of the second touch screen display based on the first operation information. A second communication device comprising:
The second communication unit transmits information on an image to be displayed on the second touch screen display to the first communication unit,
The first control unit causes the first touch screen display to display an image received by the first communication unit and displayed on the second touch screen display,
The second control unit determines an operation included in the first operation information based on a delay time serving as a reference for determining that the operation is a continuous contact operation, and performs the second touch based on the operation of the determination result. Control the display of the screen display,
And the mirroring system which sets the said delay time based on the delay time of the said 1st operation information acquired at the time of the start of the process which displays the said image information on a said 2nd touch screen display.   The first operation information acquired at the start of the process of displaying the image information on the second touch screen display causes the first touch screen display to display an image instructing input of a preset continuous operation. The mirroring system according to claim 1, wherein the operation is detected in a state. The second control unit determines whether the first operation information is an incomplete operation that does not match any of the image functions based on the function of the image displayed on the second touch screen display,
The mirroring system according to claim 1, wherein when it is determined that the operation is incomplete, the delay time is changed to a longer time. The second control unit determines whether the first operation information is an incomplete operation that does not match any of the image functions based on the function of the image displayed on the second touch screen display,
When it is determined that the operation is incomplete, the first operation information is detected for a set time, and when the newly detected first operation information is an operation that complements the incomplete operation, the operation is continuously performed. The mirroring system according to claim 1, wherein the mirroring system is detected as an operation.   When the second control unit detects the end of the operation with the first operation information based on the function of the image displayed on the second touch screen display, the operation detection interval corresponds to the delay time. The mirroring system according to any one of claims 1 to 4, wherein when the delay time is shorter than a predetermined time, the delay time is changed to a shorter time. A first touch screen display; a first control unit that controls display of the first touch screen display; and a first communication unit that outputs first operation information detected by the first touch screen display. Controlling display of the second touch screen display based on the first operation information, a communication device, a second communication unit for transmitting and receiving data to and from the first communication unit, a second touch screen display for displaying an image A second control unit, and a second communication device comprising: a mirroring system operation detection method comprising:
A reference for determining that the second control unit is a continuous contact operation based on a delay time of the first operation information acquired at the start of a process of displaying the image information on the second touch screen display; Setting the delay time to be
Transmitting information on an image to be displayed on the second touch screen display and causing the first touch screen display to display an image to be displayed on the second touch screen display;
The second control unit determining an operation included in the first operation information based on the delay time, and controlling display of the second touch screen display based on the operation of the determination result. Operation detection method.

Images