CN101753473A - Method for instantly transmitting interactive image and system applying same - Google Patents

Abstract

The invention provides a method for instantly transmitting an interactive image and a system applying the method. The method comprises the following steps: a: transmitting, by the first computer, the master image to the second computer such that the master image is simultaneously presented on the first and second computers; and B: drawing a tag image on the main image by one of the first computer and the second computer and instantly transmitting the tag image to the other, wherein the method further comprises the step of instantly transmitting the tag image: b1: extracting track information of the marked image at a specific frequency, wherein the marked image is substantially composed of a plurality of track points; b2: deleting track points which are too close according to a difference degree between any two adjacent track points so as to reduce the extracted track information; and B3: and compressing the deleted track information and transmitting the track information to the opposite side through a network with low bandwidth. The invention can transmit the mark symbol or the doodle on the shared image in real time under the limited bandwidth.

Description

Method for real-time transmission of an interactive image and system applying the method

technical field

The invention relates to a method for real-time transmission of an interactive image and a system applying the method, in particular to a method for real-time transmission of an interactive image and a system for applying the method, which can reduce the amount of computation under limited bandwidth.

Background technique

With the advancement of network technology, long-distance communication is no longer a problem. However, the biggest problem in network transmission is the bandwidth of the line. In other words, if you want to obtain clear transmission images and good transmission quality, you must have sufficient network bandwidth. Generally speaking, increasing network bandwidth is expensive, so most network transmissions still rely on good packet and compression technology.

The instant sharing of photos is one of the benefits of remote communication now, and there are various instant signal transmission applications (such as MSN, Yahoo messenger, etc.) on the market, so while sharing photos, you can also Suggest discussions with each other about shared photos. However, if the two parties in the communication want to discuss the detailed content of the shared photos, the most important key is to first confirm that the detailed content to be discussed by the two parties in the communication is the same part. In order to achieve this purpose, a marked image or graffiti is usually generated on the shared photo in real time, and then the shared photo is transmitted to the other party before the discussion begins.

Please refer to FIG. 1 , which is an image or photo showing that two parties in communication are discussing together. As shown in the figure, the photo contains a main image 11 including a house in the snow and a puppy. At this time, if one of the parties wants to discuss a specific part of the photo 1, such as the puppy on the right side of the photo, the most convenient way is to use a graffiti symbol 12 to mark the puppy on the right side of the photo . Only when one party will add a graffiti symbol or marked image on the shared picture or photo, the entire modified photo needs to be sent to the other party so that the other party can understand the marked part. However, the only difference between the modified photo and the unmodified photo is the appearance of graffiti or logo images, while the part about the main image has not changed at all. Therefore, retransmitting an image just for a partial change in the image or photo not only wastes bandwidth, but also reduces the feeling of instant communication during conversation.

Contents of the invention

In order to overcome the above shortcomings, the present invention proposes a system and method capable of transmitting marker symbols or graffiti on shared images in real time under limited bandwidth.

The main purpose of the present invention is to provide a method for real-time image transmission and a computer system using the method under limited bandwidth.

Another object of the present invention is to provide a method for real-time image transmission and a computer system using the method, which can reduce the computation load of the processor during real-time image transmission. The method of the present invention is applied to low-bandwidth network transmission, so that the image can be shared between two computers in real time, wherein the image includes a main image and a marked image, and the marked image is created by any one of the two computers produce. Method of the present invention comprises the following steps:

A: transmitting the main image from the first computer to a second computer so that the main image is simultaneously presented on the first and second computers; and

B: draw the marked image on the main image by one of the first computer and the second computer, and transmit the marked image to the other party in real time, wherein, in the step of real-time transmission of the marked image, the method further include:

B1: Extracting the trajectory information of the marked image at a specific frequency, wherein the marked image is essentially composed of a plurality of trajectory points;

B2: According to a degree of difference between any two adjacent track points, delete track points that are too close to reduce the extracted track information; and

B3: Compress the deleted trajectory information, and transmit it to the other party through the low-bandwidth network.

In a preferred embodiment, the specific frequency is 15 traces per second. The trajectory information includes coordinate information, trajectory shape information or color information. The trajectory information of the marked image is extracted from an operating system, such as Microsoft Windows (Windows) operating system. Furthermore, by using the operation of the computer pointer control device (such as a mouse), the computer operating system can detect its movement information, and thereby obtain coordinate information.

In one embodiment, the degree of difference is essentially based on the coordinate information of any two adjacent track points to obtain the degree of difference, thereby deleting track points that are too close to reduce track information. In this embodiment, a slope can be obtained based on the coordinate information of any two adjacent track points, thereby calculating the slopes of all adjacent track points, and the degree of difference can be obtained based on the slopes of any two adjacent points, Delete track points that are too close to reduce track information.

In a preferred embodiment, step B3 of the present invention may also include step B31: rearranging the deleted track information, so that the coordinate information and characteristic information of each track point can be converted into multiple sets of classification data, so as to further compress the deleted track information. The subtracted trajectory information. In addition, in addition to the starting point, the coordinate information of any two adjacent track points can be converted into a change amount, thereby further reducing the information to be stored, so step B3 of the present invention can also include step B32: the previous track A trajectory information change amount between the trajectory information of a point and the next trajectory point stores the trajectory information of the next trajectory point, thereby compressing the truncated trajectory information to reduce the amount of data to be transmitted.

In another embodiment, the present invention provides a method for instantly transmitting a tagged image on a shared image, the method comprising the following steps:

A provides shared images to be displayed on a first computer and a second computer at the same time;

B. Form a marked image on the shared image by the first computer, and transmit the marked image to the second computer in real time, wherein, in the instant transmission step of the marked image, the method further includes:

B1 extracts the trajectory information of each trajectory point forming the marked image at a frequency;

B2 performs data reorganization and compression on the extracted trajectory information; and

B3 transmits the compressed and reformed trajectory information to the second computer in real time.

In addition, the present invention also provides an interactive image real-time transmission system, which can share an interactive image between two computers in real time by using the method of the present invention. The real-time transmission system of the present invention includes a sending computer and a receiving computer, wherein the sending computer includes a drawing module and a first image processing module. The drawing module can cause a marker image to be formed on the main image. The first image processing module extracts and compresses the trajectory information constituting the marked image, so as to transmit the compressed trajectory information to the receiving computer in real time through the low-bandwidth network. Preferably, the first image processing module extracts the track information of the marked image at a frequency of 15 strokes per second.

The receiver computer includes a second image processing module. The second image processing module decompresses the received trajectory information to restore the marked image on the main image.

In a preferred embodiment, the sending computer further includes a computer index control device, and the drawing module forms a mark image on the main image through the computer index control device. In this embodiment, the trajectory information includes coordinate information and characteristic information of each trajectory point constituting the marker image. The coordinate information is the movement information detected by the computer pointer control device in the computer operating system, that is, the operation of the computer pointer control device (such as a mouse) can make the computer operating system detect its movement information, and thereby obtain the coordinate information. The characteristic information refers to one of the color, thickness and shape of the marked image.

In another preferred embodiment, the first image processing module further includes a downsampling module for deleting too close track points. In other words, the present invention may utilize the downsampling module to perform the step of reducing the extracted trajectory information as described above.

In addition, the first image processing module may further include a data reorganization module, which is used to convert the coordinate information and characteristic information of each trajectory point into multiple sets of classification data, so as to further compress the deleted trajectory information.

The first image processing module also includes a compression module for storing the track of the next track point by using the track information of the previous track point and a track information change amount between the track information of the previous track point and the track point of the next track point information, thereby compressing the truncated trajectory information to reduce the amount of data to be transmitted.

The second image processing module also includes an image compositor, whereby the marked image is displayed on the main image in the receiving computer. Therefore, both the computer at the sending end and the computer at the receiving end will display images with the main image and the marked image at the same time, thereby achieving the effect of real-time transmission of interactive images.

The invention can transmit marker symbols or graffiti on shared images in real time under limited bandwidth.

Description of drawings

Figure 1 shows a photo to share and its doodles.

FIG. 2 and FIG. 3 are a system diagram and a block diagram respectively showing the real-time transmission system of the interactive image according to an embodiment of the present invention.

FIG. 4 and FIG. 5 are block diagrams of the first image processing module and the second image processing module according to FIG. 3 , respectively.

FIG. 6 is a flowchart showing a method for real-time transmission of an interactive image according to an embodiment of the present invention.

Fig. 7 shows that the coordinates and characteristics of each point are converted into multiple sets of classification data.

FIG. 8 is a flowchart showing a method for real-time transmission of an interactive image according to another embodiment of the present invention.

Wherein, the reference signs are explained as follows:

1. Interactive image 2. Instant transmission

11. Main image 12. Marker image

21. 22. Computer 25. Cursor

211. Drawing module 210, 220. Processor

212, 222, image processing module 2121, reduction module

2122. Data reorganization module 2123. Compression module

221. Image synthesis module

Detailed ways

In order to make the above and other objects, features and advantages of the present invention more comprehensible, specific embodiments of the present invention are specifically cited below and described in detail in conjunction with the accompanying drawings.

The present invention provides a real-time transmission system of interactive images, which can instantly share an interactive image 1 (such as shown in FIG. main image 11 and marker image 12). Specifically, please refer to FIG. 2 and FIG. 3 . The instant transmission system 2 of the present invention includes a sending computer 21 and a receiving computer 22 , wherein the sending computer 21 and the receiving computer 22 are sharing an image. The sending computer 21 includes a graphics module 211 and a first image processing module 212 . The drawing module 211 can form the tag image on a main image of the shared image (for example, the main image 11 and the tag image 12 shown in FIG. 12). The first image processing module 212 is used for extracting and compressing the trajectory information constituting the marked image 12 , so as to transmit the compressed trajectory information to the receiving computer 22 through the low-bandwidth network in real time. Preferably, the first image processing module 212 extracts the track information of the marked image at a frequency of 15 strokes per second. The marked image is essentially composed of multiple track points. Since human eyes have persistence of vision, at an update frequency of about 15 images per second (15frames/per second), the traces of the marked images transmitted in real time at this time will also have the feeling of real-time drawing without causing discomfort. Continuous effect, so in a preferred embodiment, the specific frequency is to extract the track information of the marker image (track point) at a frequency of 15 strokes per second, but this is not intended to limit the present invention.

In the real-time transmission system 2 of the present invention, the sender computer 21 essentially includes a first processor 210 for executing an application program to perform the aforementioned drawing and image processing functions of the drawing module 211 and the first image processing module 212 respectively .

The receiving computer 22 also includes a second image processing module 222 , the second image processing module 222 decompresses the received trajectory information to restore the main image 11 to form the tagged image 12 transmitted from the sending computer. The receiver computer 22 essentially also includes a second processor 220 to execute an application program to achieve the image decompression function of the second image processing module.

In a preferred embodiment, the sender computer 21 also includes a computer pointer control device 25 (such as a mouse, a recognition writing pad, a recognition writing pen or a drawing board, and similar input devices). The drawing module 211 forms the mark image 12 on the main image 11 through the computer indicator control device 25 . In this embodiment, the trajectory information includes coordinate information and characteristic information of each trajectory point constituting the marker image 12 . The coordinate information is the movement information detected by the computer index control device 25 in the computer operating system (such as Windows operating system), that is, the operation of the computer index control device 25 can make the computer operating system detect its movement information, and thereby Get coordinate information. The characteristic information refers to one of the color, thickness and shape of the marking image 25 .

Please refer to FIG. 4 , in another preferred embodiment, the first image processing module 212 further includes a down sampling module 2121 for deleting too close track points. This function will be further described in the method flow below.

In addition, the first image processing module 212 may further include a data reorganization module 2122 for converting the coordinate information and characteristic information of each trajectory point into multiple sets of classified data, so as to further rearrange the deleted trajectory information. This function will also be further described in the following method flow.

The first image processing module 212 also includes a compression module 2123 . The compression module 2123 uses the track information of the previous track point and a track information change amount between the track information of the previous track point and the next track point to store the track information of the next track point, thereby compressing the deleted track information to reduce the amount of data that needs to be transferred. In this embodiment, this function will be further described in the following method flow.

As shown in FIG. 5 , the second image processing module 222 also includes an image compositor (imagecompositor) 2221 , so that the tagged image 12 is composited on the main image 11 to be shared in the receiving computer 22 . Therefore, both the sending computer 21 and the receiving computer 22 can display the main image 11 and the marked image 12 at the same time, so as to achieve the purpose of real-time transmission of interactive images.

The invention also discloses a method for real-time transmission of an interactive image, which is applied to a low-bandwidth network transmission to enable a first computer and a second computer to share the interactive image in real time. The interactive images also refer to the symbols in FIG. 1 , that is, the interactive image 1 includes a main image 11 and a tag image 12 , and the tag image 12 is generated by either the first computer or the second computer. In detail, the method for real-time transmission of interactive images of the present invention can utilize the above-mentioned system to achieve all the mechanisms. For example, in order to simplify the description, in this embodiment, the first computer is used as the above-mentioned sending computer 21, that is, the tagged image 12 is generated by the first computer, and the second computer can be the above-mentioned receiving computer 22, so although they are referred to as the first and second computers below, the same reference numerals 21 and 22 are still quoted for simplicity of description. Please refer to Fig. 6, the method of the present invention comprises the following steps:

Step S1: The first computer 21 transmits the main image 11 to the second computer 22, so that the main image 11 is displayed on the first and second computers 21 and 22 at the same time. For example, the first and second image processing modules 212, 222 of the first and second computers 21, 22 can achieve this function.

Step S2: Draw the marked image 12 on the main image 11 by one of the first computer 21 and the second computer 22, and transmit the marked image 12 to the other party in real time. Here, for simplicity of description, the first computer 21 is used as the sending computer, but this is not intended to limit the present invention, and any computer can be used as the sending computer. In the step of immediate transmission of the marked image 12, the method of the present invention also includes:

Step S21: Record the track information of the marker image 12, wherein the marker image 12 is substantially composed of a plurality of track points. Furthermore, the marker image 12 is substantially composed of a plurality of continuous trajectory points. This step may be performed substantially automatically by the operating system; for example, the operating system automatically records the position of the cursor, thereby obtaining the track information of the marker image 12 .

Step S22: Extract the recorded track information at a specific frequency. In a preferred embodiment, the specific frequency is 15 traces per second. As mentioned above, since human eyes have persistence of vision, at the extraction frequency of 15 images per second (15 frames/per second), the images displayed in real time at this time will hardly feel discontinuous, so it is better In the embodiment, the specific frequency is to extract the track information of the marker image (track point) at a frequency of 15 strokes per second, but this is not intended to limit the present invention.

The track information includes coordinate information, track shape information or color information. The trajectory information of the marker image 12 can be extracted from an operating system, such as Microsoft Windows (Windows) operating system. Furthermore, by using the operation of the computer pointer control device (such as the above-mentioned mouse 25 ), the computer operating system can detect its movement information (such as the position of the cursor), and thereby obtain coordinate information. The track shape information may include the thickness of the track. Because the operating system can provide a lot of information about the computer index control device, even up to 80 pieces of information per second, but as mentioned above, the adaptation frequency of human eyes is sufficient at 15 images per second, so the present invention uses a specific frequency to extract, That is to first delete all the trajectory information to 15 pens per second.

In addition, other characteristic information such as the thickness of each track point or the time of appearance can also be obtained from the information about the computer index control device 25 (such as a mouse) in the operating system. For example, according to the time information, the present invention can judge which computer started the graffiti first, and according to the time sequence, the latest graffiti is displayed on the top layer.

Next is step S23: according to a degree of difference between any two adjacent track points, delete track points that are too close to reduce the extracted track information. For example, as mentioned above, the present invention can utilize the down sampling module 2121 of the first image processing module 212 to delete track points that are too close.

In one embodiment, the degree of difference is essentially based on the coordinate information of any two adjacent track points to obtain the degree of difference, thereby deleting track points that are too close to reduce track information. In this embodiment, since the coordinate information of each track point forming the marker image 12 can be obtained from the operating system, a slope can be obtained based on the coordinate information of any two adjacent track points, thereby calculating all adjacent track points. The slope of the trajectory point, and according to the slope of any two adjacent points, the difference degree is obtained, and the trajectory points that are too close are deleted, thereby reducing the trajectory information.

Then there is step S24: compressing the truncated track information, and transmitting it to the other party through the low-bandwidth network.

Please refer to FIG. 6. In a preferred embodiment, in order to have better transmission efficiency, step S24 of the present invention may also include step S241: reorganize the deleted track information, so that the coordinate information of each track point and feature information into multiple sets of categorical data to further rearrange the pruned trajectory information. For example, as mentioned above, the present invention can use the data reorganization module 2122 of the first image processing module 212 to convert the coordinate information and characteristic information of each trajectory point into multiple sets of classification data, so as to further rearrange the deleted trajectory information.

To describe step S241 in detail, please refer to the example in FIG. 7 . The coordinate information of each trajectory point forming the marker image 12 can be defined as X, Y, and the characteristic information can be defined as Z. As shown in step S71, the first trajectory point is marked as (X1, Y1, Z1). If the information is transmitted directly, it may be necessary to transmit all the trajectory information of a very large number of trajectory points. However, as shown in step S72, if each trajectory point is classified, that is, X is divided into one category, Y is one category, and Z is one category, then only three types of trajectory information need to be transmitted when transmitting trajectory data, so The amount of transmitted data can be reduced.

In addition, in addition to the starting point of the track, the coordinate information of any two adjacent track points can be converted into a variation, thereby further reducing the information to be stored, so step S24 (as shown in Figure 6) of the present invention further Including the data compression step of step S242, that is, storing the track information of the next track point by a change amount of track information between the track information of the previous track point and the next track point, thereby compressing the deleted track information, To reduce the amount of data that needs to be transferred. For example, the first image processing module 212 further includes a compression module 2123 . The compression module 2123 uses the track information of the previous track point and a track information change amount between the track information of the previous track point and the next track point to store the track information of the next track point, thereby compressing the deleted track information to reduce the amount of data that needs to be transferred.

In detail, the position coordinates of the second track point among the multiple continuous track points forming the marker image 12 are actually the position coordinates of the first track point plus the amount of change (the change from the first point to the second point). amount), the third track point is actually the position coordinate of the second track point plus its change amount (the change amount from the second track point to the third track point)... until the last track point. In this way, it is obvious that from the coordinate information after the second track point, only the coordinate position of the first point and the variation of the subsequent points need to be stored. Therefore, the amount of data to be transferred can be further reduced. As shown in step S73 of FIG. 7 , only three types of information remain in the track information after classification, and besides the starting point, only the change amounts R and R' need to be recorded for each type of information.

Please refer to FIG. 8 . In another embodiment, the present invention provides a method for instantly transmitting a tagged image on a shared image. The method includes the following steps:

Step S81: Provide shared images to be displayed on a first computer and a second computer at the same time.

Step S82: forming a marked image on the shared image by the first computer, and transmitting the marked image to the second computer in real time, wherein the step of instantly transmitting the marked image further includes a step.

Step S821 : Extract the trajectory information of each trajectory point of the marker image at a frequency.

Step S822: Perform data reorganization and compression on the extracted trajectory information.

Step S823: Immediately transmit the compressed and reformed track information to the second computer.

It should be understood that the first computer, the second computer or the sending end computer and the receiving end computer of the present invention are not limited to any type of computer, although the illustration is a general traditional desktop computer, but in fact the present invention should be applicable In various computers such as notebook computers, tablet computers, or palmtop computers.

To sum up, the present invention shows its characteristics different from the known technologies in terms of purpose, means and effects. It should be noted that although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some changes without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the protection scope defined by the appended claims.

Claims (20)

1. A method for real-time transmission of an interactive image, which is applied to a low-bandwidth network transmission, so that the interactive image can be shared between a first computer and a second computer in real time, wherein the interactive image includes a main image and a marking an image, wherein the marking image is generated by either the first or second computer, the method comprising: A: transmitting the main image from the first computer to a second computer so that the main image is simultaneously presented on the first and second computers; and B: draw the marked image on the main image by one of the first computer and the second computer, and transmit the marked image to the other party in real time, wherein, in the step of real-time transmission of the marked image, the method further include: B1: Extracting the trajectory information of the marked image at a specific frequency, wherein the marked image is essentially composed of a plurality of trajectory points; B2: According to a degree of difference between any two adjacent track points, delete track points that are too close to reduce the extracted track information; and B3: Compress the deleted trajectory information, and transmit it to the other party through the low-bandwidth network. 2. The method as claimed in claim 1, wherein the specific frequency is 15 pieces of trace information per second. 3. The method of claim 1, wherein the track information of the marker image comprises one of coordinate information, track shape information, and color information. 4. The method of claim 3, wherein the coordinate information is extracted from a movement information of a computer index control device detected in a computer operating system. 5. The method according to claim 3, wherein in step B2, the degree of difference is substantially based on the coordinate information of any two adjacent track points, so as to obtain the degree of difference and delete track points that are too close. 6. The method as claimed in claim 5, wherein in step B2, according to the coordinate information of any two adjacent track points, a slope is obtained, thereby calculating the slope of all adjacent track points, and according to any Two adjacent slopes are used to obtain the degree of difference, and the track points that are too close are deleted. 7. The method according to claim 1, wherein the trajectory information comprises coordinate information and characteristic information of each trajectory point forming the marked image, and the compressing step B3 further comprises: B31: rearrange the deleted trajectory information, so that the coordinate information and characteristic information of each trajectory point can be converted into multiple sets of classification data, so as to further compress the deleted trajectory information. 8. The method of claim 7, further comprising: B32: The track information of the previous track point and the track information variation between the previous track point and the track information of the next track point store the track information of the next track point, thereby compressing and deleting the track information to obtain Reduce the amount of data that needs to be transferred. 9. An instant transmission system for interactive images, the system comprising: A sending computer is used to transmit a main image to a receiving computer through a low-bandwidth network, so that the main image is simultaneously displayed on the sending computer and the receiving computer, wherein The sender computer also includes: a drawing module for forming a marker image on the main image; and A first image processing module is used for extracting and compressing the trajectory information constituting the marked image, so as to transmit the compressed trajectory information to the receiving computer through the low-bandwidth network in real time. 10. The system according to claim 9, wherein the first image processing module extracts the trajectory information of the marked image at a frequency of 15 strokes per second. 11. The computer system as claimed in claim 9, wherein the drawing module forms the mark image on the main image through a computer index control device. 12. The system of claim 11, wherein the trajectory information includes coordinate information and characteristic information of each trajectory point constituting the marker image. 13. The system of claim 12, wherein the coordinate information is a movement information detected by the computer index control device in a computer operating system. 14. The system as claimed in claim 12, wherein the characteristic information refers to one of color, thickness and shape of the marked image. 15. The system of claim 12, wherein the first image processing module further comprises a down-sampling module for deleting track points that are too close together. 16. The system as claimed in claim 12, wherein the first image processing module further comprises a data reorganization module for converting the coordinate information and characteristic information of each track point into multiple sets of classified data for further compression and deletion subsequent trajectory information. 17. The system according to claim 16, wherein the first image processing module further comprises a compression module for utilizing the track information of the previous track point and the difference between the track information of the previous track point and the next track point A trajectory information change amount stores the trajectory information of the next trajectory point, thereby compressing the truncated trajectory information to reduce the amount of data to be transmitted. 18. The system as claimed in claim 9, wherein the receiver computer further comprises a second image processing module, and the second image processing module further comprises an image synthesizer to display the marked image on the main image . 19. The system of claim 9, wherein the receiving computer further comprises: a second image processing module for decompressing the received trajectory information to restore the marker image on the main image. 20. A method of instantly transmitting a tagged image on a shared image, the method comprising: A: Provide the shared image so that it can be displayed on a first computer and a second computer at the same time; B: forming a marked image on the shared image by the first computer, and transmitting the marked image to the second computer in real time, wherein, in the instant transmission step of the marked image, the method further includes: B1: Extracting the trajectory information of each trajectory point forming the marked image at a frequency; B2: Reshape and compress the extracted trajectory information; and B3: Immediately transmit the compressed and reformed track information to the second computer.

Images