JPH11308271A - Data communication device, receiving device, control method, storage medium, and data communication system - Google Patents

Abstract

(57) [Summary] [PROBLEMS] When performing data communication between two parties via a network, the transfer rate is dynamically changed and controlled in accordance with the situation on the network interposed between the two parties, and optimal data transfer is performed. Can be performed. Therefore, it is particularly effective when a real-time property such as transferring a raw video taken by a camera is required. A data transmission unit of a transmission side terminal adds a transmission time when transmitting data. The receiving terminal returns to the transmitting terminal a reception report added with information indicating the transmission time included in the received data and the time required for internal processing. The transmitting terminal measures the pure transfer time required for the round trip based on the reception time of the reception report information and the information in the reception report. Then, the buffer amount on the network is calculated based on the measured transfer time. Then, the transfer rate of the data transmission unit is controlled based on the calculated buffer amount.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data communication device and a receiving device for transmitting or receiving constantly generated data such as video and audio through a network, a control method, a storage medium, and a system.

[0002]

2. Description of the Related Art When data is transmitted and received via a network, the speed at which the data is transmitted over the network becomes a problem. If data is transmitted beyond the transmission capacity available on the network, data loss will occur.If the transmission rate is set too low, the available bandwidth of the network will not be fully used and satisfactory quality will not be obtained. turn into. Conventionally, in such a case, in order to adjust the transmission rate, a method of monitoring data loss in a network and lowering the transmission rate when a loss occurs has been adopted.

[0003]

However, the nodes distributed between the transmitting side and the receiving side of the network each have a buffer capable of storing data, and excessively sending out data causes loss. Is likely, one of the buffers on the network is overflowing. Therefore, even if the transmission rate is reduced after the loss, some buffer on the network is full, and the data stored in that buffer must be transmitted for the time that the network can process. Due to the delay, data from the transmission side arrives at the reception side, which is a very serious problem for data communication that emphasizes real-time properties.

[0004] That is, it is desired to transmit data at a transmission rate that makes full use of the transmittable network bandwidth while keeping the transmission delay within a certain fixed value.

As described above, when data is exchanged between two terminals via a network, the data transmitted to the network exists in a buffer or a transmission line of a node in the middle of the network, and has not yet reached the receiving side. There is no data. Data accumulated in the buffer of a node in the middle also causes transmission delay. Therefore, in order to suppress the transmission delay within a certain value, it is necessary to control the transmission rate so that the data amount does not become too large. However, in many cases, the amount of data stored in the buffer of a node in the middle of the network cannot be known by the terminal on the transmission / reception side. Especially on the Internet.

The present invention relates to a data communication apparatus and a data communication apparatus capable of dynamically changing and controlling a transfer rate according to a situation on an intervening network when performing data communication via a network, thereby enabling optimal data transfer. An apparatus, a control method, a storage medium, and a data communication system are provided.

[0007]

To solve this problem, for example, a data communication apparatus according to the present invention has the following configuration. That is, a data communication device that transfers data to a terminal via a network, a measuring unit that measures a data transfer time with the terminal, and a data transfer time that is measured by the measuring unit. Detecting means for detecting information relating to the amount of data buffered on the network, in data transferred to the terminal,
The apparatus includes means for obtaining a data transfer rate based on the result of the detection means, and transfer means for transferring data in accordance with the obtained transfer rate.

[0008]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is a diagram showing the connection relationship and structure of each device when data transmitted by the transmitting terminal 1-1 is received by the receiving terminal 1-2 via the network 1-3. Here, the network 1-3 includes a LAN operated in the organization and a large-scale network in which an unspecified number of networks such as the so-called Internet are connected, and is not limited in form. Hereinafter, each terminal in FIG. 1 and its operation will be described.

[0010] The transmitting terminal 1-1 is provided with a data generating unit 1-11.
Generate data at. Specifically, the data may be, for example, video data captured by a video camera. If the data is video, the data generation unit 1
In the case of -11, video is taken in or compressed. However, the content of the data here is not limited to video.

The data generated by the data generator 1-11 is sent to the data transmitter 1-12. Data transmission unit 1-
12 divides the transmitted data into an appropriate size, and transmits the divided data to the network 1-3 with the information of the sequence number and the time of transmitting the data added. At this time, the data transmission unit 1-12 adjusts the transmission rate to the rate specified by the transmission rate change unit 1-13 by adjusting the size of the data to be divided and the interval at which the divided data is sent.

The reception report receiving section 1-15 receives the reception report transmitted from the receiving terminal 1-2, and sends the contents of the report to the network buffer data amount calculation section 1-14. The network buffer data amount calculation unit 1-14 calculates the round trip time of data, and obtains the network buffer data amount based on the calculated round trip time. Based on the amount of network buffer data,
The transmission rate changing unit 1-13 determines the transmission rate, and specifies the transmission rate to the data transmission unit 1-12.

On the other hand, in the receiving terminal 1-2, the data (here, video data + sequence number + transmission time information) transmitted through the network 1-3 is received by the data receiving section 1-21. The received data is sent to data processing 1-22 for processing. For example, when the data is a video, a process for displaying the video (such as a decoding process and a display process) is performed by the data processing unit 1-22.

The data receiving section 1-21 measures information on the sequence number of the data received, the time at which the data was received, the amount of data received, and the like, and sends the information to the reception report generating section 1-23 periodically. The reception report generation unit 1-23 calculates the reception rate required for the reception report based on the information, adds information of the reception sequence number, the data transmission time, the data reception time, and the reception report transmission time, and transmits the reception report. Notify section 1-24.

The reception report transmitting section 1-24 transmits a reception report to the reception report receiving section 1-14 of the transmitting terminal 1-1 via the network 1-3.

Hereinafter, the operation procedure of the transmitting / receiving terminal and the method of determining the transmission rate will be described in detail with reference to FIGS.

The operation of the transmitting terminal will be described with reference to FIG. Note that the processes in steps S201 and S202 and the processes in steps S203 to S205 are separate tasks as will be apparent from the following description.

First, data to be transmitted is generated (step S201). For example, when transmitting a video, video capture and compression are performed. Next, the data is divided into data (packets) of an appropriate size, and transmitted to the network at intervals determined at that time (step S202). At this time, the transmission rate is determined by the size of the packet and the transmission interval. That is,
In this step 202, the data is transmitted to the network at the specified transmission rate. How to determine the transmission rate will be described later. When the data transmission is completed, the data generation is performed again (step S201), and the data is transmitted (step S201).
202) is repeated. FIG. 4 shows a format example of one packet constituting transmission data. One packet of data includes a transmission sequence number, a data transmission time, a packet size, and data to be transmitted (here, video data).

On the other hand, the transmitting terminal transmits the data as described above, and at the same time, waits for the reception report to be reported from the receiving terminal 1-2 (step S205). FIG. 5 shows a format example of the reception report. It is assumed that the reception report includes a reception sequence number (the transmission sequence number is diverted as it is), a reception rate, a reception time, and a reception report transmission time, as illustrated.

When the transmitting terminal receives the reception report, the data round trip time between the transmitting and receiving terminals is measured based on the report, and the data is transmitted from the transmitting terminal to the network based on the round trip time. The amount of data that has not yet arrived at the receiving terminal is calculated (step S204). Hereinafter, this data amount is referred to as a network buffer data amount. The calculation method of the network buffer data amount will be described below.

The time required for the round trip includes the time required from when the receiving terminal receives the data to when the receiving terminal sends out the reception report, and it is necessary to deduct that amount. In other words, the pure round trip time RTTcu for data transmission
r is RTTcur = (Ts2-Ts1)-(Tr2-Tr1). Here, Ts1 is the transmission time included in the reception report (also the time added to the transmission packet), and Ts2 is the time when the transmission terminal received the reception report. Also, T
r1 is the data reception time included in the reception report, and Tr2 is the reception report transmission time included in the reception report.

For example, the time when the reception report as shown in FIG. 5 is received is 10: 11: 13.526 on April 15, 1997.
If the time is in seconds, RTTcur = (13.526-12.988)-(13.512-13.302) = 0.328 sec (in this case, the year, month, day, hour and minute are omitted and calculated).

By calculating the data round-trip time using such a method, an accurate round-trip time required for data transmission is calculated even if the timers of the receiving end and the transmitting terminal do not match. be able to. Since the transmitting terminal only needs to know the time required inside the receiving terminal, the receiving terminal may add the time between the receiving time and the transmitting time instead of adding the receiving time and the transmitting time.

In the transmitting terminal (terminal which performs the video distribution service) in the embodiment, the minimum value of the data round trip time thus obtained is always set to the reference round trip time TRRba.
Keep as se. After the start of data transmission / reception, the reference round trip time TRRbase is set to the round trip time TRRcur when the first reception report is received.

That is, at the stage of starting transmission, RTTbase = RTTcur, and each time a reception report is received, RTTcu
R is calculated, and the round trip time RTTcur calculated each time is compared with RTTbase, and if RTTcur is smaller than RTTbase, RTTbase is updated with RTTcur.

RTTbase = RTTcur (IF RTTc
ur <RTTbase) The network buffer data amount is obtained based on the difference between the reference value RTTbase of the data round trip time and the latest measured round trip time RTTcur.

Actually, assuming that the reception rate (bits / sec) included in the reception report is Rrecv, the current network buffer data amount B is calculated as BUFcur = Rrecv * (RTTcur−RTTbase).
Find UFcur.

As will be understood by those skilled in the art from the above description, RTTcur and the like are the time required for a round trip, and data normally arrives from the transmitting terminal to the receiving terminal. It is necessary to calculate the time (outgoing time). However, the forward time and the round-trip time have a linear relationship, and in the embodiment, the round-trip time itself is not reflected in the transmission rate.
, And there is no conceptual difference between the forward time and the round-trip time, which is sufficient.

In the embodiment, the transmission rate Rnew is determined so that the network buffer data amount BUFcur thus obtained approaches the target value BUFdes of the network buffer data amount (step S203). The calculation formula is as follows.

Rnew = Rcur + C * (BUFdes−BUFcur) Rcur is the current transmission rate, and Rnew is the newly determined transmission rate. C is an appropriate constant. The transmission rate Rnew determined by this processing is transmitted to the data transmission unit, and specifies the transmission rate in the data transmission step (step S202).

Here, the transmission rate is determined based on the transmission rate Rcur, but the rate may be determined based on the reception rate Rrecv. Also, the reception report interval Inte
If rval is known, Rnew = Rrecv + ｛(BUFdes-BUFcur) / Interva so that the network buffer data amount matches the target value at the time of the next reception report arrival.
As in l｝, a method of determining the transmission rate Rnew can also be used.

As the target network buffer data amount BUFdes used at this time, it is desirable to specify a value that does not exceed the buffer amount actually mounted on network equipment such as a router.

Next, the operation process of the receiving terminal according to the embodiment will be described with reference to the flowchart of FIG.

The data received via the network is received by the data receiving section (S301). The transmitted data is passed to the data processing unit (Step S304).
For example, when video data is sent, the received data is passed to the video display processing. Also, the time when the data was received,
Information on the data amount and the reception sequence number is sent to the reception report generation unit. Then, a reception report is generated periodically (for example, every three seconds) based on the information (step S303). The reception report contains the reception sequence number,
It is generated including information on the reception rate, data reception time, and reception report transmission time. The generated reception report is transmitted to the reception report receiving unit of the transmitting terminal through the network (step S303). The format example of the reception report is as shown in FIG.

The above steps are repeated so that the receiving terminal periodically transmits a reception report while transmitting and receiving data between the transmitting and receiving terminals, and reports the reception report to the transmitting terminal. On the other hand, the transmitting terminal calculates the data round trip time based on the record of the transmission / reception time of the reception report, and calculates the network buffer data amount based on the growth of the data round trip time from the minimum value. Then, the transmission rate is determined so that the obtained network buffer data amount becomes constant.

In the embodiment, the receiving rate is calculated at the receiving terminal, included in the receiving report, and reported to the transmitting terminal. However, since the reception report includes the sequence number of the latest received data, the transmitting terminal holds the information of the reception sequence number, reception report time, and packet size reported in the previous reception report. If so, the receiving terminal can calculate the receiving rate. Thus, it is clear that the reception rate can be calculated at the transmitting terminal instead of being included in the reception report.

Next, a specific application example of the embodiment is shown in FIG. 6, and the operation thereof will be described below.

In the figure, reference numeral 10 denotes a camera server, which transfers video data captured by the camera 100 to the client 20 via the network 300. According to the description in the above embodiment, the camera server 10 corresponds to the transmitting terminal 1-1, and the client 20 corresponds to the receiving terminal 1
It can be understood that this corresponds to -2.

Now, the camera server 10 and the client 2
A hardware difference of 0 is a difference between whether or not a camera and a capture unit are provided, and both can be realized by, for example, a personal computer. That is, the code 10
3 to 109 and 203 to 209 have substantially the same configuration, and each is a general-purpose computer (for example, a personal computer).

On the other hand, in terms of software, on the camera server 10, software for transferring the captured video data to the client (stored in the external storage device 106, loaded into the RAM 104 and executed) is operating. The client 20 is different from the client 20 in that software (received in the external storage device 206, loaded into the RAM 304 and executed) for receiving and displaying video data operates.

However, here, for convenience, only the camera server and the client are separately shown, and when a video capture function is added to both, both can function as a camera server and a client. is there.

When the operation of the embodiment described above is applied to this system, the control target is the camera server 10.
Is the video data transfer rate. There are various ways to determine the transmission rate of the video data, but here, it is assumed that the capture time interval is appropriately changed.

The operation of the camera server will now be described.

FIGS. 7 and 8 are flowcharts showing the operation processing in the camera server. FIG. 7 determines a capture time interval (interval) based on a reception report transmitted from a client, and performs capture per unit time. FIG. 8 shows a process in which the number of images is determined, and as a result, the data transfer rate per unit time can be changed.
Indicates processing for transferring captured video data.

First, in step S81, the client 20
Wait for receiving the reception report (see FIG. 5) from
Upon receiving this, the process proceeds to step S82, where the data round trip time RTTcur is calculated, and the network buffer capacity BUFcur is calculated based on it. And step S
Proceeding to 83, the transmission rate Rnew (transfer data amount per unit time) is determined so as to approach the target buffer amount BUFdes.

Then, the process proceeds to a step S84 to determine the number of images that can be transferred per unit time at the calculated transfer rate, and sets it as a capture interval. Hereinafter, the processing of steps S81 to S84 is repeated.

On the other hand, regarding the transfer process (FIG. 8), this process operates at the time intervals determined in step S84.

First, in step S91, the capture unit 10
1 captures the video captured from the camera 100,
In step S92, the image is compression-encoded, and in step S93, the compression-encoded video including the transmission sequence number and the transmission time packet size is transferred in packet units as described above.

As a result, data transfer between two parties via the network can be performed optimally according to the target buffer data amount. In addition, even if the network is very vacant and the transfer speed can be increased, the transfer rate is not increased unnecessarily, but the amount of data stored as buffer data on the network is kept constant. Since the data transfer is controlled, the load on the network can be reduced, and the use of the network by a third party is not hindered.

The method described in the present embodiment can be implemented in various networks, for example, the Internet or LAN based on the IP protocol. 96
RTP standardized as RFC 1889 in January
al Time Transport Protocol). Note that RTP is a protocol for real-time applications, and is used instead of TCP in the transport layer. Such RTP
FIG. 9 shows an example of the receiver report packet determined by the above.

Here, in order to obtain the RTT (data round trip time) and Rrecv (reception rate) from the RTCP receiver report packet in FIG. 9, RTTcur = (Ts2-Ts1)-(Tr2-Tr1) .

Here, Ts2 is the time at which the RTCP receiver report packet was received, Ts1 is the “time (time) of the last sender report packet” in the RTCP receiver report packet,
“Tr2-Ts1” is “time since last sender packet” in the RTCP receiver report packet. Note that the above points are clearly shown in parentheses in FIG.

On the other hand, the reception rate is not specified in RFC1889. Therefore, there are several methods for obtaining Rrecv described above.

For example, assuming that the average size of one packet is P, Rrecv = P × (1−L) × (Sn−Sn−1) / (Ts2n−
Ts2n-1). Here, Ts2n is the time at which the RTCP receiver report packet at time n was received, and Ts2n-1 is the time at which the immediately preceding RTCP receiver report packet was received.

Alternatively, Rrecv = (total size of packets sent out from time (n-1) to n) × (1−L) / (Ts2n−Ts2n−1) or another method. I do not care.

As described above, in this embodiment, it is clear that RTP can be used.

The processing on the client side can be easily understood from the above description, and is not substantially different from that shown in FIG. 3, so that the description here is omitted.

Also, as described above, the transfer rate can be changed by changing the packet size. This is because the ratio of the original data amount to the control of the address information and the like included in the packet changes.

The initial transfer rate in the above embodiment is desirably set appropriately according to the type of network and the type of interface connected to the network. For example, if the above system is built in-house, Ethernet (10M
bps or 100 Mbps), a high transfer rate could be set initially.

The present invention is not limited only to the apparatus and method for realizing the above embodiment,
A computer program (CPU or MPU) in the system or the apparatus is supplied with a program code of software for realizing the above-described embodiment, and the computer of the system or the apparatus operates the various devices according to the program code. Implementation of the embodiment is also included in the scope of the present invention.

In this case, the program code of the software implements the functions of the above-described embodiment, and the program code itself and means for supplying the program code to the computer, specifically, the program A storage medium storing the code is included in the scope of the present invention.

Although the embodiment has been described with an example of two-party communication, the present invention is not limited to two-party communication, but may be communication of three or more parties.

As a storage medium for storing such a program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-RO
M, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

The computer controls various multiplications in accordance with only the supplied program code, so that not only the functions of the above-described embodiment are realized, but also the program code operates on the computer. Such a program code is included in the scope of the invention even when the above-described embodiment is realized in cooperation with an operating system (OS) or other application software.

Further, after the supplied program code is stored in a memory provided in a function expansion board of the computer or a function expansion unit connected to the computer, the function expansion board or the function is stored based on the instruction of the program code. The scope of the present invention also includes a case where a CPU or the like provided in the extension unit performs part or all of the actual processing, and the above-described embodiment is realized by the processing.

As described above, according to the present invention, according to the extension of the data round-trip time, data is transmitted from the transmitting terminal to the network so as to keep the data amount (network buffer amount) which has not yet arrived at the receiving terminal constant. Adjust rate. This makes it possible to accurately determine the amount of data accumulated on a route in the middle of the network, and it is expected that transmission delay can be suppressed within an allowable range by adjusting the amount of data. Further, since data is transmitted and received so that data is always stored in the buffer, it is expected that data can be transmitted and received using the available bandwidth of the network sufficiently. Further, instead of using the delay time as an index for controlling the transmission rate as it is, using the amount of data buffered in the network, the amount of buffer that will be sufficiently present on the network is defined as target buffer data. It is possible to prevent data from being excessively transmitted on the network.

The present invention is highly effective when applied to a large-scale network such as the Internet.

In the embodiment, the camera server has been described as an example of the transmitting terminal. However, the present invention is not limited thereto. For example, the present invention can also be applied to a case where a moving image file stored in an external storage device is played to provide services to a client.

[0069]

As described above, according to the present invention, when data communication is performed via a network, the transfer rate is dynamically changed and controlled in accordance with the situation on the intervening network, and optimal data transfer is performed. Becomes possible. Therefore, it is particularly effective when a real-time property such as transfer of a raw video taken by a camera is required.

[0070]

[Brief description of the drawings]

FIG. 1 is a block diagram of an apparatus according to an embodiment.

FIG. 2 is a flowchart illustrating an operation processing content on a transmission side according to the embodiment.

FIG. 3 is a flowchart illustrating an operation processing content on a receiving side in the embodiment.

FIG. 4 is a format example of transmission data.

FIG. 5 is a format example of a reception report.

FIG. 6 is a system configuration diagram showing a specific application example in the embodiment.

FIG. 7 is a flowchart showing processing contents of the camera server in FIG. 6;

FIG. 8 is a flowchart showing the processing contents of the camera server in FIG. 6;

FIG. 9 is a diagram illustrating a format of an RTCP receiver report packet as an application example of the embodiment;

Claims (15)

[Claims] 1. A data communication device for transferring data to a terminal via a network, comprising: a measuring means for measuring a data transfer time with the terminal; and a data transfer time measured by the measuring means. Detecting means for detecting information on the amount of data buffered on the network in the data transferred to the terminal, based on the result of the detecting means; and obtaining a data transfer rate based on the result of the detecting means. Transfer means for transferring data according to a given transfer rate. 2. The data communication apparatus according to claim 1, wherein the transfer unit transfers, for each unit of data transfer, transmission time information for transmitting the unit. 3. The terminal according to claim 2, wherein the terminal returns transmission time information included in the received data, and reception report information including processing time information required internally after receiving the data. 3. The data communication device according to item 2. 4. The data communication apparatus according to claim 3, wherein the transmission interval of the reception report information is asynchronous with the data reception and is returned at a predetermined time interval. 5. The measuring means calculates a time required for a round-trip of data communication based on a time at which reception report information is received from the terminal, transmission time information included in the reception report information, and processing time information. The data communication device according to claim 3, wherein the data communication device performs the operation. 6. A means for obtaining the transfer rate, calculating means for calculating a buffer amount on the network based on the data transmission time measured by the measuring means, and calculating the calculated network buffer amount to a predetermined target value. 2. The data communication apparatus according to claim 1, further comprising: a correction unit configured to correct a previous transfer rate based on a difference from the data transfer rate. 7. The data communication apparatus according to claim 6, wherein said correction means corrects a transfer rate so as to reduce a difference from said target value. 8. The data communication apparatus according to claim 7, wherein the target value is set according to a transmission band of a network. 9. The data communication device according to claim 1, wherein the data transmitted to the terminal is video data captured by a predetermined image capturing unit. 10. A receiving device as a terminal for receiving data from the data communication device according to claim 2, wherein the receiving device receives data transferred from the data communication device; Information including transmission time information included in the data received by the means, and information including time information required from the reception to the transmission of the reply information to the data communication apparatus, to assist the measurement by the measurement means. And a reply means. 11. The data communication apparatus according to claim 1, wherein the network is a network conforming to RTP standardized by RFC1889. 12. A data communication method for transferring data to a terminal via a network, comprising: a measuring step of measuring a data transfer time with the terminal; and a data transfer time measured in the measuring step. A detection step of detecting information on the amount of data buffered on the network in the data transferred to the terminal, and a step of obtaining a data transfer rate based on a detection result by the detection step; A control step of controlling data transfer according to the obtained transfer rate. 13. A storage medium storing a program code that functions as a data communication device that transfers data to a terminal via a network by being read and executed by a computer, wherein the data transfer is performed between the terminal and the terminal. Measuring means for measuring time, based on the data transfer time measured by the measuring means, detecting means for detecting information on the amount of data buffered on the network, among data transferred to the terminal, A storage medium storing a means for obtaining a data transfer rate based on a result of the detection means, and a program code functioning as a transfer means for transferring data in accordance with the obtained transfer rate. 14. A data communication system comprising a transmitting device for transmitting data via a network and a receiving device for receiving data, wherein the transmitting device measures a data transfer time with the receiving device. Measuring means for detecting, based on the data transfer time measured by the measuring means, detecting means for detecting information on the amount of data buffered on the network in data transferred to the receiving device; Means for obtaining a data transfer rate based on the result of the means, and transfer means for transferring data according to the obtained transfer rate, wherein the receiving device receives data from the transmitting device, Replying means for creating information that assists the measurement by the measuring means with respect to the data received by the receiving means, and returning the information to the transmitting device. Data communication system, characterized in that to obtain. 15. The transfer means adds and transmits transmission time information for transmitting the unit for each data transfer unit, and the reply means transmits the data included in the data received by the reception means. 15. The data communication system according to claim 14, wherein information including time information and time information required for a reply is created and returned.