WO2024250827A1 - 流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品 - Google Patents

流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品 Download PDF

Info

Publication number
WO2024250827A1
WO2024250827A1 PCT/CN2024/086929 CN2024086929W WO2024250827A1 WO 2024250827 A1 WO2024250827 A1 WO 2024250827A1 CN 2024086929 W CN2024086929 W CN 2024086929W WO 2024250827 A1 WO2024250827 A1 WO 2024250827A1
Authority
WO
WIPO (PCT)
Prior art keywords
packet loss
time
media stream
stream data
confirmation message
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/086929
Other languages
English (en)
French (fr)
Inventor
吴波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP24818364.2A priority Critical patent/EP4629596A4/en
Publication of WO2024250827A1 publication Critical patent/WO2024250827A1/zh
Priority to US19/234,177 priority patent/US20250310393A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80Responding to QoS
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00Arrangements for detecting or preventing errors in the information received
    • H04L1/12Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/1607Details of the supervisory signal
    • H04L1/1671Details of the supervisory signal the supervisory signal being transmitted together with control information
    • H04L1/1678Details of the supervisory signal the supervisory signal being transmitted together with control information where the control information is for timing, e.g. time stamps
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00Arrangements for detecting or preventing errors in the information received
    • H04L1/12Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1825Adaptation of specific ARQ protocol parameters according to transmission conditions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00Arrangements for detecting or preventing errors in the information received
    • H04L1/12Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1829Arrangements specially adapted for the receiver end
    • H04L1/1864ARQ related signaling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/28Flow control; Congestion control in relation to timing considerations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/30Flow control; Congestion control in combination with information about buffer occupancy at either end or at transit nodes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/32Flow control; Congestion control by discarding or delaying data units, e.g. packets or frames
    • H04L47/323Discarding or blocking control packets, e.g. ACK packets

Definitions

  • the present application relates to the field of computer technology, and in particular to a flow data transmission method, apparatus, computer equipment, computer-readable storage medium, and computer program product.
  • the quality of experience (QoE) on the user side is an important basis for measuring the service quality of different cloud service providers. Effectively improving the user experience quality is the goal pursued by major cloud platforms.
  • the main approaches to current traffic data transmission are concentrated in the following two aspects: First, research, design and deploy congestion control algorithms that are more suitable for network environments and business types, such as BBR. The core idea of these methods is to try to "accurately" identify whether the current network is congested. If congestion occurs, timely adjustments are made to the congestion problem in the hope that packet loss will not occur or will occur as little as possible during subsequent traffic transmission. Second, from the perspective of protocol innovation, the latest network transmission protocol is designed to improve traffic transmission efficiency through the advantages of the protocol itself. For example, multi-path transmission speeds up the flow completion time by adding a traffic transmission path, which will significantly improve the user-side service experience in most scenarios.
  • All the solutions adjust the sending strategy from the perspective of network congestion to reduce the packet loss rate and improve data transmission efficiency, but fail to fully consider the service experience on the user terminal side. That is, in actual situations, sometimes even if the packet loss rate is high during traffic transmission, the service experience of the user terminal is not necessarily very poor.
  • the key factor affecting the service experience on the user terminal side is whether the network packet loss is repaired in time within a certain period of time. If it is not repaired in time, the service experience on the user terminal side, such as audio and video playback, will experience freezes, black screens, and other features that seriously affect the user experience. Therefore, how to quickly and effectively repair network packet loss while ensuring that it does not affect the service experience on the user terminal side has become an urgent problem to be solved.
  • a flow data transmission method, apparatus, computer equipment, computer-readable storage medium, and computer program product are provided.
  • the present application provides a method for transmitting traffic data, which is executed by a terminal.
  • the method includes: in the process of receiving media stream data sent by a server, querying the cache time of the media stream data; encapsulating the cache time in a confirmation message to obtain a target confirmation message; sending the target confirmation message to the server, so that when the server determines that the media stream data has packet loss, the server adapts the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, and transmits the media loss packet according to the packet loss retransmission parameter before repairing the media stream data; receiving the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • the present application also provides a traffic data transmission device.
  • the device includes: a query module, which is used to query the cache time of the media stream data in the process of receiving the media stream data sent by the server; an encapsulation module, which is used to encapsulate the cache time in a confirmation message to obtain a target confirmation message; a sending module, which is used to send the target confirmation message to the server, so that when the server determines that the media stream data has packet loss, it will adapt the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, and before repairing the media stream data, it will adapt the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message.
  • the packet loss retransmission parameter is used to transmit the media loss packet; and the receiving module is used to receive the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • the present application also provides a computer device.
  • the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program: in the process of receiving media stream data sent by a server, query the cache time of the media stream data; encapsulate the cache time in a confirmation message to obtain a target confirmation message; send the target confirmation message to the server, so that when the server determines that the media stream data has packet loss, the server adapts the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, and transmits the media loss packet according to the packet loss retransmission parameter before repairing the media stream data; receive the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • the present application also provides a computer program product.
  • the computer program product includes a computer program, which implements the following steps when executed by a processor: in the process of receiving media stream data sent by a server, query the cache time of the media stream data; encapsulate the cache time in a confirmation message to obtain a target confirmation message; send the target confirmation message to the server, so that when the server determines that the media stream data has packet loss, it adapts the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, and transmits the media loss packet according to the packet loss retransmission parameter before repairing the media stream data; receive the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • the present application provides a method for transmitting traffic data, which is executed by a server.
  • the method includes: in the process of transmitting media stream data to a terminal, receiving a target confirmation message returned by the terminal; the target confirmation message carries the cache time of the terminal for the media stream data; when it is determined based on the target confirmation message that the media stream data has packet loss, adaptively adjusting the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message; transmitting the media loss packet corresponding to the media stream data according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • the present application also provides a flow data transmission device.
  • the device includes: a receiving module, which is used to receive a target confirmation message returned by the terminal during the process of transmitting media stream data to the terminal; the target confirmation message carries the cache time of the terminal for the media stream data; an adaptive module, which is used to adapt the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message when it is determined that the media stream data has packet loss based on the target confirmation message; a transmission module, which is used to transmit the media loss packet corresponding to the media stream data according to the packet loss retransmission parameters, so that the terminal receives the media loss packet before repairing the media stream data.
  • the present application further provides a computer device.
  • the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program: in the process of transmitting media stream data to a terminal, receiving a target confirmation message returned by the terminal; the target confirmation message carries the cache time of the terminal for the media stream data; when it is determined based on the target confirmation message that the media stream data is lost, adaptively retransmitting packet loss parameters according to the packet loss repair time and the cache time in the target confirmation message; according to the The packet loss retransmission parameter transmits the media loss packet corresponding to the media stream data, so that the terminal receives the media loss packet before repairing the media stream data.
  • the present application further provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented: in the process of transmitting media stream data to a terminal, receiving a target confirmation message returned by the terminal; the target confirmation message carries the cache time of the terminal for the media stream data; when it is determined based on the target confirmation message that the media stream data has packet loss, adaptively adjusting a packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message; transmitting a media loss packet corresponding to the media stream data according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • the present application also provides a computer program product.
  • the computer program product includes a computer program, which implements the following steps when executed by a processor: in the process of transmitting media stream data to a terminal, receiving a target confirmation message returned by the terminal; the target confirmation message carries the cache time of the terminal for the media stream data; when it is determined based on the target confirmation message that the media stream data has packet loss, adaptively adjusting a packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message; transmitting a media loss packet corresponding to the media stream data according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • FIG1 is an application environment diagram of a flow data transmission method according to an embodiment
  • FIG2a is a schematic flow chart of a flow data transmission method according to an embodiment
  • FIG2 b is a schematic flow chart of a flow data transmission method according to another embodiment
  • FIG3 is a schematic diagram of an application status information table of a user terminal in one embodiment
  • FIG4 is a schematic diagram of a user terminal periodically acquiring cache information of an application receiving queue in one embodiment
  • FIG5 is a flow chart of a step of determining cache information based on media stream data in an application receiving queue in one embodiment
  • FIG6 is a flow chart of a method for transmitting flow data in another embodiment
  • FIG7 is a schematic diagram of packet loss and user-side freeze in one embodiment
  • FIG8 is a schematic diagram of a flow chart of an adaptive packet loss repair and traffic transmission method based on multi-party collaboration in one embodiment
  • FIG9 is a schematic diagram of the overall process of an adaptive packet loss repair and traffic transmission method based on multi-party collaboration in one embodiment
  • FIG10 is a schematic diagram of a data transmitting end adjusting a bit rate in one embodiment
  • FIG11 is a structural block diagram of a flow data transmission device in one embodiment
  • FIG12 is a structural block diagram of a flow data transmission device in another embodiment
  • FIG. 13 is a diagram showing the internal structure of a computer device in one embodiment.
  • first, second, and third are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first, second, and third” can be interchanged where permitted, so that the embodiments of the present application described herein can be used in addition to those illustrated or described herein. The operations may be performed in any order other than that described above.
  • the flow data transmission method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1.
  • the terminal 102 communicates with the server 104 through the network.
  • the data storage system can store the data that the server 104 needs to process.
  • the data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers.
  • the terminal 102 queries the cache time of the media stream data, and encapsulates the cache time in the confirmation message to obtain the target confirmation message; the terminal 102 sends the target confirmation message to the server 104, so that when the server 104 determines that the media stream data has been lost, the server 104 adapts the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, and transmits the media loss packet according to the packet loss retransmission parameters before repairing the media stream data; the terminal 102 receives the media loss packet transmitted by the server 104 based on the packet loss retransmission parameters.
  • the terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an IoT device, and a portable wearable device.
  • the IoT device may be a smart speaker, a smart TV, a smart air conditioner, and a smart vehicle-mounted device, etc.
  • the portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc.
  • Server 104 can be an independent physical server or a service node in a blockchain system.
  • a peer-to-peer (P2P) network is formed between the service nodes in the blockchain system.
  • the P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP).
  • server 104 can also be a server cluster composed of multiple physical servers, and can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery NetworkCDN), as well as big data and artificial intelligence platforms.
  • cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery NetworkCDN), as well as big data and artificial intelligence platforms.
  • the terminal 102 and the server 104 can be connected via Bluetooth, USB (Universal Serial Bus) or network and other communication connection methods, which are not limited in this application.
  • USB Universal Serial Bus
  • a method for transmitting traffic data is provided.
  • the method can be executed by a server or a terminal alone, or by the server and the terminal together.
  • the method is applied to the terminal in FIG. 1 as an example for description, and includes the following steps:
  • Step 202 In the process of receiving the media stream data sent by the server, query the cache time of the media stream data.
  • stream data refers to a set of data sequences that arrive in order, in large quantities, quickly and continuously, and the data sequence can be a sequence composed of multiple data packets.
  • stream data can be regarded as a dynamic data set that grows infinitely over time.
  • the media stream data in this application is streaming media data, which can also be called media data.
  • the media stream data in this application can be a media data sequence composed of multiple media data packets, including multiple types of data.
  • the media stream data can include at least one of video data, audio data, image data, application installation data, etc.
  • the video data in this embodiment can also include at least one of on-demand video, live video, etc.
  • the media stream data in this embodiment can be at least one of point-to-point streaming media data or video call data, etc.
  • the cache time refers to the cache duration of the media stream data cached in the application layer.
  • the cache time in this application can be expressed as a cache moment or a cache duration.
  • the terminal can determine the cache duration by the difference between different cache moments.
  • the terminal can query the cache time of the media stream data in the application layer. For example, a user can play an audio or video in an application through a trigger operation, and the terminal responds to the above-mentioned trigger operation of the user and sends a traffic request message to the server, so that the server transmits the requested media stream data to the terminal, that is, the server can send a traffic request message to the terminal.
  • the end transmits a traffic message carrying the requested media stream data.
  • the media stream data is audio and video data.
  • the user can select audio and video A in application A through a selection operation, and terminal A responds to the user's selection operation and sends a traffic request message to the server, so that the server transmits the media stream data corresponding to the requested audio and video A to terminal A, that is, the server can transmit a traffic message carrying the media stream data corresponding to the requested audio and video A to terminal A.
  • the terminal A queries the cache time of the media stream data corresponding to the audio and video A to be played in the local application layer. For example, the terminal A can query the cache time of the media stream data corresponding to the application that plays the audio and video A from the state information table maintained by itself.
  • the state information table is used to store the cache information info_l7_buffer of different applications running in the terminal A.
  • the cache information info_l7_buffer of different applications in this application includes but is not limited to the cache time, and may also include other cache information.
  • the cache information may also include the number of complete audio and video frames l7_buffer_frame_amount in the application receiving queue corresponding to the application, the information frame_last of the last audio and video frame in the application cache queue, etc.
  • Step 204 encapsulate the cache time into a confirmation message to obtain a target confirmation message.
  • the confirmation message refers to the message confirmation message that the data receiving end (terminal) receives the traffic message from the data sending end (server) and periodically sends to the server.
  • the message confirmation message is the confirmation message.
  • the target confirmation message refers to a message confirmation message carrying a cache time. That is, before the terminal in this application sends a message confirmation message pkt_ack to the server, the terminal can first query the cache time of the media stream data from the status information table maintained by itself, and encapsulate the queried cache time in the message confirmation message pkt_ack, and then obtain the target confirmation message pkt_ack_l7_buffer carrying the cache time.
  • the terminal may encapsulate the inquired cache time in a confirmation message to obtain a target confirmation message carrying the cache time.
  • the media stream data is audio and video data.
  • info_l7_buffer (l7_buffer_time_len)
  • the cache information info_l7_buffer includes the cache time l7_buffer_time_len
  • the terminal A can encapsulate the queried cache information info_l7_buffer (l7_buffer_time_len) in the message confirmation message pkt_ack, and obtain the target confirmation message pkt_ack_l7_buffer carrying the cache time l7_buffer_time_len.
  • the terminal when receiving media stream data sent by a server, determines whether there is packet loss based on the received media data packet. If there is packet loss, the terminal determines the identifier of the media loss packet (such as a sequence number), and encapsulates the identifier of the media loss packet and the cache time in a confirmation message, so that the server determines the media loss packet based on the identifier of the media loss packet.
  • the identifier of the media loss packet such as a sequence number
  • the server sends media data packets with sequence numbers 1 to 10 and the terminal receives media data packets with sequence numbers 1 to 4 and media data packets with sequence numbers 6 to 10, it can be determined that the media data packet with sequence number 5 is lost.
  • the sequence number of the lost media data packet and the cache time can be encapsulated in a confirmation message, and then the encapsulated target confirmation message is sent to the server, so that the server can determine the lost media data packet based on the sequence number.
  • Step 206 sending a target confirmation message to the server, so that when the server determines that the media stream data is lost, it can adapt the packet loss retransmission parameters according to the packet loss repair time and the buffer time in the target confirmation message, and Media lost packets are transmitted based on the packet loss retransmission parameters.
  • Packet loss refers to the inability of one or more data packets to reach the destination through the Internet. For example, in the process of the server sending media stream data to the terminal in this application, some data packets may not reach the terminal.
  • the packet loss repair time refers to the preset time threshold for packet loss repair.
  • the packet loss retransmission parameter refers to a parameter used to reflect the retransmission strategy.
  • the packet loss retransmission parameter in the present application may include packet loss retransmission parameters corresponding to different retransmission strategies.
  • the packet loss retransmission parameter corresponding to the aggressive retransmission strategy is the first packet loss retransmission parameter; the packet loss retransmission parameter corresponding to the redundant backhaul strategy is the second packet loss retransmission parameter.
  • Repairing media stream data refers to repairing packet loss data during the transmission of media stream data.
  • the packet loss rate is high during traffic data transmission, the service experience on the terminal side may not be very poor.
  • the key factor for the user experience on the terminal side is whether the network packet loss is repaired in time within a certain period of time. If it is repaired in time, there will be no lag when playing audio and video data on the terminal side.
  • Media loss packets refer to media data packets lost during the transmission of media stream data. For example, if the server sends media data packets with sequence numbers 1 to 10, and the terminal receives media data packets with sequence numbers 1 to 4 and media data packets with sequence numbers 6 to 10, then the media data packet with sequence number 5 is the media loss packet.
  • Step 208 receiving the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • the terminal encapsulates the queried cache time in a confirmation message.
  • the terminal can send a target confirmation message carrying the cache time to the server, so that when the server determines that packet loss occurs in the media stream data, the server can adapt the packet loss retransmission parameters based on the packet loss repair time and the cache time in the target confirmation message, and transmit the media loss packet based on the packet loss retransmission parameters before repairing the media stream data; further, the terminal can receive the media loss packet transmitted by the server based on the packet retransmission parameters.
  • terminal A receives the media stream data corresponding to audio and video A sent by the server, it is assumed that terminal A encapsulates the queried cache information info_l7_buffer (l7_buffer_time_len) in the message confirmation message pkt_ack, and after obtaining the target confirmation message pkt_ack_l7_buffer carrying the cache time, terminal A can send the target confirmation message pkt_ack_l7_buffer to the server. After the server receives the above target confirmation message pkt_ack_l7_buffer sent by terminal A, the server can determine whether packet loss occurs when transmitting the media stream data.
  • info_l7_buffer l7_buffer_time_len
  • the server can adaptively adjust the retransmission strategy of the lost packet according to the preset packet loss repair time and the cache time l7_buffer_time_len in the target confirmation message pkt_ack_l7_buffer, and obtain the packet loss retransmission parameters corresponding to the retransmission strategy; further, the server can retransmit the media loss packet according to the obtained packet loss retransmission parameters, so that the terminal receives the media loss packet before repairing the media stream data, and repairs the media stream data (i.e., repairs the lost data) according to the media loss packet, so as to ensure as much as possible that there is enough media stream data for the next stage of use (such as audio and video playback) before the cache time is consumed.
  • the server can adaptively adjust the retransmission strategy of the lost packet according to the preset packet loss repair time and the cache time l7_buffer_time_len in the target confirmation message pkt_ack_l7_buffer, and obtain the packet loss retransmission parameters corresponding
  • the receiving end i.e. the terminal
  • the sending end i.e. the server
  • the receiving end i.e. the terminal
  • the receiving end is in a state of "always waiting" for the lost message until the receiving end, i.e. the terminal, receives the retransmitted lost message and plays the subsequent traffic messages for audio and video.
  • the cache time of the media stream data is queried; the cache time is encapsulated in a confirmation message to obtain a target confirmation message; the target confirmation message is sent to the server, so that when the server determines that the media stream data has been lost, it adapts the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, and transmits the media loss packet according to the packet loss retransmission parameters before repairing the media stream data; and the media loss packet transmitted by the receiving server based on the packet loss retransmission parameters.
  • the server can adaptively adjust the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message. That is, after receiving the above cache time, the server can adaptively adjust the packet loss retransmission parameters corresponding to the retransmission strategy of the lost message.
  • the packet loss repair method involved in this embodiment can well balance the relationship between traffic sending and packet loss repair. While optimizing the user-side freeze caused by packet loss, it reduces the impact on the subsequent traffic transmission efficiency. It can quickly and effectively repair network packet loss while ensuring that the service experience on the user terminal side is not affected, which is conducive to improving the user-side experience quality of Internet services mainly based on audio and video.
  • the media stream data is cached in an application receiving queue corresponding to the application program; before querying the cache time of the media stream data, the method further includes:
  • the application refers to a target application that uses the transmitted media stream data. For example, after receiving the media stream data sent by the server, the terminal plays the received media stream data through application A, and application A is the target application.
  • the application receiving queue refers to the application receiving queue corresponding to the application program, which may also be called the application cache queue in some cases.
  • each application program corresponds to an application receiving queue, for example, the application receiving queue corresponding to application program A is Q1.
  • the cache information refers to the cache information of the application receiving queue corresponding to the application.
  • the cache information in this application may include:
  • the cache duration of the data in the application cache queue is l7_buffer_time_len;
  • frame_last Information about the last audio and video frame in the application cache queue, frame_last, specifically including:
  • info_l7_buffer ⁇ l7_buffer_time_len,l7_buffer_frame_amount, frame_last ⁇ (2)
  • the status information table refers to a table used to store cache information of different applications running in the terminal.
  • the status information table in this application may include: the application identifier of the application, the cache information corresponding to the application, and the update time.
  • the update time refers to the update time of the cache information corresponding to the application.
  • Figure 3 it is a schematic diagram of the application status information table of the user terminal. It can be seen from Figure 3 that the cache information of the application corresponding to the application identifier 1 currently running in the terminal is Info_l7_buffer_1, and the update time corresponding to the cache information Info_l7_buffer_1 is Time_1.
  • the terminal can cache the received media stream data in the application receiving queue corresponding to the application.
  • the terminal can periodically obtain the cache information info_l7_buffer of each application receiving queue through the monitoring module, and feed back the obtained cache information info_l7_buffer to the transport layer.
  • the terminal can maintain the status table shown in Figure 3 through the monitoring module, and periodically update the cache information info_l7_buffer of different applications in the table.
  • the terminal can monitor the media stream data in the application receiving queue corresponding to each application through the monitoring module, that is, the monitoring module in the terminal can determine the cache information info_l7_buffer of each application based on the media stream data in the application receiving queue corresponding to each application, and periodically update the cache information info_l7_buffer of each application in the status information table maintained by itself as shown in Figure 3.
  • the terminal can periodically query the cache time of the application playing the media stream data from the status information table shown in FIG3 according to a preset strategy or preset protocol.
  • the time interval T_gain_l7 for the terminal to periodically obtain the cache information info_l7_buffer of each application can be pre-configured by the administrator and reflected in the configuration file.
  • the media stream data is audio and video data.
  • FIG 4 it is a schematic diagram of the user terminal periodically obtaining the cache information of the application receiving queue.
  • the monitoring module in Figure 4 is located between the application layer and the transport layer, and its main goal is to obtain the cache information info_l7_buffer of different applications in a timely manner, that is, in the method provided in the present application, in order to obtain the cache information info_l7_buffer in the application receiving queue corresponding to each application, a monitoring module can be added in the application layer, or a monitoring module can be added between the application layer and the transport layer.
  • terminal A In the process of terminal A receiving the media stream data corresponding to the audio and video A sent by the server, assuming that terminal A caches the received media stream data corresponding to the audio and video A in the application receiving queue Q1 corresponding to application 1, then terminal A can determine the cache information as Info_l7_buffer_1 based on the media stream data corresponding to the audio and video A to be played in the application receiving queue Q1, wherein the cache information Info_l7_buffer_1 includes the cache duration l7_buffer_time_len1.
  • terminal A can store cache information Info_l7_buffer_1 in the state information table shown in FIG3, that is, terminal A can update cache information Info_l7_buffer_1 in the entry of application identifier 1 corresponding to application 1 as shown in FIG3, so that when terminal A needs to send a target confirmation message to the server, terminal A can query the cache time of the media stream data corresponding to audio and video A from the state information table shown in FIG3 as l7_buffer_time_len1, and encapsulate the cache time l7_buffer_time_len1 found in the confirmation message pkt_ack, and obtain the target confirmation message pkt_ack_l7_buffer.
  • the data sending end i.e., the server
  • the server can adaptively adjust the packet loss repair strategy according to the cache information sent by the receiving end, i.e., the terminal.
  • the server can timely grasp whether the terminal side (taking audio and video as an example) will (will) have a stuck phenomenon, and adjust the aggressiveness of the packet loss repair according to this information, effectively reducing the waiting time for a certain packet loss data on the user terminal side, thereby optimizing the service experience on the user terminal side.
  • the method further includes: when the application is switched to a closed state, deleting the application status bar of the application from the status information table; when other applications are switched to an open state, adding application status bars corresponding to application identifiers of other applications to the status information table.
  • the closed state refers to a state in which the application stops running. For example, the user closes the application A through a trigger operation, and the terminal switches the application A from the open state to the closed state in response to the user's trigger operation.
  • the application status bar refers to the application status entry in the status information table.
  • each row in the status information table can be used as the application status bar of the application.
  • the information contained in the application status bar of the application includes the application identifier of the application, cache information (Info_l7_buffer) and update time.
  • Other applications refer to applications other than the application currently using the transmitted media stream data, that is, other applications and applications in this application are used to distinguish different applications.
  • application A in the terminal is the application that plays the media stream data
  • other applications can be other applications in the terminal except application A, for example, other applications can be application B and application C in the terminal.
  • the open state refers to the state in which the application is open and running. For example, the user opens application A through a trigger operation, and the terminal switches application A from the closed state to the open state in response to the user's trigger operation.
  • the application identifier is an identifier used to uniquely identify an application.
  • the other applications may be application B and application C in the terminal.
  • the application identifier of other application B is B
  • the application identifier of other application C is C.
  • the terminal can maintain the status table as shown in Figure 3 through the monitoring module, and periodically update the cache information info_l7_buffer of different applications in the table, that is, the terminal can monitor the media stream data in the application receiving queue corresponding to each application through the monitoring module, and determine the cache information info_l7_buffer of each application based on the media stream data in the application receiving queue corresponding to each application.
  • the terminal After the terminal periodically updates the cache information info_l7_buffer of each application in the status information table maintained by itself as shown in Figure 3, when the terminal detects that the target application is switched to the closed state, the terminal can delete the application status bar of the target application from the status information table as shown in Figure 3; or, when the terminal detects that other applications are switched to the open state, the terminal can add the application status bar corresponding to the application identifier of the other application in the status information table as shown in Figure 3.
  • the other applications may be other applications in the terminal except application 1, for example, the other applications may be application 2 and application 3 in the terminal.
  • the terminal may delete the application status bar of application 1 from the status information table as shown in FIG3, that is, the terminal may delete the application status bar of the row corresponding to application identifier 1 of application 1 from the status information table as shown in FIG3;
  • the terminal may add the application status bar corresponding to application identifier 3 of other application 3 in the status information table as shown in FIG3, for example, the terminal may add the application status bar of the row corresponding to application identifier 3 of application 3 from the status information table as shown in FIG3.
  • the data sending end i.e. the server
  • the server can promptly understand whether there will be (or will be) a freeze on the terminal side (taking audio and video as an example), and adjust the aggressiveness of the packet loss repair based on this information, effectively reducing the waiting time for a certain packet loss data on the user terminal side, thereby optimizing the service experience on the user terminal side.
  • the step of encapsulating the cache time in a confirmation message to obtain a target confirmation message includes: when it is detected that the cache time in the status information table is updated, encapsulating the updated cache time in the confirmation message to obtain a target confirmation message.
  • the updated cache time refers to the updated cache time. For example, at time T1, the cache time of application 1 in the status information table is S1. At time T2, the cache time of application 1 in the status information table is updated and the updated cache time is S2.
  • the terminal in the process of receiving media stream data sent by the server, when the terminal detects that the cache time of the media stream data in the status information table is updated, the terminal can obtain the updated cache time and encapsulate the updated cache time in the confirmation message to obtain the target confirmation message; further, after the terminal obtains the target confirmation message, the process of the terminal sending the target confirmation message to the server can be immediately triggered. That is, the cache time in the latest cache information in this application will trigger the automatic generation of the target confirmation message.
  • the terminal can obtain the updated cache time S2, and encapsulate the updated cache time S2 in the confirmation message pkt_ack to obtain the target confirmation message pkt_ack_l7_buffer; further, when the terminal obtains the target confirmation message pkt_ack_l7_buffer, the terminal can obtain the updated cache time S2.
  • the process of the terminal sending the target confirmation message pkt_ack_l7_buffer to the server can be immediately triggered, so that the server can receive the target confirmation message in time, and based on the cache time carried in the target confirmation message, timely and accurately grasp whether the terminal side is (about to) experience a freeze, and adjust the aggressiveness of packet loss repair based on this information, thereby optimizing the service experience.
  • the method before querying the cache time of the media stream data, the method further includes: generating a confirmation message when the media stream data is received; and querying the cache time of the media stream data includes: when the confirmation message is detected, querying the cache time of the media stream data.
  • the terminal can generate a confirmation message.
  • the terminal can query the latest cache time of the transmitted media stream data from the state information table maintained by itself, and encapsulate the latest cache time queried in the confirmation message to obtain the target confirmation message.
  • the process of the terminal sending the target confirmation message to the server can be immediately triggered. That is, the latest generated confirmation message in this application will trigger the automatic acquisition of the latest cache time and automatically generate the target confirmation message.
  • the terminal can generate a pkt_ack message.
  • the terminal can query the latest cache information info_l7_buffer corresponding to the application that plays the media stream data A from the status information table maintained by itself as shown in Figure 3.
  • the latest cache information info_l7_buffer includes the latest cache time S2, and the queried latest cache time S2 is encapsulated in the pkt_ack message to obtain the target confirmation message pkt_ack_l7_buffer.
  • the process of the terminal sending the target confirmation message pkt_ack_l7_buffer to the server can be immediately triggered, so that the server can receive the target confirmation message in time, and based on the cache time carried in the target confirmation message, timely and accurately grasp whether the terminal side is (about to) experience a freeze, and adjust the aggressiveness of packet loss repair based on this information, thereby optimizing the service experience.
  • the step of determining cache information based on the media stream data in the application receiving queue includes:
  • Step 502 Obtain the start frame data and the end frame data of the media stream data in the application receiving queue
  • Step 504 determining media cache data according to the start frame data and the end frame data
  • Step 506 Determine the buffering time according to the time interval between the start frame data and the end frame data.
  • the starting frame data refers to the first available data in the application receiving queue.
  • the starting frame data in the present application may be the first available frame data in the application receiving queue.
  • the end frame data refers to the last available data in the application receiving queue.
  • the end frame data in the present application may be the last available frame data in the application receiving queue.
  • Media cache data refers to data used to reflect relevant information of media stream data in the application receiving queue.
  • the media cache data in this application includes at least: a) the cache length of data in the application receiving queue l7_buffer_time_len; b) the number of complete audio and video frames in the application receiving queue l7_buffer_frame_amount; c) the information frame_last of the last audio and video frame in the application receiving queue.
  • the terminal can cache the received media stream data in the application receiving queue corresponding to the application, and the terminal can periodically obtain the cache information info_l7_buffer of each application receiving queue through the monitoring module, that is, the terminal can obtain the starting frame data and the ending frame data of the media stream data from the application receiving queue corresponding to each application through the monitoring module, and determine the media cache data according to the starting frame data and the ending frame data.
  • the terminal can determine the number of complete audio and video frames l7_buffer_frame_amount in the application receiving queue according to the frame sequence number of the starting frame data and the ending frame data; further, the terminal can determine the cache time according to the time interval between the starting frame data and the ending frame data.
  • the terminal determines that the time interval between the starting frame data and the ending frame data is S according to the timestamp of the starting frame data and the timestamp of the ending frame data, and the terminal can use the time interval S as the cache time. It can be understood that in some cases, the terminal can also use the timestamp of the starting frame data and the timestamp of the ending frame data as the cache time.
  • the identifier of the audio and video frame frame_last_id f6 and the reception degree of the audio and video frame frame_last_rcv, where frame_last_rcv specifically includes the number of messages shared by the audio and video frame frame_last_pkt_amount and the number of messages currently received frame_last_pkt_rcv_amount;
  • the information frame_last of the last audio and video frame in the application receiving queue can be expressed by the aforementioned formula (1), and the reception degree frame_last_rcv of the audio and video frame can be calculated based on the two parameters of the number of packets currently received frame_last_pkt_rcv_amount and the number of packets shared by the audio and video frame frame_last_pkt_amount.
  • the calculation method can be shown in the following formula (3):
  • the terminal can maintain the status table shown in Figure 3 through the monitoring module, and periodically update the cache information info_l7_buffer of different applications in the table.
  • the data sending end i.e. the server
  • the server can adaptively adjust the packet loss repair strategy according to the cache information sent by the receiving end, i.e. the terminal.
  • the server can promptly understand whether there will be (or will be) a freeze on the terminal side (taking audio and video as an example), and adjust the aggressiveness of the packet loss repair based on this information, effectively reducing the waiting time for a certain packet loss data on the user terminal side, thereby optimizing the service experience on the user terminal side.
  • the buffering time is determined according to the time interval between the start frame data and the end frame data. It includes: obtaining a first timestamp of the starting frame data and a second timestamp of the ending frame data; the first timestamp is the playback time point of the starting frame data in the player of the application, and the second timestamp is the playback time point of the ending data in the player; based on the difference between the second timestamp and the first timestamp, determining the cache time.
  • the first timestamp and the second timestamp in the present application are only used to distinguish the timestamps corresponding to different frame data.
  • the terminal can cache the received media stream data in the application receiving queue corresponding to the application, and the terminal can periodically obtain the cache information info_l7_buffer of each application receiving queue through the monitoring module, that is, the terminal can obtain the start frame data and end frame data of the media stream data from the application receiving queue corresponding to each application through the monitoring module, and obtain the first timestamp of the start frame data and the second timestamp of the end frame data. Further, the terminal can calculate the difference between the second timestamp and the first timestamp, and take the absolute value of the difference as the cache time. It can be understood that in some cases, the terminal can also use the first timestamp of the start frame data and the second timestamp of the end frame data as the cache time.
  • s of the difference d 20s as the cache time.
  • the terminal can maintain the status table shown in Figure 3 through the monitoring module, and periodically update the cache time in the cache information info_l7_buffer of different applications in the table.
  • the data sending end i.e. the server
  • the server can adaptively adjust the packet loss repair strategy according to the cache information sent by the receiving end, i.e. the terminal.
  • the server can promptly understand whether there will be (or will be) a freeze on the terminal side (taking audio and video as an example), and adjust the aggressiveness of the packet loss repair based on this information, effectively reducing the waiting time for a certain packet loss data on the user terminal side, thereby optimizing the service experience.
  • the method further includes: obtaining first start frame data and first end frame data of the media stream data in an application receiving queue of the application layer; obtaining second start frame data and second end frame data of the media stream data in a data receiving queue of the transport layer; and determining the cache time based on a first time interval between the first start frame data and the first end frame data and a second time interval between the second start frame data and the second end frame data.
  • the first start frame data and the second start frame data are used to distinguish the start frame data in different queues, and the first end frame data and the second end frame data are used to distinguish the end frame data in different queues.
  • the terminal can cache the received media stream data in a queue.
  • the queue in the embodiment of the present application includes a cache queue of the application layer player (also referred to as an application receiving queue) and a data receiving queue of the transport layer.
  • the terminal can obtain the first start frame data and the first end frame data of the media stream data in the application receiving queue of the application layer; at the same time, if the data in the data receiving queue of the transport layer is not empty, the terminal can obtain the second start frame data and the second end frame data of the media stream data in the data receiving queue of the transport layer; further, the terminal can determine the cache time based on the first time interval between the first start frame data and the first end frame data, and the second time interval between the second start frame data and the second end frame data, that is, the terminal can calculate the first time interval between the first start frame data and the first end frame data, and the second time interval between the second start frame data and the second end frame data, and determine the sum value between the first time interval and the second time interval, and use the determined sum value as the final cache time; or, in some cases, the terminal can take the absolute values of the first time interval and the second time interval respectively, determine the sum value between the absolute values of the first time interval and the second time interval, and use the determined sum value as the final cache time
  • the calculation method adopted by the terminal to calculate the first time interval between the first start frame data and the first end frame data, and the second time interval between the second start frame data and the second end frame data includes: the terminal calculates the difference between the timestamp of the first start frame data and the timestamp of the first end frame data, and takes the absolute value of the difference as the first time interval; similarly, the terminal calculates the difference between the timestamp of the second start frame data and the timestamp of the second end frame data, and takes the absolute value of the difference as the second time interval.
  • the user terminal carries the information of the current application layer (such as a player) data cache time when replying to the message confirmation message, so that the data sending end (cloud server) can adaptively adjust the retransmission strategy of the previously lost message after receiving the above cache time, and can well balance the relationship between traffic sending and packet loss repair. While optimizing the user-side lag caused by packet loss, it reduces the impact on subsequent traffic transmission efficiency, and can quickly and effectively repair network packet loss while ensuring that the service experience on the user terminal side is not affected, which is conducive to improving the user-side experience quality of Internet services mainly based on audio and video.
  • the data sending end cloud server
  • transmitting the media loss packet according to the packet loss retransmission parameter includes: before repairing the media stream data, transmitting the media loss packet to the routing node according to the packet loss retransmission parameter, and when the packet loss retransmission parameter meets the priority condition, instructing the routing node to forward the media loss packet according to the priority condition.
  • the routing node refers to the intermediate routing node in the process of the terminal receiving the media stream data sent by the server.
  • the routing node in this application can be a gateway, an intermediate node router, etc.
  • the routing node is used to forward the received media stream data.
  • the priority condition refers to the condition used to filter the packet loss retransmission parameter.
  • the priority condition in the present application can be set to: determine whether the packet loss retransmission parameter meets the preset target value.
  • the preset target value can be 1, that is, when the packet loss retransmission parameter meets the preset target value of 1, it means that the packet loss retransmission parameter meets the priority condition.
  • the terminal sends a target confirmation message to the server, so that when the server determines that the media stream data has packet loss, the server can adapt the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, and before repairing the media stream data, the server transmits the media loss packet according to the adaptive packet loss retransmission parameter, that is, before repairing the media stream data, the server can transmit the media loss packet to the routing node according to the adaptive packet loss retransmission parameter, and when the routing node detects that the packet loss retransmission parameter meets the priority condition, it instructs the routing node to forward the media loss packet according to the priority condition, that is, when the routing node detects that the packet loss retransmission parameter meets the priority condition, the routing node can adjust the processing priority of the message containing the media loss packet, for example, the routing node can set the processing priority of the message containing the media loss packet to high, so that the routing node forwards the message through a high priority forwarding queue.
  • the server
  • the step of sending a target confirmation message to the server includes: when the cache time is less than the packet loss repair time, returning a preset number of target confirmation messages to the server; the packet loss repair time is determined based on the delay information of the media stream data during the transmission process.
  • the delay information in this application may include SRTT and RTT, and RTT (Rround-Trip Time) represents the round-trip delay.
  • the terminal encapsulates the queried cache time in the confirmation message.
  • the terminal can adopt an aggressive return strategy when returning the target confirmation message to the server. For example, the terminal can return a preset number of target confirmation messages to the server. That is, when the cache time is less than the packet loss repair time, the terminal can return the target confirmation message to the server.
  • the repair time When the repair time is reached, it means that the application playing the media stream data will soon be able to play the remaining cached data; in particular, if the player of the application has finished playing all the data in its cache, and the terminal has not promptly "informed" the sender (server), problems such as freezing may occur; in the actual transmission process, the target confirmation message fed back by the terminal may be delayed, for example, the terminal may reply to a target confirmation message for every 5 messages received, and the target confirmation message may also be lost. Therefore, in the embodiment of the present application, when the terminal determines that the cache time is less than the packet loss repair time, the terminal can return the same M copies of the target confirmation message to the server to prevent the target confirmation message from being lost during the return process.
  • the terminal encapsulates the queried cache time l7_buffer_time_len in the confirmation message, and after obtaining the target confirmation message pkt_ack_l7_buffer, when the terminal determines that the cache time l7_buffer_time_len is less than the packet loss repair time Time_threshold, the terminal will adopt an "aggressive" redundant return strategy when returning pkt_ack_l7_buffer, that is, the terminal returns the same M copies of pkt_ack_l7_buffer to prevent pkt_ack_l7_buffer from being lost during the return process;
  • the data sending end i.e. the server
  • the server can promptly understand whether there will be (or will be) a freeze on the terminal side (taking audio and video as an example), and adjust the aggressiveness of the packet loss repair based on this information, effectively reducing the waiting time for a certain packet loss data on the user terminal side, thereby optimizing the service experience.
  • FIG6 another method for transmitting traffic data is provided.
  • the method can be executed by a server or a terminal alone, or by a server and a terminal together.
  • the method is applied to the server in FIG1 as an example for description, and includes the following steps:
  • Step 602 During the process of transmitting the media stream data to the terminal, a target confirmation message returned by the terminal is received; the target confirmation message carries the buffering time of the terminal for the media stream data.
  • Step 604 When it is determined based on the target confirmation message that the media stream data has packet loss, adaptive packet loss retransmission parameters are set according to the packet loss repair time and the buffer time in the target confirmation message.
  • Step 606 Transmit the media loss packet corresponding to the media stream data according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • the server in the process of transmitting media stream data from the server to the terminal, the server periodically receives a target confirmation message returned by the terminal, and the target confirmation message carries the terminal's cache time for the media stream data.
  • the server determines that the media stream data has packet loss based on the target confirmation message, the server can adaptively adjust the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, and transmit the media loss packet corresponding to the media stream data according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • the media stream data is audio and video data.
  • terminal A receives the media stream data corresponding to the audio and video A sent by the server, it is assumed that terminal A encapsulates the queried cache information info_l7_buffer (l7_buffer_time_len) in the message confirmation message pkt_ack, and obtains the target confirmation message pkt_ack_l7_buffer carrying the cache time.
  • Terminal A can send the target confirmation message pkt_ack_l7_buffer to the server.
  • the server After the server receives the above-mentioned target confirmation message pkt_ack_l7_buffer sent by terminal A, the server can determine whether the media stream data has packet loss based on the target confirmation message.
  • the server can adaptively adjust the retransmission strategy of the lost message according to the preset packet loss repair time and the cache time l7_buffer_time_len in the target confirmation message pkt_ack_l7_buffer, and obtain the packet loss retransmission parameter corresponding to the retransmission strategy. Further, the server can transmit the media loss corresponding to the media stream data according to the obtained packet loss retransmission parameter.
  • the terminal receives the media loss packet before repairing the media stream data, and repairs the lost data based on the media loss packet, so as to ensure that there is enough media data for the next stage of use (such as audio and video playback) before the cache time is consumed.
  • the judgment method in the embodiment of FIG2a can be used to judge whether the media stream data has packet loss.
  • the receiving end i.e. the terminal
  • the sending end i.e. the server
  • the receiving end i.e. the terminal
  • the receiving end is in a state of "always waiting" for the lost message, and can only play the subsequent traffic messages for audio and video after the receiving end, i.e. the terminal, receives the retransmitted lost message.
  • a target confirmation message returned by the terminal is received; the target confirmation message carries the cache time of the terminal for the media stream data; when it is determined based on the target confirmation message that the media stream data has packet loss, a packet loss retransmission parameter is adaptively set according to the packet loss repair time and the cache time in the target confirmation message; and a media loss packet corresponding to the media stream data is transmitted according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • the server can adaptively adjust the packet loss retransmission parameters based on the packet loss repair time and the cache time in the target confirmation message. That is, after receiving the above cache time, the server can adaptively adjust the packet loss retransmission parameters corresponding to the retransmission strategy of the lost message.
  • the packet loss repair method involved in this embodiment can well balance the relationship between traffic sending and packet loss repair. While optimizing the user-side freeze caused by packet loss, it reduces the impact on subsequent traffic transmission efficiency. It can quickly and effectively repair network packet loss while ensuring that the service experience on the user terminal side is not affected, which is conducive to improving the user-side experience quality of Internet services mainly based on audio and video.
  • the step of adaptively adjusting the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message includes: comparing the packet loss repair time with the cache time in the target confirmation message to obtain a comparison result; when the comparison result indicates that the cache time is less than the packet loss repair time, adaptively adjusting the first packet loss retransmission parameters; when the comparison result indicates that the cache time is greater than or equal to the packet loss repair time, adaptively adjusting the second packet loss retransmission parameters.
  • the comparison result refers to the size relationship between the packet loss repair time and the cache time in the target confirmation message.
  • the comparison results in this application may include three situations: the first situation is that the cache time is less than the packet loss repair time, the second situation is that the cache time is equal to the packet loss repair time, and the third situation is that the cache time is greater than the packet loss repair time.
  • the first packet loss retransmission parameter and the second packet loss retransmission parameter are used to distinguish the packet loss retransmission parameters corresponding to different backhaul strategies.
  • the packet loss retransmission parameter when the "aggressive" redundant backhaul strategy is adopted is the first packet loss retransmission parameter
  • the packet loss retransmission parameter when the default backhaul strategy is adopted is the second packet loss retransmission parameter.
  • the server can adapt the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, that is, the server compares the packet loss repair time with the cache time in the target confirmation message to obtain a comparison result; when the comparison result indicates that the cache time is less than the packet loss repair time, the server adapts the first packet loss retransmission parameter; when the comparison result indicates that the cache time is greater than or equal to the packet loss repair time, the server adapts the second packet loss retransmission parameter.
  • the cache time T_buffer ⁇ packet loss repair time T_recovery it means that the packet loss can be repaired within the cache time, and the player on the terminal side will not be stuck at this time; if the cache time T_buffer ⁇ packet loss repair time T_recovery, it means that the packet loss has not been repaired when the data in the queue is consumed, and the player on the terminal side will be stuck at this time; in summary, different packet losses will have different impacts on the QoE on the terminal side, so the server needs to adapt the packet loss repair mechanism to this differentiated impact.
  • T_buffer is much larger than T_recovery, for example, T_buffer ⁇ 3 times T_recovery, it means that the packet loss is not urgent to repair.
  • the sender i.e., the server
  • the sender will adopt a relatively mild packet loss retransmission strategy, i.e., the server will automatically Adapt to the second packet loss retransmission parameter corresponding to the relatively mild packet loss retransmission strategy; if T_buffer is slightly greater than or equal to T_recovery, it means that the packet loss must be repaired within T_buffer.
  • the sender i.e.
  • the server will adopt a relatively aggressive packet loss retransmission strategy to ensure that the packet loss will not be lost again, that is, the server adapts to the second packet loss retransmission parameter corresponding to the relatively aggressive packet loss retransmission strategy; if T_buffer is less than T_recovery, it means that the packet loss is facing a more severe repair problem.
  • the sender i.e. the server, will adopt a more aggressive packet loss retransmission strategy, that is, the server adapts to the first packet loss retransmission parameter corresponding to the more aggressive packet loss retransmission strategy.
  • the end-cloud collaborative method used in this embodiment can accurately grasp the service experience on the user side, which is beneficial to improving the competitiveness of cloud server providers in optimizing QoE (Quality of Experience) indicators such as audio and video freezes, screen distortion, and mosaics.
  • QoE Quality of Experience
  • the method further includes: when the comparison result indicates that the cache time is less than the packet loss repair time, retransmitting a preset number of media loss packets according to a preset sending time interval; wherein the sending time interval is determined based on the delay information of the media stream data during the transmission process and the preset number; when the comparison result indicates that the cache time is greater than or equal to the packet loss repair time, retransmitting the media loss packet once.
  • the delay information may include SRTT and RTT
  • RTT Red-Trip Time
  • SRTT represents the smooth round-trip time
  • the server when the server determines that packet loss occurs in the media stream data based on the target confirmation message, the server can adaptively adjust the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, that is, the server compares the packet loss repair time with the cache time in the target confirmation message to obtain a comparison result; when the comparison result indicates that the cache time is less than the packet loss repair time, the server can retransmit a preset number of media loss packets according to the preset sending time interval.
  • the sender i.e., the server
  • the sender retransmits 4 identical pkt_loss messages to the terminal
  • the sender i.e., the server
  • the sender retransmits one pkt_loss message to the terminal.
  • the packet loss repair method involved in this embodiment has a good balance between traffic transmission and packet loss repair. While optimizing the freeze on the terminal side caused by packet loss, it reduces the impact on the subsequent traffic transmission efficiency, which is conducive to improving the user experience of existing Internet services based on audio and video, and enhancing the competitiveness of server manufacturers.
  • the method after receiving the target confirmation message returned by the terminal, the method further includes: when it is determined based on the target confirmation message that the media stream data has not been lost, adaptively transmitting the bit rate according to the packet loss repair time and the buffer time in the target confirmation message; or adaptively transmitting the bit rate according to the preset number of frames and the number of buffered frames in the target confirmation message; based on Transmission bit rate, transmitting media stream data to the terminal.
  • the transmission bit rate refers to the bit rate for transmitting media stream data, that is, the server can select different bit rates to transmit the target media stream data. For example, at time T1, the server transmits media stream data A to the terminal according to the original bit rate A1; after a period of time, at time T2, the server can select a bit rate A2 lower than the original bit rate A1 to transmit media stream data A to the terminal, so that the server can send as many audio and video frames as possible to the terminal under limited network resources.
  • the preset frame number refers to the preset frame number.
  • the preset frame number can be set to 10 in advance. That is, when the number of cached frames in the target confirmation message is less than the preset frame number 10, it means that there are fewer audio and video frames cached on the terminal side. At this time, the server needs to send as many audio and video frames as possible to the production terminal under limited network resources to prevent freezes and other phenomena.
  • the number of cached frames in the target confirmation message refers to the number of frames cached on the terminal side. For example, if the number of cached frames in the application receiving queue of the application that plays the media stream data is 30, then the number of cached frames in the target confirmation message is 30.
  • the server can adapt the transmission bit rate according to the packet loss repair time and the cache time in the target confirmation message; or, the server can adapt the transmission bit rate according to the preset number of frames and the number of cached frames in the target confirmation message; further, the server can transmit the media stream data to the terminal based on the adaptive transmission bit rate.
  • the server determines that the media stream data A has not been lost based on the target confirmation message, that is, when the server does not detect the loss of the message, if the cache time l7_buffer_time_len in the cache information info_l7_buffer ⁇ the packet loss repair time Time_threshold; or, if the cache frame number l7_buffer_frame_amount in the cache information info_l7_buffer ⁇ the preset frame number l7_buffer_frame_amount_threshold, it means that the cache time and cached audio and video frames on the terminal side are small. At this time, the server needs to send as many audio and video frames as possible under limited network resources.
  • the server can Select audio and video frames with a lower bitrate as the data source. For example, the server can record the bitrate_(i-1) of the audio and video frames sent in the past period of time.
  • the server will select the latest audio and video bitrate bitrate_i, where bitrate_i ⁇ bitrate_(i-1), that is, the server will send media stream data A to the terminal based on the latest audio and video bitrate bitrate_i.
  • l7_buffer_frame_amount_threshold can be configured by the administrator and declared in the configuration file.
  • the data sending end i.e. the server
  • the server can adaptively adjust the packet loss repair strategy based on the cache information sent by the receiving end, i.e. the terminal.
  • the server can promptly understand whether there will (or will be) a freeze on the terminal side (taking audio and video as an example), and automatically adjust the transmission bit rate when no packet loss occurs, thereby optimizing the service experience.
  • the step of adaptively transmitting the bit rate according to the packet loss repair time and the cache time in the target confirmation message includes: when the cache time in the target confirmation message is less than the packet loss repair time, reducing the original bit rate value of the media stream data to obtain the transmission bit rate; the step of adaptively transmitting the bit rate according to the preset frame number and the cache frame number in the target confirmation message includes: when the cache frame number in the target confirmation message is less than the preset frame number, reducing the original bit rate value of the media stream data to obtain the transmission bit rate.
  • the original bit rate value refers to the initial bit rate value in the process of the server transmitting media stream data to the terminal.
  • the server responds to the traffic request message of the terminal, selects the data source with the bit rate value A1, and transmits the media stream data to the terminal based on the bit rate value A1, then the original bit rate value is A1.
  • the server can adapt the transmission bit rate according to the packet loss repair time and the cache time in the target confirmation message, that is, when the cache time in the target confirmation message is less than the packet loss repair time, the server can reduce the original bit rate value of the media stream data to obtain the reduced transmission bit rate; or, when the number of cached frames in the target confirmation message is less than the preset number of frames, the server can also reduce the original bit rate value of the media stream data.
  • the original bit rate value is used to obtain the reduced transmission bit rate.
  • the method provided by the present application when the cache time is less than the packet loss repair time, and the number of cached frames is less than the preset number of frames, indicates that the cache time and cached audio and video frames on the terminal side are less.
  • the server needs to send as many audio and video frames as possible under limited network resources. Therefore, the server needs to reduce the original bit rate value of the media stream data and send the media stream data to the terminal based on the reduced transmission bit rate.
  • the data sending end i.e., the server
  • the server can adaptively adjust the packet loss repair strategy according to the cache information sent by the receiving end, i.e., the terminal.
  • the server can promptly grasp whether the terminal side is (about to) experience a freeze (taking audio and video as an example), and automatically adjust the transmission bit rate when no packet loss occurs, thereby optimizing the service experience.
  • the transmission code rate is a first code rate value; the method also includes: when the cache time in the target confirmation message is greater than the packet loss repair time, and the number of cached frames in the target confirmation message is greater than the preset number of frames, selecting a second code rate value as the transmission code rate; wherein the second code rate value is greater than the first code rate value.
  • the second bit rate value and the first bit rate value are used to distinguish different bit rate values.
  • the server can adaptively adjust the transmission bit rate according to the packet loss repair time and the cache time in the target confirmation message, that is, when the cache time in the target confirmation message is greater than the packet loss repair time, and the number of cached frames in the target confirmation message is greater than the preset number of frames, the server can select the second bit rate value as the transmission bit rate; wherein the second bit rate value is greater than the first bit rate value. That is, the method provided by the present application, when the cache time is greater than the packet loss repair time, and the number of cached frames is greater than the preset number of frames, indicates that the cache time and cached audio and video frames on the terminal side are large.
  • the server does not need to send as many audio and video frames as possible under limited network resources. Therefore, the server can increase the original bit rate value of the media stream data, and send the media stream data to the terminal based on the increased transmission bit rate. Therefore, the data sending end, i.e., the server, can adaptively adjust the packet loss repair strategy according to the cache information sent by the receiving end, i.e., the terminal. By adding the cache information on the terminal side in the message confirmation message, the server can timely understand whether there will be (or will be) a freeze on the terminal side (taking audio and video as an example), and automatically adjust the transmission bit rate when no packet loss occurs, thereby optimizing the service experience.
  • the target confirmation message carries the end frame identifier and reception degree parameter of the media stream data; the method further includes: when the reception degree parameter meets the preset conditions, the end frame data corresponding to the end frame identifier has not been sent completely, and the transmission window of the media stream data is limited, the end frame data continues to be transmitted to the terminal.
  • the end frame identifier refers to the identifier of the last audio or video frame in the application receiving queue of the application that plays or uses the media stream data.
  • the identifier of the last audio or video frame of media stream data A is f203
  • the end frame identifier of the media stream data carried by the target confirmation message is f203.
  • the reception degree parameter refers to the reception degree parameter of the last audio or video frame in the application receiving queue of the application that plays or uses the media stream data.
  • the reception degree parameter can be represented by frame_last_rcv.
  • the reception degree frame_last_rcv of the audio or video frame can be calculated based on two parameters: the number of messages currently received, frame_last_pkt_rcv_amount, and the number of messages shared by the audio or video frame, frame_last_pkt_amount. For example, the calculation method can be as shown in the aforementioned formula (3).
  • Window limitation refers to the window limitation specified in the congestion control method. If the window is limited, no traffic packets can be sent until the window is no longer limited.
  • the server can temporarily ignore the influence of the window limit, that is, the server can continue to transmit the end frame data to the terminal until all the remaining messages of the audio and video frame are sent to the terminal. That is, in this implementation, when the server is transmitting the media stream data, the server will optimize the last audio and video frame frame_last received by the user terminal.
  • the server can determine whether the last audio and video frame frame_last_id has been sent. If the last audio and video frame frame_last_id has not been sent, the server can continue to send the remaining content corresponding to the last audio and video frame frame_last_id.
  • the above process is not affected by window limitations, that is, if the window is limited and messages cannot be sent, the last audio and video frame on the terminal side cannot be completely received.
  • the terminal side will cause problems such as freeze, screen distortion, and mosaic. It can be understood that before sending a certain audio and video frame, the server will know how many messages there are in this frame. For example, a certain video frame contains a total of 20 messages, and its message numbers are: 10-19. The server sends messages (numbers: 10-16) in sequence. The remaining messages (numbers: 17-19) may not be sent in time due to window restrictions and other reasons.
  • the method provided in the embodiment of the present application enables the server to continue to implement an aggressive traffic sending strategy when the window is limited, that is, if the window is limited and more than half of the messages of the audio and video frame have been sent, the server will automatically ignore the impact of the window restriction and continue to send the remaining messages of the audio and video frame. Therefore, by optimizing the last audio and video frame received by the user terminal to ensure that the last audio and video frame on the terminal side can be fully received, the phenomenon of freeze, screen distortion, mosaic, etc. on the terminal side is effectively avoided, which is conducive to improving the user experience of existing Internet services based on audio and video and enhancing the competitiveness of server manufacturers.
  • the step of transmitting a media loss packet corresponding to media stream data according to a packet loss retransmission parameter includes: transmitting the media loss packet to a routing node according to the packet loss retransmission parameter, and when the packet loss retransmission parameter meets a priority condition, instructing the routing node to forward the media loss packet according to the priority condition.
  • the terminal sends a target confirmation message to the server, so that when the server determines that the media stream data has packet loss, the server can adapt the packet loss retransmission parameter according to the packet loss repair time and the cache time in the target confirmation message, and before repairing the media stream data, the server transmits the media loss packet according to the adaptive packet loss retransmission parameter, that is, before repairing the media stream data, the server can transmit the media loss packet to the routing node according to the adaptive packet loss retransmission parameter, when the routing node detects that the packet loss retransmission parameter meets the priority condition, instruct the routing node to forward the media loss packet according to the priority condition, that is, when the routing node detects that the packet loss retransmission parameter meets the priority condition, the routing node can adjust the processing priority of the message containing the media loss packet, for example, the routing node can set the processing priority of the message containing the media loss packet to high, so that the routing node forwards the message through a forwarding queue with a high priority, thereby
  • a media loss packet is transmitted to a routing node according to a packet loss retransmission parameter, and when the packet loss retransmission parameter satisfies a priority condition, the step of instructing the routing node to forward the media loss packet according to the priority condition includes: adding a target identification field to the media loss packet based on the packet loss retransmission parameter; transmitting the media loss packet to the routing node according to the packet loss retransmission parameter, and when the target identification field is a target value, instructing the routing node to preferentially forward the media loss packet; wherein, when the target identification field is the target value, it indicates that the packet loss retransmission parameter satisfies the priority condition.
  • the target identification field refers to a pre-set identification field as the target identification field.
  • the target identification field in this application can be set to loss_l7_buffer.
  • the target value refers to the field value of the target identification field being a specific value.
  • the target value can be set to 1.
  • the field value of the target identification field is 1, it means that the field value of the target identification field is the target value.
  • the server can add a target identification field in the media loss packet based on the packet loss retransmission parameters, and transmit the media loss packet to the routing node according to the packet loss retransmission parameters.
  • the target identification field is a target value
  • the routing node is instructed to give priority to forwarding the media loss packet; wherein, when the target identification field is a target value, it indicates that the packet loss retransmission parameters meet the priority conditions.
  • the routing node sets the forwarding priority of the message pkt_loss to high, which is specifically manifested in that the routing node forwards the message pkt_loss through a forwarding queue with a high priority;
  • the intermediate routing node if the intermediate routing node encounters a packet loss repair message with a specific identifier used to avoid terminal freezes, the forwarding priority of the message is increased, the forwarding efficiency of such messages is improved, the waiting time of the user terminal for certain packet loss data is reduced, and the service experience is optimized.
  • a flow data transmission system comprising:
  • a server used for transmitting media stream data to a terminal
  • the terminal is used to query the cache time of the media stream data during the process of receiving the media stream data sent by the server; encapsulate the cache time in a confirmation message to obtain a target confirmation message; and send the target confirmation message to the server;
  • the server is further configured to, when determining that packet loss occurs in the media stream data, adaptively configure packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, and transmit the media loss packet corresponding to the media stream data according to the packet loss retransmission parameters before repairing the media stream data;
  • the terminal is further used to receive the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • system further comprises:
  • the server is further configured to transmit the media loss packet to the routing node according to the packet loss retransmission parameter before repairing the media stream data;
  • the routing node is used to forward the media loss packet according to the priority condition when the packet loss retransmission parameter meets the priority condition.
  • the media stream data is cached in an application receiving queue corresponding to the application program; the system further includes:
  • the terminal is further used to determine cache information based on the media stream data in the application receiving queue; store the cache information in a status information table corresponding to the application; and query the cache time of the media stream data from the status information table.
  • system further comprises:
  • the terminal is further configured to delete the application status bar of the application from the status information table when the application is switched to the closed state; and to add the application status bar of the other application to the status information table when the other application is switched to the open state.
  • the application status bar corresponding to the application logo of the program.
  • system further comprises:
  • the terminal is further configured to encapsulate the updated cache time in a confirmation message when detecting that the cache time in the status information table is updated, so as to obtain a target confirmation message.
  • system further comprises:
  • the terminal is further configured to generate the confirmation message when receiving the media stream data; and query the cache time of the media stream data when detecting the confirmation message.
  • system further comprises:
  • the terminal is also used to obtain the starting frame data and the ending frame data of the media stream data in the application receiving queue; determine the media cache data according to the starting frame data and the ending frame data; and determine the cache time according to the time interval between the starting frame data and the ending frame data.
  • system further comprises:
  • the terminal is also used to obtain a first timestamp of the start frame data and a second timestamp of the end frame data; the first timestamp is the playback time point of the start frame data in the player of the application, and the second timestamp is the playback time point of the end data in the player; based on the difference between the second timestamp and the first timestamp, the cache time is determined.
  • system further comprises:
  • the terminal is also used to return a preset number of the target confirmation messages to the server when the cache time is less than the packet loss repair time; the packet loss repair time is determined based on the delay information of the media stream data during the transmission process.
  • system further comprises:
  • the server is also used to compare the packet loss repair time with the cache time in the target confirmation message to obtain a comparison result; when the comparison result indicates that the cache time is less than the packet loss repair time, the first packet loss retransmission parameter is adaptively used; when the comparison result indicates that the cache time is greater than or equal to the packet loss repair time, the second packet loss retransmission parameter is adaptively used.
  • system further comprises:
  • the server is further configured to, when it is determined based on the target confirmation message that no packet loss occurs in the media stream data, adaptively adjust the transmission bit rate according to the packet loss repair time and the buffering time in the target confirmation message; or adaptively adjust the transmission bit rate according to a preset number of frames and the number of buffered frames in the target confirmation message; and transmit the media stream data to the terminal based on the transmission bit rate;
  • the terminal is further used to receive the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • system further comprises:
  • the server is also used to reduce the original bit rate value of the media stream data to obtain the transmission bit rate when the cache time in the target confirmation message is less than the packet loss repair time; when the number of cached frames in the target confirmation message is less than the preset number of frames, reduce the original bit rate value of the media stream data to obtain the transmission bit rate.
  • the transmission code rate is a first code rate value; the system further comprises:
  • the server is also used to select a second code rate value as the transmission code rate when the cache time in the target confirmation message is greater than the packet loss repair time and the number of cached frames in the target confirmation message is greater than the preset number of frames; wherein the second code rate value is greater than the first code rate value.
  • the target confirmation message carries an end frame identifier and a reception degree parameter of the media stream data;
  • the system further comprises:
  • the server is further configured to continue transmitting the end frame data to the terminal when the acceptance degree parameter meets a preset condition, the end frame data corresponding to the end frame identifier has not been completely sent, and the window for transmitting the media stream data is limited.
  • system further comprises:
  • the server is further configured to add a target identification field to the media loss packet based on the packet loss retransmission parameter; and transmit the media loss packet to the routing node according to the packet loss retransmission parameter;
  • the routing node is used to preferentially forward the media loss packet when the target identification field is a target value; wherein, when the target identification field is the target value, it indicates that the packet loss retransmission parameter meets the priority condition.
  • the present application further provides an application scenario, and the application scenario applies the above-mentioned flow data transmission method.
  • the application of the flow data transmission method in the application scenario is as follows:
  • the above-mentioned traffic data transmission method can be used, that is, in scenarios such as audio and video on demand, live broadcast services, and real-time communication, the user terminal sends a request message to the server, so that the server transmits the corresponding media stream data to the terminal in response to the request message.
  • the user terminal can query the cache time of the application that plays the media stream data, and encapsulate the cache time in a confirmation message to obtain a target confirmation message; further, the user terminal sends a target confirmation message to the server, so that when the server determines that packet loss occurs in the media stream data, the server can adaptively adjust the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, and before repairing the media stream data, the server transmits the media loss packet according to the packet loss retransmission parameters, and the user terminal receives the media loss packet transmitted by the server based on the packet loss retransmission parameters.
  • the adaptive packet loss repair method based on multi-party collaboration designed in the embodiment of the present application strikes a good balance between traffic sending and packet loss repair. While optimizing the user-side lag caused by packet loss, it reduces the impact on subsequent traffic transmission efficiency, which is beneficial to improving the user experience of existing Internet services based on audio and video, and enhancing the competitiveness of server manufacturers.
  • the method provided in the embodiment of the present application can be applied to improve audio and video on demand, live broadcast services, real-time communication and other scenarios.
  • the following takes the Internet service scenario mainly based on audio and video as an example to illustrate the traffic data transmission method provided in the embodiment of the present application.
  • TCP Transport Control Protocol, transmission control protocol.
  • QUIC Quick UDP Internet Connection, fast UDP Internet connection.
  • QoE Quality of Experience, (user) experience quality.
  • QoS Quality of Services, quality of service.
  • RTT Rround-Trip Time, round-trip time delay.
  • IP Internet Protocol
  • Network packet loss is an important factor affecting traffic transmission efficiency, and is also a key indicator that affects user service experience and evaluates the quality of cloud service providers.
  • a high packet loss rate will lead to a deterioration in traffic transmission performance, that is, the amount of data transmitted by the user terminal in the same period of time is relatively low, which may cause buffering, freezing, screen distortion, mosaics, black screens, and other features that seriously affect the user experience during the use of terminal applications. Therefore, improving the packet loss rate of network transmission and thus increasing the effective data throughput has become a key breakthrough point for optimizing the terminal-side service experience.
  • the main ways to reduce packet loss rate and improve data transmission efficiency are mainly concentrated in the following two aspects: First, research, design and deploy congestion control algorithms that are more suitable for network environment and service type, such as BBR, Reno, TIMELY, etc.
  • the core idea of these methods is to try to "accurately" identify whether the current network is congested. If congestion occurs, appropriately reduce the sending parameters (such as sending rate or sending window) in the hope that there will be no or as little packet loss as possible in the subsequent traffic transmission process.
  • packet second, from the perspective of protocol innovation, design the latest network transmission protocol, and improve the traffic transmission efficiency through the advantages of the protocol itself.
  • multi-path transmission (such as MPTCP) speeds up the flow completion time by adding a traffic transmission path, which will significantly improve the user-side service experience in most scenarios;
  • UDP-based QUIC protocol that improves end-to-end transmission performance by supporting 0-RTT and alleviating head-of-line blocking.
  • the above solution only looks at network congestion from the perspective of the data sender and adjusts the sending strategy, but fails to fully consider the service experience of the user terminal.
  • the service experience of the user terminal is the most critical factor in measuring a CDN manufacturer. In this case, if the packet loss rate is high during traffic transmission, the service experience of the user terminal is not necessarily very bad.
  • the key factor of user experience lies in whether the network packet loss is repaired in a certain period of time. If it is repaired in time, the audio and video player of the terminal will not be stuck.
  • FIG. 7 it is a schematic diagram of packet loss and user-side jamming.
  • reliable transmission protocols such as TCP, QUIC, etc.
  • the receiving end if a traffic message is lost during transmission, the receiving end "informs" the sending end to retransmit the lost message through a message confirmation message, and the receiving end is in a state of "always waiting" for the lost message until the lost message is received and the subsequent traffic messages are played for audio and video.
  • T_recovery N*(RTT+T_detect), where N is the number of repairs;
  • T_buffer can be represented by the time interval between the timestamp of the first available data in the queue and the timestamp of the last available data, where the above-mentioned "queue” includes the application layer player cache queue and the transport layer data receiving queue; the data's "timestamp” refers to the playback time point of the data in the player, generally expressed by the time interval from a certain (such as the first) audio or video frame.
  • the timestamp of the data can be understood as the playback time of the data in the player.
  • the timestamp of the first data in the player queue is time_0, indicating that the player will play the data immediately
  • the timestamp of the last data in the player queue is time_1, indicating that the data will be played at time_1, and the time interval between the above two data is time_1-time_0
  • the "queue” here includes at least one of the cache queue of the application layer player or the queue of the transport layer.
  • the former is used in the audio and video application to store the audio and video data to be played soon, and the latter is used in the mobile device to store the data received from the sender and pass the data to the audio and video application queue.
  • T_recovery T_buffer
  • T_recovery>T_buffer it means that the packet loss in the queue has not been repaired when the data is consumed, and the player on the terminal will freeze
  • the present application provides a solution, that is, the present application proposes an adaptive packet loss repair and traffic transmission method based on multi-party collaboration, which enables the user terminal to carry information such as cache time in the message confirmation message through end-cloud collaboration, so that the sending end (cloud server) can adaptively decide the packet loss retransmission strategy based on the relationship between the above cache time and the packet loss repair time.
  • the method proposed in the present application can be used to improve the user experience in scenarios such as audio and video on demand, live broadcast services, and real-time communications, and belongs to the field of computer network transmission optimization.
  • the user terminal when the user terminal feeds back a message confirmation message to the data sending end, it carries the cache length of the application buffer; when the data sending end detects packet loss, it can adaptively adjust the packet loss repair strategy according to the data cache length on the user terminal side; in this process, if the cache time is longer, the data sending end adopts a "mild" packet loss retransmission strategy; if the cache time is shorter, the data sending end adopts an "aggressive" packet loss retransmission strategy; if the cache time is moderate, the data sender will adaptively adjust the packet loss retransmission strategy based on the latest measured end-to-end delay, packet loss rate and other information.
  • the packet loss repair method involved in the embodiment of the present application has a good balance between traffic sending and packet loss repair. While optimizing the user-side freeze caused by packet loss, it reduces the impact on the subsequent traffic transmission efficiency, which is conducive to improving the user experience of existing Internet services mainly based on audio and video, and enhancing the competitiveness of CDN manufacturers.
  • the purpose of the method provided in this application is to solve the problem that the existing transmission protocol and congestion control algorithm did not consider the "linkage” with the user terminal service experience at the beginning of the design, which led to the data sender "blindly” adjusting the sending parameters and failing to truly ensure that the client service experience is in a good state.
  • An adaptive packet loss repair method based on end-cloud collaboration is proposed.
  • the user terminal carries the information of the current application layer (such as the player) data cache time when replying to the message confirmation message; after receiving the above cache time, the data sender (cloud server) adaptively adjusts the retransmission strategy of the previously lost message; specifically, when the cache time is long, the task of packet loss repair is not urgent, and the sender will retransmit the lost packet data once; when the cache time is short, the task of packet loss repair is more urgent, and the real-time performance of the lost packet data reaching the user terminal must be improved, otherwise the user terminal will experience a jamming phenomenon.
  • the sender will use a more aggressive method to repair the previously lost data, so as to ensure that there is enough data for the next stage of use (such as audio and video playback) before the above cache time is consumed.
  • FIG 8 it is a schematic diagram of the adaptive packet loss repair and traffic transmission method based on multi-party collaboration.
  • the data receiving end queries the cache information (such as cache time, number of cache messages, size) of the application corresponding to these traffic messages; 2 the data receiving end embeds the above cache information into the message confirmation message and sends the above message confirmation message to the data sending end; 3 after receiving the message confirmation message, the data sending end adjusts the packet loss repair and sending strategy according to the cache information of the above user terminal; 4 after receiving the packet loss repair message due to the relatively tight terminal cache information, the intermediate routing node increases the forwarding priority of the message, enhances the transmission efficiency of the retransmitted message, reduces the time the user terminal waits for the data packet loss, and optimizes QoE indicators such as audio and video freeze.
  • the cache information such as cache time, number of cache messages, size
  • the server in the embodiments of the present application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
  • the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this.
  • the terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
  • FIG. 9 it is a schematic diagram of the entire process of the adaptive packet loss repair and traffic transmission method based on multi-party collaboration provided by this application.
  • the idea of the method provided by this application is to enable the data sender to adaptively adjust the packet loss repair strategy according to the cache information of the receiving end, and to add the cache information of the user terminal in the message confirmation message, so that the server can timely understand whether the user terminal is (about to) experience a freeze (taking audio and video as an example), and adjust the aggressiveness of the packet loss repair based on this information; at the same time, during the data forwarding process, if the intermediate routing node encounters a packet loss repair message with a specific identifier used to avoid terminal freezes, the forwarding priority of the message is increased, the forwarding efficiency of such messages is improved, and the waiting time of the user terminal for a certain packet loss data is reduced, thereby optimizing the service experience.
  • the data receiving end receives the traffic message from the data sending end (cloud server) and periodically sends a message confirmation message to the server, where the message carries the data cache information of the user terminal, as follows:
  • the reception level of the audio and video frame, frame_last_rcv specifically includes the number of packets in the audio and video frame, frame_last_pkt_amount, and the number of packets currently received, frame_last_pkt_rcv_amount;
  • info_l7_buffer ⁇ l7_buffer_time_len,l7_buffer_frame_amount, frame_last ⁇ (2)
  • a monitoring module is added to the application layer to periodically obtain the cache information info_l7_buffer of the application receiving queue and feed it back to the transport layer, as shown in the user terminal monitoring schematic diagram in FIG4 .
  • the time interval T_gain_17 for the user terminal to obtain info_17_buffer can be configured by the administrator and reflected in the configuration file.
  • T_gain_17 5ms
  • the monitoring module of the user terminal is located between the application layer and the transport layer, and its main goal is to obtain the info_l7_buffer of different applications in a timely manner;
  • the monitoring module maintains a status table as shown in FIG. 3 for the currently used application, and periodically updates the info_l7_buffer of different applications in the table:
  • the user terminal When an application is closed, the user terminal deletes the corresponding application status entry from the above status information table; when an application is opened, the user terminal adds the corresponding application status entry to the above status information table;
  • the user terminal before sending a message confirmation message pkt_ack to the server, the user terminal obtains the info_l7_buffer of the corresponding application through the monitoring module, and embeds the info_l7_buffer into the pkt_ack to obtain the final message confirmation message pkt_ack_l7_buffer;
  • the user terminal decides the pkt_ack_l7_buffer feedback strategy based on the status of its own info_l7_buffer, as follows:
  • the user terminal adopts an "aggressive" redundant return strategy when returning pkt_ack_l7_buffer, that is, returning the same M copies of pkt_ack_l7_buffer to prevent the loss of pkt_ack_l7_buffer during the return process;
  • step 2) If the monitoring module detects that the l7_buffer_time_len of a certain application is small, and the judgment condition is as shown in step 1), the transport layer will be triggered to reply pkt_ack_l7_buffer immediately, without waiting for the pkt_ack to be generated in the existing method before making a judgment, thus avoiding the phenomenon of delayed data transmission due to slow pkt_ack reply;
  • the user terminal After receiving the traffic, the user terminal does not reply with a pkt_ack for each traffic message, but “aggregates” and “packs” together to reply whether certain messages have been received.
  • the above step 2) It can effectively avoid the problem of untimely reply to message confirmation messages;
  • the pkt_ack message will trigger the acquisition of the latest info_l7_buffer and form pkt_ack_l7_buffer; it can also be: the l7_buffer_time_len in the latest info_l7_buffer will also trigger pkt_ack to generate pkt_ack_l7_buffer;
  • the data sender (cloud server) adaptively decides the packet loss retransmission strategy and traffic sending strategy based on the info_l7_buffer information, as follows:
  • the data sender Before retransmitting the lost message pkt_loss, the data sender adds a new identification field loss_l7_buffer in the message, where the field occupies 1 bit. If the value is binary 1, it means that the retransmitted packet loss is used to alleviate the problem that the terminal application cache time is too short and may cause jamming;
  • the data sender needs to overcome the limited network resources and send as many audio and video frames as possible; among which, l7_buffer_frame_amount_threshold can be configured by the administrator and declared in the configuration file; it can be understood that if the data cached in the player queue in L7 is short, it means that the player can quickly play the remaining buffered data; in particular, if the player does not "inform" the sender in time when it has played all the data in its cache, it may be stuck; in fact, the message confirmation message fed back by the user terminal may be delayed (such as replying an ACK message for every 5 messages received), and the returned message may be lost.
  • l7_buffer_frame_amount_threshold can be configured by the administrator and declared in the configuration file; it can be understood that if the data cached in the player queue in L7 is short, it means that the player can quickly play the remaining buffered data; in particular, if the player does not "inform" the sender in time when it has played all the
  • the data sending end selects the audio and video frames with a lower bit rate as the data source and sends them to the data receiving end, see step b) for details;
  • the data transmitter records the audio and video frame bitrate bitrate_(i-1) sent in the past period of time. When the conditions in step a) are met, the data transmitter will select the latest audio and video bitrate bitrate_i, where bitrate_i ⁇ bitrate_(i-1);
  • FIG. 10 it is a schematic diagram of the data transmitter adjusting the bit rate.
  • the data transmitter initially sends high-bitrate audio and video content to the user terminal. After a period of time, if the following conditions are met: l7_buffer_time_len ⁇ Time_threshold or l7_buffer_frame_amount ⁇ l7_buffer_frame_amount_threshold, the data transmitter switches the audio and video content from high bit rate to medium bit rate; after a period of time, if l7_buffer_time_len ⁇ Time_threshold or l7_buffer_frame_amount ⁇ l7_buffer_frame_amount_threshold, the data transmitter Then it switches from medium bit rate to low bit rate; after a period of time, if l7_buffer_time_len>Time_threshold and l7_buffer_frame_amount>l7_buffer_frame_amount_thres
  • the intermediate network node After receiving the traffic message, the intermediate network node checks whether the message carries loss_l7_buffer with a value of binary 1, as follows:
  • the packet loss repair method involved in the present invention has a good balance between traffic sending and packet loss repair. While optimizing the user-side lag caused by packet loss, it reduces the impact on subsequent traffic transmission efficiency, which is beneficial to improving the user experience of existing Internet services mainly based on audio and video, and enhancing the competitiveness of CDN manufacturers.
  • the embodiment of the present application also provides a flow data transmission device for implementing the flow data transmission method involved above.
  • the implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more flow data transmission device embodiments provided below can refer to the limitations of the flow data transmission method above, and will not be repeated here.
  • the encapsulation module 1104 is used to encapsulate the cache time into a confirmation message to obtain a target confirmation message.
  • the sending module 1106 is used to send the target confirmation message to the server, so that when the server determines that packet loss occurs in the media stream data, it adapts the packet loss retransmission parameters according to the packet loss repair time and the cache time in the target confirmation message, and transmits the media loss packet according to the packet loss retransmission parameters before repairing the media stream data.
  • the receiving module 1108 is configured to receive the media loss packet transmitted by the server based on the packet loss retransmission parameter.
  • the media stream data is cached in an application receiving queue corresponding to the application; the device also includes: a determination module and a storage module, the determination module is used to determine the cache information based on the media stream data in the application receiving queue; the storage module is used to store the cache information in a status information table corresponding to the application; the query module is also used to query the cache time of the media stream data from the status information table.
  • the device also includes: a deletion module and an addition module, the deletion module is used to delete the application status bar of the application from the status information table when the application is switched to a closed state; the addition module is used to add an application status bar corresponding to the application identifier of the other application to the status information table when the other application is switched to an open state.
  • the encapsulation module is further configured to encapsulate the updated cache time in a confirmation message when it is detected that the cache time in the status information table is updated, so as to obtain a target confirmation message.
  • the device further comprises: a generating module, configured to generate the confirmation message when the media stream data is received; and a querying module, configured to query the cache time of the media stream data when the confirmation message is detected.
  • the device also includes: an acquisition module, used to obtain the starting frame data and the ending frame data of the media stream data in the application receiving queue; the determination module is also used to determine the media cache data based on the starting frame data and the ending frame data; and determine the cache time based on the time interval between the starting frame data and the ending frame data.
  • the acquisition module is also used to obtain a first timestamp of the start frame data and a second timestamp of the end frame data; the first timestamp is the playback time point of the start frame data in the player of the application, and the second timestamp is the playback time point of the end data in the player; the determination module is also used to determine the cache time based on the difference between the second timestamp and the first timestamp.
  • the device also includes: a transmission module, which is used to transmit the media loss packet to the routing node according to the packet loss retransmission parameter before repairing the media stream data, and when the packet loss retransmission parameter meets the priority condition, instruct the routing node to forward the media loss packet according to the priority condition.
  • a transmission module which is used to transmit the media loss packet to the routing node according to the packet loss retransmission parameter before repairing the media stream data, and when the packet loss retransmission parameter meets the priority condition, instruct the routing node to forward the media loss packet according to the priority condition.
  • the transmission module is also used to transmit a preset number of the target confirmation messages back to the server when the cache time is less than the packet loss repair time; the packet loss repair time is determined based on the delay information of the media stream data during the transmission process.
  • another flow data transmission device including: a receiving module 1202, an adaptive module 1204 and a transmission module 1206, wherein:
  • the receiving module 1202 is configured to receive a target confirmation message returned by a terminal during the process of transmitting media stream data to the terminal; the target confirmation message carries a cache time of the terminal for the media stream data.
  • the adaptive module 1204 is configured to adaptively adjust packet loss retransmission parameters according to packet loss repair time and the buffer time in the target confirmation message when it is determined based on the target confirmation message that packet loss occurs in the media stream data.
  • the transmission module 1206 is configured to transmit the media loss packet corresponding to the media stream data according to the packet loss retransmission parameter, so that the terminal receives the media loss packet before repairing the media stream data.
  • the adaptive module is also used to adapt the transmission bit rate according to the packet loss repair time and the cache time in the target confirmation message when it is determined based on the target confirmation message that the media stream data has not been lost; or, to adapt the transmission bit rate according to a preset number of frames and the number of cached frames in the target confirmation message; the transmission module is also used to transmit the media stream data to the terminal based on the transmission bit rate.
  • the device also includes: an adjustment module, which is used to reduce the original bit rate value of the media stream data to obtain the transmission bit rate when the cache time in the target confirmation message is less than the packet loss repair time; when the number of cached frames in the target confirmation message is less than the preset number of frames, reduce the original bit rate value of the media stream data to obtain the transmission bit rate.
  • an adjustment module which is used to reduce the original bit rate value of the media stream data to obtain the transmission bit rate when the cache time in the target confirmation message is less than the packet loss repair time; when the number of cached frames in the target confirmation message is less than the preset number of frames, reduce the original bit rate value of the media stream data to obtain the transmission bit rate.
  • the transmission bit rate is a first bit rate value
  • the device also includes: a selection module, which is used to select a second bit rate value as the transmission bit rate when the cache time in the target confirmation message is greater than the packet loss repair time and the number of cached frames in the target confirmation message is greater than the preset number of frames; wherein the second bit rate value is greater than the first bit rate value.
  • the target confirmation message carries the end frame identifier and reception degree parameter of the media stream data; the transmission module is also used to continue transmitting the end frame data to the terminal when the reception degree parameter meets a preset condition, the end frame data corresponding to the end frame identifier has not been sent completely, and the window for transmitting the media stream data is limited.
  • the transmission module is further used to transmit the media loss packet to the routing node according to the packet loss retransmission parameter, and when the packet loss retransmission parameter meets the priority condition, instruct the routing node to forward the media loss packet according to the priority condition.
  • the device also includes: a new adding module, used to add a target identification field in the media loss packet based on the packet loss retransmission parameter; the transmission module is also used to transmit the media loss packet to the routing node according to the packet loss retransmission parameter, and when the target identification field is a target value, instruct the routing node to preferentially forward the media loss packet; wherein, when the target identification field is the target value, it indicates that the packet loss retransmission parameter meets the priority condition.
  • a new adding module used to add a target identification field in the media loss packet based on the packet loss retransmission parameter
  • the transmission module is also used to transmit the media loss packet to the routing node according to the packet loss retransmission parameter, and when the target identification field is a target value, instruct the routing node to preferentially forward the media loss packet; wherein, when the target identification field is the target value, it indicates that the packet loss retransmission parameter meets the priority condition.
  • Each module in the above-mentioned flow data transmission device can be implemented in whole or in part by software, hardware and a combination thereof.
  • Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.
  • a computer device is provided.
  • the computer device may be a terminal or a server.
  • the computer device is a terminal as an example for explanation.
  • Its internal structure diagram may be as shown in FIG13.
  • the computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device.
  • the processor, The memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and a computer program.
  • the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium.
  • the input/output interface of the computer device is used to exchange information between the processor and the external device.
  • the communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies.
  • the display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device, the display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.
  • FIG. 13 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied.
  • the specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
  • a computer device including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
  • a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
  • a computer program product including a computer program, which implements the steps in the above method embodiments when executed by a processor.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory.
  • Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc.
  • Volatile memory can include random access memory (RAM) or external cache memory, etc.
  • RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • the database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database.
  • Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this.
  • the processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本申请涉及一种流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品。该方法可以应用于对车载终端、云服务器、区块链节点或其它设备进行流量数据传输的应用场景,包括:在接收服务器发送媒体流数据的过程中,查询媒体流数据的缓存时间(202);将缓存时间封装于确认报文,得到目标确认报文(204);向服务器发送目标确认报文,以使服务器在确定媒体流数据出现丢包时,依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复媒体流数据之前依据丢包重传参数传输媒体丢失包(206);接收服务器基于丢包重传参数传输的媒体丢失包(208)。

Description

流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品
本申请要求于2023年06月08日提交中国专利局,申请号为2023106768269,发明名称为“流量数据传输方法、装置、计算机设备、存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域,特别是涉及一种流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品。
背景技术
随着计算机技术以及互联网技术的发展,用户侧的体验质量(Quality of Experience,QoE)是衡量不同云服务提供商的服务质量的重要依据,有效提升用户的体验质量是当前各大云平台所追求的目标。
目前的流量数据传输方式中,主要途径集中在以下两个方面:一是研究设计并部署更适配网络环境与业务类型的拥塞控制算法,如BBR等,这些方法的核心思想在于试图“准确地”识别当前网络是否拥塞,如果出现拥塞,则针对拥塞问题及时进行调整,以期望在后续的流量传输过程中不再出现或尽可能少地出现丢包;二是从协议创新的角度出发,设计最新的网络传输协议,通过协议本身自带的优势提升流量传输效率,例如多路径传输通过增加一条流量传输路径,加快流完成时间,在多数场景下会显著改善用户侧的业务体验;上述解决方案都是从网络拥塞的角度来调整发送策略,以达到降低丢包率提升数据传输效率的目的,未能充分考虑到用户终端侧的业务体验,即在实际情况中,有时候即使流量传输过程中丢包率偏高,但用户终端的业务体验不一定很差,影响用户终端侧的业务体验的关键因素在于出现的网络丢包是否在一定时间内得到及时修复,如果未能得到及时修复,用户终端侧的业务体验比如音视频播放就会出现卡顿、黑屏等严重影响用户体验的特征,因此,如何在保证不影响用户终端侧的业务体验的同时又能快速有效的修复网络丢包成为亟需解决的问题。
发明内容
根据本申请的各种实施例,提供了一种流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品。
第一方面,本申请提供了一种流量数据传输方法,所述方法由终端执行。所述方法包括:在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;将所述缓存时间封装于确认报文,得到目标确认报文;向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包;接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
第二方面,本申请还提供了一种流量数据传输装置。所述装置包括:查询模块,用于在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;封装模块,用于将所述缓存时间封装于确认报文,得到目标确认报文;发送模块,用于向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述 丢包重传参数传输媒体丢失包;接收模块,用于接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
第三方面,本申请还提供了一种计算机设备。所述计算机设备包括存储器和处理器,所述存储器存储有计算机程序,所述处理器执行所述计算机程序时实现以下步骤:在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;将所述缓存时间封装于确认报文,得到目标确认报文;向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包;接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
第四方面,本申请还提供了一种计算机可读存储介质。所述计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现以下步骤:在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;将所述缓存时间封装于确认报文,得到目标确认报文;向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包;接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
第五方面,本申请还提供了一种计算机程序产品。所述计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现以下步骤:在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;将所述缓存时间封装于确认报文,得到目标确认报文;向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包;接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
第六方面,本申请提供了一种流量数据传输方法,所述方法由服务器执行。所述方法包括:在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
第七方面,本申请还提供了一种流量数据传输装置。所述装置包括:接收模块,用于在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;自适应模块,用于当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;传输模块,用于根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
第八方面,本申请还提供了一种计算机设备。所述计算机设备包括存储器和处理器,所述存储器存储有计算机程序,所述处理器执行所述计算机程序时实现以下步骤:在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;根据所述 丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
第九方面,本申请还提供了一种计算机可读存储介质。所述计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现以下步骤:在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
第十方面,本申请还提供了一种计算机程序产品。所述计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现以下步骤:在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
图1为一个实施例中流量数据传输方法的应用环境图;
图2a为一个实施例中流量数据传输方法的流程示意图;
图2b为另一个实施例中流量数据传输方法的流程示意图;
图3为一个实施例中用户终端应用状态信息表的示意图;
图4为一个实施例中用户终端周期性的获取应用接收队列的缓存信息的示意图;
图5为一个实施例中基于应用接收队列中的媒体流数据确定缓存信息步骤的流程示意图;
图6为另一个实施例中流量数据传输方法的流程示意图;
图7为一个实施例中丢包与用户侧卡顿的示意图;
图8为一个实施例中基于多方协同的自适应丢包修复与流量传输方法的流程示意图;
图9为一个实施例中基于多方协同的自适应丢包修复与流量传输方法的整体流程示意图;
图10为一个实施例中数据发送端调整码率的示意图;
图11为一个实施例中流量数据传输装置的结构框图;
图12为另一个实施例中流量数据传输装置的结构框图;
图13为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。
需要说明的是,在以下的描述中,所涉及的术语“第一、第二和第三”仅仅是区别类似的对象,不代表针对对象的特定排序,可以理解地,“第一、第二和第三”在允许的情况下可以互换特定的顺序或先后次序,以使这里描述的本申请实施例能够以除了在这里图示或描 述的以外的顺序实施。
本申请实施例提供的流量数据传输方法,可以应用于如图1所示的应用环境中。其中,终端102通过网络与服务器104进行通信。数据存储系统可以存储服务器104需要处理的数据。数据存储系统可以集成在服务器104上,也可以放在云上或其他网络服务器上。终端102在接收服务器104发送媒体流数据的过程中,终端102查询媒体流数据的缓存时间,并将缓存时间封装于确认报文,得到目标确认报文;终端102向服务器104发送目标确认报文,以使服务器104在确定媒体流数据出现丢包时,服务器104依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复媒体流数据之前依据丢包重传参数传输媒体丢失包;终端102接收服务器104基于丢包重传参数传输的媒体丢失包。
其中,终端102可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、物联网设备和便携式可穿戴设备,物联网设备可为智能音箱、智能电视、智能空调和智能车载设备等。便携式可穿戴设备可为智能手表、智能手环、头戴设备等。
服务器104可以是独立的物理服务器,也可以是区块链系统中的服务节点,该区块链系统中的各服务节点之间形成点对点(P2P,Peer To Peer)网络,P2P协议是一个运行在传输控制协议(TCP,Transmission Control Protocol)协议之上的应用层协议。
此外,服务器104还可以是多个物理服务器构成的服务器集群,可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(CDN,Content Delivery NetworkCDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。
终端102与服务器104之间可以通过蓝牙、USB(Universal Serial Bus,通用串行总线)或者网络等通讯连接方式进行连接,本申请在此不做限制。
在一个实施例中,如图2a和图2b所示,提供了一种流量数据传输方法,该方法可以由服务器或终端单独执行,也可以由服务器和终端共同执行,以该方法应用于图1中的终端为例进行说明,包括以下步骤:
步骤202,在接收服务器发送媒体流数据的过程中,查询媒体流数据的缓存时间。
其中,流数据是指一组顺序、大量、快速、连续到达的数据序列,该数据序列可以是由多个数据包组成的序列。一般情况下,流数据可被视为一个随时间延续而无限增长的动态数据集合。本申请中的媒体流数据即流媒体数据,也可以称为媒体数据。可以理解,本申请中的媒体流数据可以是由多个媒体数据包所组成的媒体数据序列,包括多种类型的数据,例如,媒体流数据可以包括视频数据、音频数据、图像数据、应用程序安装数据等中的至少一种。本实施例中的视频数据又可以包括点播视频、直播视频等中的至少一种。比如,本实施例中媒体流数据可以为点对点的流媒体数据或视频通话数据等中的至少一种。
缓存时间是指缓存于应用层中的媒体流数据的缓存时长,本申请中的缓存时间可以用缓存时刻来表示,也可以用缓存时长来表示。比如,当终端查询到的缓存时间为缓存时刻时,终端可以通过不同的缓存时刻之间的差值确定缓存时长。
具体地,在音视频点播、直播业务、实时通信等场景下,比如,当用户启动终端中的应用程序进行视频通话、视频直播时,即在不同的媒体流数据的使用场景下,在终端接收服务器发送媒体流数据的过程中,终端可以查询该媒体流数据在应用层中的缓存时间,比如,用户可以通过触发操作播放应用程序中的某个音视频,则终端响应于用户的上述触发操作,向服务器发送流量请求报文,以使服务器向终端传输所请求的媒体流数据,即服务器可以向终 端传输携带所请求的媒体流数据的流量报文。
举个例子,以媒体流数据为音视频数据为例进行说明。当用户启动终端A中的应用程序A进行音视频点播时,用户可以通过选取操作选择应用程序A中音视频A,则终端A响应于用户的上述选取操作,向服务器发送流量请求报文,以使服务器向终端A传输所请求的音视频A对应的媒体流数据,即服务器可以向终端A传输携带所请求的音视频A对应的媒体流数据的流量报文。
进一步的,在终端A接收服务器发送的音视频A对应的媒体流数据的过程中,终端A查询本地应用层中待播放的音视频A对应的媒体流数据的缓存时间,比如,终端A可以从自身维护的状态信息表中,查询播放音视频A的应用程序对应的媒体流数据的缓存时长。其中,该状态信息表用于存储终端A中运行的不同应用程序的缓存信息info_l7_buffer。
可以理解,本申请中不同应用程序的缓存信息info_l7_buffer,包括但不限于是缓存时间,还可以包括其他的缓存信息,比如,缓存信息中还可以包括应用程序对应的应用接收队列中完整的音视频帧数量l7_buffer_frame_amount、应用缓存队列中最后一个音视频帧的信息frame_last等。
步骤204,将缓存时间封装于确认报文,得到目标确认报文。
其中,确认报文是指数据接收端(终端)收到来自数据发送端(服务器)的流量报文,并周期性地向服务器发送的消息确认报文,该消息确认报文即为确认报文。
目标确认报文是指携带了缓存时间的消息确认报文,即本申请中的终端在向服务器发送消息确认报文pkt_ack之前,终端可以先从自身维护的状态信息表中查询该媒体流数据的缓存时间,并将查询到的缓存时间封装于消息确认报文pkt_ack中,即可得到携带了缓存时间的目标确认报文pkt_ack_l7_buffer。
具体地,在终端接收服务器发送媒体流数据的过程中,终端查询到该媒体流数据的缓存时间之后,终端可以将查询到的该缓存时间封装于确认报文中,以得到携带了缓存时间的目标确认报文。
举个例子,以媒体流数据为音视频数据为例进行说明。在终端A接收服务器发送的音视频A对应的媒体流数据的过程中,假设终端A查询到本地应用层中待播放的音视频A对应的媒体流数据的缓存信息为info_l7_buffer(l7_buffer_time_len),即该缓存信息info_l7_buffer中包括缓存时长l7_buffer_time_len,则终端A可以将查询到的缓存信息info_l7_buffer(l7_buffer_time_len)封装于消息确认报文pkt_ack中,即可得到携带了缓存时间l7_buffer_time_len的目标确认报文pkt_ack_l7_buffer。
在一个实施例中,终端在接收服务器发送媒体流数据的过程中,根据接收到的媒体数据包判断是否存在丢包情况,若存在丢包情况,则确定媒体丢失包的标识(如序号),并将该媒体丢失包的标识与缓存时间一并封装于确认报文中,以使服务器根据该媒体丢失包的标识确定媒体丢失包。
例如,服务器发送了序号为1至10的媒体数据包,终端接收到序号为1至4的媒体数据包和序号为6至10的媒体数据包,则可以确定序号为5的媒体数据包丢失,此时可以将丢失的媒体数据包的序号与缓存时间一并封装于确认报文中,然后发送封装所得的目标确认报文至服务器,从而服务器可以根据该序号确定所丢失的媒体数据包。
步骤206,向服务器发送目标确认报文,以使服务器在确定媒体流数据出现丢包时,依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复媒体流数据之 前依据丢包重传参数传输媒体丢失包。
其中,丢包(Packet loss)是指一个或多个数据数据包(packet)的数据无法透过网上到达目的地,比如,本申请中在服务器向终端发送媒体流数据的过程中,可能会出现部分数据包并没有到达终端的现象。
丢包修复时间是指预设的丢包修复的时间阈值,比如,本申请中的丢包修复时间可以设置为Time_threshold,Time_threshold可以设置为固定数值Time_threshold_value,如Time_threshold_value=200ms;或者,Time_threshold也可设置为时延信息的倍数关系,如Time_threshold=N*SRTT;其中,SRTT表示平滑往返时间,N表示预设数量,N是由管理员进行配置的,比如N=2。
丢包重传参数是指用于反映重传策略的参数,比如,本申请中的丢包重传参数可以包括不同重传策略所对应的丢包重传参数,比如,激进重传策略对应的丢包重传参数为第一丢包重传参数;冗余回传策略对应的丢包重传参数为第二丢包重传参数。
修复媒体流数据是指修复传输媒体流数据过程中的丢包数据,在一些情况下,如果流量数据传输过程中丢包率偏高,终端侧的业务体验不一定很差,而终端侧的用户体验的关键因素在于:出现的网络丢包是否在一定时间内得到及时修复,如果得到及时修复,终端侧在播放音视频数据时也不会出现卡顿等现象。
媒体丢失包是指在传输媒体流数据过程中丢失的媒体数据包。例如,服务器发送了序号为1至10的媒体数据包,终端接收到序号为1至4的媒体数据包和序号为6至10的媒体数据包,则序号为5的媒体数据包为媒体丢失包。
步骤208,接收服务器基于丢包重传参数传输的媒体丢失包。
具体地,终端将查询到的缓存时间封装于确认报文,得到目标确认报文之后,终端可以向服务器发送携带了缓存时间的目标确认报文,以使服务器在确定媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复媒体流数据之前依据丢包重传参数传输媒体丢失包;进一步的,终端可以接收服务器基于包重传参数传输的媒体丢失包。
举个例子,以媒体流数据为音视频数据为例进行说明。在终端A接收服务器发送的音视频A对应的媒体流数据的过程中,假设终端A将查询到的缓存信息info_l7_buffer(l7_buffer_time_len)封装于消息确认报文pkt_ack中,得到携带了缓存时间的目标确认报文pkt_ack_l7_buffer之后,终端A可以向服务器发送该目标确认报文pkt_ack_l7_buffer,当服务器收到终端A发送的上述目标确认报文pkt_ack_l7_buffer之后,服务器可以判断传输该媒体流数据时是否出现丢包,当服务器确定传输该媒体流数据出现丢包时,服务器可以依据预设的丢包修复时间和目标确认报文pkt_ack_l7_buffer中的缓存时间l7_buffer_time_len自适应地调整丢失报文的重传策略,即可得到与重传策略对应的丢包重传参数;进一步的,服务器可以依据所得到的丢包重传参数重新传输媒体丢失包,以使终端在修复该媒体流数据之前接收到媒体丢失包,并根据媒体丢失包修复媒体流数据(即修复丢失的数据),以尽可能地保证在缓存时间消耗完之前有足够的媒体流数据来用于下一阶段的使用(如音视频播放)。
传统方式中,在可靠传输协议中,以音视频数据传输为例,若流量报文在传输过程中出现丢失,则接收端即终端需要通过消息确认报文“告知”发送端即服务器重传丢失的报文,且接收端即终端处于“一直等待”该丢失的报文的状态,直至接收端即终端接收到重传的丢失的报文后将后续的流量报文进行音视频播放。
而本实施例中,通过在接收服务器发送媒体流数据的过程中,查询媒体流数据的缓存时间;将缓存时间封装于确认报文,得到目标确认报文;向服务器发送目标确认报文,以使服务器在确定媒体流数据出现丢包时,依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复媒体流数据之前依据丢包重传参数传输媒体丢失包;接收服务器基于丢包重传参数传输的媒体丢失包。由于目标确认报文是将缓存时间封装于确认报文中得到的,故服务器在确定媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,即服务器在收到上述缓存时间后,能够自适应地调整丢失报文的重传策略所对应的丢包重传参数,相较于传统方式中无差别的丢包重传策略,本实施例中涉及的丢包修复方法能够很好地权衡流量发送与丢包修复的关系,在优化因丢包造成用户侧卡顿的同时,减少了对后续流量传输效率的影响,能够在保证不影响用户终端侧的业务体验的同时又能快速有效的修复网络丢包,有利于改善以音视频为主的互联网业务的用户侧的体验质量。
在一个实施例中,媒体流数据缓存于应用程序对应的应用接收队列中;查询媒体流数据的缓存时间之前,所述方法还包括:
基于应用接收队列中的媒体流数据确定缓存信息;将缓存信息存储于应用程序对应的状态信息表;所述查询所述媒体流数据的缓存时间,包括:从状态信息表查询媒体流数据的缓存时间。
其中,应用程序是指使用传输的媒体流数据的目标应用程序,比如,终端在接收到服务器发送媒体流数据之后,终端通过应用程序A播放接收到的媒体流数据,则应用程序A为目标应用程序。
应用接收队列是指应用程序对应的应用接收队列,在某些情况下也可以称为应用缓存队列。本申请中每个应用程序对应一个应用接收队列,比如,应用程序A对应的应用接收队列为Q1。
缓存信息是指应用程序对应的应用接收队列的缓存信息,比如,本申请中的缓存信息可以包括:
a)该应用缓存队列中数据的缓存时长l7_buffer_time_len;
b)该应用缓存队列中完整的音视频帧数量l7_buffer_frame_amount;
c)该应用缓存队列中最后一个音视频帧的信息frame_last,具体包括:
该音视频帧的标识frame_last_id;
该音视频帧的接收程度frame_last_rcv,具体包含该音视频帧共有的报文数量frame_last_pkt_amount,以及当前已经收到的报文数量frame_last_pkt_rcv_amount;因此,该应用缓存队列中最后一个音视频帧的信息frame_last可用如下公式(1)表示:
frame_last = {frame_last_id, frame_last_rcv}   (1)
缓存信息info_l7_buffer可用如下公式(2)表示:
info_l7_buffer = {l7_buffer_time_len,l7_buffer_frame_amount, frame_last}   (2)
状态信息表是指用于存储终端中运行的不同应用程序的缓存信息的表,本申请中的状态信息表中可以包括:应用程序的应用标识、应用程序对应的缓存信息以及更新时间。其中,更新时间是指应用程序对应的缓存信息的更新时间。比如,如图3所示,为用户终端应用状态信息表的示意图。图3中可以看出终端中当前时刻运行的应用标识1对应的应用程序的缓存信息为Info_l7_buffer_1,该缓存信息Info_l7_buffer_1对应的更新时间为Time_1。
具体地,在终端接收服务器发送媒体流数据的过程中,终端可以将已接收到的媒体流数据缓存于应用程序对应的应用接收队列中,在终端查询传输的该媒体流数据的缓存时间之前,终端可以通过监测模块周期性的获取各个应用接收队列的缓存信息info_l7_buffer,并将获取到的缓存信息info_l7_buffer反馈至传输层,同时,终端可以通过监测模块维护如图3中所示的状态表,并周期性地更新该表中不同应用程序的缓存信息info_l7_buffer,终端可以通过监测模块对各个应用程序对应的应用接收队列中的媒体流数据进行监测,即终端中的监测模块可以基于各个应用程序对应的应用接收队列中的媒体流数据确定各个应用程序的缓存信息info_l7_buffer,并将各个应用程序的缓存信息info_l7_buffer周期性的更新于自身维护的如图3所示的状态信息表中。
进一步的,在终端接收服务器发送媒体流数据的过程中,终端可以按照预设策略或者预设协议周期性的从如图3中所示的状态信息表中查询播放该媒体流数据的应用程序的缓存时间。比如,终端周期性的获取各个应用程序的缓存信息info_l7_buffer的时间间隔T_gain_l7可由管理员预先进行配置,并在配置文件中体现,在默认情况下,时间间隔可以设置为T_gain_l7=5ms。
举个例子,以媒体流数据为音视频数据为例进行说明。如图4所示,为用户终端周期性的获取应用接收队列的缓存信息的示意图。图4中的监测模块位于应用层与传输层之间,其主要目标在于及时获取不同应用程序的缓存信息info_l7_buffer,即在本申请提供的方法中,为了获取各个应用程序对应的应用接收队列中缓存信息info_l7_buffer,可以在应用层中新增监测模块,也可以在应用层与传输层之间新增监测模块。在终端A接收服务器发送的音视频A对应的媒体流数据的过程中,假设终端A将已接收的音视频A对应的媒体流数据缓存于应用程序1对应的应用接收队列Q1中,则终端A可以基于该应用接收队列Q1中待播放的音视频A对应的媒体流数据确定缓存信息为Info_l7_buffer_1,其中,该缓存信息Info_l7_buffer_1中包括缓存时长l7_buffer_time_len1。
进一步的,终端A可以将缓存信息Info_l7_buffer_1存储于如图3所示的状态信息表中,即终端A可以将缓存信息Info_l7_buffer_1更新于如图3中所示的应用程序1对应的应用标识1的条目中,以使当终端A需要向服务器发送目标确认报文时,终端A可以从如图3所示的状态信息表中查询到音视频A对应的媒体流数据的缓存时间为l7_buffer_time_len1,并将查询到的该缓存时间l7_buffer_time_len1封装于确认报文pkt_ack中,即可得到目标确认报文pkt_ack_l7_buffer。由此使得,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并依此信息调整丢包修复的激进程度,有效减少了用户终端侧针对某丢包数据的等待时间,进而优化了用户终端侧的业务体验。
在其中一个实施例中,所述方法还包括:当应用程序切换到关闭状态时,从状态信息表中删除应用程序的应用状态条;当其它应用程序切换到开启状态时,在状态信息表中增加其它应用程序的应用标识对应的应用状态条。
其中,关闭状态是指应用程序停止运行的状态,比如,用户通过触发操作将应用程序A关闭,则终端响应于用户的上述触发操作,将应用程序A从开启状态切换到关闭状态。
应用状态条是指状态信息表中的应用状态条目,比如,可以将状态信息表中的每一行作为应用程序的应用状态条,如图3中所示,应用程序的应用状态条中包含的信息包括应用程序的应用标识、缓存信息即Info_l7_buffer和更新时间。
其它应用程序是指除了当前使用传输的媒体流数据的应用程序之外的应用程序,即本申请中的其它应用程序和应用程序用于区分不同的应用程序,比如,终端中的应用程序A为播放该媒体流数据的应用程序,则其它应用程序可以是该终端中除了应用程序A之外的其他应用程序,比如,其他应用程序可以为该终端中的应用程序B和应用程序C。
开启状态是指应用程序开启运行的状态,比如,用户通过触发操作将应用程序A打开,则终端响应于用户的上述触发操作,将应用程序A从关闭状态切换到开启状态。
应用标识是指用于唯一标识应用程序的标识,比如,其他应用程序可以为该终端中的应用程序B和应用程序C,其他应用程序B的应用标识为B,其他应用程序C的应用标识为C。
具体地,终端可以通过监测模块维护如图3中所示的状态表,并周期性地更新该表中不同应用程序的缓存信息info_l7_buffer,即终端可以通过监测模块对各个应用程序对应的应用接收队列中的媒体流数据进行监测,并基于各个应用程序对应的应用接收队列中的媒体流数据确定各个应用程序的缓存信息info_l7_buffer,终端将各个应用程序的缓存信息info_l7_buffer周期性的更新于自身维护的如图3中所示的状态信息表中之后,当终端检测到目标应用程序切换到关闭状态时,终端可以从如图3所示的状态信息表中删除该目标应用程序的应用状态条;或者,当终端检测到其它应用程序切换到开启状态时,终端可以在如图3所示的状态信息表中增加其它应用程序的应用标识对应的应用状态条。
举个例子,假设终端中的应用程序1为播放传输的媒体流数据1的应用程序,则其它应用程序可以是该终端中除了应用程序1之外的其他应用程序,比如,其他应用程序可以为该终端中的应用程序2和应用程序3。终端将各个应用程序的缓存信息info_l7_buffer周期性的更新于自身维护的如图3中所示的状态信息表中之后,当终端检测到应用程序1切换到关闭状态时,终端可以从如图3所示的状态信息表中删除应用程序1的应用状态条,即终端可以从如图3中所示的状态信息表中删除应用程序1的应用标识1对应的这一行的应用状态条;当终端检测到其它应用程序3切换到开启状态时,终端可以在如图3所示的状态信息表中增加其它应用程序3的应用标识3对应的应用状态条,比如,终端可以从如图3所示的状态信息表中新增应用程序3的应用标识3对应的这一行的应用状态条。由此使得,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并依此信息调整丢包修复的激进程度,有效减少了用户终端侧针对某丢包数据的等待时间,进而优化了用户终端侧的业务体验。
在一个实施例中,将缓存时间封装于确认报文,得到目标确认报文的步骤,包括:当检测到状态信息表中的缓存时间发生更新时,将更新的缓存时间封装于确认报文,得到目标确认报文。
其中,更新的缓存时间是指更新后的缓存时间,比如,在T1时刻,状态信息表中应用程序1的缓存时间为S1,在T2时刻,状态信息表中应用程序1的缓存时间发生了更新,更新后的缓存时间为S2。
具体地,在终端接收服务器发送媒体流数据的过程中,当终端检测到状态信息表中的该媒体流数据的缓存时间发生更新时,终端可以获取更新的缓存时间,并将更新的缓存时间封装于确认报文,以得到目标确认报文;进一步的,在终端得到目标确认报文之后,可以立即触发终端向服务器发送该目标确认报文的流程。即本申请中最新的缓存信息中的缓存时间会触发自动生成目标确认报文。
举个例子,在终端接收服务器发送媒体流数据A的过程中,当终端检测到如图3中所示的状态信息表中的播放该媒体流数据A的应用程序1(应用标识1)的缓存时间发生更新时,即当终端检测到如图3中所示的状态信息表中的播放该媒体流数据A的应用程序1的最新的缓存信息Info_l7_buffer_1中的缓存时间由S1更新为S2时,终端可以获取更新后的缓存时间S2,并将更新后的缓存时间S2封装于确认报文pkt_ack中,以得到目标确认报文pkt_ack_l7_buffer;进一步的,在终端得到目标确认报文pkt_ack_l7_buffer之后,可以立即触发终端向服务器发送该目标确认报文pkt_ack_l7_buffer的流程,以使服务器能够及时的接收到目标确认报文,并基于目标确认报文中携带的缓存时间及时准确的掌握终端侧是否(即将)出现卡顿现象,并依此信息调整丢包修复的激进程度,进而优化业务体验,同时也可有效避免因pkt_ack回复过慢而造成数据传输不及时的问题,能够在保证不影响用户终端侧的业务体验的同时又能快速有效的修复网络丢包,减少了对后续流量数据传输效率的影响,有利于改善以音视频为主的互联网业务的用户侧的体验质量。
在其中一个实施例中,查询媒体流数据的缓存时间之前,所述方法还包括:在接收到媒体流数据时,生成确认报文;所述查询所述媒体流数据的缓存时间,包括:当检测到确认报文时,查询媒体流数据的缓存时间。
具体地,在终端接收服务器发送媒体流数据的过程中,在终端接收到媒体流数据时,终端可以生成确认报文,当终端检测到已生成的该确认报文时,终端可以从自身维护的状态信息表中查询传输的媒体流数据的最新的缓存时间,并将查询到的最新的缓存时间封装于该确认报文中,以得到目标确认报文。进一步的,在终端得到目标确认报文之后,可以立即触发终端向服务器发送该目标确认报文的流程。即本申请中最新生成的确认报文会触发自动获取最新的缓存时间,并自动生成目标确认报文。
举个例子,在终端接收服务器发送媒体流数据A的过程中,在终端接收到媒体流数据A时,终端可以生成pkt_ack报文,当终端检测到已生成的pkt_ack报文时,终端可以从自身维护的如图3所示的状态信息表中查询播放媒体流数据A的应用程序所对应的最新的缓存信息info_l7_buffer,最新的缓存信息info_l7_buffer中包含最新的缓存时间S2,并将查询到的最新的缓存时间S2封装于pkt_ack报文中,以得到目标确认报文pkt_ack_l7_buffer。进一步的,在终端得到目标确认报文pkt_ack_l7_buffer之后,可以立即触发终端向服务器发送该目标确认报文pkt_ack_l7_buffer的流程,以使服务器能够及时的接收到目标确认报文,并基于目标确认报文中携带的缓存时间及时准确的掌握终端侧是否(即将)出现卡顿现象,并依此信息调整丢包修复的激进程度,进而优化业务体验,同时也可有效避免因pkt_ack回复过慢而造成数据传输不及时的问题,能够在保证不影响用户终端侧的业务体验的同时又能快速有效的修复网络丢包,减少了对后续流量数据传输效率的影响,有利于改善以音视频为主的互联网业务的用户侧的体验质量。
在一个实施例中,如图5所示,基于应用接收队列中的媒体流数据确定缓存信息的步骤,包括:
步骤502,在应用接收队列中,获取媒体流数据的起始帧数据和结尾帧数据;
步骤504,根据起始帧数据和结尾帧数据,确定媒体缓存数据;
步骤506,根据起始帧数据和结尾帧数据之间的时间间隔,确定缓存时间。
其中,起始帧数据是指应用接收队列中第一个可用数据,比如,本申请中的起始帧数据可以是应用接收队列中的第一个可用的帧数据。
结尾帧数据是指应用接收队列中最后一个可用数据,比如,本申请中的结尾帧数据可以是应用接收队列中的最后一个可用的帧数据。
媒体缓存数据是指用于反映应用接收队列中的媒体流数据的相关信息的数据,比如,本申请中的媒体缓存数据至少包括:a)应用接收队列中数据的缓存时长l7_buffer_time_len;b)应用接收队列中完整的音视频帧数量l7_buffer_frame_amount;c)应用接收队列中最后一个音视频帧的信息frame_last。
具体地,在终端接收服务器发送媒体流数据的过程中,终端可以将已接收到的媒体流数据缓存于应用程序对应的应用接收队列中,终端可以通过监测模块周期性的获取各个应用接收队列的缓存信息info_l7_buffer,即终端可以通过监测模块从各个应用程序对应的应用接收队列中,获取媒体流数据的起始帧数据和结尾帧数据,并根据起始帧数据和结尾帧数据,确定媒体缓存数据,比如,终端可以根据起始帧数据和结尾帧数据的帧序号,确定该应用接收队列中完整的音视频帧数量l7_buffer_frame_amount;进一步的,终端可以根据起始帧数据和结尾帧数据之间的时间间隔,确定缓存时间,比如,终端根据起始帧数据的时间戳和结尾帧数据的时间戳,确定起始帧数据和结尾帧数据之间的时间间隔为S,终端可以将时间间隔S作为缓存时间。可以理解,在一些情况下,终端也可以将起始帧数据的时间戳和结尾帧数据的时间戳作为缓存时间。
举个例子,假设终端通过监测模块从应用程序1对应的应用接收队列Q中,获取到媒体流数据的起始帧数据为f3和结尾帧数据为f6,则终端可以根据起始帧数据和结尾帧数据的帧序号f3和f6,确定该应用接收队列中完整的音视频帧数量l7_buffer_frame_amount=4;同时,终端还可以根据结尾帧数据f6确定该应用接收队列中最后一个音视频帧的信息frame_last,具体包括:
该音视频帧的标识frame_last_id=f6和该音视频帧的接收程度frame_last_rcv,frame_last_rcv具体包含该音视频帧共有的报文数量frame_last_pkt_amount,以及当前已经收到的报文数量frame_last_pkt_rcv_amount;
因此,该应用接收队列中最后一个音视频帧的信息frame_last可用前述公式(1)来表示,该音视频帧的接收程度frame_last_rcv可以是基于当前已经收到的报文数量frame_last_pkt_rcv_amount和该音视频帧共有的报文数量frame_last_pkt_amount这两个参数计算得到的,比如,计算方式可以如下公式(3)所示:
frame_last_rcv=frame_last_pkt_rcv_amount÷frame_last_pkt_amount(3)
进一步的,终端可以根据起始帧数据f3的时间戳和结尾帧数据f6的时间戳,确定起始帧数据f3和结尾帧数据f6之间的时间间隔为S=20s,终端可以将时间间隔S=20s作为缓存时间,并将获取到的该应用接收队列中的音视频帧数量l7_buffer_frame_amount=4、最后一个音视频帧的信息frame_last={frame_last_id=f6,frame_last_rcv}和缓存时间S=20s作为缓存信息info_l7_buffer反馈至传输层,同时,终端可以通过监测模块维护如图3中所示的状态表,并周期性地更新该表中不同应用程序的缓存信息info_l7_buffer。由此使得,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并依此信息调整丢包修复的激进程度,有效减少用户终端侧针对某丢包数据的等待时间,进而优化了用户终端侧的业务体验。
在其中一个实施例中,根据起始帧数据和结尾帧数据之间的时间间隔,确定缓存时间, 包括:获取起始帧数据的第一时间戳和结尾帧数据的第二时间戳;第一时间戳为起始帧数据在应用程序的播放器中的播放时间点,第二时间戳为结尾数据在播放器中的播放时间点;基于第二时间戳与第一时间戳之间的差值,确定缓存时间。
其中,本申请中的第一时间戳和第二时间戳只是用于区分不同的帧数据所对应的时间戳。
具体地,在终端接收服务器发送媒体流数据的过程中,终端可以将已接收到的媒体流数据缓存于应用程序对应的应用接收队列中,终端可以通过监测模块周期性的获取各个应用接收队列的缓存信息info_l7_buffer,即终端可以通过监测模块从各个应用程序对应的应用接收队列中,获取媒体流数据的起始帧数据和结尾帧数据,并获取起始帧数据的第一时间戳和结尾帧数据的第二时间戳。进一步的,终端可以计算第二时间戳与第一时间戳之间的差值,并取该差值的绝对值作为缓存时间。可以理解,在一些情况下,终端也可以将起始帧数据的第一时间戳和结尾帧数据的第二时间戳作为缓存时间。
举个例子,假设终端通过监测模块从应用程序1对应的应用接收队列Q中,获取到媒体流数据A的起始帧数据为f3和结尾帧数据为f6,则终端可以根据起始帧数据f3的第一时间戳t1和结尾帧数据f6的第二时间戳t2,确定第二时间戳t2与第一时间戳t1之间的差值为d=20s,终端可以取该差值d=20s的绝对值d=|20|s作为缓存时间,同时,终端可以通过监测模块维护如图3所示的状态表,并周期性地更新该表中不同应用程序的缓存信息info_l7_buffer中的缓存时间,比如,终端可以将最新得到的缓存时间d=|20|s更新于如图3所示的状态表中应用程序1(应用标识1)对应的缓存信息Info_l7_buffer_1中。由此使得,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并依此信息调整丢包修复的激进程度,有效减少用户终端侧针对某丢包数据的等待时间,进而优化业务体验。
在其中一个实施例中,所述方法还包括:在应用层的应用接收队列中,获取媒体流数据的第一起始帧数据和第一结尾帧数据;在传输层的数据接收队列中,获取媒体流数据的第二起始帧数据和第二结尾帧数据;基于第一起始帧数据和第一结尾帧数据之间的第一时间间隔、第二起始帧数据和第二结尾帧数据之间的第二时间间隔,确定缓存时间。
其中,第一起始帧数据和第二起始帧数据用于区分不同队列中的起始帧数据,第一结尾帧数据和第二结尾帧数据用于区分不同队列中的结尾帧数据。
具体地,在终端接收服务器发送媒体流数据的过程中,终端可以将接收到的媒体流数据缓存于队列中,本申请实施例中的队列包含应用层播放器的缓存队列(也可称为应用接收队列)和传输层的数据接收队列,则终端可以在应用层的应用接收队列中,获取该媒体流数据的第一起始帧数据和第一结尾帧数据;同时,若传输层的数据接收队列中的数据不为空,则终端可以在传输层的数据接收队列中,获取该媒体流数据的第二起始帧数据和第二结尾帧数据;进一步的,终端可以基于第一起始帧数据和第一结尾帧数据之间的第一时间间隔、第二起始帧数据和第二结尾帧数据之间的第二时间间隔,确定缓存时间,即终端可以计算第一起始帧数据和第一结尾帧数据之间的第一时间间隔、以及第二起始帧数据和第二结尾帧数据之间的第二时间间隔,并确定第一时间间隔和第二时间间隔之间的和值,将所确定的和值作为最终得到的缓存时间;或者,在一些情况下,终端可以分别取第一时间间隔和第二时间间隔的绝对值,确定第一时间间隔和第二时间间隔的绝对值之间的和值,并将所确定的和值作为最终得到的缓存时间。
其中,终端计算第一起始帧数据和第一结尾帧数据之间的第一时间间隔、以及第二起始帧数据和第二结尾帧数据之间的第二时间间隔所采用的计算方式包括:终端计算第一起始帧数据的时间戳和第一结尾帧数据的时间戳之间的差值,并取该差值的绝对值作为第一时间间隔;同理,终端计算第二起始帧数据的时间戳和第二结尾帧数据的时间戳之间的差值,并取该差值的绝对值作为第二时间间隔。
本实施例中,通过用户终端在回复消息确认报文时携带当前应用层(如播放器)数据缓存时间的信息,以使得数据发送端(云服务器)在收到上述缓存时间后,能够自适应地调整先前丢失报文的重传策略,能够很好地权衡流量发送与丢包修复的关系,在优化因丢包造成用户侧卡顿的同时,减少了对后续流量传输效率的影响,能够在保证不影响用户终端侧的业务体验的同时又能快速有效的修复网络丢包,有利于改善以音视频为主的互联网业务的用户侧的体验质量。
在一个实施例中,在修复媒体流数据之前依据丢包重传参数传输媒体丢失包,包括:在修复媒体流数据之前,依据丢包重传参数传输媒体丢失包至路由节点,当丢包重传参数满足优先级条件时,指示路由节点按照优先级条件转发媒体丢失包。
其中,路由节点是指在终端接收服务器发送媒体流数据的过程中的中间路由节点,比如,本申请中的路由节点可以是网关、中间节点路由器等,路由节点用于转发接收到的媒体流数据。
优先级条件是指用于筛选丢包重传参数的条件,比如,本申请中的优先级条件可以设置为:判断丢包重传参数是否满足预设目标值,例如,预设目标值可以是1,即当丢包重传参数满足预设目标值为1时,则表示该丢包重传参数满足优先级条件。
具体地,终端向服务器发送目标确认报文,以使服务器在确定该媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复该媒体流数据之前服务器依据自适应的丢包重传参数传输媒体丢失包,即在修复媒体流数据之前,服务器可以依据自适应的丢包重传参数传输媒体丢失包至路由节点,当路由节点检测到丢包重传参数满足优先级条件时,指示该路由节点按照优先级条件转发媒体丢失包,即当路由节点检测到丢包重传参数满足优先级条件时,该路由节点可以对包含该媒体丢失包的报文的处理优先级进行调整,比如,该路由节点可以将该包含该媒体丢失包的报文的处理优先级设置为高,以使路由节点将该报文通过优先级高的转发队列进行转发。由此,能够增强重传报文的传输效率,降低用户侧的终端等待该数据丢包的时间,优化音视频卡顿等指标。
在一个实施例中,向服务器发送目标确认报文的步骤,包括:当缓存时间小于丢包修复时间时,向服务器回传预设数量的目标确认报文;丢包修复时间是基于媒体流数据在传输过程中的时延信息确定的。
其中,丢包修复时间可以是预先设置好的时间阈值,比如,丢包修复时间可以设置为固定数值Time_threshold_value,也可设置为时延信息的倍数关系,如丢包修复时间Time_threshold=N*SRTT,其中,SRTT表示平滑往返时间,N是由管理员进行配置的,比如N=2。本申请中的时延信息可以包括SRTT和RTT,RTT(Rround-Trip Time)表示往返时延。
具体地,终端将查询到的缓存时间封装于确认报文,得到目标确认报文之后,当终端确定该缓存时间小于丢包修复时间时,终端向服务器回传该目标确认报文时可以采用激进的回传策略,比如,终端可以向服务器回传预设数量的该目标确认报文。即当缓存时间小于丢包 修复时间时,说明播放该媒体流数据的应用程序很快就能播放完剩余缓存的数据;特别地,若该应用程序的播放器就其缓存中的数据都播放完的时候,终端还没有及时地“告知”发送端(服务器),则可能会出现卡顿等问题;在实际传输过程中,终端反馈的目标确认报文可能会有延时,比如终端每收到5个报文才回复一个目标确认报文,也可能回传的目标确认报文会出现丢失。因此,本申请实施例中,当终端确定该缓存时间小于丢包修复时间时,终端可以向服务器回传相同的M份目标确认报文,以防止目标确认报文在回传过程中丢失。
举个例子,终端将查询到的缓存时间l7_buffer_time_len封装于确认报文,得到目标确认报文pkt_ack_l7_buffer之后,当终端确定该缓存时间l7_buffer_time_len小于丢包修复时间Time_threshold时,终端在回传pkt_ack_l7_buffer时会采用“激进”的冗余回传策略,即终端将pkt_ack_l7_buffer回传相同的M份,以防止pkt_ack_l7_buffer在回传过程中丢失;其中,Time_threshold可设置为固定数值Time_threshold_value,也可设置为时延信息的倍数关系,如Time_threshold=N*SRTT;可以理解,Time_threshold_value、N、M是由管理员进行配置的,并在配置文件中生效;在默认情况下,Time_threshold_value=200ms、N=2、M=3。由此使得,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并依此信息调整丢包修复的激进程度,有效减少用户终端侧针对某丢包数据的等待时间,进而优化业务体验。
在一个实施例中,如图6所示,提供了另一种流量数据传输方法,该方法可以由服务器或终端单独执行,也可以由服务器和终端共同执行,以该方法应用于图1中的服务器为例进行说明,包括以下步骤:
步骤602,在向终端传输媒体流数据的过程中,接收终端返回的目标确认报文;目标确认报文携带终端的针对媒体流数据的缓存时间。
步骤604,当基于目标确认报文确定媒体流数据出现丢包时,依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数。
步骤606,根据丢包重传参数传输媒体流数据对应的媒体丢失包,以使终端在修复媒体流数据之前接收到媒体丢失包。
具体地,在服务器向终端传输媒体流数据的过程中,服务器周期性的接收终端返回的目标确认报文,该目标确认报文携带终端针对该媒体流数据的缓存时间。当服务器基于目标确认报文确定该媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并根据丢包重传参数传输媒体流数据对应的媒体丢失包,以使终端在修复媒体流数据之前接收到媒体丢失包。
举个例子,以媒体流数据为音视频数据为例进行说明。在终端A接收服务器发送的音视频A对应的媒体流数据的过程中,假设终端A将查询到的缓存信息info_l7_buffer(l7_buffer_time_len)封装于消息确认报文pkt_ack中,得到携带了缓存时间的目标确认报文pkt_ack_l7_buffer之后,终端A可以向服务器发送该目标确认报文pkt_ack_l7_buffer,当服务器收到终端A发送的上述目标确认报文pkt_ack_l7_buffer之后,服务器可以基于目标确认报文确定该媒体流数据是否出现丢包,当服务器确定传输该媒体流数据出现丢包时,服务器可以依据预设的丢包修复时间和目标确认报文pkt_ack_l7_buffer中的缓存时间l7_buffer_time_len自适应地调整丢失报文的重传策略,即可得到与重传策略对应的丢包重传参数,进一步的,服务器可以依据所得到的丢包重传参数传输该媒体流数据对应的媒体丢失 包,以使终端在修复该媒体流数据之前接收到媒体丢失包,并基于媒体丢失包修复丢失的数据,以尽可能地保证在缓存时间消耗完之前有足够的媒体数据来用于下一阶段的使用(如音视频播放)。其中,对于媒体流数据是否出现丢包的情况,可以采用图2a的实施例中的判断方式进行判断。
传统方式中,在可靠传输协议中,以音视频数据传输为例,若流量报文在传输过程中出现丢失,则接收端即终端通过消息确认报文“告知”发送端即服务器重传丢失的报文,且接收端即终端处于“一直等待”该丢失的报文的状态,直至接收端即终端接收到重传的丢失的报文后才能将后续的流量报文进行音视频播放。
而本实施例中,通过在向终端传输媒体流数据的过程中,接收终端返回的目标确认报文;目标确认报文携带终端的针对媒体流数据的缓存时间;当基于目标确认报文确定媒体流数据出现丢包时,依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数;根据丢包重传参数传输媒体流数据对应的媒体丢失包,以使终端在修复媒体流数据之前接收到媒体丢失包。由于目标确认报文携带终端的针对媒体流数据的缓存时间,故服务器在确定媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,即服务器在收到上述缓存时间后,能够自适应地调整丢失报文的重传策略所对应的丢包重传参数,相较于传统方式中无差别的丢包重传策略,本实施例中涉及的丢包修复方法能够很好地权衡流量发送与丢包修复的关系,在优化因丢包造成用户侧卡顿的同时,减少了对后续流量传输效率的影响,能够在保证不影响用户终端侧的业务体验的同时又能快速有效的修复网络丢包,有利于改善以音视频为主的互联网业务的用户侧的体验质量。
在一个实施例中,依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数的步骤,包括:对比丢包修复时间与目标确认报文中的缓存时间,得到对比结果;当对比结果表示缓存时间小于丢包修复时间时,自适应第一丢包重传参数;当对比结果表示缓存时间大于或者等于丢包修复时间时,自适应第二丢包重传参数。
其中,对比结果是指丢包修复时间与目标确认报文中的缓存时间之间的大小关系,比如,本申请中的对比结果可以包括三种情况:第一种情况是缓存时间小于丢包修复时间,第二种情况是缓存时间等于丢包修复时间,第三种情况是缓存时间大于丢包修复时间。
第一丢包重传参数和第二丢包重传参数用于区分不同的回传策略所对应的丢包重传参数,比如,采用“激进”的冗余回传策略时的丢包重传参数为第一丢包重传参数,采用默认的回传策略的丢包重传参数为第二丢包重传参数。
具体地,当服务器基于目标确认报文确定媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,即服务器对比丢包修复时间与目标确认报文中的缓存时间,得到对比结果;当对比结果表示缓存时间小于丢包修复时间时,服务器自适应第一丢包重传参数;当对比结果表示缓存时间大于或者等于丢包修复时间时,服务器自适应第二丢包重传参数。即在本实施例中,若缓存时间T_buffer≥丢包修复时间T_recovery,则说明丢包在缓存时间之内就可以得到修复,此时终端侧的播放器不会出现卡顿;若缓存时间T_buffer<丢包修复时间T_recovery,则说明队列中的数据在消耗完的时候丢包还没有得到修复,此时终端侧的播放器会出现卡顿;综上可知,不同的丢包将会对终端侧的QoE造成不同的影响,因此,服务器需要针对这种差异化的影响来自适应丢包修复机制。
举个例子,若T_buffer远远大于T_recovery,比如,T_buffer≥3倍的T_recovery,则说明该丢包并不着急修复,此时发送端即服务器将采用相对缓和的丢包重传策略,即服务器自 适应与相对缓和的丢包重传策略对应的第二丢包重传参数;若T_buffer略微大于或等于T_recovery,则说明该丢包必须在T_buffer内修复,此时发送端即服务器将采用相对激进的丢包重传策略,以保证该丢包不会再次出现丢失,即服务器自适应与相对激进的丢包重传策略对应的第二丢包重传参数;若T_buffer小于T_recovery,则说明该丢包面临比较严峻的修复问题,此时发送端即服务器将采用更为激进的丢包重传策略,即服务器自适应与更为激进的丢包重传策略对应的第一丢包重传参数。
本实施例中,相比传统方式中所采用的发送端单端来探测网络状态与调控发送策略的方法,本实施例中采用的端云协同的方法,能够准确掌握用户侧的业务体验,有利于提升云服务器提供商在优化音视频卡顿、花屏、马赛克等QoE(Quality of Experience,用户体验质量)指标上的竞争力。
在其中一个实施例中,所述方法还包括:当对比结果表示缓存时间小于丢包修复时间时,按照预设发送时间间隔,重传预设数量的媒体丢失包;其中,发送时间间隔是基于媒体流数据在传输过程中的时延信息和预设数量确定的;当对比结果表示缓存时间大于或者等于丢包修复时间时,重传一次媒体丢失包。
其中,时延信息可以包括SRTT和RTT,RTT(Rround-Trip Time)表示往返时延,SRTT表示平滑往返时间,即本申请实施例中的发送时间间隔可以是预先设置好的时间间隔,比如,发送时间间隔可以设置为固定数值30s,即本申请中的发送时间间隔time_interval可通过管理员进行配置,并在配置手册中体现;或者根据当前时延信息来配置,并通过time_interval=1/P*SRTT计算获得,其中,SRTT表示平滑往返时间,P表示预设数量,P是由管理员进行配置的,比如P=4。可以理解,本申请实施例中P的数值默认为4,该数值可由管理员进行配置,并在配置文件中声明。
具体地,当服务器基于目标确认报文确定媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,即服务器对比丢包修复时间与目标确认报文中的缓存时间,得到对比结果;当对比结果表示缓存时间小于丢包修复时间时,服务器可以按照预设发送时间间隔,重传预设数量的媒体丢失包,比如,当对比结果表示缓存信息info_l7_buffer中的缓存时间l7_buffer_time_len<丢包修复时间Time_threshold时,发送端即服务器重传4份相同的pkt_loss报文至终端;当对比结果表示缓存信息info_l7_buffer中的缓存时间l7_buffer_time_len≥丢包修复时间Time_threshold时,发送端即服务器重传一份pkt_loss报文至终端。
本实施例中,当缓存时间较长时,丢包修复的任务并不紧迫,此时发送端将采用重传一次丢包数据;当缓存时间较短时,丢包修复的任务较为紧迫,必须提升该丢包数据到达用户终端的实时性,不然用户终端将会出现卡顿现象,此时发送端将采用较为激进的方法修复之前丢失的数据,以尽可能地保证在缓存时间消耗完之前有足够的数据来用于下一阶段的使用(如音视频播放)。相较于传统方式中无差别的丢包重传策略,本实施例中涉及的丢包修复方法很好地权衡了流量发送与丢包修复的关系,在优化因丢包造成终端侧卡顿的同时,减少了对后续流量传输效率的影响,有利于改善现有以音视频为主的互联网业务的用户体验,提升服务器厂商的竞争力。
在一个实施例中,接收终端返回的目标确认报文之后,所述方法还包括:当基于目标确认报文确定媒体流数据未出现丢包时,依据丢包修复时间和目标确认报文中的缓存时间自适应传输码率;或者,依据预设帧数量和目标确认报文中的缓存帧数量自适应传输码率;基于 传输码率,向终端传输媒体流数据。
其中,传输码率是指传输媒体流数据的码率,即服务器可以选择不同的码率来传输目标媒体流数据,比如,在T1时刻,服务器按照原始码率A1向终端传输媒体流数据A;经过一段时间后,在T2时刻,服务器可以选取低于原始码率A1的码率A2向终端传输媒体流数据A,以使服务器在有限的网络资源下发送尽可能多的音视频帧至终端。
预设帧数量是指预设的帧数量,比如,可以预先设置预设帧数量为10,即当目标确认报文中的缓存帧数量小于该预设帧数量10时,说明终端侧中缓存的音视频帧较少,此时服务器需要在有限的网络资源下发送尽可能多的音视频帧制作终端,以防止出现卡顿等现象。
目标确认报文中的缓存帧数量是指终端侧中缓存的帧数量,比如,在播放该媒体流数据的应用程序的应用接收队列中的缓存帧数量为30,则目标确认报文中的缓存帧数量为30。
具体地,服务器接收到终端返回的目标确认报文之后,当服务器基于目标确认报文确定该媒体流数据未出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应传输码率;或者,服务器可以依据预设帧数量和目标确认报文中的缓存帧数量自适应传输码率;进一步的,服务器可以基于自适应的传输码率,向终端传输该媒体流数据。
举个例子,当服务器基于目标确认报文确定该媒体流数据A未出现丢包时,即在服务器没有检测到报文丢失的情况下,若缓存信息info_l7_buffer中的缓存时间l7_buffer_time_len<丢包修复时间Time_threshold;或者,若缓存信息info_l7_buffer中的缓存帧数量l7_buffer_frame_amount<预设帧数量l7_buffer_frame_amount_threshold,说明终端侧的缓存时间和缓存的音视频帧较少,此时服务器需要在有限的网络资源下发送尽可能多的音视频帧,因此,服务器可以选择码率低一档次的音视频帧作为数据源,比如,服务器可以记录过去一段时间发送的音视频帧的码率bitrate_(i-1),当缓存时间l7_buffer_time_len<丢包修复时间Time_threshold时;或者,当缓存帧数量l7_buffer_frame_amount<预设帧数量l7_buffer_frame_amount_threshold时,服务器将选择最新的音视频码率bitrate_i,其中,bitrate_i<bitrate_(i-1),即服务器会基于最新的音视频码率bitrate_i向终端发送媒体流数据A。其中,l7_buffer_frame_amount_threshold可由管理员自行配置,并在配置文件中声明。由此使得,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并在未出现丢包时,自动调整传输码率,进而优化业务体验。
在一个实施例中,依据丢包修复时间和目标确认报文中的缓存时间自适应传输码率的步骤,包括:当目标确认报文中的缓存时间小于丢包修复时间时,调小媒体流数据的原始码率值,得到传输码率;所述依据预设帧数量和所述目标确认报文中的缓存帧数量自适应所述传输码率,包括:当目标确认报文中的缓存帧数量小于预设帧数量时,调小媒体流数据的原始码率值,得到传输码率。
其中,原始码率值是指在服务器向终端传输媒体流数据的过程中的初始码率值,比如,服务器响应于终端的流量请求报文,选取了码率值为A1的数据源,并基于码率值A1向终端传输媒体流数据,则原始码率值为A1。
具体地,当服务器基于目标确认报文确定媒体流数据未出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应传输码率,即当目标确认报文中的缓存时间小于丢包修复时间时,服务器可以调小该媒体流数据的原始码率值,得到调小后的传输码率;或者,当目标确认报文中的缓存帧数量小于预设帧数量时,服务器也可以调小媒体流数据的 原始码率值,得到调小后的传输码率。即本申请提供的方法,在缓存时间小于丢包修复时间、以及缓存帧数量小于预设帧数量的情况下,说明终端侧的缓存时间和缓存的音视频帧较少,此时服务器需要在有限的网络资源下发送尽可能多的音视频帧,因此,服务器需要调小媒体流数据的原始码率值,并基于调小后的传输码率向终端发送媒体流数据。由此,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并在未出现丢包时,自动调整传输码率,进而优化业务体验。
在一个实施例中,传输码率为第一码率值;所述方法还包括:当目标确认报文中的缓存时间大于丢包修复时间、且目标确认报文中的缓存帧数量大于预设帧数量时,选择第二码率值作为传输码率;其中,第二码率值大于第一码率值。
其中,第二码率值和第一码率值用于区分不同的码率值,
具体地,当服务器基于目标确认报文确定媒体流数据未出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应传输码率,即当目标确认报文中的缓存时间大于丢包修复时间、且目标确认报文中的缓存帧数量大于预设帧数量时,服务器可以选择第二码率值作为传输码率;其中,第二码率值大于第一码率值。即本申请提供的方法,在缓存时间大于丢包修复时间、以及缓存帧数量大于预设帧数量的情况下,说明终端侧的缓存时间和缓存的音视频帧较多,此时服务器无需在有限的网络资源下发送尽可能多的音视频帧,因此,服务器可以调大媒体流数据的原始码率值,并基于调大后的传输码率向终端发送媒体流数据。由此,数据发送端即服务器可以根据接收端即终端发送的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加终端侧的缓存信息,使得服务器及时掌握终端侧是否(即将)出现卡顿(以音视频为例)现象,并在未出现丢包时,自动调整传输码率,进而优化业务体验。
在一个实施例中,目标确认报文携带媒体流数据的结尾帧标识和接收程度参数;所述方法还包括:当接受程度参数满足预设条件、结尾帧标识对应的结尾帧数据未发送完毕、且传输媒体流数据窗口受限时,将结尾帧数据继续传输至终端。
其中,结尾帧标识是指播放或者使用该媒体流数据的应用程序的应用接收队列中最后一个音视频帧的标识,比如,媒体流数据A的最后一个音视频帧的标识为f203,则目标确认报文携带媒体流数据的结尾帧标识为f203。
接收程度参数是指播放或者使用该媒体流数据的应用程序的应用接收队列中最后一个音视频帧的接收程度参数,接收程度参数可以用frame_last_rcv来表示,音视频帧的接收程度frame_last_rcv可以是基于当前已经收到的报文数量frame_last_pkt_rcv_amount和该音视频帧共有的报文数量frame_last_pkt_amount这两个参数计算得到的,比如,计算方式可以如前述公式(3)所示。
窗口受限是指拥塞控制方法中规定的窗口受限,如果窗口受限,则不能继续发送任何流量报文,直至窗口不再受限为止。
具体地,当服务器确定传输的媒体流数据的结尾帧数据的接受程度参数满足预设条件、结尾帧标识对应的结尾帧数据未发送完毕、且传输该媒体流数据窗口受限时,服务器可以暂时忽略窗口限制的影响,即服务器可以将结尾帧数据继续传输至终端,直到将该音视频帧剩余的报文全部发送至终端。即本实施中,服务器在传输媒体流数据的过程中,服务器会针对用户终端收到的最后一个音视频帧frame_last进行优化。
比如,若当前已经收到的报文数量frame_last_pkt_rcv_amount≥1/2*该音视频帧共有的报文数量frame_last_pkt_amount,即表明最后一个音视频帧中有超过一半的报文已经被终端接收到,此时服务器可以判断是否已经将最后一个音视频帧frame_last_id发送完毕,若最后一个音视频帧frame_last_id没有发送完毕,则服务器可以将最后一个音视频帧frame_last_id对应的剩余的内容继续发送出去,上述过程不受窗口受限的影响,即如果遇到窗口受限而无法继续发送报文,进而导致终端侧的最后一个音视频帧无法完整接收,严重情况下,终端侧会造成卡顿、花屏、马赛克等问题。可以理解,服务器在发送某一个音视频帧之前,会清楚这个帧总共有多少个报文,比如某个视频帧总共包含20个报文,其报文编号为:10~19,服务器依次发送报文(编号:10~16),剩余的报文(编号:17~19)由于窗口受限等原因可能无法及时发出,在这种情况下,本申请实施例中提供的方法,使得服务器在窗口受限时可以继续执行激进的流量发送策略,即如果窗口受限且音视频帧有超过一半的报文已被发出,则此时服务器会自动忽略窗口限制的影响,继续将该音视频帧剩余的报文发出为止。由此,通过对用户终端收到的最后一个音视频帧进行优化,以确保终端侧的最后一个音视频帧能够完整接收,有效避免了终端侧出现卡顿、花屏、马赛克等现象,有利于改善现有以音视频为主的互联网业务的用户体验,提升了服务器厂商的竞争力。
在一个实施例中,根据丢包重传参数传输媒体流数据对应的媒体丢失包的步骤,包括:依据丢包重传参数传输媒体丢失包至路由节点,当丢包重传参数满足优先级条件时,指示路由节点按照优先级条件转发媒体丢失包。
具体地,终端向服务器发送目标确认报文,以使服务器在确定该媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复该媒体流数据之前服务器依据自适应的丢包重传参数传输媒体丢失包,即在修复媒体流数据之前,服务器可以依据自适应的丢包重传参数传输媒体丢失包至路由节点,当路由节点检测到丢包重传参数满足优先级条件时,指示该路由节点按照优先级条件转发媒体丢失包,即当路由节点检测到丢包重传参数满足优先级条件时,该路由节点可以对包含该媒体丢失包的报文的处理优先级进行调整,比如,该路由节点可以将该包含该媒体丢失包的报文的处理优先级设置为高,以使路由节点将该报文通过优先级高的转发队列进行转发,由此,能够增强重传报文的传输效率,降低用户侧的终端等待该数据丢包的时间,优化音视频卡顿等指标。
在其中一个实施例中,依据丢包重传参数传输媒体丢失包至路由节点,当丢包重传参数满足优先级条件时,指示路由节点按照优先级条件转发媒体丢失包的步骤,包括:基于丢包重传参数,在媒体丢失包中新增目标标识字段;依据丢包重传参数传输媒体丢失包至路由节点,当目标标识字段为目标值时,指示路由节点优先转发媒体丢失包;其中,目标标识字段为目标值时,表示丢包重传参数满足优先级条件。
其中,目标标识字段是指预先设置的某个标识字段作为目标标识字段,比如,本申请中的目标标识字段可以设置为loss_l7_buffer。
目标值是指目标标识字段的字段值为某个特定的值,比如,目标值可以设置为1,则当目标标识字段的字段值为1时,表示该目标标识字段的字段值为目标值。
具体地,服务器可以基于丢包重传参数,在媒体丢失包中新增目标标识字段,并依据丢包重传参数传输媒体丢失包至路由节点,当目标标识字段为目标值时,指示路由节点优先转发媒体丢失包;其中,目标标识字段为目标值时,表示丢包重传参数满足优先级条件。
举个例子,当服务器在确定媒体流数据出现丢包需要回传丢包数据时,服务器可以在该 报文pkt_loss中新增目标标识字段loss_l7_buffer,其中该字段占1bit,若目标标识字段loss_l7_buffer的值为二进制1,则说明重传的丢包数据用于缓解终端侧缓存时间偏小而可能造成卡顿的问题,即在丢失的报文pkt_loss在重传之前,服务器可以在该报文pkt_loss中新增目标标识字段loss_l7_buffer=1,中间网络节点在收到该流量报文pkt_loss后,检查报文pkt_loss中是否携带值为二进制1的loss_l7_buffer,具体如下:
1)若报文pkt_loss中携带的标识字段loss_l7_buffer的值为二进制0,即loss_l7_buffer=0,则该路由节点按照正常报文的处理方式,将该报文pkt_loss转发至下一跳路由节点;
2)若报文pkt_loss中携带的标识字段loss_l7_buffer的值为二进制1,即loss_l7_buffer=1,则该路由节点提升该报文pkt_loss的处理优先级(数值越小越好),包括但不限于:
a)该路由节点将该报文pkt_loss的转发优先级设置为高,具体表现为该路由节点将该报文pkt_loss通过优先级高的转发队列进行转发;
b)记录该报文pkt_loss的原始优先级pri_origin,计算最新的报文处理优先级pri_new,如下述公式(4)所示:
pri_new=pri_origin–Q(4)
其中Q>0,由管理员进行配置,若pri_origin–Q<0,则pri_new=0,此时代表优先级最高。
本实施例中,中间路由节点在数据转发过程中,若遇到用于避免终端卡顿的带有特定标识的丢包修复报文,则提升该报文的转发优先级,提升这类报文的转发效率,减少用户终端针对某丢包数据的等待时间,进而优化业务体验。
在一个实施例中,提供了一种流量数据传输系统,该系统包括:
服务器,用于向终端传输媒体流数据;
终端,用于在接收所述服务器发送所述媒体流数据的过程中,查询所述媒体流数据的缓存时间;将所述缓存时间封装于确认报文,得到目标确认报文;向所述服务器发送所述目标确认报文;
服务器,还用于在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包;
终端,还用于接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
在一个实施例中,所述系统还包括:
服务器,还用于在修复所述媒体流数据之前,依据所述丢包重传参数传输所述媒体丢失包至路由节点;
路由节点,用于当所述丢包重传参数满足优先级条件时,按照所述优先级条件转发所述媒体丢失包。
在一个实施例中,所述媒体流数据缓存于应用程序对应的应用接收队列中;所述系统还包括:
终端,还用于基于所述应用接收队列中的媒体流数据确定缓存信息;将所述缓存信息存储于所述应用程序对应的状态信息表;从所述状态信息表查询所述媒体流数据的缓存时间。
在一个实施例中,所述系统还包括:
终端,还用于当所述应用程序切换到关闭状态时,从所述状态信息表中删除所述应用程序的应用状态条;当其它应用程序切换到开启状态时,在所述状态信息表中增加所述其它应 用程序的应用标识对应的应用状态条。
在一个实施例中,所述系统还包括:
终端,还用于当检测到所述状态信息表中的缓存时间发生更新时,将更新的所述缓存时间封装于确认报文,得到目标确认报文。
在一个实施例中,所述系统还包括:
终端,还用于在接收到所述媒体流数据时,生成所述确认报文;当检测到所述确认报文时,查询所述媒体流数据的缓存时间。
在一个实施例中,所述系统还包括:
终端,还用于在所述应用接收队列中,获取所述媒体流数据的起始帧数据和结尾帧数据;根据所述起始帧数据和所述结尾帧数据,确定媒体缓存数据;根据所述起始帧数据和所述结尾帧数据之间的时间间隔,确定缓存时间。
在一个实施例中,所述系统还包括:
终端,还用于获取所述起始帧数据的第一时间戳和所述结尾帧数据的第二时间戳;所述第一时间戳为所述起始帧数据在所述应用程序的播放器中的播放时间点,所述第二时间戳为所述结尾数据在所述播放器中的播放时间点;基于所述第二时间戳与所述第一时间戳之间的差值,确定缓存时间。
在一个实施例中,所述系统还包括:
终端,还用于当所述缓存时间小于所述丢包修复时间时,向所述服务器回传预设数量的所述目标确认报文;所述丢包修复时间是基于所述媒体流数据在传输过程中的时延信息确定的。
在一个实施例中,所述系统还包括:
服务器,还用于对比所述丢包修复时间与所述目标确认报文中的缓存时间,得到对比结果;当所述对比结果表示所述缓存时间小于所述丢包修复时间时,自适应第一丢包重传参数;当所述对比结果表示所述缓存时间大于或者等于所述丢包修复时间时,自适应第二丢包重传参数。
在一个实施例中,所述系统还包括:
服务器,还用于当基于所述目标确认报文确定所述媒体流数据未出现丢包时,依据所述丢包修复时间和所述目标确认报文中的缓存时间自适应传输码率;或者,依据预设帧数量和所述目标确认报文中的缓存帧数量自适应所述传输码率;基于所述传输码率,向所述终端传输所述媒体流数据;
终端,还用于接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
在一个实施例中,所述系统还包括:
服务器,还用于当所述目标确认报文中的缓存时间小于所述丢包修复时间时,调小所述媒体流数据的原始码率值,得到所述传输码率;当所述目标确认报文中的缓存帧数量小于预设帧数量时,调小所述媒体流数据的原始码率值,得到所述传输码率。
在一个实施例中,所述传输码率为第一码率值;所述系统还包括:
服务器,还用于当所述目标确认报文中的缓存时间大于所述丢包修复时间、且所述目标确认报文中的缓存帧数量大于预设帧数量时,选择第二码率值作为所述传输码率;其中,所述第二码率值大于所述第一码率值。
在一个实施例中,所述目标确认报文携带所述媒体流数据的结尾帧标识和接收程度参数; 所述系统还包括:
服务器,还用于当所述接受程度参数满足预设条件、所述结尾帧标识对应的结尾帧数据未发送完毕、且传输所述媒体流数据窗口受限时,将所述结尾帧数据继续传输至所述终端。
在一个实施例中,所述系统还包括:
服务器,还用于基于所述丢包重传参数,在所述媒体丢失包中新增目标标识字段;依据所述丢包重传参数传输媒体丢失包至路由节点;
所述路由节点,用于当所述目标标识字段为目标值时,优先转发所述媒体丢失包;其中,所述目标标识字段为所述目标值时,表示所述丢包重传参数满足优先级条件。
在一个实施例中,本申请还提供一种应用场景,该应用场景应用上述的流量数据传输方法。具体地,该流量数据传输方法在该应用场景的应用如下:
当想要实现在不影响用户终端侧的业务体验的同时,又能快速有效的修复网络丢包,可以采用上述的流量数据传输方法,即在音视频点播、直播业务、实时通信等场景下,用户终端向服务器发送请求报文,以使服务器响应于该请求报文向终端传输对应的媒体流数据,在用户终端接收服务器发送媒体流数据的过程中,用户终端可以查询播放该媒体流数据的应用缓存时间,并将缓存时间封装于确认报文,得到目标确认报文;进一步的,用户终端向服务器发送目标确认报文,以使服务器在确定媒体流数据出现丢包时,服务器可以依据丢包修复时间和目标确认报文中的缓存时间自适应丢包重传参数,并在修复媒体流数据之前服务器依据丢包重传参数传输媒体丢失包,用户终端接收服务器基于丢包重传参数传输的媒体丢失包。相较于传统方法,本申请实施例中设计的基于多方协同的自适应丢包修复方法很好地权衡了流量发送与丢包修复的关系,在优化因丢包造成用户侧卡顿的同时,减少了对后续流量传输效率的影响,有利于改善现有以音视频为主的互联网业务的用户体验,提升了服务器厂商的竞争力。
本申请实施例提供的方法,可以应用于改善音视频点播、直播业务、实时通信等场景中。以下以音视频为主的互联网业务场景为例,对本申请实施例提供的流量数据传输方法进行说明。
TCP:Transport Control Protocol,传输控制协议。
QUIC:Quick UDP Internet Connection,快速UDP互联网连接。
QoE:Quality of Experience,(用户)体验质量。
QoS:Quality of Services,服务质量。
RTT:Rround-Trip Time,往返时延。
IP:Internet Protocol,互联网协议。
网络丢包是影响流量传输效率的重要因素,也是影响用户业务体验、评估云服务提供商优劣的关键指标。较高的丢包率会带来流量传输性能的恶化,也即在相同的时间内用户终端手段的数据量偏低,进而可能造成终端应用在使用过程中出现缓冲、卡顿、花屏、马赛克、黑屏等严重影响用户体验的特征。因此,改善网络传输的丢包率进而提升数据有效吞吐量成为优化终端侧业务体验的关键突破点。
传统的降低丢包率提升数据传输效率的主要途径集中在以下两个方面:一是研究设计并部署更适配网络环境与业务类型的拥塞控制算法,如BBR、Reno、TIMELY等,这些方法的核心思想在于试图“准确地”识别当前网络是否拥塞,如果出现拥塞,适当地降低发送参数(如发送速率或发送窗口),以期望在后续的流量传输过程中不再出现或尽可能少地出现丢 包;二是从协议创新的角度出发,设计最新的网络传输协议,通过协议本身自带的优势提升流量传输效率,例如多路径传输(如MPTCP)通过增加一条流量传输路径,加快流完成时间,在多数场景下会显著改善用户侧的业务体验;再如基于UDP的QUIC协议通过支持0-RTT、缓解队头阻塞的方式提升端到端的传输性能。
遗憾地是,上述解决方案仅从数据发送端的角度来看网络拥塞并调整发送策略,未能充分考虑到用户终端地业务体验如何,而事实上,用户终端的业务体验是衡量一个CDN厂商最关键的因素。在这种情况下,如果流量传输过程中丢包率偏高,用户终端的业务体验不一定很差,而用户体验的关键因素在于出现的网络丢包是否在一定时间内得到及时修复,如果得到及时修复,终端的音视频播放器就不会出现卡顿。
如图7所示,为丢包与用户侧卡顿的示意图。在可靠传输协议(如TCP、QUIC等)中,以音视频传输为例,若流量报文在传输过程中出现丢失,则接收端通过消息确认报文“告知”发送端重传丢失的报文,且接收端处于“一直等待”该丢失的报文,直至收到丢失的报文后将后续的流量报文进行音视频播放。
当一个报文出现丢失时,修复时间为T_recovery,其数值为可用RTT和丢包检测时间T_detect来衡量,具体为:T_recovery=N*(RTT+T_detect),其中,N为修复的次数;与此同时,当一个报文出现丢失时,其数据缓存时间T_buffer可用队列中第一个可用数据的时间戳与最后一个可用数据的时间戳之间的时间间隔表示,其中上述“队列”包含应用层播放器缓存队列和传输层的数据接收队列;数据的“时间戳”指数据在播放器中的播放时间点,一般用距离某一个(如第一个)音视频帧的时间间隔来表示。
其中,数据的时间戳可以理解为该数据在播放器的播放时间,例如,播放器队列中的第一个数据的时间戳为time_0,说明播放器马上播放该数据;播放器队列中的最后一个数据的时间戳为time_1,说明该数据将在time_1进行播放,则上述两个数据之间的时间间隔为time_1-time_0;这里的“队列”包含应用层播放器的缓存队列或传输层的队列中的至少一种,前者在音视频应用程序中,用于存储马上要播放的音视频数据,后者在手机设备中,用于存储从发送端接收到的数据,并将这些数据传递给音视频应用程序队列。
在上述示例中,可以根据以下条件判断终端在面临报文丢失时是否会出现卡顿:
若T_recovery≤T_buffer,则说明丢包在缓存时间之内就可以得到修复,此时终端的播放器不会出现卡顿;
若T_recovery>T_buffer,则说明队列中的数据在消耗完的时候丢包还没有得到修复,此时终端的播放器出现卡顿;
综上可知,不同的丢包将会对客户端的QoE造成不同的影响,如何针对这种差异化的影响来设计丢包修复机制将是本申请设计的方法的重点。
因此,为了解决上述问题,本申请提供了一种解决方案,即本申请提出一种基于多方协同的自适应丢包修复与流量传输方法,通过端云协同的方式,赋能用户终端在消息确认报文中携带缓存时间等信息,进而使得发送端(云服务器)可以根据上述缓存时间与丢包修复时间的关系自适应决策丢包重传策略。本申请中提出的方法,可用于改善音视频点播、直播业务、实时通信等场景下的用户体验,属于计算机网络传输优化领域。即用户终端向数据发送端反馈消息确认报文时,携带应用缓冲区的缓存时长;数据发送端在检测到丢包时,可以根据用户终端侧的数据缓存时长自适应调整丢包修复的策略;在该过程中,若缓存时间较长,则数据发送端采取“缓和”的丢包重传策略;若缓存时间较短,则数据发送端采取“激进” 的丢包重传策略;若缓存时间适中,则数据发送端根据最新测量的端到端时延、丢包率等信息自适应调整丢包重传策略。相较于传统方式中无差别的丢包重传策略,本申请实施例中涉及的丢包修复方法很好地权衡了流量发送与丢包修复的关系,在优化因丢包造成用户侧卡顿的同时,减少了对后续流量传输效率的影响,有利于改善现有以音视频为主的互联网业务的用户体验,提升CDN厂商的竞争力。
在产品侧,本申请提供的方法的目的是解决现有传输协议和拥塞控制算法在设计之初未考虑与用户终端业务体验“联动”而导致数据发送端“一味”地调节发送参数,无法真正保证客户端业务体验处于较好状态的问题,提出了一种基于端云协同的自适应丢包修复方法。在本申请中,用户终端在回复消息确认报文时携带当前应用层(如播放器)数据缓存时间的信息;数据发送端(云服务器)在收到上述缓存时间后,自适应地调整先前丢失报文的重传策略;具体地,当缓存时间较长时,丢包修复的任务并不紧迫,此时发送端将采用重传一次丢包数据;当缓存时间较短时,丢包修复的任务较为紧迫,必须提升该丢包数据到达用户终端的实时性,不然用户终端将会出现卡顿现象,此时发送端将采用较为激进的方法修复之前丢失的数据,以尽可能地保证上述缓存时间在消耗完之前有足够的数据来用于下一阶段的使用(如音视频播放)。
如图8所示,为基于多方协同的自适应丢包修复与流量传输方法的示意图。在图8中,①数据接收端在收到流量报文并向发送端反馈消息确认报文之前,查询这些流量报文对应应用的缓存信息(如缓存时间、缓存报文数量、大小)等信息;②数据接收端将上述缓存信息嵌入至消息确认报文,并向数据发送端发送上述消息确认报文中;③数据发送端在收到消息确认报文后,根据上述用户终端的缓存信息调整丢包修复以及发送策略;④中间路由节点在收到因终端缓存信息较为紧张情况下的丢包修复报文后,提升该报文的转发优先级,增强重传报文的传输效率,降低用户终端等待该数据丢包的时间,优化音视频卡顿等QoE指标。
可以理解,本申请实施例中的服务器可以是独立的物理服务器,也可以是多个物理服务器构成的服务器集群或者分布式系统,还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人工智能平台等基础云计算服务的云服务器。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等,但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接,本申请在此不做限制。
在技术侧,如图9所示,为本申请提供的基于多方协同的自适应丢包修复与流量传输方法的整个流程示意图。本申请提供的方法的思想是赋能数据发送端根据接收端的缓存信息自适应调整丢包修复策略,通过在消息确认报文中添加用户终端的缓存信息,使得服务器及时掌握用户终端是否(即将)出现卡顿(以音视频为例)现象,并依此信息调整丢包修复的激进程度;与此同时,中间路由节点在数据转发过程中,若遇到用于避免终端卡顿的带有特定标识的丢包修复报文,则提升该报文的转发优先级,提升这类报文的转发效率,减少用户终端针对某丢包数据的等待时间,进而优化业务体验。
具体是通过以下技术方案实现的:
(1)数据接收端(用户终端)收到来自数据发送端(云服务器)的流量报文,并周期性地向服务器发送消息确认报文,其中该报文中携带用户终端的数据缓存信息,具体如下:
1)用户终端在向云服务器发送消息确认报文pkt_ack之前,获取该应用层对应应用接收队列的缓存信息info_l7_buffer,包括但不限于以下:
a)该应用缓存队列中数据的缓存时长l7_buffer_time_len;
b)该应用缓存队列中完整的音视频帧数量l7_buffer_frame_amount;
c)该应用缓存队列中最后一个音视频帧的信息frame_last,具体包括:
·该音视频帧的标识frame_last_id;
·该音视频帧的接收程度frame_last_rcv,具体包含该音视频帧共有的报文数量frame_last_pkt_amount,以及当前已经收到的报文数量frame_last_pkt_rcv_amount;
·因此,frame_last可用公式(1)表示:
frame_last = {frame_last_id, frame_last_rcv}   (1)
d)因此,应用缓存信息info_l7_buffer可用公式(2)表示:
info_l7_buffer = {l7_buffer_time_len,l7_buffer_frame_amount, frame_last}   (2)
2)为实现上述指标的获取,在本发明中,应用层新增监测模块,用于周期性的获取应用接收队列的缓存信息info_l7_buffer,并将这些反馈至传输层,如图4中所示的用户终端监测示意图。
a)在本实施例中,用户终端获取info_l7_buffer的时间间隔T_gain_l7可由管理员进行配置,并在配置文件中体现,在默认情况下,T_gain_l7=5ms;
b)在本实施例中,用户终端的监测模块位于应用层与传输层之间,其主要目标在于及时获取不同应用的info_l7_buffer;
在本实施例中,监测模块为当前使用的应用维护如图3中所示的状态表,并周期性地更新该表中不同应用的info_l7_buffer:
当某个应用关闭时,用户终端从上述状态信息表中删除对应的应用状态条目;当某个应用开启时,用户终端在上述状态信息表中增加对应的应用状态条目;
c)在本申请中,用户终端在向服务器发送消息确认报文pkt_ack之前,通过监测模块获取对应应用的info_l7_buffer,并将info_l7_buffer嵌入至pkt_ack中,得到最终的消息确认报文pkt_ack_l7_buffer;
(2)用户终端根据自身的info_l7_buffer的情况决策pkt_ack_l7_buffer的反馈策略,具体如下:
1)若info_l7_buffer中的l7_buffer_time_len较小,则用户终端回传pkt_ack_l7_buffer时将采用激进的策略,具体地:
a)若l7_buffer_time_len<Time_threshold时,则用户终端在回传pkt_ack_l7_buffer时采用“激进”的冗余回传策略,即将pkt_ack_l7_buffer回传相同的M份,以防止pkt_ack_l7_buffer再回传过程中丢失;
b)Time_threshold可设置为固定数值Time_threshold_value,也可设置为时延信息的倍数关系,如Time_threshold=N*SRTT;
c)其中,Time_threshold_value、N、M是由管理员进行配置的,并在配置文件中生效;在默认情况下,Time_threshold_value=200ms、N=2、M=3;
2)若监测模块检测到某个应用的l7_buffer_time_len较小,判定条件如上述步骤1)所示,则会触发传输层立刻回复pkt_ack_l7_buffer,而不用等待现有方法中pkt_ack生成后才进行判断,避免因pkt_ack回复过慢而造成数据传输不及时的现象;
a)在现有传输协议中,用户终端在收到流量后,并不是针对每个流量报文都回复一个pkt_ack,而是通过“聚合”的方式“打包”一起回复某些报文是否已经收到,上述步骤2) 可有效避免消息确认报文回复不及时的问题;
b)也就是说,在本申请中,pkt_ack报文会触发获取最新的info_l7_buffer并形成pkt_ack_l7_buffer;也可以是:最新的info_l7_buffer中的l7_buffer_time_len也会触发pkt_ack生成pkt_ack_l7_buffer;
(3)数据发送端(云服务器)在收到上述pkt_ack_l7_buffer后,根据info_l7_buffer信息自适应决策丢包重传策略与流量发送策略,具体如下:
1)如果数据发送端检测到有报文丢失(丢包记为pkt_loss),则:
a)若info_l7_buffer中l7_buffer_time_len<Time_threshold,则发送端重传P份相同的pkt_loss报文;
b)上述P份pkt_loss报文的发送时间间隔time_interval可通过管理员进行配置,并在配置手册中体现;或者根据当前网络时延来配置,并通过time_interval=1/P*SRTT计算获得;
c)P的数值默认为4,该数值可由管理员进行配置,并在配置文件中声明;
d)丢失的报文pkt_loss在重传之前,数据发送端在该报文中新增标识字段loss_l7_buffer,其中该字段占1bit,若值为二进制1,则说明重传的丢包用于缓解终端应用缓存时间偏小而可能造成卡顿的问题;
2)如果数据发送端没有检测到报文丢失,则:
a)若info_l7_buffer中l7_buffer_time_len<Time_threshold或l7_buffer_frame_amount<l7_buffer_frame_amount_threshold,说明用户终端的缓存时间和音视频帧较少,此时数据发送端需要克服在有限的网络资源下发送尽可能多的音视频帧;其中,l7_buffer_frame_amount_threshold可由管理员自行配置,并在配置文件中声明;可以理解,如果L7中播放器队列中缓存的数据时长较小,说明该播放器很快就能播放完剩余缓存的数据;特别地,如果播放器就其缓存中的数据都播放完的时候,如果没有及时地“告知”发送端,则可能会出现卡顿;事实上,用户终端反馈的消息确认报文可能会有延时(如每收到5个报文才回复一个ACK报文),也可能回传的报文会出现丢失。
具体地,数据发送端将选择码率低一档次的音视频帧作为数据源,并向数据接收端发送,详见步骤b);
b)数据发送端记录过去一段时间发送的音视频帧码率bitrate_(i-1),当步骤a)中的条件满足时,数据发送端将选择最新的音视频码率bitrate_i,其中,bitrate_i<bitrate_(i-1);
c)当info_l7_buffer中l7_buffer_time_len>Time_threshold且l7_buffer_frame_amount>l7_buffer_frame_amount_threshold时,此时数据发送端将选择最新的音视频码率bitrate_(i+1),其中bitrate_(i+1)>bitrate_i;
d)如图10所示,为数据发送端调整码率的示意图。数据发送端一开始向用户终端发送高码率的音视频内容,经过一段时间后,满足如下条件:l7_buffer_time_len<Time_threshold或l7_buffer_frame_amount<l7_buffer_frame_amount_threshold时,数据发送端将音视频内容从高码率切换为中码率;在经过一段时间后,若l7_buffer_time_len<Time_threshold或l7_buffer_frame_amount<l7_buffer_frame_amount_threshold,则数据发送端则从中码率切换为低码率;在经过一段时间后,若l7_buffer_time_len>Time_threshold且l7_buffer_frame_amount>l7_buffer_frame_amount_threshold,则数据发送端则将音视频内容从中码率切换为高码率;比如,若l7_buffer_time_len>Time_threshold、且小于3倍的Time_threshold时,数据发送端选择中码率;若l7_buffer_time_len>大于3倍的Time_threshold时,数据发送端选择高码率。
3)除此之外,本申请提供的方法将针对用户终端收到的最后一个音视频帧frame_last进行优化,具体如下:
a)若frame_last_pkt_rcv_amount≥1/2*frame_last_pkt_amount,即最后一个音视频帧中有超过一半的报文已经被用户终端收到,此时数据发送端判断是否已经将最后一个音视频帧frame_last_id发送完毕,若没有,则将frame_last_id对应的音视频帧剩余的内容继续发送出去,值得一提的是,上述过程不受窗口受限的影响,即发送端如果遇到窗口受限而无法继续发送报文,进而导致用户终端侧的最后一个音视频帧没有完整接收,严重情况下,用户终端会造成卡顿、花屏、马赛克等;
(4)中间网络节点在收到流量报文后,检查报文中是否携带值为二进制1的loss_l7_buffer,具体如下:
1)若报文中携带loss_l7_buffer=0,则按照正常报文的处理方式将报文转发至下一跳路由节点;
2)若报文中携带loss_l7_buffer=1,则该路由节点提升该报文的处理优先级(数值越小越好),包括但不限于:
a)将该报文的转发优先级设置为高,具体表现为将该报文通过优先级高的转发队列进行转发;
b)记录该报文的原始优先级pri_origin,计算最新的报文处理优先级pri_new,如下公式所示:
pri_new=pri_origin–Q
其中Q>0,由管理员进行配置,若pri_origin–Q<0,则pri_new=0,此时代表优先级最高。
本申请技术方案所产生的有益效果包括:
本申请中技术方案的目的是解决现有传输协议和拥塞控制算法在设计之初未考虑与用户终端业务体验“联动”而导致数据发送端“一味”地调节发送参数而无法真正保证客户端业务体验处于较好状态的问题,提出一种基于端云协同的自适应丢包修复方法。在本申请中,用户终端在回复消息确认报文时携带当前应用层(如播放器)数据缓存时间的信息;数据发送端(云服务器)在收到上述缓存时间后,自适应地调整先前丢失报文的重传策略;具体地,当缓存时间较长时,丢包修复的任务并不紧迫,此时发送端将采用重传一次丢包数据;当缓存时间较短时,丢包修复的任务较为紧迫,必须提升该丢包数据到达用户终端的实时性,不然用户终端将会出现卡顿现象,此时发送端将采用较为激进的方法修复之前丢失的数据,以尽可能地保证上述缓存时间在消耗完之前有足够的数据来用于下一阶段的使用(如音视频播放)。对比传统方式中无差别的丢包重传策略,本发明涉及的丢包修复方法很好地权衡了流量发送与丢包修复的关系,在优化因丢包造成用户侧卡顿的同时,减少了对后续流量传输效率的影响,有利于改善现有以音视频为主的互联网业务的用户体验,提升CDN厂商的竞争力。
应该理解的是,虽然如上所述的各实施例所涉及的流程图中的各个步骤按照箭头的指示依次显示,但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,如上所述的各实施例所涉及的流程图中的至少一部分步骤可以包括多个步骤或者多个阶段,这些步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤中的步骤或者阶段 的至少一部分轮流或者交替地执行。
基于同样的发明构思,本申请实施例还提供了一种用于实现上述所涉及的流量数据传输方法的流量数据传输装置。该装置所提供的解决问题的实现方案与上述方法中所记载的实现方案相似,故下面所提供的一个或多个流量数据传输装置实施例中的具体限定可以参见上文中对于流量数据传输方法的限定,在此不再赘述。
在一个实施例中,如图11所示,提供了一种流量数据传输装置,包括:查询模块1102、封装模块1104、发送模块1106和接收模块1108,其中:
查询模块1102,用于在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间。
封装模块1104,用于将所述缓存时间封装于确认报文,得到目标确认报文。
发送模块1106,用于向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包。
接收模块1108,用于接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
在一个实施例中,所述媒体流数据缓存于应用程序对应的应用接收队列中;所述装置还包括:确定模块和存储模块,确定模块用于基于所述应用接收队列中的媒体流数据确定缓存信息;存储模块用于将所述缓存信息存储于所述应用程序对应的状态信息表;查询模块还用于从所述状态信息表查询所述媒体流数据的缓存时间。
在一个实施例中,所述装置还包括:删除模块和增加模块,删除模块用于当所述应用程序切换到关闭状态时,从所述状态信息表中删除所述应用程序的应用状态条;增加模块用于当其它应用程序切换到开启状态时,在所述状态信息表中增加所述其它应用程序的应用标识对应的应用状态条。
在一个实施例中,封装模块还用于当检测到所述状态信息表中的缓存时间发生更新时,将更新的所述缓存时间封装于确认报文,得到目标确认报文。
在一个实施例中,所述装置还包括:生成模块,用于在接收到所述媒体流数据时,生成所述确认报文;查询模块还用于当检测到所述确认报文时,查询所述媒体流数据的缓存时间。
在一个实施例中,所述装置还包括:获取模块,用于在所述应用接收队列中,获取所述媒体流数据的起始帧数据和结尾帧数据;确定模块还用于根据所述起始帧数据和所述结尾帧数据,确定媒体缓存数据;根据所述起始帧数据和所述结尾帧数据之间的时间间隔,确定缓存时间。
在一个实施例中,获取模块还用于获取所述起始帧数据的第一时间戳和所述结尾帧数据的第二时间戳;所述第一时间戳为所述起始帧数据在所述应用程序的播放器中的播放时间点,所述第二时间戳为所述结尾数据在所述播放器中的播放时间点;确定模块还用于基于所述第二时间戳与所述第一时间戳之间的差值,确定缓存时间。
在一个实施例中,所述装置还包括:传输模块,用于在修复所述媒体流数据之前,依据所述丢包重传参数传输媒体丢失包至路由节点,当所述丢包重传参数满足优先级条件时,指示所述路由节点按照所述优先级条件转发所述媒体丢失包。
在一个实施例中,传输模块还用于当所述缓存时间小于所述丢包修复时间时,向所述服务器回传预设数量的所述目标确认报文;所述丢包修复时间是基于所述媒体流数据在传输过程中的时延信息确定的。
在一个实施例中,如图12所示,提供了另一种流量数据传输装置,包括:接收模块1202、自适应模块1204和传输模块1206,其中:
接收模块1202,用于在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间。
自适应模块1204,用于当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数。
传输模块1206,用于根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
在一个实施例中,所述装置还包括:对比模块,用于对比所述丢包修复时间与所述目标确认报文中的缓存时间,得到对比结果;自适应模块还用于当所述对比结果表示所述缓存时间小于所述丢包修复时间时,自适应第一丢包重传参数;当所述对比结果表示所述缓存时间大于或者等于所述丢包修复时间时,自适应第二丢包重传参数。
在一个实施例中,自适应模块还用于当基于所述目标确认报文确定所述媒体流数据未出现丢包时,依据所述丢包修复时间和所述目标确认报文中的缓存时间自适应传输码率;或者,依据预设帧数量和所述目标确认报文中的缓存帧数量自适应所述传输码率;传输模块还用于基于所述传输码率,向所述终端传输所述媒体流数据。
在一个实施例中,所述装置还包括:调节模块,用于当所述目标确认报文中的缓存时间小于所述丢包修复时间时,调小所述媒体流数据的原始码率值,得到所述传输码率;当所述目标确认报文中的缓存帧数量小于预设帧数量时,调小所述媒体流数据的原始码率值,得到所述传输码率。
在一个实施例中,所述传输码率为第一码率值;所述装置还包括:选择模块,用于当所述目标确认报文中的缓存时间大于所述丢包修复时间、且所述目标确认报文中的缓存帧数量大于预设帧数量时,选择第二码率值作为所述传输码率;其中,所述第二码率值大于所述第一码率值。
在一个实施例中,所述目标确认报文携带所述媒体流数据的结尾帧标识和接收程度参数;传输模块还用于当所述接受程度参数满足预设条件、所述结尾帧标识对应的结尾帧数据未发送完毕、且传输所述媒体流数据窗口受限时,将所述结尾帧数据继续传输至所述终端。
在一个实施例中,传输模块还用于依据所述丢包重传参数传输媒体丢失包至路由节点,当所述丢包重传参数满足优先级条件时,指示所述路由节点按照所述优先级条件转发所述媒体丢失包。
在一个实施例中,所述装置还包括:新增模块,用于基于所述丢包重传参数,在所述媒体丢失包中新增目标标识字段;传输模块还用于依据所述丢包重传参数传输媒体丢失包至路由节点,当所述目标标识字段为目标值时,指示所述路由节点优先转发所述媒体丢失包;其中,所述目标标识字段为所述目标值时,表示所述丢包重传参数满足优先级条件。
上述流量数据传输装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中,也可以以软件形式存储于计算机设备中的存储器中,以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中,提供了一种计算机设备,该计算机设备可以是终端或服务器,在本实施例中,以该计算机设备是终端为例进行说明,其内部结构图可以如图13所示。该计算机设备包括处理器、存储器、输入/输出接口、通信接口、显示单元和输入装置。其中,处理器、 存储器和输入/输出接口通过系统总线连接,通信接口、显示单元和输入装置通过输入/输出接口连接到系统总线。其中,该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的输入/输出接口用于处理器与外部设备之间交换信息。该计算机设备的通信接口用于与外部的终端进行有线或无线方式的通信,无线方式可通过WIFI、移动蜂窝网络、NFC(近场通信)或其他技术实现。该计算机程序被处理器执行时以实现一种流量数据传输方法。该计算机设备的显示单元用于形成视觉可见的画面,可以是显示屏、投影装置或虚拟现实成像装置,显示屏可以是液晶显示屏或电子墨水显示屏,该计算机设备的输入装置可以是显示屏上覆盖的触摸层,也可以是计算机设备外壳上设置的按键、轨迹球或触控板,还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解,图13中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。
在一个实施例中,提供了一种计算机设备,包括存储器和处理器,存储器中存储有计算机程序,该处理器执行计算机程序时实现上述各方法实施例中的步骤。
在一个实施例中,提供了一种计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现上述各方法实施例中的步骤。
在一个实施例中,提供了一种计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现上述各方法实施例中的步骤。
需要说明的是,本申请所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等),均为经用户授权或者经过各方充分授权的信息和数据,且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一非易失性计算机可读取存储介质中,该计算机程序在执行时,可包括如上述各方法的实施例的流程。其中,本申请所提供的各实施例中所使用的对存储器、数据库或其它介质的任何引用,均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory,ROM)、磁带、软盘、闪存、光存储器、高密度嵌入式非易失性存储器、阻变存储器(ReRAM)、磁变存储器(Magnetoresistive Random Access Memory,MRAM)、铁电存储器(Ferroelectric Random Access Memory,FRAM)、相变存储器(Phase Change Memory,PCM)、石墨烯存储器等。易失性存储器可包括随机存取存储器(Random Access Memory,RAM)或外部高速缓冲存储器等。作为说明而非局限,RAM可以是多种形式,比如静态随机存取存储器(Static Random Access Memory,SRAM)或动态随机存取存储器(Dynamic Random Access Memory,DRAM)等。本申请所提供的各实施例中所涉及的数据库可包括关系型数据库和非关系型数据库中至少一种。非关系型数据库可包括基于区块链的分布式数据库等,不限于此。本申请所提供的各实施例中所涉及的处理器可为通用处理器、中央处理器、图形处理器、数字信号处理器、可编程逻辑器、基于量子计算的数据处理逻辑器等,不限于此。
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各 个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对本申请专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请的保护范围应以所附权利要求为准。

Claims (20)

  1. 一种流量数据传输方法,由终端执行,其中,所述方法包括:
    在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;
    将所述缓存时间封装于确认报文,得到目标确认报文;
    向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包;
    接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
  2. 根据权利要求1所述的方法,其中,所述媒体流数据缓存于应用程序对应的应用接收队列中;所述查询所述媒体流数据的缓存时间之前,所述方法还包括:
    基于所述应用接收队列中的媒体流数据确定缓存信息;
    将所述缓存信息存储于所述应用程序对应的状态信息表;
    所述查询所述媒体流数据的缓存时间,包括:
    从所述状态信息表查询所述媒体流数据的缓存时间。
  3. 根据权利要求2所述的方法,其中,所述查询所述媒体流数据的缓存时间之前,所述方法还包括:
    在接收到所述媒体流数据时,生成所述确认报文;
    所述从所述状态信息表查询所述媒体流数据的缓存时间,包括:
    当检测到所述确认报文时,从所述状态信息表查询所述媒体流数据的缓存时间;
    所述将所述缓存时间封装于确认报文,得到目标确认报文,包括:
    当检测到所述状态信息表中的缓存时间发生更新时,将更新的所述缓存时间封装于确认报文,得到目标确认报文。
  4. 根据权利要求2或3所述的方法,其中,所述基于所述应用接收队列中的媒体流数据确定缓存信息,包括:
    在所述应用接收队列中,获取所述媒体流数据的起始帧数据和结尾帧数据;
    根据所述起始帧数据和所述结尾帧数据,确定媒体缓存数据;
    根据所述起始帧数据和所述结尾帧数据之间的时间间隔,确定缓存时间。
  5. 根据权利要求4所述的方法,其中,所述根据所述起始帧数据和所述结尾帧数据之间的时间间隔,确定缓存时间,包括:
    获取所述起始帧数据的第一时间戳和所述结尾帧数据的第二时间戳;所述第一时间戳为所述起始帧数据在所述应用程序的播放器中的播放时间点,所述第二时间戳为所述结尾数据在所述播放器中的播放时间点;
    基于所述第二时间戳与所述第一时间戳之间的差值,确定缓存时间。
  6. 根据权利要求1至5任一项所述的方法,其中,所述向所述服务器发送所述目标确认报文,包括:
    当所述缓存时间小于所述丢包修复时间时,向所述服务器回传预设数量的所述目标确认报文;所述丢包修复时间是基于所述媒体流数据在传输过程中的时延信息确定的;
    所述在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包,包括:
    在修复所述媒体流数据之前,依据所述丢包重传参数传输媒体丢失包至路由节点,当所述丢包重传参数满足优先级条件时,指示所述路由节点按照所述优先级条件转发所述媒体丢 失包。
  7. 一种流量数据传输方法,由服务器执行,其中,所述方法包括:
    在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;
    当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;
    根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
  8. 根据权利要求7所述的方法,其中,所述依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,包括:
    对比所述丢包修复时间与所述目标确认报文中的缓存时间,得到对比结果;
    当所述对比结果表示所述缓存时间小于所述丢包修复时间时,自适应第一丢包重传参数;
    当所述对比结果表示所述缓存时间大于或者等于所述丢包修复时间时,自适应第二丢包重传参数。
  9. 根据权利要求7或8所述的方法,其中,所述接收所述终端返回的目标确认报文之后,所述方法还包括:
    当基于所述目标确认报文确定所述媒体流数据未出现丢包时,依据所述丢包修复时间和所述目标确认报文中的缓存时间自适应传输码率;或者,依据预设帧数量和所述目标确认报文中的缓存帧数量自适应所述传输码率;
    基于所述传输码率,向所述终端传输所述媒体流数据。
  10. 根据权利要求9所述的方法,其中,所述依据所述丢包修复时间和所述目标确认报文中的缓存时间自适应传输码率,包括:
    当所述目标确认报文中的缓存时间小于所述丢包修复时间时,调小所述媒体流数据的原始码率值,得到所述传输码率;
    所述依据预设帧数量和所述目标确认报文中的缓存帧数量自适应所述传输码率,包括:
    当所述目标确认报文中的缓存帧数量小于预设帧数量时,调小所述媒体流数据的原始码率值,得到所述传输码率。
  11. 根据权利要求10所述的方法,其中,所述传输码率为第一码率值;所述方法还包括:
    当所述目标确认报文中的缓存时间大于所述丢包修复时间、且所述目标确认报文中的缓存帧数量大于预设帧数量时,选择第二码率值作为所述传输码率;其中,所述第二码率值大于所述第一码率值。
  12. 根据权利要求7至11任一项所述的方法,其中,所述目标确认报文携带所述媒体流数据的结尾帧标识和接收程度参数;所述方法还包括:
    当所述接受程度参数满足预设条件、所述结尾帧标识对应的结尾帧数据未发送完毕、且传输所述媒体流数据窗口受限时,将所述结尾帧数据继续传输至所述终端。
  13. 根据权利要求7至11任一项所述的方法,其中,所述根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,包括:
    依据所述丢包重传参数传输媒体丢失包至路由节点,当所述丢包重传参数满足优先级条件时,指示所述路由节点按照所述优先级条件转发所述媒体丢失包。
  14. 根据权利要求13所述的方法,其中,所述依据所述丢包重传参数传输媒体丢失包至 路由节点,当所述丢包重传参数满足优先级条件时,指示所述路由节点按照所述优先级条件转发所述媒体丢失包,包括:
    基于所述丢包重传参数,在所述媒体丢失包中新增目标标识字段;
    依据所述丢包重传参数传输媒体丢失包至路由节点,当所述目标标识字段为目标值时,指示所述路由节点优先转发所述媒体丢失包;其中,所述目标标识字段为所述目标值时,表示所述丢包重传参数满足优先级条件。
  15. 一种流量数据传输装置,其中,所述装置包括:
    查询模块,用于在接收服务器发送媒体流数据的过程中,查询所述媒体流数据的缓存时间;
    封装模块,用于将所述缓存时间封装于确认报文,得到目标确认报文;
    发送模块,用于向所述服务器发送所述目标确认报文,以使所述服务器在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输媒体丢失包;
    接收模块,用于接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
  16. 一种流量数据传输装置,其中,所述装置包括:
    接收模块,用于在向终端传输媒体流数据的过程中,接收所述终端返回的目标确认报文;所述目标确认报文携带所述终端的针对所述媒体流数据的缓存时间;
    自适应模块,用于当基于所述目标确认报文确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数;
    传输模块,用于根据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包,以使所述终端在修复所述媒体流数据之前接收到所述媒体丢失包。
  17. 一种流量数据传输系统,其中,所述系统包括:
    服务器,用于向终端传输媒体流数据;
    终端,用于在接收所述服务器发送所述媒体流数据的过程中,查询所述媒体流数据的缓存时间;将所述缓存时间封装于确认报文,得到目标确认报文;向所述服务器发送所述目标确认报文;
    所述服务器,还用于在确定所述媒体流数据出现丢包时,依据丢包修复时间和所述目标确认报文中的缓存时间自适应丢包重传参数,并在修复所述媒体流数据之前依据所述丢包重传参数传输所述媒体流数据对应的媒体丢失包;
    所述终端,还用于接收所述服务器基于所述丢包重传参数传输的所述媒体丢失包。
  18. 一种计算机设备,包括存储器和处理器,所述存储器存储有计算机程序,其中,所述处理器执行所述计算机程序时实现权利要求1至6,或7至14中任一项所述的方法的步骤。
  19. 一种计算机可读存储介质,其上存储有计算机程序,其中,所述计算机程序被处理器执行时实现权利要求1至6,或7至14中任一项所述的方法的步骤。
  20. 一种计算机程序产品,包括计算机程序,其中,该计算机程序被处理器执行时实现权利要求1至6,或7至14中任一项所述的方法的步骤。
PCT/CN2024/086929 2023-06-08 2024-04-10 流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品 Ceased WO2024250827A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP24818364.2A EP4629596A4 (en) 2023-06-08 2024-04-10 METHOD AND APPARATUS FOR TRANSMITTING TRAFFIC DATA, COMPUTER DEVICE, COMPUTER-READABLE STORAGE MEDIA AND COMPUTER PROGRAM PRODUCT
US19/234,177 US20250310393A1 (en) 2023-06-08 2025-06-10 Method and apparatus for transmitting streammethod for transmitting stream data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310676826.9 2023-06-08
CN202310676826.9A CN116405470B (zh) 2023-06-08 2023-06-08 流量数据传输方法、装置、计算机设备、存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/234,177 Continuation US20250310393A1 (en) 2023-06-08 2025-06-10 Method and apparatus for transmitting streammethod for transmitting stream data

Publications (1)

Publication Number Publication Date
WO2024250827A1 true WO2024250827A1 (zh) 2024-12-12

Family

ID=87018435

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/086929 Ceased WO2024250827A1 (zh) 2023-06-08 2024-04-10 流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品

Country Status (4)

Country Link
US (1) US20250310393A1 (zh)
EP (1) EP4629596A4 (zh)
CN (1) CN116405470B (zh)
WO (1) WO2024250827A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119420673B (zh) * 2024-10-30 2026-03-20 浙江吉利控股集团有限公司 报文传输的管控方法、装置、电子设备及存储介质
CN120017224B (zh) * 2025-04-17 2025-07-25 中国铁塔股份有限公司云南省分公司 一种双通道无人机视距通信系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106850595A (zh) * 2017-01-17 2017-06-13 烽火通信科技股份有限公司 一种流媒体传输优化方法及装置
US20190104079A1 (en) * 2016-09-22 2019-04-04 Tencent Technology (Shenzhen) Company Limited Calling method and device, computer storage medium, and terminal
CN111385241A (zh) * 2018-12-27 2020-07-07 北京紫荆视通科技有限公司 多媒体数据的丢包修复方法、装置、系统和可读存储介质
CN114025389A (zh) * 2021-11-01 2022-02-08 网易(杭州)网络有限公司 数据传输方法、装置、计算机设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1200368C (zh) * 2000-08-18 2005-05-04 清华大学 一种将tcp用于不可靠传输网络的局域重传方法
GB2485765B (en) * 2010-11-16 2014-02-12 Canon Kk Client based congestion control mechanism
CN112073331A (zh) * 2017-05-31 2020-12-11 华为技术有限公司 一种流量控制方法、设备及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190104079A1 (en) * 2016-09-22 2019-04-04 Tencent Technology (Shenzhen) Company Limited Calling method and device, computer storage medium, and terminal
CN106850595A (zh) * 2017-01-17 2017-06-13 烽火通信科技股份有限公司 一种流媒体传输优化方法及装置
CN111385241A (zh) * 2018-12-27 2020-07-07 北京紫荆视通科技有限公司 多媒体数据的丢包修复方法、装置、系统和可读存储介质
CN114025389A (zh) * 2021-11-01 2022-02-08 网易(杭州)网络有限公司 数据传输方法、装置、计算机设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4629596A4 *

Also Published As

Publication number Publication date
CN116405470A (zh) 2023-07-07
CN116405470B (zh) 2023-08-04
EP4629596A4 (en) 2025-12-31
EP4629596A1 (en) 2025-10-08
US20250310393A1 (en) 2025-10-02

Similar Documents

Publication Publication Date Title
JP7664407B2 (ja) データ再送処理方法、装置、コンピュータ機器及びコンピュータプログラム
Matsuzono et al. Low latency low loss streaming using in-network coding and caching
CN101834879B (zh) 一种适应不同网络环境的智能高效视音频数据传输方法
WO2024250827A1 (zh) 流量数据传输方法、装置、计算机设备、计算机可读存储介质和计算机程序产品
US12010016B2 (en) Data stream transmission method and device
CN106341738B (zh) 流媒体网络传输的带宽计算方法、服务器端和系统
Xu et al. CMT-NC: Improving the concurrent multipath transfer performance using network coding in wireless networks
CN111740808A (zh) 一种数据传输方法及装置
JP6685621B2 (ja) マルチキャストネットワークにおいてデータの再送を要求すること
CN102843257A (zh) 一种路径评估方法及装置
WO2011000200A1 (zh) 一种高效的流媒体传输方法
WO2020078448A1 (zh) 一种报文处理方法和装置
CN105392157A (zh) 一种拥塞窗口的调整方法、相关装置和系统
CN115767143A (zh) 播放卡顿的判断方法、装置、电子设备和可读存储介质
CN110602568A (zh) 一种基于rtp的视频流传输丢包重传方法、设备及存储设备
CN113783942A (zh) 基于优先分级队列的mpquic数据包快速传输方法和系统
WO2011009403A1 (zh) 一种实现流业务的方法及通信系统以及相关设备
CN101060717B (zh) 服务质量保障方法与装置
CN105262643B (zh) 一种基于td-lte移动网络传输特征的丢包区分方法
WO2013036453A1 (en) Methods, system and apparatus for packet routing using a hop-by-hop protocol in multi-homed environments
JP3793446B2 (ja) パケット交換通信ネットワークのための中継装置及び中継方法
CN106954240B (zh) 智能移动端接口选择系统与方法
HK40089606A (zh) 流量数据传输方法、装置、计算机设备、存储介质
HK40089606B (zh) 流量数据传输方法、装置、计算机设备、存储介质
WO2024244337A1 (zh) 媒体数据传输方法、装置、电子设备、存储介质及程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24818364

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024818364

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2024818364

Country of ref document: EP

Effective date: 20250630

WWP Wipo information: published in national office

Ref document number: 2024818364

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE