WO2022017454A1 - 一种报文处理方法、网络设备以及相关设备 - Google Patents

一种报文处理方法、网络设备以及相关设备 Download PDF

Info

Publication number
WO2022017454A1
WO2022017454A1 PCT/CN2021/107828 CN2021107828W WO2022017454A1 WO 2022017454 A1 WO2022017454 A1 WO 2022017454A1 CN 2021107828 W CN2021107828 W CN 2021107828W WO 2022017454 A1 WO2022017454 A1 WO 2022017454A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
data
message
network device
memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/107828
Other languages
English (en)
French (fr)
Inventor
廖志坚
包锦程
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP21846476.6A priority Critical patent/EP4175231A4/en
Publication of WO2022017454A1 publication Critical patent/WO2022017454A1/zh
Priority to US18/153,618 priority patent/US12088688B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/34—Flow control; Congestion control ensuring sequence integrity, e.g. using sequence numbers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H04L47/82—Miscellaneous aspects
    • H04L47/827—Aggregation of resource allocation or reservation requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/30—Peripheral units, e.g. input or output ports
    • H04L49/3009—Header conversion, routing tables or routing tags
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/90—Buffering arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/90—Buffering arrangements
    • H04L49/9042—Separate storage for different parts of the packet, e.g. header and payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22—Parsing or analysis of headers

Definitions

  • the present application relates to the field of communications, and in particular, to a message processing method, a network device, and related devices.
  • Transmission Control Protocol is one of the core protocols of the Internet. Because the TCP protocol can ensure the integrity and reliability of data communication, it is widely used in scenarios that require relatively high accuracy, such as file transfer scenarios.
  • TCP applications can realize the function of file transfer based on the File Transfer Protocol (File Transfer Protocol, FTP) and Hyper Text Transfer Protocol (Hyper Text Transfer Protocol, HTTP) in the TCP protocol group;
  • the TCP application can realize the function of sending and receiving mail based on the Simple Mail Transfer Protocol (Simple Mail Transfer Protocol, SMTP) or the Interactive Mail Access Protocol (Interactive Mail Access Protocol, IMAP) in the TCP protocol group.
  • Simple Mail Transfer Protocol Simple Mail Transfer Protocol
  • IMAP Interactive Mail Access Protocol
  • the external network card connected to it often receives the message first, and after the external network card writes the message into the server's network card driver memory, the TCP application then sends the message from the The network card driver obtains the application data in the packet from the memory, and then copies the application data to the application memory.
  • the redundant copying steps result in high memory usage for TCP packet processing and limited processing efficiency.
  • the present application provides a packet processing method, network equipment and related equipment, which can reduce the memory occupancy rate in the packet processing process and improve the packet processing efficiency.
  • a first aspect provides a message processing method, the method is applied to a network device, the network device is connected to a server, the method includes the following steps: receiving a message of an application running in the server, and separating the message from the message Application data, write the application data into the memory interval allocated for the application in the server.
  • the network device Before writing the packet to the server, the network device first separates the application data from the packet, and then writes the application data into the application memory. The entire packet processing process does not need to duplicate the application. This reduces the memory usage during packet processing and improves packet processing efficiency.
  • the method further includes: separating the message header and the metadata of the application data from the message; storing the message header and the metadata of the application data in the server to allocate the data for the driver of the network device. in the memory range.
  • the network device separates the application data from the message, separates the message header and the metadata of the application data from the message, and then writes the application data into the application memory, and writes the message to the application memory.
  • the header and the metadata of the application data are written into the network card driver memory.
  • the TCP application of the server can determine the application memory address where the application data should be stored according to the metadata in the network card driver memory, and then store the application data in the memory page.
  • the process of data communication can be completed by exchanging the memory page corresponding to the memory address of the application. This method avoids the occurrence of multiple copies in the process of transmitting the message from the network device to the memory page of the TCP application, and improves the message processing efficiency. Reduce memory usage.
  • separating the data of the application from the message includes: separating the data of the application from the message according to a delimitation template, and the delimitation template defines the separation of the data of the application and other data in the message rule.
  • the metadata of the same application is generated according to unified rules, so the delimitation templates of the same application are the same, and the delimitation templates of different applications may be the same or different, and the delimitation templates may also be determined according to the rules for generating metadata.
  • the application may deliver a delimitation template corresponding to its application type to the network device.
  • the length of the metadata of the TCP application is uniformly L1
  • the length of the header of the message is also fixed
  • the data with the length of L1 after the header is the metadata
  • the data length of the application is determined according to the metadata.
  • L2 then the data with a length of L2 after the metadata is the data of the application, so that the data of the application is separated from the message.
  • the delimitation template can obtain the metadata according to the delimiter, and then according to the metadata Describes the data length of the application, which separates the application data from the message.
  • the length of the header of the message is fixed, then the data after the header and before the delimiter is the metadata.
  • the length after the metadata is L2. That is, the data of the application, so that the data of the application is separated from the message.
  • the delimitation template is delivered to the network device, so that the network device can split the packet according to the delimitation template, and the data of the application is sent from the packet to the network device.
  • the text is separated and written into the application memory, which avoids the redundant steps of copying the application data into the application memory after the packet is written to the server, and reduces the memory occupation in the process of packet processing and improves the performance of the packet. processing efficiency.
  • receiving the message of the application running in the server includes: aggregating multiple sub-packets belonging to the same data flow into a packet, wherein the source Internet of the multiple sub-packets belonging to the same data flow is The Internet Protocol IP address and the destination IP address are the same.
  • the number of packets to be processed is reduced, and the number of times the network device writes the packets to the server is also reduced, thereby improving the reporting rate of the network device. text processing efficiency.
  • the method before separating the application data from the message, the method further includes: determining that the message includes complete data in a data stream.
  • TCP Offload Engine.TOE TCP Offload Engine
  • TCP protocol processing can be performed in advance within the network device, such as out-of-order processing, congestion control, retransmission, etc.
  • a message includes the complete data in one data stream. It is understandable that after the network device determines that the packet includes the complete data in a data stream, the packet will not be out of sequence or duplicated. The accuracy of the score is improved.
  • a second aspect provides a packet processing method, the method is applied to a server, the server is connected to a network device, and the method includes the following steps: the server receives application data sent by the network device, and stores it in the server as an application In the allocated memory range, receive the message header sent by the network device and the metadata of the application data, store it in the memory range allocated by the server for the driver of the network device, and then determine the memory range allocated by the server for the application. Whether the data of the application is complete, and if the application data is complete, determine the application memory address of the application data according to the metadata of the application data, and exchange the memory page corresponding to the address with the memory page where the application data is located, Causes the application data to be stored in the application memory address.
  • the memory page swap may be to exchange the pointer to the data of the application with the pointer to the memory address of the application, or replace the virtual address of the memory page where the application data is located with the memory address of the application.
  • page swapping is limited.
  • the TCP application running in the server confirms the application memory address of the application data according to the metadata, and then exchanges the application memory page corresponding to the application memory address with the memory page where the application data is located, thereby The resource waste caused by multiple copies of application data is avoided, and the packet processing efficiency is improved.
  • the server may also determine, according to the delimitation template, the packet header written by the network device to the server, the metadata of the application data, and the metadata It is the complete data of the same data stream. If it is determined to be complete data, the application memory address of the application data is determined according to the metadata. If it is determined that it is not complete data, the server writes the packet header, Metadata and metadata of the application data, obtain the application data and the application memory address of the application data, and then copy the application data to the application memory address.
  • the network device since the packets received by the network device may be out of order, repeated, etc., when the network device separates the application data from the packets, the separated application data may be incomplete. After the network device writes the data of the application to the server, the application can re-confirm whether the data of the application is complete according to the delimitation template.
  • the delimitation template here is the same as the delimitation template used by the network device when splitting packets.
  • a TCP application can obtain metadata according to a delimitation template, such as determining the metadata according to the delimiter, or determining the metadata according to the fixed length of the metadata, and then obtain the length of the application data according to the metadata, if the length is different from the If the data length of the application written in the application memory is the same, it means that the network device splits the packet correctly. On the contrary, if the length of the data is different from the length of the application data written into the application memory, it means that the network device does not divide the packet correctly. For the splitting error, it should be understood that the above-mentioned process of determining whether the network device correctly splits the packet according to the delimitation template is used for illustration, which is not limited in this application.
  • the server re-splits the message header, the metadata of the application data, and the application data by using the delimitation template, thereby avoiding the incomplete data written to the application memory address due to the splitting error of the network device.
  • the occurrence of the situation improves the reliability of data transmission.
  • a network device is provided, the network device is connected to a server, and the network device includes: a receiving unit for receiving a message of an application running in the server; a separating unit for separating data of the application from the message ; The writing unit is used to write the data of the application into the memory interval allocated for the application in the server.
  • the separation unit is further configured to separate the message header and the metadata of the application data from the message;
  • the writing unit is further configured to store the message header and the metadata of the application data into a In the memory range allocated by the server for the driver of the network device.
  • the separation unit is configured to separate application data from the packet according to a delimitation template, where the delimitation template defines a separation rule between application data and other data in the packet.
  • the receiving unit is configured to aggregate multiple sub-packets belonging to the same data flow into a packet, wherein the source Internet Protocol IP addresses and the destination of the multiple sub-packets belonging to the same data flow The IP addresses are the same.
  • the network device further includes a determination unit configured to determine that the packet includes complete data in a data stream before the separation unit separates application data from the packet.
  • a server is provided, the server is connected to a network device, and the server includes: an application module, configured to receive application data sent by the network device, and store it in a memory interval allocated by the server for the application, a network card driver , which is used to receive the message header sent by the network device and the metadata of the application data, and store it in the memory interval allocated by the server for the driver of the network device.
  • the kernel protocol stack is used to determine the memory interval allocated for the application in the server. Whether the data of the application in the application is complete, the application module is also used to determine the application memory address of the application data according to the metadata of the application data when the application data is complete, and the memory page corresponding to the address is located with the application data. The memory page is swapped, so that the data of the application is stored in the memory address of the application.
  • the application module is further configured to, before the switching unit determines the application memory address of the application data according to the metadata, according to the delimitation template, determine the packet header written by the network device into the server and the address of the application data. Metadata and metadata are the complete data of the same data stream. If it is determined to be complete data, the application memory address of the application data is determined according to the metadata. If it is determined not to be complete data, the server writes to the server according to the network device. The message header, metadata and metadata of the application data, obtain the application data and the application memory address of the application data, and then copy the application data to the application memory address.
  • a fifth aspect provides a message processing system, including a server and a network device, wherein the server is used to implement the operation steps of the method described in the second aspect or any possible implementation manner of the second aspect, and the network device Operation steps for implementing the method described in the first aspect or any possible implementation manner of the first aspect.
  • a computer program product comprising a computer program that, when read and executed by a computing device, implements the method as described in the first aspect or the second aspect.
  • a computer-readable storage medium comprising instructions that, when executed on a computing device, cause the computing device to implement the method as described in the first aspect or the second description.
  • a network device including a processor and a communication interface, where the communication interface is configured to receive a message, and the processor is configured to execute the method described in the first aspect to process the message.
  • a server including a processor and a memory, and the processor executes code in a storage area to implement the method described in the second aspect.
  • FIG. 1 is a schematic structural diagram of a message processing system provided by the present application.
  • Fig. 2 is a schematic flow chart of the steps of a message processing method in the related art
  • FIG. 3 is a schematic flowchart of steps of a message processing method provided by the present application.
  • FIG. 4 is a schematic flowchart of separating application data from a message under an application scenario provided by the present application
  • Fig. 5 is a kind of step flow schematic diagram of message aggregation provided by the present application.
  • FIG. 6 is a schematic flowchart of steps of the message processing method provided by the present application in an application scenario
  • FIG. 7 is a schematic flowchart of steps of a message processing method provided by the present application.
  • FIG. 8 is a schematic structural diagram of a network device provided by the present application.
  • FIG. 9 is a schematic diagram of a hardware structure of a network device provided by the present application.
  • Protocol Stack Also known as the protocol stack, it is a specific software implementation of the computer network protocol suite.
  • a protocol in a protocol suite is usually designed for only one purpose, which can make the design easier. Because each protocol module usually communicates with two other protocol modules above and below, they can usually be thought of as layers in the protocol stack.
  • the lowest-level protocol always describes the physical interaction with hardware.
  • the three computers are A, B and C.
  • computer A and computer B have radio equipment, which can communicate through the network protocol IEEE802.11, and computer B and computer C are connected by cables to exchange data, such as Ethernet In this way, the data communication between computer A and computer C can only be transmitted through computer B, but cannot be transmitted directly.
  • a new protocol can be established on top of the two protocols, such as IP protocol, In this way, two protocol stacks are formed to realize data communication between computer A and computer C.
  • TCP provides a connection-oriented reliable byte stream service.
  • TCP packages user data to form segments to be processed. When it sends data, it starts a timer, and the other end confirms after receiving the data, and then rearranges out-of-order data and discards duplicate data.
  • TCP-based data communication has High security and reliability are widely used in scenarios with relatively high accuracy requirements, such as file transfer scenarios
  • TCP applications can be based on the File Transfer Protocol (FTP), Hyper Text Transfer Protocol (Hyper Text Transfer Protocol, HTTP) realizes the function of file transfer; for example, in the scenario of sending or receiving mail, TCP applications can be based on the Simple Mail Transfer Protocol (Simple Mail Transfer Protocol, SMTP) in the TCP protocol group or The Interactive Mail Access Protocol (IMAP) implements the function of sending and receiving mail.
  • FTP File Transfer Protocol
  • HTTP Hyper Text Transfer Protocol
  • SMTP Simple Mail Transfer Protocol
  • IMAP Interactive Mail Access Protocol
  • DMA Direct Memory Access
  • Memory page The address space of the memory is artificially divided into several parts of equal size, one corresponding to a memory page, and the processor writes and reads the memory in units of pages.
  • TOE is generally composed of software and hardware components. It extends the functions of the traditional TCP/IP protocol stack, and transfers all the TCP protocol processing work of network data traffic to the integrated hardware of the network card. The server only undertakes TCP/IP application processing tasks, thereby reducing the processing pressure on the server.
  • FIG. 1 is a schematic structural diagram of a server connected to a network, wherein the server 100 is connected to a network device 200, and the network device 200 is connected to a network 300.
  • the network device 200 When other servers in the network 300 send to the server 100 a message to be processed , the network device 200 first receives the to-be-processed packet, and then sends the to-be-processed packet to the server 100, and the server 100 completes one packet processing after processing the to-be-processed packet.
  • the network device 200 is a hardware device that connects the server 100 with the network 300 , and may specifically be a network card (NIC) or a TOE network card, which is not specifically limited in this application.
  • NIC network card
  • TOE TOE network card
  • One server 100 may be connected to one or more network devices 200.
  • FIG. 1 takes the connection of the server 100 to one network device 200 as an example for illustration, which is not limited in this application.
  • the server 100 is a general physical server, for example, an ARM server or an X86 server.
  • the server 100 includes a processor 110 and a memory 120, wherein the processor 110 and the memory 120 are connected to each other through an internal bus 130, which may be an interconnection of peripheral components.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the processor 110 may be composed of at least one general-purpose processor, such as a central processing unit (Central Processing Unit, CPU), or a combination of a CPU and a hardware chip.
  • the above-mentioned hardware chip can be an application-specific integrated circuit (Application-Specific Inegrated Circuit, ASIC), a programmable logic device (Programmable Logic Device, PLD) or a combination thereof.
  • the above-mentioned PLD can be a complex programmable logic device (Complex Programmable Logic Device, CPLD), a field programmable gate array (Field-Programmable Gate Array, FPGA), a general array logic (Generic Array Logic, GAL) or any combination thereof.
  • the processor 110 executes various types of digitally stored instructions, such as software or firmware programs stored in the memory 120, which enable the server 1 to provide a wide variety of services.
  • the memory 120 may be a volatile memory (Volatile Memory), such as random access memory (Random Access Memory, RAM), dynamic random access memory (Dynamic RAM, DRAM), static random access memory (Static RAM, SRAM), synchronous dynamic random access memory (Synchronous Dynamic RAM, SDRAM), double-rate synchronous dynamic random access memory (Double Data Rate RAM, DDR), cache (Cache), etc.
  • the memory 120 may also include a combination of the above types.
  • the memory 120 includes program code 121, wherein the program code 121 may include one or more software modules, such as the code of the TCP application 1211, the kernel protocol stack 1212 and the network card driver 1213 shown in FIG.
  • the TCP application 1211 is implemented based on the TCP protocol Application modules of various functions
  • the kernel protocol stack 1212 can be understood as a part of the operating system, and is mainly used to process the messages in the kernel protocol stack. Or the special program of the TOE network card), which is equivalent to the hardware interface of the network device 200, and the operating system can control the network device 200 through the interface.
  • the memory 120 also includes an application memory 1221 and a network card driver memory 1222, wherein the network card driver memory 1222 is a section of memory applied by the network card driver 1213 to the memory 120.
  • the kernel protocol stack 1212 can process the data in the network card driver memory 1222.
  • the application memory 1221 It is a segment of memory applied by the TCP application 1211 to the memory 120 , and the TCP application 1211 can process the data in the application memory 1221 .
  • the network device 200 When other servers send data to the server 100, the network device 200 first receives the TCP packet from the network 300, and the TCP packet includes the packet header, metadata and application data. Header precedes metadata, which precedes application data.
  • the packet header includes at least a quadruple (source IP address, destination IP address, source port, and destination port). Groups are the same.
  • the metadata includes at least the length of the data of the application and the control information of the data of the application, wherein the control information is used for the TCP application to determine the application memory address of the data of the application.
  • the control information may include context information
  • the TCP application receives After the message arrives, the TCP application can determine, according to the context information, that the application memory addresses where the context data D1 and D3 of the application data D2 in the message are located are Add1 and Add3, respectively, so as to determine that the application memory address of the application data is Add2. , among them, Add1, Add2 and Add3 are a continuous memory.
  • the application memory address determined by the TCP application according to the metadata may be the address of a certain memory page in the application memory 1221, and the present application does not limit the metadata and the specific form of the application memory address determined according to the metadata.
  • the data of the application is the original data sent by other servers to the server 100 , that is, the payload. It should be understood that, during data transmission, in order to make the data transmission more reliable, certain auxiliary information will be added to the head and/or tail of the original data, such as the size of the data amount, check digit, etc., so that the original data is It is not easy to be lost during the transmission process.
  • the original data plus auxiliary information forms the basic transmission unit of the transmission channel, that is, data frames, data packets or TCP/IP messages, etc., and the original data is the application data.
  • each TCP packet is fixed, generally about 1480 bytes, if a large amount of data is to be sent at one time, the data needs to be fragmented and divided into multiple packets for transmission, such as a 10MB packet. file, more than 7100 messages need to be sent.
  • the TCP protocol assigns each message number (Sequence Number, SEQ), so that the receiving-side server can restore the received multiple messages in the order of SEQ. 10M of the original file.
  • SEQ Sequence Number
  • application data 1 is fragmented into three application data (application data 1A, application data 1B, and application data 1C) as an example, wherein, packet 1 includes a packet header, metadata 1 and application data 1A, message 2 includes a message header and application data 1B, message 3 includes a message header, application data 1C, metadata 2 and application data 2, and metadata 1 includes application data 1 (The application data 1A, the application data 1B and the application data 1C) control information, the TCP application can determine the application memory address of the application data 1 according to the control information, and the metadata 2 includes the control information of the application data 2, The TCP application can determine the application memory address of the application data 2 according to the control information.
  • packet 1 includes a packet header, metadata 1 and application data 1A
  • message 2 includes a message header and application data 1B
  • message 3 includes a message header, application data 1C, metadata 2 and application data 2
  • metadata 1 includes application data 1 (The application data 1A, the application data 1B and the application data 1C) control information
  • the TCP application
  • the server shown in FIG. 1 receives and processes the specific flow of the TCP packet from the network 300. Include the following steps:
  • Step 1 The network device 200 writes the message 1, the message 2, and the message 3 into the network card driver memory 1222 of the server 100 through the DMA technology.
  • the network card driver memory 1222 is a segment of memory that the network card driver 1213 applies to the server 100.
  • the network device 200 DMAs the message to the network card driver memory 1222, it can send a protocol processing request to the kernel protocol stack.
  • the protocol processing request includes The address of this message.
  • Step 2 The kernel protocol stack 1212 performs TCP protocol processing on the packets in the network card driver memory 1222 .
  • the kernel protocol stack 1212 can perform TCP protocol processing on the packet according to the address of the packet in the protocol processing request to ensure the integrity of data communication .
  • the kernel protocol stack 1212 can read 20kb of data from the header of the memory page of the network card, so as to obtain the message Then complete the TCP protocol processing steps such as congestion control, out-of-order processing, retransmission, etc. according to the packet header, rearrange the out-of-sequence data, discard the duplicate data, and ensure that the TCP flow corresponding to the packet header is in the All of the messages have been written to the server.
  • the kernel protocol stack 1212 performs TCP protocol processing on the packet header, and determines whether the packets 1 to 3 have been written into the network card driver memory 1222 according to the SEQ in the packet header. If the packet 2 is lost, the kernel The protocol stack 1212 may send a request to retransmit the packet 2 to the network device, and in the case of confirming that no packet is lost, send an application processing request to the TCP application, where the application processing request includes the address of the above-mentioned packet.
  • Step 3 The TCP application processes the metadata in the network card driver memory 1222 to determine the application memory address of the application data.
  • the metadata of the data of each application is generated according to a fixed format, and the metadata can be obtained according to the format characteristics of the metadata.
  • the length of the metadata is fixed, after the TCP application receives the application processing request sent by the kernel protocol stack, it first reads 40kb of data from the header to the metadata memory according to the length of the metadata, such as 40kb, and the 40kb
  • the data is metadata, and then the metadata is parsed to determine the length of the application data and the application memory address; if there are special symbols as separators at the end of the metadata, such as newlines, spaces, colons, etc., After the TCP application receives the application processing request sent by the kernel protocol stack, it first determines the position of the delimiter, then reads the data after the header and before the delimiter into the metadata memory, and then parses it to determine the application data length and application memory address.
  • the metadata memory may be a segment of memory that the TCP application applies to the memory in advance for temporarily storing metadata. After the TCP application parses the metadata in the application metadata memory, the metadata in the memory can be deleted and released. out storage space. It should be understood that FIG. 3 is only used for illustration, and in a specific implementation, the metadata may also include more contents, which will not be illustrated one by one here.
  • Step 4 The TCP application copies the application data in the network card driver memory 1222 into the application memory address.
  • the TCP application after the TCP application confirms that the length of the application data is 1400kb and the application memory address of the application data is Add1 according to the metadata, it can copy the 1400kb data after the 40kb metadata to Add1, and the 1400kb data is Application data required by the TCP application.
  • the network device 200 DMAs the packet to the network card driver memory 1222 that the server has divided for the network card driver in advance.
  • the application data in the message is copied to the application memory 1221 .
  • the message needs to be written to the memory of the network card first, and then the TCP application copies the application data in the message from the memory of the network card to the application memory.
  • the redundant copying steps lead to high memory usage for TCP message processing and high processing efficiency. restricted.
  • the present application provides a packet processing method. As shown in Figure 3, the method includes the following steps:
  • the network device 200 receives the message of the application running in the server 100 .
  • the packet of the above application may be a packet received by the network device 200, such as packet 1, or packet 2, or a packet in FIG. 2, or may be received by the network device 200.
  • the obtained sub-packets belonging to the same data flow (such as TCP flow) are obtained after packet aggregation processing, wherein the source IP addresses and destination IP addresses of the multiple sub-packets belonging to the same data flow are the same.
  • the network device 200 may determine multiple packets of the same TCP stream according to the quadruple in the TCP header of each packet, wherein the quadruples of the multiple packets in the same TCP stream are the same, and then the packets belonging to the same TCP stream are assigned the same quadruple.
  • Multiple packets of a TCP stream are aggregated into one packet, which is also composed of a TCP packet header, metadata, and application data. Each metadata is followed by the corresponding application data. The data of the application is immediately followed by the metadata of the data of the next application.
  • the network device 200 aggregates the packets 1 to 3
  • the obtained packet 0 may be shown in FIG. 5 , where the application data 1 includes the application data 1A in the packet 1 , the application data 1B in the packet 2 , and the application data 1C in the packet 3 .
  • the network device 200 may implement packet aggregation through algorithms such as Large Receive Offload (LRO) and Receive Side Coalescing (RSC), which are not specifically limited here. It can be understood that after the network device 200 aggregates multiple packets belonging to the same TCP flow, the number of times the network device 200 writes the packets to the server can be reduced, and the packet processing efficiency of the network device 200 can be improved.
  • LRO Large Receive Offload
  • RSSC Receive Side Coalescing
  • S320 The network device 200 separates application data from the message.
  • the network device 200 may separate application data from the packet according to a delimitation template, where the delimitation template defines a separation rule between application data and other data in the packet.
  • the packet received by the network device 200 is a packet that has not been processed by the TCP protocol
  • the packet may be an out-of-order or packet-loss packet.
  • splitting there may be a splitting error, that is, the data of the split application is incomplete, for example, only part of the application data is included, or other data is included in addition to the application data.
  • the network device 200 correctly splits the packet the separated application data only includes the complete application data, and the remaining packets only include the complete packet header and complete metadata.
  • the metadata of the same TCP application is generated according to unified rules, the delimitation template of the same TCP application is the same, and the delimitation template of different TCP applications may be the same or different.
  • the network device 200 delivers a delimitation template corresponding to its application type. Specifically, the TCP application can send an interface calling request to the driver of the network device 200 (such as a network card driver), and the driver of the network device 200 responds to the request and provides the TCP application with the interface of the network device 200, and the TCP application calls the interface to delimit the The template is delivered to the network device 200 .
  • the driver of the network device 200 such as a network card driver
  • delimitation templates corresponding to the two metadata formats are illustrated below.
  • the first delimitation template If the length of the metadata of the TCP application is uniformly L1, and the length of the packet header is also fixed, then the data with the length of L1 after the packet header is the metadata, and the application is determined according to the metadata. The data length is L2, then the data with the length L2 after the metadata is the data of the application, so that the data of the application is separated from the message.
  • FIG. 4 is a schematic diagram of a message format.
  • the length of the metadata of the application data of application 1 is 40kb, wherein the first 10kb describes the length of the data of the application, such as The data length of the application is 1400kb, and the last 30kb describes the control information of the data of the application.
  • the TCP application receives the application processing request sent by the kernel protocol stack, it first reads 40kb of data from the header to the application memory according to the metadata length of 40kb, and the 40kb of data is the metadata of application data 1.
  • the data length of the application is 1400kb, and then according to the control information in the last 30kb of the metadata, the application memory address Add1 of the data of the application is determined.
  • FIG. 4 is only used for illustration, and in a specific implementation, the metadata may also include more contents, which will not be illustrated one by one here.
  • the second delimitation template If the metadata of the TCP application is uniformly terminated with a certain delimiter, for example, the delimiter can be a newline character, a space character, a colon, etc., then the delimitation template can obtain the metadata according to the delimiter, and then According to the data length of the application described by the metadata, the data of the application is separated from the message. In the same way, the length of the header of the message is fixed, then the data after the header and before the delimiter is the metadata. After the data length L2 of the application is determined according to the metadata, the length after the metadata is L2. That is, the data of the application, so that the data of the application is separated from the message.
  • delimitation templates of different TCP applications can be determined according to the format of the metadata of the TCP application, which is not specifically limited in this application.
  • the packet is a packet that has not undergone packet aggregation processing, such as packet 1 to packet 3 in the embodiment of FIG. It is necessary to identify the packet header first, determine multiple packets belonging to the same TCP flow, and then use the delimitation template to split multiple packets in the same TCP flow, and extract each packet from the same TCP flow.
  • the application data is separated in the text, and then all the separated application data in the same TCP stream is written into the network card driver memory 1222 , and the packet header and metadata are all written into the application memory 1221 .
  • the network device 200 can first determine according to the packet header that the packets 1 to 3 belong to the same TCP flow, and the reading order of the packets in the TCP flow is the packet 1.
  • Packet 2 and Packet 3 then read the metadata in Packet 1, determine that the length of the application data 1 is 3000KB, and read the application data 1A in Packet 1 (assuming that the data length of the application is 1000KB), at this time, the data 1 of the application has 2000KB unread, and the data 1B of the application can be read from the packet 2 (assuming the data length of the application is 1400KB). At this time, the data 1 of the application still has 600KB unread. You can continue to read the application data 1C from the message 3 (assuming the application data length is 600KB), so as to obtain the application data 1 with a length of 3000KB, and continue to read the 40kb data to obtain the application data 2 metadata.
  • the metadata 1 and the message header are finally written into the network card driver cache memory, and the application data 1A, the application data 1B, and the application data 1C are written into the application memory.
  • the reading method of the second delimited template is similar, and details are not repeated here.
  • the network device 200 writes the data of the application into the memory interval allocated for the application in the server 100 , that is, the application memory 1221 shown in FIG. 1 .
  • the network device 200 may write the data of the separated application into the application memory 1221 by using the DMA technology.
  • the storage address of the application data may be sent by the network card driver of the server to the network device 200 in advance, and the storage address is the address sent by the TCP application to the network card driver after being determined according to the idle condition of the application memory 1221 .
  • S340 The network device 200 separates the message header and the metadata of the application data from the message.
  • the message includes a message header, application data and metadata.
  • step S320 uses the delimitation template to separate the application data, the message header and the metadata of the application data can also be separated.
  • the message header and the metadata of the application data can also be separated.
  • the network device 200 writes the metadata of the message header and the application data into the memory interval allocated for the driver of the network device 200 in the server 100 , that is, the network card driver memory 1222 shown in FIG. 1 .
  • the network device 200 may write the metadata of the message header and application data into the network card driver memory 1222 by using the DMA technology.
  • the storage address of the message header and the metadata of the application data may be sent by the network card driver of the server to the network device 200 in advance, and the storage address is determined by the network card driver according to the idle condition of the network card driver memory 1222.
  • steps S340 to S350 may occur simultaneously with steps S320 to S330, or may occur sequentially, which is not specifically limited in this application.
  • S360 The kernel protocol stack of the server 100 performs TCP protocol processing on the packet header, metadata of the application data, and application data, where the TCP protocol processing is used to make the packet include complete data in a data stream.
  • the kernel protocol stack can perform out-of-order processing, congestion processing, retransmission, etc. on the packet header. If out-of-order, the application data is rearranged according to the packet header. The header discards the repeated data in the application data, so that the packet header, application data, and metadata of the application data processed by the protocol include all the data in the same TCP data stream, ensuring complete metadata in the TCP stream. And the data of the complete application has been written to the server, which improves the reliability of message processing.
  • the kernel protocol stack can send a request to retransmit Packet 3 to the network device 200.
  • the network device 200 receives the packet.
  • the application data X2 in the message 3 is separated according to the delimitation template, and then the application data X2 is written into the application memory 1221, and the remaining message Y2 is written into the network card driver memory 1222, so that the server 100 obtains the TCP All packets of the flow.
  • the data structure of the kernel protocol stack is usually a socket buffer (SKB) structure, which can process data of multiple different addresses by hanging multiple pointer items, such as processing packets.
  • SKB socket buffer
  • the pointer 1 of the SKB1 of 1 points to the network card driver memory 1222.
  • the kernel protocol stack can use the pointer 1 to perform TCP protocol processing on the packet header in the network card driver memory.
  • the pointer 2 of the SKB1 can also point to the application memory 1221. Through the pointer 2. Monitor the data in the application memory 1221, determine that the application data has been written into the memory, and make corresponding adjustments to the application data in the case of data duplication and disorder in the packet header.
  • a retransmission request can be sent to the network device until all the packets have been written to the server.
  • a retransmission request can be sent to the network device until all the packets have been written to the server.
  • the kernel protocol stack sends a confirmation request to the TCP application, where the confirmation request carries the address of the metadata of the application data in the network card driver memory and the address information of the application data in the application memory.
  • the TCP application determines whether the application data is complete application data in the same TCP stream, if it is complete application data, execute step S390, and if it is not complete application data, execute step S311.
  • the TCP application may reconfirm whether the data of the application is complete according to the delimitation template, and the delimitation template here is the same as the delimitation template used by the network device 200 in step S310.
  • a TCP application can obtain metadata according to a delimitation template, such as determining the metadata according to the delimiter, or determining the metadata according to the fixed length of the metadata, and then obtain the length of the application data according to the metadata, if the length is different from the If the data length of the application written in the application memory is the same, it means that the network device 200 splits correctly at step S310, and then step S390 can be executed.
  • step S310 if the length of the data is the same as the length of the data of the application written in the application memory If it is different, it means that the network device 200 has a split error in step S310, and then step S311 can be executed. It should be understood that the above-mentioned process of determining whether the network device 200 correctly splits the packet according to the delimitation template is used for illustration and is not limited in this application.
  • the TCP application determines the application memory address of the application data according to the metadata, and exchanges the memory page corresponding to the application memory address with the memory page where the application data is located.
  • the memory page swap may be to exchange a pointer to the data of the application with a pointer to the memory address of the application, or to replace the virtual address of the memory page where the data of the application is located with the memory address of the application.
  • the specific implementation of the exchange is limited.
  • the TCP application confirms the application memory address of the application data according to the metadata, and then exchanges the application memory page corresponding to the application memory address with the memory page where the application data is located, thus avoiding copying the application data multiple times.
  • the resulting waste of resources improves the efficiency of packet processing.
  • step S390 may not be performed, thereby further improving the packet processing efficiency.
  • the TCP application obtains the metadata of the application data and the application data, parses the metadata, determines the application memory address of the application data, and then copies the application data into the application memory address. For details, reference may be made to steps 3 to 4 in the embodiment of FIG. 1 , and details are not repeated here.
  • the TCP application records the confirmation result of step S380, that is, the confirmation result of whether the network device 200 splits correctly. If the number of consecutive split errors exceeds a threshold, the TCP application can send a re-delimitation request to the network device 200. , the re-delimitation request includes information about the TCP stream to which the packet with the split error belongs, and the network device 200 re-aggregates and splits all packets in the TCP stream in response to the request, thereby avoiding continuous splitting In an error scenario, the TCP application repeatedly executes steps S370 to S311, which further improves the packet processing efficiency and reduces the memory occupancy rate of the server processing packets.
  • the TCP application when it sends a re-delimitation request to the network device 200, it may first send an interface calling request to the network card driver, and the network card driver responds to the request and provides the interface of the network card to the TCP application, and the TCP application calls the interface to deliver Synchronization command (SYNC), wherein the SYNC command is used to force the data in the memory buffer to be written into the network device 200 immediately, so that the network device 200 performs a restart and delimitation operation, thereby avoiding the scenario of continuous packet splitting errors,
  • SYNC Synchronization command
  • the TCP application repeatedly performs steps S370 to S11, the packet processing efficiency is further improved, and the memory occupancy rate of the server processing packets is reduced.
  • the network device 200 receives the packets 1 to 3. Assuming that the packets 1 to 3 belong to the same TCP flow, the network device 200 performs packet aggregation on the packets 1 to 3. After the packet 0 shown in FIG. 5 is obtained, the specific flow of processing the packets 1 to 3 using the above steps S310 to S311 may be as shown in FIG. 6 .
  • the network device 200 separates the application data from the packet 0 through steps S320 and S340, and separates the packet header and metadata from the packet 0, such as As shown in FIG. 6 , application data 1 and application data 2 are separated from packet 0 , and the packet header, metadata 1 and metadata 2 are separated from packet 0 .
  • the delimitation template is determined according to the format of the metadata. For example, the length of the metadata is L1.
  • the network device 200 reads the data with the length of L1 from the packet header, obtains the metadata 1, and then determines the application according to the metadata.
  • the data length is L2, and then the data of length L2 is read from the metadata 1 to obtain the application data 1, and then the data of length L1 is read from the application data 1 to obtain the metadata 2. According to the metadata 2 After the length L3 of the data 2 of the application is determined, the data of the length L3 is finally read from the metadata 2 to obtain the data 2 of the application. It should be understood that the above examples are for illustration only, and do not constitute specific limitations.
  • the network device 200 DMAs the application data to the server's network card driver memory 1222 through steps S330 and S350, and DMAs the message header and metadata to the server's application memory 1211, wherein the application data 1 is written into the application memory page Y1, application data 2 is written to application memory page Y2.
  • the network device 200 can write data of different applications into different application memory pages as shown in FIG. 6, or can write the data of all applications into the same page.
  • An application memory page that is, writing both application data 1 and application data 2 into application memory page Y1 or application memory page Y2, can be specifically determined according to the processing logic of the TCP application, which is not limited in this application.
  • step S360 to process the packet headers in the network card driver memory 1222, such as out-of-order processing, retransmission, etc., rearrange the out-of-order data, discard the duplicate data, and ensure the integrity of the TCP stream.
  • step S370 is performed to send a confirmation request to the TCP application, and the TCP application responds to the confirmation request and performs step S380 according to the delimitation template to determine that the application data written into the application memory is
  • step S390 determines the application memory page Y3 corresponding to the application memory address of the application data 1 according to the metadata 1, and then assigns the application memory page Y1 where the application data 1 is located.
  • This process does not require repeated copying of application data, which reduces the memory usage of the server for processing TCP packets and improves packet processing efficiency.
  • some network devices 200 support the TOE function, which enables the network device 200 to perform TCP protocol processing on the packet after receiving the packet, and then process the complete The TCP stream is written to the server 100 .
  • the specific process of using the packet processing method provided by the present application for this type of TOE network card may be as shown in FIG. 7 .
  • the network device 200 performs TCP protocol processing on to-be-processed packets of the application running in the server 100, such as out-of-order processing, retransmission, congestion processing, etc., so that the to-be-processed packets include complete data in a data stream.
  • TCP protocol processing on to-be-processed packets of the application running in the server 100, such as out-of-order processing, retransmission, congestion processing, etc.
  • the network device 200 may be a TOE network card supporting the TOE function.
  • the message of the application running in the above server 100 may be a single sub-package, such as message 1 to message 3 in the embodiment of FIG. 5 , or may be multiple sub-packages received by the network device 200
  • a to-be-processed packet is obtained, such as packet 0 in the embodiment of FIG. 5 .
  • the network device 200 can aggregate multiple sub-packets belonging to the same TCP flow into a packet after performing TCP protocol processing on all received packets, and can also aggregate multiple received sub-packets into a packet.
  • the to-be-processed packet is processed by the TCP protocol, which is not limited in this application. Wherein, for the specific description of the TCP protocol processing, reference may be made to step 2, step S330, etc. in the foregoing content, which will not be repeated here.
  • S420 The network device 200 separates application data from the message.
  • the network device 200 can separate the application data from the message according to the delimitation template, wherein the delimitation template is the TCP application in the server 100 before step S410, by calling the interface driven by the network card to the network device 200.
  • the delimitation template is the TCP application in the server 100 before step S410.
  • the network device 200 supports the TOE function, the message has been processed by the TCP protocol in step S420, so the message will not be out of sequence, repeated, etc. data without splitting errors.
  • the network device 200 writes the data of the application into the memory interval allocated for the application in the server, which may be the application memory 1221. For the description of this step, refer to step S330 in the foregoing content, which will not be repeated here. In a specific implementation, the network device 200 may write application data into the application memory 1221 through the DMA technology.
  • step S440 Separate the message header and the metadata of the application data from the message. For a specific description of this step, reference may be made to step S340 in the foregoing content, which will not be repeated here.
  • step S450 Write the message header and the metadata of the application data into one of the memory intervals allocated by the server for the driver of the network device, specifically, the network card driver memory 1222.
  • the network device 200 may write the message header and metadata of the application data into the network card driver memory 1222 by using the DMA technology.
  • step S350 for the description of this step, reference may be made to step S350 in the foregoing content, which will not be repeated here.
  • the kernel protocol stack sends an exchange request to the TCP application, where the exchange request includes the address of the separated application data and the address of the metadata.
  • the exchange request includes the address of the separated application data and the address of the metadata.
  • the TCP application determines the application memory address of the application data according to the metadata, and exchanges the memory page where the application data is located with the memory page corresponding to the application memory address. Specifically, the TCP application can obtain metadata according to the address of the metadata in the exchange request, and then determine the application memory address of the application data according to the metadata, wherein the application memory address of the application data and the specific memory page exchange are determined according to the metadata. For the description, reference may be made to step S390 in the signed content, which will not be repeated here.
  • the TCP application can secondly confirm whether the network device 200 is split correctly through the delimitation template, and if the split is correct, then determine the application according to the metadata.
  • the application memory address of the data, and the memory page where the application data is located is exchanged with the memory page corresponding to the application memory address, so as to avoid other reasons such as DMA errors that cause the application data to be incomplete.
  • the TCP application gets wrong data. The occurrence of the situation further improves the reliability of the packet processing method provided by this application.
  • the server sends a delimitation template to the network device in advance, so that the network device splits the application data from the message through the delimitation template before sending the message to the server. and metadata, and write the metadata into the network card driver memory, and write the application data into the application memory, so that the TCP application of the server determines the application memory address of the application data according to the metadata in the network card driver memory, and then stores the application data.
  • a data communication process can be completed by exchanging the memory page where the memory page is located with the memory page corresponding to the memory address of the application. This method avoids the occurrence of multiple copies in the process of transmitting the message from the network device to the memory page of the TCP application, and improves the reporting rate. Improve file processing efficiency and reduce memory usage.
  • FIG. 8 is a schematic structural diagram of a network device 200 provided by the present application.
  • the network device 200 is applied to the packet processing system shown in FIG. 1 , wherein the network device 200 is connected to the server 100.
  • the network The apparatus 200 may include a receiving unit 810 , a separating unit 820 , and a writing unit 830 .
  • the receiving unit 810 is configured to receive the message of the application running in the server.
  • the separation unit 820 is used to separate the data of the application from the message.
  • the separation unit 820 is used to separate the data of the application from the message.
  • step S320 in the embodiment of FIG. 3 and step S420 in the embodiment of FIG. 4 and details are not repeated here;
  • the writing unit 830 is used to write the data of the application into the memory interval allocated for the application in the server.
  • step S330 in the embodiment in FIG. 3 and step S430 in the embodiment in FIG. 4 Repeat.
  • the separating unit 820 is further configured to separate the metadata of the header and the data of the application from the message; the writing unit is further configured to store the metadata of the header and the data of the application to the server as:
  • the memory interval allocated by the driver of the network device for the specific implementation, reference may be made to the detailed descriptions of steps S340 to S350 in the embodiment of FIG. 3 and steps S440 to S450 in the embodiment of FIG.
  • the separation unit 820 is configured to separate the data of the application from the message according to the delimitation template, and the delimitation template defines the separation rule of the data of the application and other data in the message.
  • the delimitation template defines the separation rule of the data of the application and other data in the message.
  • the receiving unit 810 is configured to aggregate multiple sub-packets belonging to the same data flow into a packet, wherein the source IP address and destination IP address of the multiple sub-packets belonging to the same data flow.
  • the network device 200 when the network device 200 is a network card that supports the TOE function, the network device 200 further includes a determining unit 840, and the determining unit 840 is configured to determine that the packet includes, before the separation unit separates the data of the application from the packet. Complete data in one data stream. Specifically, TCP protocol processing may be performed on the packet, such as congestion control, retransmission, out-of-order processing, etc. For the specific implementation method, reference may be made to step S410 in the above embodiment of FIG. 4 , which will not be repeated here.
  • each module of the network device 200 shown in FIG. 8 may also be divided into multiple divisions, and each module may be a software module, a hardware module, or a part of a software module and a part of a hardware module.
  • FIG. 8 is an exemplary division manner, which is not specifically limited in this application.
  • the network device provided by this application can receive the delimitation template issued by the server in advance, so that the network device splits the application data and metadata from the message through the delimitation template before sending the message to the server. data, and write the metadata into the network card driver memory, and write the application data into the application memory, so that the TCP application of the server determines the application memory address of the application data according to the metadata in the network card driver memory, and stores the application data in the application memory address.
  • the process of data communication can be completed by exchanging the memory page with the memory page corresponding to the memory address of the application. This method avoids the occurrence of multiple copies in the process of transmitting the message from the network device to the memory page of the TCP application, and improves the message processing. efficiency and reduce memory usage.
  • FIG. 9 is a schematic diagram of the hardware structure of a network device 200 provided by the present application.
  • the network device 200 includes: a processor 910 , a communication interface 920 and a memory 930 .
  • the processor 910, the communication interface 920 and the memory 930 can be connected to each other through the internal bus 940, and can also communicate through other means such as wireless transmission.
  • the embodiment of the present application takes the connection through the bus 940 as an example, and the bus 940 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus or the like.
  • PCI peripheral component interconnect standard
  • EISA Extended Industry Standard Architecture
  • the bus 940 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 9, but it does not mean that there is only one bus or one type of bus.
  • the processor 910 may be composed of at least one general-purpose processor, such as a central processing unit (Central Processing Unit, CPU), or a combination of a CPU and a hardware chip.
  • the above-mentioned hardware chip may be an application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), a programmable logic device (Programmable Logic Device, PLD) or a combination thereof.
  • the above-mentioned PLD can be a complex programmable logic device (Complex Programmable Logic Device, CPLD), a field programmable gate array (Field-Programmable Gate Array, FPGA), a general array logic (Generic Array Logic, GAL) or any combination thereof.
  • Processor 910 executes various types of digitally stored instructions, such as software or firmware programs stored in memory 930, which enable network device 200 to provide a wide variety of services.
  • the memory 930 is used for storing program codes, and is controlled and executed by the processor 910, so as to execute the processing steps of the network device 200 in any of the embodiments in FIG. 1 to FIG. 7 .
  • One or more software modules may be included in the program code.
  • the one or more software modules may be the software modules provided in the embodiment shown in FIG.
  • the receiving unit may be used to receive messages of applications running in the server; separation; unit, which can be used to separate the data of the application from the message; the writing unit is used to write the data of the application into the memory interval allocated for the application in the server, and specifically can be used to perform steps S310-S350 of the foregoing method, Steps S410 to S450 and their optional steps may also be used to perform other steps performed by the network device 200 described in the embodiments of FIG. 2 to FIG. 7 , which will not be repeated here.
  • the memory 930 may include a volatile memory (Volatile Memory), such as a random access memory (Random Access Memory, RAM); the memory 1030 may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read- Only Memory (ROM), flash memory (Flash Memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); the memory 930 may also include a combination of the above types.
  • the memory 930 may store program codes, and may specifically include program codes for the processor 910 to perform other steps described in the embodiments of FIGS. 2-7 , which will not be repeated here.
  • the communication interface 920 may be a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIe) bus interface), a wired interface (such as an Ethernet interface), or a wireless interface (for example, a cellular network interface or a wireless local area network interface) is used to communicate with other servers or modules.
  • the communication interface 920 can be used to receive a message for the processor 910 to process the message.
  • FIG. 9 is only a possible implementation manner of the embodiment of the present application.
  • the network device may further include more or less components, which is not limited here.
  • the embodiment of the present application further provides a computer program product, when the computer program product runs on the processor, the method flow shown in FIG. 2-FIG. 7 is realized.
  • the above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination.
  • the above-described embodiments may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes at least one computer instruction.
  • the procedures or functions according to the embodiments of the present invention result in whole or in part.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server, or data center over a wire (e.g.
  • Coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) means to transmit to another website site, computer, server or data center.
  • a computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that contains at least one set of available media.
  • Useful media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, Digital Video Disc (DVD), or semiconductor media.
  • the semiconductor media may be SSDs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Computer And Data Communications (AREA)

Abstract

本申请提供了一种报文处理的方法、网络设备以及相关设备，该方法包括以下步骤：网络设备接收服务器中运行的应用的报文，从报文中分离出应用的数据，将应用的数据写入服务器中为应用分配的内存区间中，使得服务器处理该报文的过程中，无需重复拷贝应用的数据，减少报文处理过程中的内存占用率，提高报文处理效率。

Description

一种报文处理方法、网络设备以及相关设备 技术领域
本申请涉及通信领域，尤其涉及一种报文处理方法、网络设备以及相关设备。
背景技术
传输控制协议(Transmission Control Protocol,TCP)是互联网核心协议之一，由于TCP协议可以保证数据通信的完整性和可靠性，被广泛应用在对准确性要求相对较高的场景下，比如文件传输场景中，TCP应用可以基于TCP协议组中的文件传输协议(File Transfer Protocol，FTP)、超文本传输协议(Hyper Text Transfer Protocol，HTTP)实现文件传输的功能；再比如发送或接收邮件的场景中，TCP应用可以基于TCP协议组中的简单邮件传输协议(Simple Mail Transfer Protocol，SMTP)或者交互邮件访问协议(Interactive Mail Access Protocol，IMAP)实现邮件收发的功能。
但是，基于TCP协议进行数据通信的服务器在接收TCP报文时，往往是与其相连的外接网卡首先接收到该报文，外接网卡将该报文写入服务器的网卡驱动内存后，TCP应用再从网卡驱动内存中获取报文中的应用的数据，然后将该应用的数据拷贝至应用内存，多余的拷贝步骤导致TCP报文处理的内存占用率高，处理效率受到限制。
发明内容
本申请提供了一种报文处理的方法、网络设备及相关设备，能够减少报文处理过程中的内存占用率，提高报文处理效率。
第一方面，提供了一种报文处理的方法，该方法应用于网络设备，该网络设备连接至服务器，该方法包括以下步骤：接收服务器中运行的应用的报文，从报文中分离出应用的数据，将应用的数据写入服务器中为应用分配的内存区间中。
实施第一方面描述的方法，网络设备在将报文写入服务器之前，先从报文中分离出应用的数据，然后将应用的数据写入应用内存，整个报文处理的过程无需重复拷贝应用的数据，降低了报文处理过程中的内存占用率，提高报文处理效率。
在一可能的实现方式中，该方法还包括：从报文中分离出报文头及应用的数据的元数据；将报文头和应用的数据的元数据存储至服务器为网络设备的驱动分配的内存区间中。
实施上述实现方式，网络设备从报文中分离出应用的数据后，再将报文头以及应用的数据的元数据也从报文中分离出，然后将应用的数据写入应用内存，将报文头以及应用的数据的元数据写入网卡驱动内存，这样，服务器的TCP应用可根据网卡驱动内存中的元数据确定应用的数据应存储的应用内存地址后，将应用的数据所在的内存页与该应用内存地址对应的内存页交换即可完成一次数据通信的过程，该方法避免了报文由网络设备传输至TCP应用内存页的过程中多次拷贝情况的发生，提高报文处理效率，降低内存占用率。
在一可能的实现方式中，从报文中分离出应用的数据，包括：根据定界模板从报文中分离出应用的数据，定界模板定义了应用的数据与报文中其他数据的分离规则。
应理解，相同应用的元数据是按照统一规则生成的，因此相同应用的定界模板相同，不同应用的定界模板可能相同也可能不同，定界模板也可根据元数据生成的规则确定。在应用启动后，应用可向网络设备下发与其应用类型对应的定界模板。
可选地，如果TCP应用的元数据长度统一为L1，而报文的报文头长度也是固定的，那么报文头之后长度为L1的数据即为元数据，根据元数据确定应用的数据长度L2，那么元数据之后长度为L2的数据即为应用的数据，从而将应用的数据从报文中分离。
可选地，如果TCP应用的元数据统一以某个分隔符结尾，比如分隔符可以是换行符、空格符、冒号等等，那么定界模板可以根据该分隔符获得元数据，再根据元数据描述的应用的数据长度，将应用的数据从报文中分离。同理，报文的报文头长度是固定的，那么报文头之后、分隔符之前的数据即为元数据，根据该元数据确定应用的数据长度L2后，元数据之后长度为L2的数据即为应用的数据，从而将应用的数据从报文中分离。
实施上述实现方式，TCP应用根据元数据的生成规则生成定界模板后，将定界模板下发至网络设备，使得网络设备可以根据定界模板对报文进行拆分，将应用的数据从报文中分离，并将其写入应用内存，避免了报文写入服务器之后，再将应用的数据拷贝入应用内存的冗余步骤，且降低了报文处理过程中的内存占用，提高报文处理效率。
在一可能的实现方式中，接收服务器中运行的应用的报文包括：将属于同一个数据流的多个子报文聚合为报文，其中，属于同一个数据流的多个子报文的源网际互联协议IP地址和目的IP地址相同。
实施上述实现方式，网络设备将属于同一个数据流的多个报文聚合后，使得待处理的报文数量减少，网络设备将报文写入服务器的次数也随之减少，提高网络设备的报文处理效率。
在一可能的实现方式中，从报文中分离出应用的数据之前，方法还包括：确定报文包括一个数据流中的完整数据。
实施上述实现方式，对于支持TCP减负引擎(TCP Offload Engine.TOE)功能的网络设备，可在网络设备内部提前进行TCP协议处理，比如乱序处理、拥塞控制、重传等等，协议处理后的报文包括一个数据流中的完整数据。可以理解的，网络设备确定报文包括一个数据流中的完整数据之后，此时报文不会出现失序、重复等情况，此时再使用定界模板对报文进行拆分，可使得报文拆分的准确率提高。
第二方面，提供了一种报文处理的方法，该方法应用于服务器，该服务器与网络设备连接，该方法包括以下步骤：服务器接收网络设备发送的应用的数据，将其存储至服务器为应用分配的内存区间中，接收网络设备发送的报文头以及应用的数据的元数据，将其存储至服务器为网络设备的驱动分配的内存区间中，然后服务器确定服务器中为应用分配的内存区间中的应用的数据是否完整，并在该应用数据完整的情况下，根据应用的数据的元数据确定应用的数据的应用内存地址，将该地址对应的内存页与应用的数据所在的内存页交换，使得应用的数据存入该应用内存地址。
具体实现中，内存页交换可以是将指向应用的数据的指针与指向该应用内存地址的指针进行交换，或者将应用的数据所在的内存页的虚拟地址替换成该应用内存地址，本申请不对内存页交换的具体实现方式进行限定。
实施第二方面描述的方法，服务器中运行的TCP应用根据元数据，确认应用的数据的应用内存地址，然后将该应用内存地址对应的应用内存页与应用的数据所在的内存页进行交换，从而避免了多次拷贝应用的数据造成的资源浪费，提高报文处理效率。
在一种可能的实现方式中，服务器根据元数据确定应用的数据的应用内存地址之前，还 可根据定界模板，确定网络设备写入服务器的报文头、应用的数据的元数据以及元数据是同一个数据流的完整数据，确定是完整数据的情况下，再根据元数据确定应用的数据的应用内存地址，确定不是完整数据的情况下，服务器根据网络设备写入服务器的报文头、应用的数据的元数据以及元数据，获取应用的数据和应用的数据的应用内存地址，再将应用的数据拷贝至应用内存地址。
其中，由于网络设备接收到的报文可能存在乱序、重复等情况，此时网络设备将应用的数据从报文中分离时，可能会出现分离出的应用的数据不完整的情况，因此在网络设备将应用的数据写入服务器之后，应用可以根据定界模板再次确认应用的数据是否完整，这里的定界模板与网络设备拆分报文时使用的定界模板相同。举例来说，TCP应用可根据定界模板获取元数据，比如根据分隔符确定元数据，或者根据元数据的固定长度确定元数据，然后根据元数据获取应用的数据的长度，如果该长度与被写入应用内存中的应用的数据长度相同，表示网络设备对报文拆分正确，相反地，如果该数据长度与被写入应用内存中的应用的数据的长度不同，表示网络设备对报文拆分错误，应理解，上述TCP应用根据定界模板确定网络设备是否正确拆分报文的过程用于举例说明，本申请不对此进行限定。
实施上述实现方式，服务器使用定界模板重新对报文头、应用的数据的元数据以及应用的数据进行拆分，从而避免了由于网络设备拆分错误导致最终写入应用内存地址的数据不完整情况的发生，提高数据传输的可靠性。
第三方面，提供了一种网络设备，网络设备连接至服务器，网络设备包括：接收单元，用于接收服务器中运行的应用的报文；分离单元，用于从报文中分离出应用的数据；写入单元，用于将应用的数据写入服务器中为应用分配的内存区间中。
在一种可能的实现方式中，分离单元还用于从报文中分离出报文头及应用的数据的元数据；写入单元还用于将报文头和应用的数据的元数据存储至服务器为网络设备的驱动分配的内存区间中。
在一种可能的实现方式中，分离单元用于根据定界模板从报文中分离出应用的数据，定界模板定义了应用的数据与报文中其他数据的分离规则。
在一种可能的实现方式中，接收单元用于将属于同一个数据流的多个子报文聚合为报文，其中，属于同一个数据流的多个子报文的源网际互联协议IP地址和目的IP地址相同。
在一种可能的实现方式中，网络设备还包括确定单元，确定单元用于在分离单元从报文中分离出应用的数据之前，确定报文包括一个数据流中的完整数据。
第四方面，提供了一种服务器，该服务器与网络设备连接，该服务器包括：应用模块，用于接收网络设备发送的应用的数据，将其存储至服务器为应用分配的内存区间中，网卡驱动，用于接收网络设备发送的报文头以及应用的数据的元数据，将其存储至服务器为网络设备的驱动分配的内存区间中，内核协议栈，用于确定服务器中为应用分配的内存区间中的应用的数据是否完整，应用模块还用于在该应用数据完整的情况下，根据应用的数据的元数据确定应用的数据的应用内存地址，将该地址对应的内存页与应用的数据所在的内存页交换，使得应用的数据存入该应用内存地址。
在一种可能的实现方式中，应用模块还用于在交换单元根据元数据确定应用的数据的应用内存地址之前，根据定界模板，确定网络设备写入服务器的报文头、应用的数据的元数据以及元数据是同一个数据流的完整数据，确定是完整数据的情况下，再根据元数据确定应用的数据的应用内存地址，确定不是完整数据的情况下，服务器根据网络设备写入服务器的报文头、应用的数据的元数据以及元数据，获取应用的数据和应用的数据的应用内存地址，再 将应用的数据拷贝至应用内存地址。
第五方面，提供了一种报文处理系统，包括服务器和网络设备，其中，服务器用于实现如第二方面或第二方面任一种可能的实现方式中描述的方法的操作步骤，网络设备用于实现如第一方面或第一方面任一种可能的实现方式中描述的方法的操作步骤。
第六方面，提供了一种计算机程序产品，包括计算机程序，当计算机程序被计算设备读取并执行时，实现如第一方面或第二方面所描述的方法。
第七方面，提供了一种计算机可读存储介质，包括指令，当指令在计算设备上运行时，使得计算设备实现如第一方面或第二描述所描述的方法。
第八方面，提供了一种网络设备，包括处理器和通信接口，该通信接口用于接收报文，该处理器用于执行如第一方面描述的方法以对报文进行处理。
第九方面，提供了一种服务器，包括处理器和存储器，处理器执行存储区中的代码实现如第二方面描述的方法。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍。
图1是本申请提供的一种报文处理系统的结构示意图；
图2是相关技术中的一种报文处理方法的步骤流程示意图；
图3是本申请提供的一种报文处理方法的步骤流程示意图；
图4是本申请提供的一种应用场景下从报文中分离应用的数据的流程示意图；
图5是本申请提供的一种报文聚合的步骤流程示意图；
图6是本申请提供的报文处理方法在一应用场景下的步骤流程示意图；
图7是本申请提供的一种报文处理方法的步骤流程示意图；
图8是本申请提供的一种网络设备的结构示意图；
图9是本申请提供的一种网络设备的硬件结构示意图。
具体实施方式
首先，对本申请涉及的部分术语进行解释说明。值得注意的，本申请的实施方式部分使用的术语仅用于对本申请的具体实施例进行解释，而非旨在限定本申请。
协议栈(Protocol Stack)：又称协议堆叠，是计算机网络协议套件的一个具体地软件实现。协议套件中的一个协议通常只为一个目的而设计的，这样可以使得设计更容易。因为每个协议模块通常都要和上下两个其他协议模块通信，它们通常可以想象成是协议栈中的层。最低级的协议总是描述与硬件的物理交互。举例来说，三台电脑分别为A、B以及C，其中，电脑A和电脑B都有无线电设备，可以通过网络协议IEEE802.11通信，电脑B和电脑C通过电缆连接来交换数据，比如以太网，这样，电脑A与电脑C之间的数据通信只能通过电脑B来传输，而无法直接传输，为了解决这一问题，可以在两个协议之上建立一个新的协议，比如IP协议，这样就形成了两个协议栈，实现了电脑A和电脑C之间的数据通信。
TCP：TCP提供一种面向连接的可靠的字节流服务。TCP将用户数据打包构成待处理报文段，它发送数据时启动一个定时器，另一端接收到数据后进行确认，然后对失序的数据重新排列，丢弃重复的数据，因此基于TCP的数据通信具有较高的安全性和可靠性，被广泛应 用在对准确性要求相对较高的场景下，比如文件传输场景中，TCP应用可以基于TCP协议组中的文件传输协议(File Transfer Protocol,FTP)、超文本传输协议(Hyper Text Transfer Protocol,HTTP)实现文件传输的功能；再比如发送或接收邮件的场景中，TCP应用可以基于TCP协议组中的简单邮件传输协议(Simple Mail Transfer Protocol,SMTP)或者交互邮件访问协议(Interactive Mail Access Protocol,IMAP)实现邮件收发的功能。应理解，上述TCP应用的类型仅用于举例说明，本申请提供的报文处理方案适用于任何TCP应用，本申请不对TCP应用进行限定。
直接内存存取(Direct Memory Access,DMA)：设备直接与计算机内存进行报文处理，实现了DMA设备直接访问服务器内存，缩短了报文处理路径，不仅提升服务器的IO性能，也降低了CPU的负载压力。
内存页：将内存的地址空间人为地划分为大小相等的若干份，一份对应一个内存页，处理器以页为单位对内存进行写入和读取。
TOE：TOE一般由软硬两部分组件构成，将传统的TCP/IP协议栈的功能进行延伸，把网络数据流量的TCP协议处理工作全部转移到网卡的集成硬件中进行，服务器只承担TCP/IP应用的处理任务，从而减轻了服务器的处理压力。
首先，对本申请适用的应用场景进行解释说明。
图1是一种与网络相连的服务器的结构示意图，其中，该服务器100与网络设备200相连，网络设备200与网络300相连，当网络300中的其他服务器向该服务器100发送待处理报文时，网络设备200首先接收到该待处理报文，然后将该待处理报文发送至服务器100，服务器100对该待处理报文进行处理后，完成一次报文处理。
网络设备200是一个使得服务器100与网络300相连接的硬件设备，具体可以是网卡(Nic)，也可以是TOE网卡，本申请不对此具体限定。其中，一个服务器100可以与一个或者多个网络设备200相连，图1以服务器100与一个网络设备200相连为例进行了说明，本申请不对此进行限定。
服务器100是通用的物理服务器，例如，ARM服务器或者X86服务器，服务器100包括处理器110和内存120，其中，处理器110和内存120通过内部总线130相互连接，内部总线130可以是外设部件互连标准(Peripheral Component Interconnect,PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture,EISA)总线等。需要说明的，图1仅仅是服务器100的一种可能的实现方式，实际应用中，服务器100还可以包括更多或更少的部件，这里不作限制。
处理器110可以由至少一个通用处理器构成，例如中央处理器(Central Processing Unit,CPU)，或者CPU和硬件芯片的组合。上述硬件芯片可以是专用集成电路(Application-Specific Inegrated Circuit,ASIC)、可编程逻辑器件(Programmable Logic Device,PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(Complex Programmable Logic Device,CPLD)、现场可编程逻辑门阵列(Field-Programmable Gate Array，FPGA)、通用阵列逻辑(Generic Array Logic,GAL)或其任意组合。处理器110执行各种类型的数字存储指令，例如存储在内存120中的软件或者固件程序，它能使服务器1提供较宽的多种服务。
内存120可以是易失性存储器(Volatile Memory)，例如随机存取存储器(Random Access Memory,RAM)、动态随机存储器(Dynamic RAM，DRAM)、静态随机存储器(Static RAM，SRAM)、同步动态随机存储器(Synchronous Dynamic RAM，SDRAM)、双倍速率同步动态 随机存储器(Double Data Rate RAM，DDR)、高速缓存(Cache)等等，内存120还可以包括上述种类的组合。内存120包括程序代码121，其中，程序代码121可以包括一个或多个软件模块，比如图1所示的TCP应用1211、内核协议栈1212以及网卡驱动1213的代码，TCP应用1211是基于TCP协议实现各种功能的应用模块，内核协议栈1212可以理解为是操作系统的一部分，主要用于处理内核协议栈中的报文，网卡驱动1213是一种可以使CPU控制和使用网络设备200(比如Nic或者TOE网卡)的特殊程序，相当于网络设备200的硬件接口，操作系统通过该接口可以控制网络设备200。内存120还包括应用内存1221和网卡驱动内存1222，其中，网卡驱动内存1222是网卡驱动1213向内存120申请的一段内存，内核协议栈1212可对网卡驱动内存1222中的数据进行处理，应用内存1221是TCP应用1211向内存120申请的一段内存，TCP应用1211可对应用内存1221中的数据进行处理。
下面结合图2，对相关技术中图1所示的服务器100接收并处理来自网络300的TCP报文的具体流程进行解释说明。
当其他服务器向该服务器100发送数据时，网络设备200最先接收到来自网络300的TCP报文，TCP报文包括报文头、元数据以及应用的数据，其中，如图2所示，报文头位于元数据之前，元数据位于应用的数据之前。
其中，报文头至少包括四元组(源IP地址、目的IP地址、源端口、目的端口)，应理解，同一个TCP流(Stream)的待处理报文，其报文头中的四元组是相同的。
元数据至少包括应用的数据的长度和应用的数据的控制信息，其中，控制信息用于供TCP应用确定该应用的数据的应用内存地址，举例来说，控制信息可以包括上下文信息，TCP应用接收到报文后，TCP应用可根据上下文信息，确定该报文中的应用的数据D2的上下文数据D1和D3所在的应用内存地址分别为Add1和Add3，从而确定应用的数据的应用内存地址为Add2，其中，Add1、Add2和Add3是一段连续的内存。具体实现中，TCP应用根据元数据确定的应用内存地址可以是应用内存1221中的某个内存页的地址，本申请不对元数据和根据元数据确定的应用内存地址的具体形式进行限制。
应用的数据是其他服务器向服务器100发送的原始数据，即负载(Payload)。应理解，通常在数据传输时，为了使数据传输更加可靠，将会把原始数据的头部和/或尾部增加一定的辅助信息，比如数据量的大小、校验位等等，使得原始数据在传输过程中不易丢失，原始数据加上辅助信息就形成了传输通道的基本传输单元，也就是数据帧、数据包或者TCP/IP报文等等，其中的原始数据即为应用的数据。
应理解，由于每个TCP报文的大小是固定的，一般为1480字节左右，如果要一次性发送大量数据，数据需要进行分片处理后，分成多个报文进行传输，比如一个10MB的文件，需要发送7100多个报文，发送侧服务器发送报文的时候，TCP协议为每个报文编号(Sequence Number,SEQ)，以便接收侧服务器按照SEQ顺序将收到的多个报文还原出10M的原文件。图2以应用的数据1被分片为3个应用的数据(应用的数据1A、应用的数据1B以及应用的数据1C)为例进行了说明，其中，报文1包括报文头、元数据1以及应用的数据1A，报文2包括报文头以及应用的数据1B，报文3包括报文头、应用的数据1C、元数据2以及应用的数据2，元数据1包括应用的数据1(应用的数据1A、应用的数据1B以及应用的数据1C)的控制信息，TCP应用可以根据该控制信息，确定应用的数据1的应用内存地址，元数据2包括应用的数据2的控制信息，TCP应用可以根据该控制信息确定应用的数据2的应用内存地址。假设报文1、报文2以及报文3属于同一个TCP流，那么报文1、报文2以及报文3 的报文头的四元组相同，但是报文1、报文2以及报文3中应用的数据1的SEQ不同。
在上述应用场景中，如图2所示，当网络设备200接收到报文1、报文2以及报文3之后，图1所示的服务器接收并处理来自网络300的TCP报文的具体流程包括以下步骤：
步骤1、网络设备200通过DMA技术将报文1、报文2以及报文3写入服务器100的网卡驱动内存1222。
具体实现中，网卡驱动内存1222是网卡驱动1213向服务器100申请的一段内存，网络设备200将报文DMA到网卡驱动内存1222之后，可向内核协议栈发送协议处理请求，该协议处理请求中包括该报文的地址。
步骤2、内核协议栈1212对网卡驱动内存1222中的报文进行TCP协议处理。
具体地，内核协议栈1212接收到网卡驱动发送的协议处理请求之后，内核协议栈1212可根据协议处理请求中的报文的地址，对该报文进行TCP协议处理，以确保数据通信的完整性。
具体实现中，由于报文头的长度和格式是固定的，比如报文头的长度为20kb，内核协议栈1212可以从该网卡内存页的头部开始读取20kb的数据，从而获得该报文的报文头，然后根据该报文头完成拥塞控制、乱序处理、重传等等TCP协议处理步骤，对失序的数据重新排列，丢弃重复的数据，保证该报文头对应的TCP流中的全部报文都已写入服务器中。
例如图2所示，内核协议栈1212对报文头进行TCP协议处理，根据报文头中的SEQ确定报文1～3是否都已被写入网卡驱动内存1222，如果报文2遗失，内核协议栈1212可以向网络设备发送重传报文2的请求，在确认没有报文遗失的情况下，向TCP应用发送应用处理请求，该应用处理请求中包括上述报文的地址。
步骤3、TCP应用对网卡驱动内存1222中的元数据进行处理，确定应用的数据的应用内存地址。
具体地，每个应用的数据的元数据是按照固定格式生成的，可根据元数据的格式特点来获取元数据。如果元数据的长度是固定的，TCP应用接收到内核协议栈发送的应用处理请求后，首先根据元数据的长度，比如40kb，从报文头之后读取40kb的数据至元数据内存，该40kb的数据即为元数据，然后再对元数据进行解析，确定应用的数据的长度和应用内存地址；如果元数据的尾部存在作为分隔符的特殊符号，比如换行符、空格符、冒号等等，TCP应用接收到内核协议栈发送的应用处理请求后，首先确定分隔符的位置，然后将报文头之后、分隔符之前的数据读取至元数据内存，再对其进行解析，确定应用的数据的长度和应用内存地址。其中，该元数据内存可以是TCP应用提前向内存申请的一段用于临时存储元数据的内存，TCP应用对应用元数据内存中的元数据进行解析后，可以将内存中的元数据删除，释放出存储空间。应理解，图3仅用于举例说明，具体实现中，元数据还可以包括更多的内容，这里不一一举例说明。
步骤4、TCP应用将网卡驱动内存1222中的应用的数据拷贝入应用内存地址。
仍以图3为例，TCP应用根据元数据确认应用的数据的长度为1400kb、应用的数据的应用内存地址为Add1后，可以将40kb元数据之后的1400kb数据拷贝至Add1，该1400kb数据即为TCP应用所需的应用的数据。
综上可知，当服务器100外接的网络设备200接收到TCP报文后，网络设备200将报文DMA到服务器提前为网卡驱动划分的网卡驱动内存1222中，服务器100的内核协议栈1212先对网卡驱动内存1222中的报文进行处理后，再将报文中的应用的数据拷贝至应用内存1221中。简单来说，报文需要先写入网卡内存，再由TCP应用从网卡内存中拷贝报文中的应用的 数据至应用内存，多余的拷贝步骤导致TCP报文处理的内存占用率高，处理效率受到限制。
为了解决上述TCP报文处理时内存占用率高、处理效率受限的问题，本申请提供了一种报文处理方法。如图3所示，该方法包括以下步骤：
S310：网络设备200接收服务器100中运行的应用的报文。
在一实施例中，上述应用的报文可以是网络设备200接收到的一个报文，比如图2中的报文1、或者报文2、或者报文，也可以是由网络设备200将接收到的属于同一个数据流(比如TCP流)的多个子报文进行报文聚合处理后得到的，其中，属于同一个数据流的多个子报文的源IP地址和目的IP地址相同。网络设备200可以根据每个报文的TCP头中的四元组确定同一个TCP流的多个报文，其中，同一个TCP流中的多个报文的四元组相同，然后将属于同一个TCP流的多个报文聚合为一个报文，该报文也是由TCP报文头、元数据和应用的数据构成，其中，每个元数据后面紧接着与其对应的应用的数据，每个应用的数据后面紧接着下一个应用的数据的元数据。
假设网络设备200接收到了如图2所示的报文1～3，且报文1～报文3是属于同一个TCP流的三个报文，网络设备200对该报文1～3进行聚合获得的报文0可以如图5所示，其中，应用的数据1包括了报文1中的应用的数据1A、报文2中的应用的数据1B以及报文3中的应用的数据1C。具体实现中，网络设备200可以通过大量接收卸载(Large Receive Offload,LRO)、接收方向聚合(Receive Side Coalescing,RSC)等算法实现报文聚合，这里不作具体限定。可以理解的，网络设备200将属于同一个TCP流的多个报文聚合后，可以减少网络设备200将报文写入服务器的次数，提高网络设备200的报文处理效率。
S320：网络设备200从报文中分离出应用的数据。
在一实施例中，网络设备200可根据定界模板从报文中分离出应用的数据，其中，该定界模板定义了应用的数据与报文中其他数据的分离规则。应理解，由于网络设备200接收到的报文是未经过TCP协议处理的报文，因此该报文可能是乱序、或者丢包的报文，该种情况下使用定界模板对报文进行拆分，可能会出现拆分错误的情况，也就是拆分出的应用的数据不完整的情况，比如只包括部分应用的数据，或者除了应用的数据还包括其他数据的情况。在网络设备200对报文拆分正确的情况下，分离出的应用的数据只包括完整的应用的数据，剩余报文只包括完整的报文头和完整元数据。
其中，相同TCP应用的元数据是按照统一规则生成的，相同TCP应用的定界模板相同，不同TCP应用的定界模板可能相同也可能不同，因此在步骤S310之前，TCP应用启动时，可以向网络设备200下发与其应用类型对应的定界模板。具体地，TCP应用可向网络设备200的驱动(比如网卡驱动)发送接口调用请求，网络设备200的驱动响应于该请求，向TCP应用提供网络设备200的接口，TCP应用调用该接口将定界模板下发至网络设备200。
为了便于本申请更好地被理解，示例性地，下面对两种元数据格式对应的定界模板进行举例说明。
第一种定界模板：如果TCP应用的元数据长度统一为L1，而报文的报文头长度也是固定的，那么报文头之后长度为L1的数据即为元数据，根据元数据确定应用的数据长度L2，那么元数据之后长度为L2的数据即为应用的数据，从而将应用的数据从报文中分离。
例如，图4是一种报文格式的示意图，在图4所示的例子中，应用1的应用的数据的元数据的长度为40kb，其中，前10kb描述了该应用的数据的长度，比如应用的数据长度为1400kb，后30kb描述了该应用的数据的控制信息。那么TCP应用接收到内核协议栈发送的 应用处理请求后，首先根据元数据的长度40kb，从报文头之后读取40kb的数据至应用内存，该40kb的数据即为应用的数据1的元数据，然后再根据该元数据的前10kb内容得到应用的数据长度为1400kb，再根据该元数据的后30kb中的控制信息，确定该应用的数据的应用内存地址Add1。应理解，图4仅用于举例说明，具体实现中，元数据还可以包括更多的内容，这里不一一举例说明。
第二种定界模板：如果TCP应用的元数据统一以某个分隔符结尾，比如分隔符可以是换行符、空格符、冒号等等，那么定界模板可以根据该分隔符获得元数据，再根据元数据描述的应用的数据长度，将应用的数据从报文中分离。同理，报文的报文头长度是固定的，那么报文头之后、分隔符之前的数据即为元数据，根据该元数据确定应用的数据长度L2后，元数据之后长度为L2的数据即为应用的数据，从而将应用的数据从报文中分离。
应理解，上述两种定界模板仅用于举例说明，具体实现中，不同TCP应用的定界模板可以根据该TCP应用的元数据的格式来确定，本申请不对此进行具体限定。
值得注意的是，如果报文是未经过报文聚合处理的报文时，比如图5实施例中的报文1～报文3，使用的定界模板对该类报文进行拆分之前，需要先识别报文头，确定属于同一个TCP流的多个报文，然后再使用定界模板，对同一个TCP流中的多个报文进行拆分，从同一个TCP流中每个报文中分离出应用的数据，然后将同一个TCP流中分离出的应用的数据全部写入网卡驱动内存1222，报文头和元数据全部写入应用内存1221。
仍以图5所示的3个报文为例，假设元数据长度固定为40kb，每个报文的总长度为1400kb，网络设备200没有对报文1～3进行聚合，在该应用场景下，使用上述第一种定界模板拆分报文时，网络设备200可先根据报文头确定报文1～3属于同一个TCP流，并且该TCP流中报文的读取顺序为报文1、报文2和报文3，然后读取报文1中的元数据，确定应用的数据1的长度为3000KB之后，读取报文1中的应用的数据1A(假设应用的数据长度为1000KB)，此时应用的数据1还有2000KB未读取，可从报文2再读应用的数据1B(假设应用的数据长度为1400KB)，此时应用的数据1还有600KB未读取，可继续从报文3读取应用的数据1C(假设应用的数据长度为600KB)，从而获得长度为3000KB的应用的数据1，继续读取40kb数据即可获得应用的数据2的元数据，以此类推，最后将元数据1以及报文头写入网卡驱动缓内存，将应用的数据1A、应用的数据1B以及应用的数据1C写入应用内存。第二种定界模板的读取方法类似，这里不再重复赘述。
S330：网络设备200将应用的数据写入服务器100中为应用分配的内存区间中，即图1所示的应用内存1221。
具体实现中，网络设备200可通过DMA技术将分离出的应用的数据写入应用内存1221。其中，应用的数据的存储地址可以是服务器的网卡驱动事先向网络设备200发送的，该存储地址是TCP应用根据应用内存1221的空闲情况确定后，向网卡驱动发送的地址。
S340：网络设备200从报文中分离出报文头以及应用的数据的元数据。
可以理解的，报文包括报文头、应用的数据以及元数据，步骤S320使用定界模板分离出应用的数据后，也可分离出报文头以及应用的数据的元数据，定界模板的具体描述可参考前述内容的步骤S320，这里不重复赘述。
S350：网络设备200将报文头和应用的数据的元数据写入服务器100中为网络设备200的驱动分配的内存区间中，即图1所示的网卡驱动内存1222。
具体实现中，网络设备200可通过DMA技术将报文头和应用的数据的元数据写入网卡驱动内存1222。其中，报文头和应用的数据的元数据的存储地址可以是服务器的网卡驱动事 先向网络设备200发送的，该存储地址是网卡驱动根据网卡驱动内存1222的空闲情况确定的。
应理解，步骤S340～步骤S350可以是与步骤S320～步骤S330同时发生的，也可以是先后发生的，本申请不作具体限定。
S360：服务器100的内核协议栈对报文头、应用的数据的元数据以及应用的数据进行TCP协议处理，其中，TCP协议处理用于使得报文包括一个数据流中的完整数据。
具体地，内核协议栈可以对报文头进行乱序处理、拥塞处理、重传等等，如果出现乱序，则根据报文头对应用的数据进行重新排列，如果出现数据重复，则根据报文头丢弃应用的数据中重复的数据，使得协议处理后的报文头、应用的数据以及应用的数据的元数据包括同一个TCP数据流中的全部数据，确保该TCP流中的完整元数据和完整应用的数据都已被写入服务器，提高报文处理的可靠性。
仍以图5为例，假设报文1～3同属一个TCP流，如果网络设备先接收到了报文1和报文2，还未接收到报文3时，网络设备将报文1和报文2聚合为待处理报文11，并通过TCP应用事先下发至网络设备的定界模板，将应用的数据X2从待处理报文11中分离，然后将该应用的数据X2写入应用内存1221，将剩余报文X1写入网卡驱动内存1222，内核协议栈从剩余报文Y1中获取TCP报文头，确定该TCP流中包括3个报文，而写入服务器的应用的数据X2和剩余报文Y1只包括报文1和报文2，因此报文3需要重传，内核协议栈可以向网络设备200发送重传报文3的请求，网络设备200响应于该请求，在接收到报文3时，根据定界模板将报文3中的应用的数据X2分离出来，然后将应用的数据X2写入应用内存1221，剩余报文Y2写入网卡驱动内存1222，使得服务器100获得该TCP流的全部报文。应理解，上述举例仅用于说明，并不能构成具体限定。
需要说明的，内核协议栈的数据结构通常为套接字缓存(Soket Buffer,SKB)结构，该结构可以通过挂多个指针项的形式，对多个不同地址的数据进行处理，比如处理报文1的SKB1的指针1指向网卡驱动内存1222，这样，内核协议栈可以通过指针1对网卡驱动内存中的报文头进行TCP协议处理，同时，SKB1的指针2还可以指向应用内存1221，通过指针2监控应用内存1221中的数据，确定应用的数据已写入该内存中，并在报文头出现数据重复和乱序等情况下，对应用的数据进行相应的调整，如果出现报文漏发或者报文乱序的情况，可以向网络设备发送重传请求，直至全部报文都已写入服务器。步骤S330未描述的内容可以参考前述图2实施例中的步骤2，这里不重复赘述。
S370：内核协议栈向TCP应用发送确认请求，该确认请求中携带有应用的数据的元数据在网卡驱动内存中的地址以及应用的数据在应用内存中的地址信息。
S380：TCP应用确定应用的数据是否是同一个TCP流中完整的应用的数据，如果是完整的应用的数据的情况下，执行步骤S390，如果不是完整的应用的数据的情况下执行步骤S311。
其中，TCP应用可以根据定界模板再次确认应用的数据是否完整，这里的定界模板与步骤S310处网络设备200使用的定界模板相同。举例来说，TCP应用可根据定界模板获取元数据，比如根据分隔符确定元数据，或者根据元数据的固定长度确定元数据，然后根据元数据获取应用的数据的长度，如果该长度与被写入应用内存中的应用的数据长度相同，表示网络设备200在步骤S310处拆分正确，即可执行步骤S390，相反地，如果该数据长度与被写入应用内存中的应用的数据的长度不同，表示网络设备200在步骤S310处拆分错误，即可执行步骤S311。应理解，上述TCP应用根据定界模板确定网络设备200是否对报文拆分正确的过程用于举例说明，本申请不对此进行限定。
S390：TCP应用根据元数据确定应用的数据的应用内存地址，将该应用内存地址对应的 内存页与应用的数据所在的内存页进行交换。
具体实现中，内存页交换可以是将指向应用的数据的指针与指向应用内存地址的指针进行交换，或者将应用的数据所在的内存页的虚拟地址替换成该应用内存地址，本申请不对内存页交换的具体实现方式进行限定。
可以理解的，TCP应用根据元数据，确认应用的数据的应用内存地址，然后将该应用内存地址对应的应用内存页与应用的数据所在的内存页进行交换，从而避免了多次拷贝应用的数据造成的资源浪费，提高报文处理效率。
值得注意的，如果应用的数据的存储地址与根据元数据确定的应用内存地址相同，那么可以不执行步骤S390，从而进一步提升报文处理效率。
S311：TCP应用从获取应用的数据的元数据以及应用的数据，对元数据进行解析后，确定应用的数据的应用内存地址，然后将应用的数据拷贝入该应用内存地址。具体可以参考图1实施例中的步骤3-步骤4，这里不重复赘述。
在一实施例中，TCP应用记录步骤S380的确认结果，也就是网络设备200是否拆分正确的确认结果，如果连续拆分错误的次数超过阈值，TCP应用可以向网络设备200发送重新定界请求，该重新定界请求包括拆分错误的报文所属的TCP流的信息，网络设备200响应于该请求，重新对该TCP流中的全部报文进行聚合和拆分，从而避免了连续拆分错误的场景下，TCP应用重复执行步骤S370～步骤S311这一情况的发生，进一步提高报文处理效率，降低服务器处理报文的内存占用率。
具体实现中，TCP应用向网络设备200发送重新定界请求时，可以首先向网卡驱动发送接口调用请求，网卡驱动响应于该请求，将网卡的接口提供给TCP应用，TCP应用调用该接口下发同步命令(SYNC)，其中，SYNC命令用于将内存缓冲区中的数据立即强制写入网络设备200，使得网络设备200执行重启定界操作，从而避免了连续报文拆分错误的场景下，TCP应用重复执行步骤S370～步骤S11这一情况的发生，进一步提高报文处理效率，降低服务器处理报文的内存占用率。
仍以图2所示的应用场景为例，网络设备200接收到了报文1～3，假设报文1～3属于同一个TCP流，网络设备200对该报文1～3进行报文聚合，获得如图5所示的报文0之后，使用上述步骤S310～步骤S311对该报文1～3进行处理的具体流程可以如图6所示。
如图6所示，首先，网络设备200根据定界模板，通过步骤S320和步骤S340将应用的数据从报文0中分离出来，将报文头和元数据从报文0中分离出来，如图6所示，应用的数据1和应用的数据2作从报文0中分离出来，报文头、元数据1和元数据2从报文0中分离出来。参考前述内容可知，定界模板根据元数据的格式确定，比如元数据的长度是L1，网络设备200从报文头之后读取长度为L1的数据，获得元数据1，然后根据元数据确定应用的数据长度L2，再从元数据1之后读取长度为L2的数据，获得应用的数据1，然后再从应用的数据1之后读取长度为L1的数据，获得元数据2，根据元数据2确定应用的数据2的长度L3后，最后从元数据2之后读取长度为L3的数据，获得应用的数据2。应理解，上述举例仅用于说明，并不能构成具体限定。
其次，网络设备200通过步骤S330和步骤S350将应用的数据DMA至服务器的网卡驱动内存1222，将报文头和元数据DMA至服务器的应用内存1211，其中，应用的数据1写入应用内存页Y1，应用的数据2写入应用内存页Y2。需要说明的，网络设备200在将应用的数据写入应用内存1221时，可以如图6所示，将不同的应用的数据写入不同的应用内存页，也可以将全部应用的数据写入同一个应用内存页，也就是将应用的数据1和应用的数据2都 写入应用内存页Y1或者应用内存页Y2，具体可以根据TCP应用的处理逻辑确定，本申请不对此进行限定。
最后，服务器100执行步骤S360，对网卡驱动内存1222中的报文头进行处理，比如乱序处理、重传等等，对失序的数据重新排列，丢弃重复的数据，确保该TCP流中的完整元数据和完整应用的数据都已被写入服务器之后，执行步骤S370向TCP应用发送确认请求，TCP应用响应于该确认请求，根据定界模板执行步骤S380确定写入应用内存的应用的数据是同一个TCP流中的完整的应用数据之后，TCP应用执行步骤S390，根据元数据1确定应用的数据1的应用内存地址对应的应用内存页Y3，然后将应用的数据1所在的应用内存页Y1与该应用内存页Y3进行交换，根据元数据2确定应用的数据2的应用内存页Y4，将应用的数据2所在的应用内存页Y2与该应用内存页Y4进行交换，从而完成一次报文的处理过程。该过程无需重复拷贝应用的数据，使得服务器处理TCP报文的内存占用率降低，提高报文处理效率。
参考前述内容可知，为了降低服务器的处理压力，部分网络设备200支持TOE功能，该功能使得在网络设备200接收到报文后可先对报文进行TCP协议处理，然后再将处理后的完整的TCP流写入服务器100。对于该类TOE网卡使用本申请提供的报文处理方法的具体流程可以如图7所示。
S410：网络设备200对服务器100中运行的应用的待处理报文进行TCP协议处理，比如乱序处理、重传、拥塞处理等等，使得待处理报文包括一个数据流中的完整数据，具体描述可参考前述内容中的步骤S360，这里不重复赘述，其中，该网络设备200可以是支持TOE功能的TOE网卡。
具体实现中，上述服务器100中运行的应用的报文可以是单个子报文，比如图5实施例中的报文1～报文3，也可以是网络设备200对接收到的多个子报文进行报文聚合后，获得的待处理报文，比如图5实施例中的报文0。值得注意的是，网络设备200可对接收到的所有报文进行TCP协议处理后，将属于同一个TCP流的多个子报文聚合为报文，也可以将接收到的多个子报文聚合为待处理报文后，再对待处理报文进行TCP协议处理，本申请不对此进行限定。其中，TCP协议处理的具体描述可以参考前述内容中的步骤2、步骤S330等等，这里不重复赘述。
S420：网络设备200从报文中分离出应用的数据。
具体实现中，网络设备200可根据定界模板从报文中分离出应用的数据，其中，定界模板是服务器100中的TCP应用在步骤S410之前，通过调用网卡驱动的接口向网络设备200下发的模板，定界模板的描述可以参考前述内容中的步骤S310，这里不重复赘述。
应理解，由于网络设备200支持TOE功能，在步骤S420已对报文进行了TCP协议处理，因此报文不会存在失序、重复等情况，使用定界模板对从报文中分离出的应用的数据，不会出现拆分错误的情况。
S430：网络设备200将应用的数据写入服务器中为应用分配的内存区间中，具体可以是应用内存1221，该步骤的描述可以参考前述内容的步骤S330，这里不重复赘述。具体实现中，网络设备200可通过DMA技术将应用的数据写入应用内存1221。
S440：从报文中分离出报文头以及应用的数据的元数据。该步骤的具体描述可参考前述内容的步骤S340，这里不重复赘述。
S450：将报文头以及应用的数据的元数据写入服务器为网络设备的驱动分配的内存区间 中个，具体可以是网卡驱动内存1222。具体实现中，网络设备200可通过DMA技术将报文头以及应用的数据的元数据写入网卡驱动内存1222。该步骤的描述可参考前述内容的步骤S350，这里不重复赘述。
S460：内核协议栈向TCP应用发送交换请求，其中，该交换请求包括分离得到的应用的数据的地址以及元数据的地址。具体可参考前述内容中的步骤S370，这里不重复赘述。
S470：TCP应用根据元数据确定应用的数据的应用内存地址，将应用的数据所在的内存页和该应用内存地址对应的内存页交换。具体地，TCP应用可以根据交换请求中元数据的地址获取元数据，然后根据元数据确定应用的数据的应用内存地址，其中，根据元数据确定应用的数据的应用内存地址以及内存页交换的具体描述可以参考签署内容中的步骤S390，这里不重复赘述。
应理解，如果该应用内存地址与分离出的应用的数据所在的内存地址相同，那么可以不再进行内存页交换，进一步提升报文处理的效率。
可选地，TCP应用可在根据元数据确定应用的数据的应用内存地址之前，通过定界模板二次确认网络设备200是否拆分正确，在拆分正确的情况下，再根据元数据确定应用的数据的应用内存地址，将应用的数据所在的内存页和该应用内存地址对应的内存页交换，从而避免由于其他原因比如DMA出错导致应用的数据不完整，内存页交换后TCP应用得到错误数据情况的发生，进一步提高本申请提供的报文处理方法的可靠性。
综上可知，本申请提供的报文处理方法，服务器提前向网络设备下发定界模板，使得网络设备在向服务器发送报文之前，先通过定界模板从报文中拆分出应用的数据和元数据，并将元数据写入网卡驱动内存，将应用的数据写入应用内存，使得服务器的TCP应用根据网卡驱动内存中的元数据确定应用的数据的应用内存地址后，将应用的数据所在的内存页与该应用内存地址对应的内存页交换即可完成一次数据通信的过程，该方法避免了报文由网络设备传输至TCP应用内存页的过程中多次拷贝情况的发生，提高报文处理效率，降低内存占用率。
上述详细阐述了本申请实施例的方法，为了便于更好的实施本申请实施例上述方案，相应地，下面还提供用于配合实施上述方案的相关设备。
图8是本申请提供的一种网络设备200的结构示意图，该网络设备200应用于如图1所示的报文处理系统，其中，网络设备200与服务器100连接，如图8所示，网络设备200可包括接收单元810、分离单元820以及写入单元830。
接收单元810用于接收服务器中运行的应用的报文，具体实现方式，请参考上述图3所示实施例中步骤S310以及上述图4实施例中步骤S410的详细描述，这里不重复赘述；
分离单元820用于从报文中分离出应用的数据，具体实现方式，可参考上述图3实施例中步骤S320以及图4实施例中步骤S420的详细描述，这里不重复赘述；
写入单元830用于将应用的数据写入服务器中为应用分配的内存区间中，具体实现方式，可参考上述图3实施例的步骤S330以及图4实施例的步骤S430的详细描述，这里不重复赘述。
在一实施例中，分离单元820还用于从报文中分离出报文头及应用的数据的元数据；写入单元还用于将报文头和应用的数据的元数据存储至服务器为网络设备的驱动分配的内存区间中，具体实现方式，可参考上述图3实施例中步骤S340～步骤S350，以及图4实施例中步骤S440～步骤S450的详细描述，这里不重复赘述。
在一实施例中，分离单元820用于根据定界模板从报文中分离出应用的数据，定界模板 定义了应用的数据与报文中其他数据的分离规则，具体实现方式，可参考上述图3实施例中的步骤S310，以及图4实施例中的步骤S420中关于网络设备200使用定界模板将应用的数据从报文中分离这一步骤的详细描述，这里不重复赘述。
在一实施例中，接收单元810用于将属于同一个数据流的多个子报文聚合为报文，其中，属于同一个数据流的多个子报文的源网际互联协议IP地址和目的IP地址相同，具体实现方式，可参考上述图3实施例中的步骤S310以及图4实施例中的步骤S410关于报文聚合步骤的详细描述，这里不重复赘述。
在一实施例中，当网络设备200是支持TOE功能的网卡时，网络设备200还包括确定单元840，确定单元840用于在分离单元从报文中分离出应用的数据之前，确定报文包括一个数据流中的完整数据。具体可以对报文进行TCP协议处理，比如拥塞控制、重传、乱序处理等等，具体实现方式，可参考上述图4实施例中的步骤S410，这里不重复赘述。
应理解，图8所示的网络设备200的内部的单元模块也可以有多种划分，各个模块可以是软件模块，也可以是硬件模块，也可以是部分软件模块部分硬件模块，本申请不对其进行限制，图8是一种示例性的划分方式，本申请不作具体限定。
可以理解的，本申请提供的网络设备可提前接收由服务器下发的定界模板，使得网络设备在向服务器发送报文之前，先通过定界模板从报文中拆分出应用的数据和元数据，并将元数据写入网卡驱动内存，将应用的数据写入应用内存，使得服务器的TCP应用根据网卡驱动内存中的元数据确定应用的数据的应用内存地址后，将应用的数据所在的内存页与该应用内存地址对应的内存页交换即可完成一次数据通信的过程，该方法避免了报文由网络设备传输至TCP应用内存页的过程中多次拷贝情况的发生，提高报文处理效率，降低内存占用率。
图9是本申请提供的一种网络设备200的硬件结构示意图，如图9所示，网络设备200包括：处理器910、通信接口920以及存储器930。其中，处理器910、通信接口920以及存储器930可以通过内部总线940相互连接，也可通过无线传输等其他手段实现通信。本申请实施例以通过总线940连接为例，总线940可以是外设部件互连标准(Peripheral Component Interconnect,PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture,EISA)总线等。总线940可以分为地址总线、数据总线、控制总线等。为便于表示，图9中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
处理器910可以由至少一个通用处理器构成，例如中央处理器(Central Processing Unit，CPU)，或者CPU和硬件芯片的组合。上述硬件芯片可以是专用集成电路(Application-Specific Integrated Circuit，ASIC)、可编程逻辑器件(Programmable Logic Device，PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(Complex Programmable Logic Device，CPLD)、现场可编程逻辑门阵列(Field-Programmable Gate Array，FPGA)、通用阵列逻辑(Generic Array Logic，GAL)或其任意组合。处理器910执行各种类型的数字存储指令，例如存储在存储器930中的软件或者固件程序，它能使网络设备200提供较宽的多种服务。
存储器930用于存储程序代码，并由处理器910来控制执行，以执行上述图1-图7中任一实施例中网络设备200的处理步骤。程序代码中可以包括一个或多个软件模块。这一个或多个软件模块可以为图8所示实施例中提供的软件模块，如接收单元，分离单元以及写入单元，其中，接收单元，可用于接收服务器中运行的应用的报文；分离单元，可用于从报文中分离出应用的数据；写入单元，用于可将应用的数据写入服务器中为应用分配的内存区间中，具体可用于执行前述方法的步骤S310-步骤S350、步骤S410-步骤S450及其可选步骤，还可 以用于执行图2-图7实施例描述的其他由网络设备200执行的步骤，这里不再进行赘述。
存储器930可以包括易失性存储器(Volatile Memory)，例如随机存取存储器(Random Access Memory，RAM)；存储器1030也可以包括非易失性存储器(Non-Volatile Memory)，例如只读存储器(Read-Only Memory，ROM)、快闪存储器(Flash Memory)、硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；存储器930还可以包括上述种类的组合。存储器930可以存储有程序代码，具体可以包括用于供处理器910执行图2-图7实施例描述的其他步骤的程序代码，这里不再进行赘述。
通信接口920可以为有线接口(例如以太网接口)，可以为内部接口(例如高速串行计算机扩展总线(Peripheral Component Interconnect express,PCIe)总线接口)、有线接口(例如以太网接口)或无线接口(例如蜂窝网络接口或使用无线局域网接口)，用于与其他服务器或模块进行通信，具体实现中，通信接口920可用于接收报文，以供处理器910对该报文进行处理。
需要说明的，图9仅仅是本申请实施例的一种可能的实现方式，实际应用中，网络设备还可以包括更多或更少的部件，这里不作限制。关于本申请实施例中未示出或未描述的内容，可参见前述图2-图7实施例中的相关阐述，这里不再赘述。
本申请实施例还提供一种计算机可读存储介质，计算机可读存储介质中存储有指令，当其在处理器上运行时，图2-图7所示的方法流程得以实现。
本申请实施例还提供一种计算机程序产品，当计算机程序产品在处理器上运行时，图2-图7所示的方法流程得以实现。
上述实施例，可以全部或部分地通过软件、硬件、固件或其他任意组合来实现。当使用软件实现时，上述实施例可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括至少一个计算机指令。在计算机上加载或执行计算机程序指令时，全部或部分地产生按照本发明实施例的流程或功能。计算机可以为通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line,DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含至少一个可用介质集合的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(Digital Video Disc,DVD)、或者半导体介质。半导体介质可以是SSD。
以上，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (11)

  1. 一种报文处理方法，其特征在于，所述方法应用于网络设备，所述网络设备连接至服务器，所述方法包括：
    接收服务器中运行的应用的报文；
    从所述报文中分离出所述应用的数据；
    将所述应用的数据写入所述服务器中为所述应用分配的内存区间中。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    从所述报文中分离出报文头及所述应用的数据的元数据；
    将所述报文头和所述应用的数据的元数据存储至所述服务器为所述网络设备的驱动分配的内存区间中。
  3. 根据权利要求1或2所述的方法，其特征在于，
    所述从所述报文中分离出所述应用的数据，包括：
    根据定界模板从所述报文中分离出所述应用的数据，所述定界模板定义了所述应用的数据与报文中其他数据的分离规则。
  4. 根据权利要求1至3任一权利要求所述的方法，其特征在于，
    所述接收服务器中运行的应用的报文包括：将属于同一个数据流的多个子报文聚合为所述报文，其中，所述属于同一个数据流的多个子报文的源网际互联协议IP地址和目的IP地址相同。
  5. 根据权利要求1至4任一权利要求所述的方法，其特征在于，从所述报文中分离出所述应用的数据之前，所述方法还包括：
    确定所述报文包括一个数据流中的完整数据。
  6. 一种网络设备，其特征在于，所述网络设备连接至服务器，所述网络设备包括：
    接收单元，用于接收服务器中运行的应用的报文；
    分离单元，用于从所述报文中分离出所述应用的数据；
    写入单元，用于将所述应用的数据写入所述服务器中为所述应用分配的内存区间中。
  7. 根据权利要求6所述的网络设备，其特征在于，
    所述分离单元还用于从所述报文中分离出报文头及所述应用的数据的元数据；
    所述写入单元还用于将所述报文头和所述应用的数据的元数据存储至所述服务器为所述网络设备的驱动分配的内存区间中。
  8. 根据权利要求6或7所述的网络设备，其特征在于，
    所述分离单元用于根据定界模板从所述报文中分离出所述应用的数据，所述定界模板定义了所述应用的数据与报文中其他数据的分离规则。
  9. 根据权利要求6至8任一权利要求所述的网络设备，其特征在于，
    所述接收单元用于将属于同一个数据流的多个子报文聚合为所述报文，其中，所述属于同一个数据流的多个子报文的源网际互联协议IP地址和目的IP地址相同。
  10. 根据权利要求6至9任一权利要求所述的网络设备，其特征在于，所述网络设备还包括确定单元，所述确定单元用于在所述分离单元从所述报文中分离出所述应用的数据之前，确定所述报文包括一个数据流中的完整数据。
  11. 一种网络设备，其特征在于，包括处理器和通信接口，所述通信接口用于接收报文，所述处理器用于执行权利要求1至5任一权利要求所述的方法以对报文进行处理。
PCT/CN2021/107828 2020-07-23 2021-07-22 一种报文处理方法、网络设备以及相关设备 Ceased WO2022017454A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21846476.6A EP4175231A4 (en) 2020-07-23 2021-07-22 Message processing method, network device and related device
US18/153,618 US12088688B2 (en) 2020-07-23 2023-01-12 Packet processing method, network device, and related device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010716127.9A CN113973091A (zh) 2020-07-23 2020-07-23 一种报文处理方法、网络设备以及相关设备
CN202010716127.9 2020-07-23

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/153,618 Continuation US12088688B2 (en) 2020-07-23 2023-01-12 Packet processing method, network device, and related device

Publications (1)

Publication Number Publication Date
WO2022017454A1 true WO2022017454A1 (zh) 2022-01-27

Family

ID=79585301

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/107828 Ceased WO2022017454A1 (zh) 2020-07-23 2021-07-22 一种报文处理方法、网络设备以及相关设备

Country Status (4)

Country Link
US (1) US12088688B2 (zh)
EP (1) EP4175231A4 (zh)
CN (1) CN113973091A (zh)
WO (1) WO2022017454A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220303230A1 (en) * 2022-03-30 2022-09-22 Intel Corporation Operations to copy portions of a packet
CN115103036B (zh) * 2022-05-20 2025-01-24 中国科学院计算技术研究所 一种高效的tcp/ip数据报处理方法及系统
CN117421112B (zh) * 2023-10-18 2024-08-06 中科驭数（北京）科技有限公司 加速单元、网卡、主机和报文处理加速方法
CN120034512B (zh) * 2025-02-24 2025-12-23 无锡众星微系统技术有限公司 报文存储装置及方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101616194A (zh) * 2009-07-23 2009-12-30 中国科学技术大学 主机网络性能优化系统及方法
US20110258337A1 (en) * 2008-08-04 2011-10-20 Zte Corporation Method and system for enabling zero-copy transmission of streaming media data
CN104506379A (zh) * 2014-12-12 2015-04-08 北京锐安科技有限公司 网络数据捕获方法和系统
CN107547623A (zh) * 2017-06-28 2018-01-05 新华三云计算技术有限公司 应用数据传输方法、装置、应用服务器和应用代理服务器
US9952979B1 (en) * 2015-01-14 2018-04-24 Cavium, Inc. Methods and systems for direct memory access operations
US20180219805A1 (en) * 2017-01-28 2018-08-02 Juniper Networks, Inc. Systems and methods for propagating metadata of in-flight packets within kernel space

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8291457B2 (en) * 2001-05-24 2012-10-16 Vixs Systems, Inc. Channel selection in a multimedia system
US7512128B2 (en) * 2002-06-12 2009-03-31 Sun Microsystems, Inc. System and method for a multi-packet data link layer data transmission
US7142540B2 (en) * 2002-07-18 2006-11-28 Sun Microsystems, Inc. Method and apparatus for zero-copy receive buffer management
US7012918B2 (en) * 2003-03-24 2006-03-14 Emulex Design & Manufacturing Corporation Direct data placement
US7826350B1 (en) * 2007-05-11 2010-11-02 Chelsio Communications, Inc. Intelligent network adaptor with adaptive direct data placement scheme
JP5962493B2 (ja) * 2012-12-20 2016-08-03 富士通株式会社 プログラム、情報処理装置およびオブジェクト送信方法
US9922102B2 (en) * 2013-07-31 2018-03-20 Splunk Inc. Templates for defining fields in machine data
US10827041B2 (en) * 2018-09-07 2020-11-03 Nokia Solutions And Networks Oy Packet fragmentation control

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110258337A1 (en) * 2008-08-04 2011-10-20 Zte Corporation Method and system for enabling zero-copy transmission of streaming media data
CN101616194A (zh) * 2009-07-23 2009-12-30 中国科学技术大学 主机网络性能优化系统及方法
CN104506379A (zh) * 2014-12-12 2015-04-08 北京锐安科技有限公司 网络数据捕获方法和系统
US9952979B1 (en) * 2015-01-14 2018-04-24 Cavium, Inc. Methods and systems for direct memory access operations
US20180219805A1 (en) * 2017-01-28 2018-08-02 Juniper Networks, Inc. Systems and methods for propagating metadata of in-flight packets within kernel space
CN107547623A (zh) * 2017-06-28 2018-01-05 新华三云计算技术有限公司 应用数据传输方法、装置、应用服务器和应用代理服务器

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4175231A4 *

Also Published As

Publication number Publication date
EP4175231A4 (en) 2023-12-27
US20230156102A1 (en) 2023-05-18
EP4175231A1 (en) 2023-05-03
CN113973091A (zh) 2022-01-25
US12088688B2 (en) 2024-09-10

Similar Documents

Publication Publication Date Title
WO2022017454A1 (zh) 一种报文处理方法、网络设备以及相关设备
US11068412B2 (en) RDMA transport with hardware integration
CN109981480A (zh) 一种数据传输方法及第一设备
EP2216955A1 (en) Network interface device
CN101827072A (zh) 虚拟内存协议分段卸载
JP2007266759A (ja) ネットワーク処理装置、マルチプロセッサシステムおよびネットワークプロトコル処理方法
JPH0981487A (ja) ネットワークデータ転送方法
US20240275740A1 (en) RDMA Data Transmission System, RDMA Data Transmission Method, and Network Device
CN104424105A (zh) 一种内存数据的读写处理方法和装置
CN117221225A (zh) 一种网络拥塞通知方法、装置及存储介质
CN117354253A (zh) 一种网络拥塞通知方法、装置及存储介质
US20260058921A1 (en) Packet transmission method and apparatus
CN107453845A (zh) 应答确认方法及设备
CN116320039A (zh) 一种报文压缩方法及装置、电子设备
CN115988108A (zh) 数据包处理方法、系统、存储介质和电子设备
CN113422792B (zh) 数据传输方法、装置、电子设备及计算机存储介质
CN111064768B (zh) 打印机数据传输控制方法、装置、设备及存储介质
CN116260887A (zh) 数据传输方法、数据发送装置、数据接收装置和存储介质
CN115080258A (zh) 数据传输系统及相关设备
US12093571B1 (en) Accelerating request/response protocols
WO2025077179A1 (zh) 一种数据传输方法、装置及相关设备
US6735649B2 (en) Multiple buffers for removing unwanted header information from received data packets
CN104615564A (zh) 一种基于qpi总线的数据传输方法及计算机系统
US20170265103A1 (en) Communication device, communication method, and non-transitory computer readable medium
CN116389378A (zh) 报文处理方法、装置、电子设备、芯片系统及存储介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021846476

Country of ref document: EP

Effective date: 20230126

NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21846476

Country of ref document: EP

Kind code of ref document: A1