WO2008089639A1 - A method for realizing back pressure of masses of ports and the device thereof - Google Patents
A method for realizing back pressure of masses of ports and the device thereof Download PDFInfo
- Publication number
- WO2008089639A1 WO2008089639A1 PCT/CN2007/071217 CN2007071217W WO2008089639A1 WO 2008089639 A1 WO2008089639 A1 WO 2008089639A1 CN 2007071217 W CN2007071217 W CN 2007071217W WO 2008089639 A1 WO2008089639 A1 WO 2008089639A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- back pressure
- idle frame
- port
- idle
- physical port
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/32—Flow control; Congestion control by discarding or delaying data units, e.g. packets or frames
- H04L47/323—Discarding or blocking control packets, e.g. ACK packets
Definitions
- the present invention relates to a control technique for back pressure of a network port, and more particularly to a method and apparatus for implementing back pressure of a massive port.
- Queue, weighted fair queue algorithm scheduling and so on. No matter what scheduling method you use, you will encounter a problem: Back pressure.
- the reserved bandwidth for the scheduling of multiple queues, it is generally necessary to configure the reserved bandwidth (for example, the bandwidth of the Aqueue is 10 Mbps, the bandwidth of the I queue is IGbps, the bandwidth of the C queue is 200 Mbps, etc.), in order to achieve the performance of the line-speed forwarding, then The sum of the reserved bandwidths of all queues will be greater than or equal to the final egress bandwidth.
- Priority, absolute priority scheduling then the user can configure CIR/PIR, which is the bandwidth promised to the user, and to some extent allows the user to burst traffic; then the user can map to different priority queues in the port, or Different services are mapped to the priority queues in the port.
- the port priority is scheduled according to PQ.
- the last is the RR scheduling between ports or WRR scheduling.
- the module that completes the QOS function shown in Figure 1 can be called a traffic management module.
- the number of ports matches the number of physical channels.
- the back pressure of the physical channel can be directly transmitted to the port, and then the port reversely transmits the back pressure in turn.
- the current technology generally sends a fifo (first in first out buffer) state to the physical sending port query through the traffic management module, or the physical sending port actively sends a f ifo state to the traffic management module, and if the sending fifo exceeds the back pressure threshold, The physical sending port outputs a back pressure signal to the traffic management module, the traffic management module stops transmitting, and then caches the packet in the scheduling queue. If the queue is full, the packet is lost according to the pre-configured parameters.
- SDH/SONET package transport packets on SDH/SONET;
- SDH is Synchronous Digital Hierarchy, synchronous digital series;
- SONET is Synchronous Optical
- NETwork, Synchronous Optical Network Ports may be up to 1K physical ports), traffic management module outbound interface (for example, SPI4.2 port can only support 256 backpressure signals) Can not respond to all channel backpressure, can not form Back pressure negative feedback, that is, according to the current implementation scheme and interface specification, the back pressure signal of the massive port cannot be transmitted.
- traffic management module outbound interface for example, SPI4.2 port can only support 256 backpressure signals
- Back pressure negative feedback that is, according to the current implementation scheme and interface specification, the back pressure signal of the massive port cannot be transmitted.
- the open loop control can be performed, that is, the outgoing interface is subjected to shaping (traffic shaping), and the rate of the outgoing interface is limited to be less than or equal to the physical port rate.
- This method is used to avoid the physical channel back pressure and the fluctuation caused by the traffic change. Solved inside the traffic management module. There are three problems with this method in practical applications:
- the general traffic management module performs traffic scheduling. It is inevitable that there is a scheduling error for some fixed-length packets. This scheduling error may cause congestion on the physical port sending queue, that is, it may be lost after the physical outgoing interface queue is full. The package cannot guarantee the configuration parameters such as the user's CIR and priority.
- the granularity of the outgoing interface flapping cannot meet the actual requirements.
- the bandwidth of the low-speed port is 64K, and the flapping granularity of the outgoing interface is 20 ⁇ . If the port line speed is to be guaranteed, only the PIR of the Shaping can be configured. 80K, if it does not respond to the backpressure, it will result in packet loss. Therefore, packet discarding on the physical port can only be performed by tail drop or by absolute priority.
- the configuration parameters such as CIR and priority of the user cannot be guaranteed.
- the channel for transmitting the massive port backpressure signal is a custom backpressure state bus. Since there is no uniform standard, it is often necessary to use the same manufacturer as the physical port and traffic management module in practical applications.
- the chip can realize the back pressure of the massive port, which limits the application range of the massive port back pressure.
- the traffic management module is implemented by a general-purpose CPU, and data is forwarded by the CPU, and the CPU directly queries the channel back pressure state through the control channel, thereby avoiding the limitation of the number of channels. .
- the forwarding performance is limited by the CPU capacity; and the CPU also participates in the processing of the control plane, causing the CPU program to be complicated and the reliability to be reduced.
- the CPU cannot handle the back pressure. , that is, the CPU cannot meet the line-speed forwarding when the number of channels is large.
- the embodiments of the present invention provide a method and a device for implementing a massive port back pressure, which has a fast back pressure and can meet the low-speed port delay requirement; CPU participation, forwarding performance is not affected by CPU performance, and can be easily applied to existing systems.
- a method for implementing massive port back pressure comprising the following steps:
- An embodiment of the present invention further provides an apparatus for implementing a back pressure of a massive port, the apparatus comprising:
- the detecting module is configured to detect whether the user data sent to the channelized physical port reaches a back pressure threshold, and if the back pressure threshold is reached, send a message;
- an idle frame generating module configured to generate one or a series of idle frames after receiving the message
- a downlink queue scheduling module configured to: use the idle frame with a channel that needs to reach a back pressure threshold User data sent by the physical port is mixed together, and the mixed data is sent;
- a downlink forwarding module configured to send, to the channelized physical port that reaches the backpressure threshold, the mixed data sent by the downlink queue scheduling module;
- the frame loss module is configured to discard the idle frame before the mixed data enters the channelized physical port that reaches the backpressure threshold.
- idle frames are used to implement back pressure, idle frames occupy a certain transmission bandwidth, reduce the bandwidth of user data, discard idle frames before entering physical ports, and idle frames do not enter physical ports. , thus achieving the purpose of back pressure.
- the embodiment of the present invention can solve the problem that the back pressure cannot be transmitted to the traffic management module because there are too many channels; whether it is due to the misalignment of the traffic management module or the back pressure caused by the sudden increase of the user data traffic,
- the solution in the embodiment of the present invention is effectively solved.
- the back pressure is fast, easy to use, and can be conveniently applied to various existing systems.
- FIG. 1 is a schematic diagram of a method for realizing back pressure in the prior art
- FIG. 2 is a schematic flow chart of a method for implementing a massive port back pressure according to an embodiment of the present invention
- FIG. 3 is a schematic flowchart diagram of an embodiment of a method for implementing massive port backpressure according to another embodiment of the present invention
- FIG. 4 is a schematic structural diagram of an embodiment of an apparatus for implementing a massive port back pressure according to the present invention
- FIG. 5 is a schematic structural view of another embodiment of an apparatus for implementing massive port back pressure according to the present invention.
- FIG. 2 it is a schematic flowchart of a method for realizing massive port backpressure in a specific embodiment of the present invention.
- implementing massive port backpressure is implemented by the following steps:
- Step 201 detecting data traffic sent to the channelized physical port, for example, detecting the transmission buffer state by detecting the status of the transmission buffer of the channelized physical port;
- Step 202 judging data traffic, that is, determining whether the transmission buffer has reached the back pressure threshold, and if so, proceeding to step 203; if not, proceeding to step 201;
- Step 203 Generate one or a series of idle frames
- Step 204 Mix the idle frame with the user data under the channelized physical port that reaches the back pressure threshold; wherein: the priority of the idle frame can be set to high; the mixed data can be sent to the traffic management module.
- the traffic management module can discard the received data in the case of congestion, and the traffic management module can perform data discarding processing according to the priority. If the priority of the idle frame is high, the traffic management module preferentially discards the mixed data. User data.
- Step 205 Send data that is mixed with user data to the channelized physical port that reaches the back pressure threshold;
- Step 206 Discard the idle frame before the idle frame and user data enter the channelized physical port.
- the initial bandwidth value of the generated idle frame may be pre-configured.
- the initial bandwidth value of the idle frame is a scheduling error of the queue in which the idle frame is located.
- the method of generating an idle frame is used to implement port backpressure: the bandwidth occupied by the idle frame is used to force the bandwidth occupied by the user data to be reduced, and the idle frame is before entering the channelized physical port. It is discarded without entering the channelized physical port, and only the user data with reduced bandwidth is entered into the channelized physical port, thereby achieving the purpose of back pressure. Therefore, the embodiment of the present invention has the advantages of simple and convenient, strong adaptability to the number of ports, and the advantages of massive port back pressure.
- each parameter is configured to improve the operating efficiency of the entire system. Specifically, the following steps are included:
- Step 301 Preserving a user queue used by an idle frame for each channelized physical port (or a channelized physical port that needs to be back pressured), proceeding to step 302;
- Step 302 Configure the CIR value of the user queue used in the idle frame as the scheduling error.
- the physical port sending rate is 64K
- the scheduling error is 5%
- the screening granularity of the port meets the requirements.
- Step 303 Configure the initial bandwidth value of the idle frame.
- Step 304 Query a transmission buffer of a channelized physical port corresponding to a user data
- Step 305 Determine whether the transmit buffer of the channelized physical port reaches the back pressure threshold, and if yes, go to step 306; if no, go back to step 304;
- Step 306 Generate one or a series of idle frames, the bandwidth of the idle frame is the starting bandwidth value configured in step 303; the generated idle frame is stored in the user queue used by the idle frame.
- Step 307 Mix the idle frame in the user queue used by the idle frame with the user data under the channelized physical port that reaches the back pressure threshold;
- Step 308 Send data that is mixed with user data to the channelized physical port that reaches the back pressure threshold;
- Step 309 Discard the idle frame before the data mixed with the idle frame and the user data enters the physical port.
- the bandwidth of the idle frame can be increased; if the back pressure disappears, the bandwidth of the idle frame is reduced until the bandwidth of the idle frame is zero. In increasing or decreasing the bandwidth of idle frames, it is best to smoothly increase or decrease.
- One method is to increase or decrease a certain bandwidth (for example, 1K) in each unit (for example, 1 second), if back pressure still exists. Or if it has disappeared, increase or decrease the bandwidth in the next day.
- the CIR value of the user queue used by the idle frame is configured.
- the pre-configured parameters are directly invoked, thereby improving the operation efficiency.
- it is implemented by detecting whether the transmission buffer of the channelized physical port reaches the back pressure threshold. If the traffic of the user data in a certain interval is greater than the total flow of the outlet, However, the transmission buffer has not reached the back pressure threshold, and no back pressure is applied, which avoids the stability of the user network due to the frequent back pressure.
- an embodiment of a device for implementing massive port backpressure includes the following modules:
- the detecting module 401 is configured to detect whether the user data sent to the channelized physical port reaches a back pressure threshold
- the idle frame generating module 403 is located at the beginning of the user data transmission, and is configured to generate one or a series of idle frames after receiving the message that the user data sent by the detecting module 401 reaches the back pressure threshold;
- the downlink queue scheduling module 404 is configured to mix the idle frame generated by the idle frame generating module with the user data, and send the mixed data to the downlink (from the user data sending end to the receiving end) to the downlink forwarding module;
- the downlink forwarding module 405 is configured to send, to the channelized physical port that reaches the backpressure threshold, data that is mixed with the user data sent by the downlink queue scheduling module 404;
- the frame loss module 406 is configured to discard the idle frame before the data mixed with the idle frame and the user data enters the channelized physical port that reaches the back pressure threshold.
- a configuration module 402 is further configured to configure a starting bandwidth value of the idle frame.
- the bandwidth of the idle frame generated by the idle frame generating module 403 is a starting bandwidth value configured by the configuration module 402.
- the idle frame generation module is located at the beginning of the user data transmission, and after the idle frame is generated, it is mixed with the user data by the downlink queue scheduling module, and then sent by the downlink forwarding module to the channelized physical port.
- the method for generating an idle frame is used to implement port backpressure: the bandwidth occupied by the idle frame is used to force the bandwidth occupied by the user data to be reduced, and the idle frame is discarded before entering the channelized physical port without entering the channel.
- the physical port is only the user data that enters the channelized physical port, thus achieving the purpose of back pressure. Therefore, the embodiment of the present invention has the advantages of simple and convenient, strong adaptability to the number of ports, and the advantages of massive port back pressure.
- the device for implementing the massive port back pressure can also be The implementation of the structure, including:
- the detecting module 501 is configured to detect whether the user data sent to the channelized physical port reaches a back pressure threshold
- the idle frame generating module 503 is located at the end of the user data transmission, and is configured to generate one or a series of idle frames after receiving the message that the user data sent by the detecting module 501 reaches the back pressure threshold; [69] The forwarding module 504 is configured to forward the idle frame generated by the idle frame generating module to the uplink queue scheduling module 505 in the upward direction (the direction from the user data receiving end to the transmitting end);
- the uplink queue scheduling module 505 is configured to forward the idle frame sent by the uplink forwarding module 504 to the downlink queue scheduling module 506;
- the downlink queue scheduling module 506 is configured to mix the idle frame sent by the uplink queue scheduling module 505 with the user data, and send the mixed data to the downlink forwarding module 507;
- the downlink forwarding module 507 is configured to forward the data that is mixed with the user data sent by the downlink queue scheduling module 506 to the channelized physical port that reaches the back pressure threshold.
- the frame loss module 508 is configured to discard the idle frame before the data of the idle frame and the user data sent by the downlink forwarding module 507 enters the channelized physical port.
- the configuration module 502 is further configured to configure a starting bandwidth value of the generated idle frame, and the bandwidth of the idle frame generated by the idle frame generating module 503 is a starting bandwidth configured by the configuration module 502.
- the uplink forwarding module 504 and the uplink queue scheduling module 505 only transparently transmit the idle frame.
- the idle frame generating module is located at the end of the user data transmission, and after the idle frame is generated, the uplink forwarding module and the uplink queue scheduling module are forwarded to the downlink queue scheduling module, and then the idle queue scheduling module sets the idle frame. Mixed with user data, and then sent by the downstream forwarding module to the channelized physical port.
- the uplink forwarding module and the uplink queue scheduling module are modules existing in the prior art.
- the present embodiment implements port backpressure by generating an idle frame method: the bandwidth occupied by the idle frame is used to force the bandwidth occupied by the user data to be reduced, and the idle frame is discarded before entering the channelized physical port.
- the embodiment of the present invention has the advantages of simple and convenient, strong adaptability to the number of ports, and the advantages of massive port back pressure.
- the software involved in the embodiments of the present invention may be stored in a computer readable storage medium.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Description
一种实现海量端口反压的方法及其装置
[1] 技术领域
[2] 本发明涉及一种网络端口反压的控制技术, 具体地说是一种实现海量端口反压 的方法及其装置。
[3] 发明背景
[4] 目前随着网络的发展, 尤其是流媒体的大量应用, 对网络的 QOS (Quality of
Sendee,服务质量) 性能提出了更高的、 差异化的需求。 无论釆用哪种策略来实 现网络的 QOS, 最终到网络的若干关键节点上, 必须能够按照预先配置的参数 带宽, 抖动, 等等) , 对多个待调度的源 (这个源一般而言是队列) 进行调 度, 选择合适队列中的数据进行传输; 如果发生拥塞需要丢包, 则按照预先配
Queue, 加权公平队列算法) 调度等等。 无论是釆用何种调度方法, 都会遇到一 个问题: 反压。 对于多个队列的调度, 一般要配置预约的带宽 (比如甲队列是 1 0Mbps的带宽, 乙队列是 IGbps的带宽, 丙队列是 200Mbps的带宽, 等等) , 为 了达到线速转发的性能, 那么所有队列的预约带宽之和会大于或者等于最终出 口带宽, 这吋因为突发流量或者调度误差等不可避免的带来了一个新问题, 即 出口允许的总流量 (比如 IGbps , 或者 2.5Gbps, 这个是受限于实际的物理出口 的) 小于所有队列的实际带宽之和的情况, 此吋不可能满足所有队列的实际带 宽。 比较理想的情况应该是:
第一, 如果实际数据的总带宽没有超过出口的总带宽, 那么希望能够按照实际 配置的预约带宽进行调度, 而且也不产生反压。
[7] 第二, 如果实际数据出现瞬间突发流量, 某吋间段内大于出口的总流量, 此吋 在出口拥塞后可以在出口队列进行缓存; 当实际数据的总带宽小于出口带宽吋 , 缓存的数据可以得到发送, 这吋也不会有反压。
[8] 第三, 如果实际数据的总带宽在较长吋间内总大于出口的总带宽则会有反压, 但是希望反压的效果能够体现在所有的参与调度的队列上, 而且最好按照预约 带宽的比例进行分配。 也就是说, 参与调度的队列的实际调度带宽最好按照预 约带宽的比例进行收缩。 注意, 不是所有的队列参与带宽比例分配, 而是存在 实际数据, 等待调度的队列之间进行带宽比例分配。
[9] 目前常用的标准 QOS调度模型如图 1所示, 共 4级调度。 首先在用户内部区分不 同的业务, 不同业务之间可以进行 WFQ+SP (Strict
Priority, 绝对优先级) 调度; 然后用户可以配置 CIR/PIR, 即给用户承诺的带宽 , 且一定程度上允许用户突发流量通过; 接着用户可以映射到端口内不同的优 先级队列, 也可以是用户内部不同业务映射到端口内的优先级队列, 端口优先 级按照 PQ调度; 最后是端口之间的 RR调度或者是 WRR调度。 完成图 1所示 QOS 功能的模块可以称为流量管理模块, 一般情况下端口数目和物理通道数目相符 , 物理通道的反压可以直接传递给端口, 然后端口依次逆向传递反压, 这吋包 会缓存在流量管理模块中; 当缓存满后按照优先级、 用户配置的 CIR或 PIR等参 数丢弃包, 在确保用户 CIR的情况下, 保证高优先级的包先得到调度, 低优先级 的包先丢弃。 目前现有技术一般通过流量管理模块主动向物理发送端口査询发 送 fifo (先入先出缓冲) 状态, 或者物理发送端口主动向流量管理模块反馈发送 f ifo状态, 如果发送 fifo超过反压门限, 则物理发送端口向流量管理模块输出反压 信号, 流量管理模块停止发送, 然后将包缓存在调度队列中, 如果队列满, 则 按照预先配置的参数进行丢包。
[10] 当物理通道数目太多 (例如通道化 POS (packet over
SDH/SONET, 在 SDH/SONET上封装传送包; 其中: SDH为 Synchronous Digital Hierarchy, 同步数字系列; SONET为 Synchronous Optical
NETwork, 同步光纤网) 端口可能达 1K个物理端口) , 流量管理模块出接口 ( 例如 SPI4.2端口只能支持 256路反压信号) 无法响应所有通道反压吋, 不能形成
反压负反馈, 即按照目前实现方案和接口规范, 无法传递海量端口的反压信号 。 那么按照当前技术下只能通过开环控制, 即对出接口做 Shaping (流量整形) , 限制其出接口速率小于等于物理端口速率, 通过这个方法来避免物理通道反 压, 将流量变化造成的波动在流量管理模块内部解决。 这种方法在实际应用中 存在 3个问题:
[11] 一般的流量管理模块在进行流量调度吋不可避免对某些固定长度的包存在调度 误差, 这种调度上的误差可能造成物理端口发送队列拥塞, 即可能在物理出接 口队列满后丢包, 无法保证用户的 CIR和优先级等配置参数。
[12] 另外还有可能出接口 Shaping的粒度无法满足实际需求, 例如低速端口的带宽 为 64K, 而出接口的 Shaping粒度为 20Κ, 则如果要保证端口线速则只能配置 Shap ing的 PIR为 80K, 如果不响应反压, 则会导致丢包; 这吋在物理端口上进行包丢 弃一般只能进行尾丢弃或按照绝对优先级丢弃, 不能保证用户的 CIR和优先级等 配置参数。
[13] 还有, 目前传输海量端口反压信号的通道为自定义的反压状态总线, 由于没有 统一的标准, 因此, 在实际应用中往往需要使用与物理端口、 流量管理模块同 一厂家的套片才能够实现海量端口反压, 从而使海量端口反压的应用范围受限
, 甚至无法应用在已经成形的系统中。
[14] 现有技术中还有实现反压的方案, 例如, 流量管理模块由通用 CPU实现, 通过 CPU进行数据转发, 由 CPU通过控制通道直接査询通道反压状态, 避免了通道数 目的限制。
[15] CPU参与转发后, 转发性能受 CPU能力限制; 而且 CPU还要参与控制平面的处 理, 造成 CPU程序复杂, 可靠性降低; 另外, 当通道数目太多, CPU对反压无法 及吋处理, 即 CPU无法满足通道数目较多情况下的线速转发。
[16] 发明内容
[17] 为了解决现有技术中无法实现海量端口反压的缺点, 本发明实施例提供一种实 现海量端口反压的方法及其装置, 反压速度快, 可以满足低速口延吋需求; 无 需 CPU参与, 转发性能不受 CPU性能影响, 且能够方便的应用于现有的各种系统 中。
[18] 本发明实施例解决其技术问题所釆用的技术方案是: 一种实现海量端口反压的 方法, 包括以下步骤:
[19] 检测发送到通道化物理端口的用户数据是否达到反压门限, 当达到反压门限吋
, 生成一个或一系列空闲帧;
[20] 将所述空闲帧与需要向达到反压门限的通道化物理端口发送的用户数据混合后 发送;
[21] 在所述空闲帧与所述用户数据混合后的数据进入通道化物理端口之前丢弃所述 空闲帧。
[22] 本发明实施例还提供一种实现海量端口反压的装置, 该装置包括:
[23] 检测模块, 用于检测发送到通道化物理端口的用户数据是否达到反压门限, 如 果达到反压门限, 发送消息;
[24] 空闲帧生成模块, 用于在接收到所述消息后, 生成一个或一系列空闲帧; [25] 下行队列调度模块, 用于将所述空闲帧与需要向达到反压门限的通道化物理端 口发送的用户数据混合在一起, 并发送所述混合后的数据;
[26] 下行转发模块, 用于向所述达到反压门限的通道化物理端口发送所述下行队列 调度模块发送的所述混合后的数据;
[27] 丢帧模块, 用于在所述混合后的数据进入所述达到反压门限的通道化物理端口 前将所述空闲帧丢弃。
[28] 本发明实施例由于使用了空闲帧来实现反压, 空闲帧占用了一定的传输宽带, 减少了用户数据的带宽, 在进入物理端口前将空闲帧丢弃, 空闲帧不会进入物 理端口, 从而达到了反压的目的。
[29] 本发明实施例可以解决因为通道过多, 使得反压无法传递到流量管理模块的问 题; 无论是因为流量管理模块调度不准还是用户数据流量突然增大造成的反压 问题, 都可以通过本发明实施例中的方案有效解决; 利用本发明实施例中的方 案, 传递反压速度快, 简单易用, 且能够方便的应用于现有的各种系统中。
[30] 附图简要说明
[31] 图 1为现有技术中实现反压的方法的示意图;
[32] 图 2为本发明的一个具体实施例中实现海量端口反压的方法的流程示意图;
[33] 图 3为本发明的另一个具体实施例中实现海量端口反压的方法的一种实施例的 流程示意图;
[34] 图 4为本发明的实现海量端口反压的装置的一种实施例的结构示意图;
[35] 图 5为本发明的实现海量端口反压的装置的另一种实施例的结构示意图。
[36] 实施本发明的方式
[37] 如图 2所示, 是本发明的一个具体实施例中实现海量端口反压的方法的流程示 意图, 该实施例中实现海量端口反压通过以下步骤实现:
[38] 步骤 201 : 对发送到通道化物理端口的数据流量进行检测, 例如, 通过检测通 道化物理端口的发送缓冲区状态来实现对发送数据流量的检测;
[39] 步骤 202: 判断数据流量, 即判断发送缓冲区是否达到了反压门限, 如果是, 进行步骤 203 ; 如果否, 进行步骤 201 ;
[40] 步骤 203: 生成一个或一系列空闲帧;
[41] 步骤 204: 将空闲帧与达到反压门限的那一通道化物理端口下的用户数据混合 ; 其中: 空闲帧的优先级可以置为高; 混合后的数据可以发送给流量管理模块 进行调度, 流量管理模块可以在拥塞等情况下对接收到的数据进行丢弃处理, 流量管理模块可以根据优先级来进行数据丢弃处理, 如空闲帧的优先级高, 则 流量管理模块优先丢弃混合数据中的用户数据。
[42] 步骤 205: 向达到反压门限的那一通道化物理端口发送空闲帧与用户数据混合 后的数据;
[43] 步骤 206: 在空闲帧与用户数据进入通道化物理端口之前, 将空闲帧丢弃。
[44] 在步骤 203中, 所生成的空闲帧的起始带宽值可以预先配置好, 例如, 空闲帧 的起始带宽值为该空闲帧所在队列的调度误差。
[45] 在该实施例中, 釆用生成空闲帧的方法来实现端口反压: 利用空闲帧抢占的带 宽来迫使用户数据所占用的带宽减小, 而空闲帧在进入通道化物理端口之前会 被丢弃而不进入通道化物理端口, 进入通道化物理端口的只有带宽减小后的用 户数据, 从而达到反压目的。 因此, 本发明的实施例具有简单方便, 对端口数 量的适应能力强, 可以实现海量端口反压等优点。
[46] 如图 3所示, 是本发明的另一个具体实施例中实现海量端口反压的方法的流程
示意图, 在该实施例中, 在对通道化物理端口的数据流量进行检测之前, 先对 各个参数进行配置, 以提高整个系统的运行效率。 具体包括以下步骤:
[47] 步骤 301 : 为每一个通道化物理端口 (或需要进行反压的通道化物理端口) 预 留一个空闲帧使用的用户队列, 进行步骤 302;
[48] 步骤 302: 配置上述空闲帧使用的用户队列的 CIR值为调度误差, 例如, 物理端 口发送速率为 64K, 调度误差为 5%, 端口的 shaping粒度满足要求, 可以配置为 6 4K, 则用于空闲帧的用户队歹 ijCIR应该配置为 64Κ*5%=3.2Κ; 如果存在粒度不 能满足需求的情况, 端口的 Shaping粒度为 20Κ, 贝 l」CIR应该配置为 20Κ*4-64Κ+8 0Κ*5 =20Κ;
[49] 步骤 303: 配置空闲帧的起始带宽值, 例如 64Κ通道应该配置为 64Κ*5%=3.2Κ ; 如果考虑粒度问题, 在粒度为 20Κ的情况下可以配置为 20Κ*4-64Κ+80Κ*5%=2 OK;
[50] 步骤 304: 査询某用户数据对应的通道化物理端口的发送缓冲区;
[51] 步骤 305: 判断通道化物理端口的发送缓冲区是否达到反压门限, 如果是, 进 行步骤 306; 如果否, 返回步骤 304;
[52] 步骤 306: 生成一个或一系列空闲帧, 该空闲帧的带宽为步骤 303中所配置的起 始带宽值; 生成的空闲帧存储在空闲帧使用的用户队列中。
[53] 步骤 307: 将空闲帧使用的用户队列中的空闲帧与达到反压门限的那一通道化 物理端口下的用户数据混合;
[54] 步骤 308: 向达到反压门限的那一通道化物理端口发送空闲帧与用户数据混合 后的数据;
[55] 步骤 309: 在空闲帧与用户数据混合后的数据进入物理端口前, 丢弃空闲帧。
[56] 如果在产生空闲帧后, 仍然存在反压, 则可以增大空闲帧的带宽; 如果反压消 失, 则减少空闲帧的带宽, 直至该空闲帧的带宽为零。 在增加或减少空闲帧的 带宽吋, 最好平滑的增加或减少, 其中一种方法是: 每个吋间单位 (例如 1秒) 内增加或减少一定带宽 (例如 1K) , 如果反压仍然存在或者已经消失, 则在下 一个吋间单位内再增加或减少一定带宽。
[57] 与上一实施例相比, 该实施例在对各通道化物理端口的反压门限进行检测之前
, 先对空闲帧使用的用户队列的 CIR值进行配置, 当检测到某通道化物理端口的 用户数据达到反压门限吋, 直接调用所预先配置好的参数, 提高了运行效率。 另外, 在检测通道化物理端口的反压门限吋, 通过检测通道化物理端口的发送 缓冲区是否达到该反压门限来实现, 如果用户数据在某一吋间段的流量大于出 口的总流量, 但发送缓冲区还未达到反压门限吋, 不进行反压, 避免了因反压 的频繁进行而影响用户网络的稳定。
[58] 如图 4所示, 实现海量端口反压的装置的一种实施例, 包括以下模块:
[59] 检测模块 401, 用于检测发送到通道化物理端口的用户数据是否达到反压门限
, 如果用户数据达到反压门限, 发送一个消息给空闲帧生成模块 403;
[60] 空闲帧生成模块 403, 该模块位于用户数据发送的起始端, 用于在接收到检测 模块 401发送的用户数据达到反压门限的消息吋, 生成一个或一系列空闲帧;
[61] 下行队列调度模块 404, 用于将空闲帧生成模块生成的空闲帧与用户数据混合 , 并将混合后的数据向下行 (由用户数据发送端到接收端的方向) 发送给下行 转发模块;
[62] 下行转发模块 405, 用于向达到反压门限的那个通道化物理端口发送下行队列 调度模块 404发来的空闲帧与用户数据混合后的数据;
[63] 丢帧模块 406, 用于在空闲帧与用户数据混合后的数据进入达到反压门限的那 个通道化物理端口前, 将空闲帧丢弃。
[64] 在实际应用中, 还可以包括一个配置模块 402, 用于配置空闲帧的起始带宽值 ; 空闲帧生成模块 403生成的空闲帧的带宽为配置模块 402配置的起始带宽值。
[65] 在本实施例中, 空闲帧生成模块位于用户数据发送的起始端, 空闲帧生成后由 下行队列调度模块将其与用户数据混合, 然后由下行转发模块向通道化物理端 口发送。 本实施例釆用生成空闲帧的方法来实现端口反压: 利用空闲帧抢占的 带宽来迫使用户数据所占用的带宽减小, 而空闲帧在进入通道化物理端口之前 会被丢弃而不进入通道化物理端口, 进入通道化物理端口的只是用户数据, 从 而达到反压目的。 因此, 本发明的实施例具有简单方便, 对端口数量的适应能 力强, 可以实现海量端口反压等优点。
[66] 在本发明的另一个具体实施例中, 实现海量端口反压的装置还可以通过图 5所
示的结构实现, 包括:
[67] 检测模块 501, 用于检测发送到通道化物理端口的用户数据是否达到反压门限
, 如果用户数据达到反压门限, 发送一个消息给空闲帧生成模块 503 ;
[68] 空闲帧生成模块 503, 该模块位于用户数据发送的末端, 用于在接收到检测模 块 501发送的用户数据达到反压门限的消息吋, 生成一个或一系列空闲帧; [69] 上行转发模块 504, 用于将空闲帧生成模块生成的空闲帧向上行 (由用户数据 接收端到发送端的方向) 转发给上行队列调度模块 505 ;
[70] 上行队列调度模块 505, 用于将上行转发模块 504发送来的空闲帧转发给下行队 列调度模块 506;
[71] 下行队列调度模块 506, 用于将上行队列调度模块 505发送来的空闲帧与用户数 据混合, 并将混合后的数据发送到下行转发模块 507;
[72] 下行转发模块 507, 用于向达到反压门限的那个通道化物理端口转发下行队列 调度模块 506发送来的空闲帧与用户数据混合后的数据;
[73] 丢帧模块 508, 用于在下行转发模块 507发送的空闲帧与用户数据混合后的数据 进入上述通道化物理端口前, 将空闲帧丢弃。
[74] 在实际应用中, 还可以包括配置模块 502, 用于配置所生成的空闲帧的起始带 宽值, 由空闲帧生成模块 503生成的空闲帧的带宽为配置模块 502配置的起始带 宽值; 在上行转发模块 504和上行队列调度模块 505对空闲帧进行转发的过程中 , 上行转发模块 504和上行队列调度模块 505只对空闲帧进行透传。
[75] 在本实施例中, 空闲帧生成模块位于用户数据发送的末端, 空闲帧生成后先由 上行转发模块和上行队列调度模块转发给下行队列调度模块, 然后由下行队列 调度模块将空闲帧与用户数据混合, 再由下行转发模块发送到通道化物理端口 。 其中上行转发模块和上行队列调度模块为现有技术中已有的模块。 同样, 本 实施例釆用生成空闲帧的方法来实现端口反压: 利用空闲帧抢占的带宽来迫使 用户数据所占用的带宽减小, 而空闲帧在进入通道化物理端口之前会被丢弃而 不进入通道化物理端口, 进入通道化物理端口的只是用户数据, 从而达到反压 目的。 因此, 本发明的实施例具有简单方便, 对端口数量的适应能力强, 可以 实现海量端口反压等优点。
[76] 本发明实施例所涉及的软件可以存储在一个计算机可读取存储介质中。
[77] 以上所述, 仅为本发明较佳的具体实施方式, 但本发明的保护范围并不局限于 此, 任何熟悉该技术的人在本发明所揭露的技术范围内, 可轻易想到的变化或 替换, 都应涵盖在本发明的保护范围之内。
Claims
[1] 1、 一种实现海量端口反压的方法, 其特征在于, 包括以下步骤:
检测发送到通道化物理端口的用户数据是否达到反压门限, 当达到反压门 限吋, 生成一个或一系列空闲帧;
将所述空闲帧与需要向达到反压门限的通道化物理端口发送的用户数据混 合后发送;
在所述空闲帧与所述用户数据混合后的数据进入通道化物理端口之前丢弃 所述空闲帧。
[2] 2、 根据权利要求 1所述的实现海量端口反压的方法, 其特征在于, 所述生 成一个或一系列空闲帧的步骤还包括:
将所述空闲帧存储在为所述通道化物理端口预留的空闲帧使用的用户队列 中。
[3] 3、 根据权利要求 2所述的实现海量端口反压的方法, 其特征在于, 所述空 闲帧使用的用户队列的承诺信息速率值为调度误差。
[4] 4、 根据权利要求 1所述的实现海量端口反压的方法, 其特征在于, 所述生 成一个或一系列空闲帧的步骤包括, 根据预先配置的空闲帧的起始带宽值 生成一个或一系列空闲帧。
[5] 5、 根据权利要求 1〜4之任意一项所述的实现海量端口反压的方法, 其特征 在于, 所述检测发送到通道化物理端口的用户数据是否达到反压门限包括
: 检测所述通道化物理端口的发送缓冲区是否达到反压门限。
[6] 6、 根据权利要求 1〜4之任意一项所述的实现海量端口反压的方法, 其特征 在于, 还包括:
产生所述空闲帧后, 如果发送到通道化物理端口的用户数据仍然达到反压 门限, 则增加所述需要进行混合的空闲帧的带宽; 如果发送到通道化物理 端口的用户数据未达到反压门限, 则减少所述需要进行混合的空闲帧的带 宽。
[7] 7、 根据权利要求 6所述的实现海量端口反压的方法, 其特征在于, 所述减 少所述需要进行混合的空闲帧的带宽包括: 将所述需要进行混合的空闲帧
的带宽减少到 0。
[8] 8、 一种实现海量端口反压的装置, 其特征在于, 包括:
检测模块, 用于检测发送到通道化物理端口的用户数据是否达到反压门限 , 如果达到反压门限, 发送消息;
空闲帧生成模块, 用于在接收到所述消息后, 生成一个或一系列空闲帧; 下行队列调度模块, 用于将所述空闲帧与需要向达到反压门限的通道化物 理端口发送的用户数据混合在一起, 并发送所述混合后的数据; 下行转发模块, 用于向所述达到反压门限的通道化物理端口发送所述下行 队列调度模块发送的所述混合后的数据;
丢帧模块, 用于在所述混合后的数据进入所述达到反压门限的通道化物理 端口前将所述空闲帧丢弃。
[9] 9、 根据权利要求 8所述的实现海量端口反压的装置, 其特征在于, 还包括 配置模块, 用于配置空闲帧的起始带宽值, 使空闲帧生成模块根据所述起 始带宽值生成一个或一系列空闲帧。
[10] 10、 根据权利要求 8或 9所述的实现海量端口反压的装置, 其特征在于, 还 包括:
上行转发模块, 用于将所述空闲帧生成模块生成的空闲帧上行转发; 上行队列调度模块, 用于将所述上行转发模块转发来的空闲帧转发到所述 下行队列调度模块。
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP07846061.5A EP2093945B1 (en) | 2007-01-18 | 2007-12-11 | A method for realizing back pressure of masses of ports and the device thereof |
| CA2675930A CA2675930C (en) | 2007-01-18 | 2007-12-11 | Method for realizing backpressure of masses of ports and device thereof |
| US12/505,312 US8189463B2 (en) | 2007-01-18 | 2009-07-17 | Method for realizing backpressure of masses of ports and device thereof |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007100011241A CN100550852C (zh) | 2007-01-18 | 2007-01-18 | 一种实现海量端口反压的方法及其装置 |
| CN200710001124.1 | 2007-01-18 |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US12/505,312 Continuation US8189463B2 (en) | 2007-01-18 | 2009-07-17 | Method for realizing backpressure of masses of ports and device thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2008089639A1 true WO2008089639A1 (en) | 2008-07-31 |
Family
ID=38744470
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2007/071217 Ceased WO2008089639A1 (en) | 2007-01-18 | 2007-12-11 | A method for realizing back pressure of masses of ports and the device thereof |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US8189463B2 (zh) |
| EP (1) | EP2093945B1 (zh) |
| CN (1) | CN100550852C (zh) |
| CA (1) | CA2675930C (zh) |
| WO (1) | WO2008089639A1 (zh) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100550852C (zh) * | 2007-01-18 | 2009-10-14 | 华为技术有限公司 | 一种实现海量端口反压的方法及其装置 |
| JP5418086B2 (ja) * | 2009-09-09 | 2014-02-19 | 富士通株式会社 | 伝送装置及び信号伝送方法 |
| CN102143053B (zh) * | 2010-11-12 | 2014-08-20 | 华为技术有限公司 | 传输数据的方法、装置和系统 |
| CN102025617B (zh) * | 2010-11-26 | 2015-04-01 | 中兴通讯股份有限公司 | 以太网拥塞控制方法及装置 |
| US8457142B1 (en) * | 2011-02-18 | 2013-06-04 | Juniper Networks, Inc. | Applying backpressure to a subset of nodes in a deficit weighted round robin scheduler |
| KR20140133295A (ko) * | 2013-05-10 | 2014-11-19 | 한국전자통신연구원 | 멀티 프로세서를 구비하는 데이터 병렬 처리 장치 및 그 방법 |
| CN103763204B (zh) * | 2013-12-31 | 2017-03-08 | 华为技术有限公司 | 一种流量控制方法及装置 |
| CN104410584A (zh) * | 2014-12-16 | 2015-03-11 | 上海斐讯数据通信技术有限公司 | 一种QoS的队列等级调度方法 |
| CN104869079B (zh) * | 2015-06-11 | 2018-03-20 | 烽火通信科技股份有限公司 | 基于动态加权循环法的队列调度方法及装置 |
| CN107846321B (zh) * | 2016-09-19 | 2022-03-11 | 阿里巴巴集团控股有限公司 | 一种接口的监控方法、装置及电子设备 |
| US11349777B2 (en) * | 2019-11-15 | 2022-05-31 | Charter Communications Operating, Llc | Network quality of service controller |
| CN117135067A (zh) * | 2022-05-19 | 2023-11-28 | 中国移动通信有限公司研究院 | 流量的速率的预测方法、装置、系统及存储介质 |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0661851A2 (en) | 1993-12-22 | 1995-07-05 | Nec Corporation | Congestion control method in ATM network |
| EP1052816A2 (en) * | 1999-05-14 | 2000-11-15 | Nortel Networks Limited | Backpressure mechanism for a network device |
| WO2001089140A2 (en) | 2000-05-19 | 2001-11-22 | Mosaid Technologies Incorporated | Method and apparatus for providing input back pressure in an output buffered switch |
| CN1325211A (zh) * | 2000-05-23 | 2001-12-05 | 日本电气株式会社 | 在矩形格栅扩展异步传输模式交换机中的信元流量控制 |
| CN1471264A (zh) * | 2002-07-27 | 2004-01-28 | 华为技术有限公司 | 基于动态包传输的动态随机存储器队列调度方法 |
| EP1489796A2 (en) | 2003-04-25 | 2004-12-22 | Alcatel IP Networks, Inc. | Fabric access node integrated circuit configured to bound cell reorder buffer depth |
| JP2006324864A (ja) * | 2005-05-18 | 2006-11-30 | Fujitsu Ltd | ネットワークスイッチおよびバックプレッシャ制御方法 |
| CN101026558A (zh) * | 2007-01-18 | 2007-08-29 | 华为技术有限公司 | 一种实现海量端口反压的方法及其装置 |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| SE510170C2 (sv) * | 1997-05-16 | 1999-04-26 | Ericsson Telefon Ab L M | Metod och anordning för styrning av datapaketflöde medelst baktrycksmeddelande |
| US7035294B2 (en) * | 2001-06-04 | 2006-04-25 | Calix Networks, Inc. | Backplane bus |
| CN1260915C (zh) * | 2002-11-19 | 2006-06-21 | 华为技术有限公司 | 一种城域网传输设备的流量控制方法 |
| CN100512280C (zh) | 2003-12-19 | 2009-07-08 | 华为技术有限公司 | 实现流量控制的方法及装置 |
-
2007
- 2007-01-18 CN CNB2007100011241A patent/CN100550852C/zh not_active Expired - Fee Related
- 2007-12-11 CA CA2675930A patent/CA2675930C/en active Active
- 2007-12-11 WO PCT/CN2007/071217 patent/WO2008089639A1/zh not_active Ceased
- 2007-12-11 EP EP07846061.5A patent/EP2093945B1/en not_active Not-in-force
-
2009
- 2009-07-17 US US12/505,312 patent/US8189463B2/en not_active Expired - Fee Related
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0661851A2 (en) | 1993-12-22 | 1995-07-05 | Nec Corporation | Congestion control method in ATM network |
| EP1052816A2 (en) * | 1999-05-14 | 2000-11-15 | Nortel Networks Limited | Backpressure mechanism for a network device |
| WO2001089140A2 (en) | 2000-05-19 | 2001-11-22 | Mosaid Technologies Incorporated | Method and apparatus for providing input back pressure in an output buffered switch |
| CN1325211A (zh) * | 2000-05-23 | 2001-12-05 | 日本电气株式会社 | 在矩形格栅扩展异步传输模式交换机中的信元流量控制 |
| US20010048664A1 (en) | 2000-05-23 | 2001-12-06 | Nec Corporation | Cell flow control in square-grid expanded ATM switch |
| CN1471264A (zh) * | 2002-07-27 | 2004-01-28 | 华为技术有限公司 | 基于动态包传输的动态随机存储器队列调度方法 |
| EP1489796A2 (en) | 2003-04-25 | 2004-12-22 | Alcatel IP Networks, Inc. | Fabric access node integrated circuit configured to bound cell reorder buffer depth |
| JP2006324864A (ja) * | 2005-05-18 | 2006-11-30 | Fujitsu Ltd | ネットワークスイッチおよびバックプレッシャ制御方法 |
| CN101026558A (zh) * | 2007-01-18 | 2007-08-29 | 华为技术有限公司 | 一种实现海量端口反压的方法及其装置 |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP2093945A4 |
Also Published As
| Publication number | Publication date |
|---|---|
| CA2675930A1 (en) | 2008-07-31 |
| US20090279435A1 (en) | 2009-11-12 |
| CN101026558A (zh) | 2007-08-29 |
| EP2093945A1 (en) | 2009-08-26 |
| CN100550852C (zh) | 2009-10-14 |
| EP2093945A4 (en) | 2009-12-16 |
| CA2675930C (en) | 2013-10-08 |
| US8189463B2 (en) | 2012-05-29 |
| EP2093945B1 (en) | 2014-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2093945B1 (en) | A method for realizing back pressure of masses of ports and the device thereof | |
| CN112887226B (zh) | 有线无线融合的卫星时间敏感网络队列管理调度方法 | |
| US11968111B2 (en) | Packet scheduling method, scheduler, network device, and network system | |
| US9479444B2 (en) | Traffic management in a multi-channel system | |
| CN101984608A (zh) | 报文拥塞避免方法及系统 | |
| CN110647071A (zh) | 一种控制数据传输的方法、装置及存储介质 | |
| CN102916880B (zh) | 一种分组交换网中数据包发送和接收的方法及装置 | |
| CN104836750B (zh) | 一种基于时间片轮转的数据中心网络流调度方法 | |
| CN101217495A (zh) | 用于t-mpls网络环境下的流量监控方法和装置 | |
| CN114615211A (zh) | 一种有线无线融合的航天器内时间敏感网络队列管理调度方法 | |
| WO2008104100A1 (en) | An apparatus and a method for realizing flow control based on rate restrict for mstp device | |
| CN118890321B (zh) | 一种基于非可靠传输的共享缓存方法和装置 | |
| CN115865810B (zh) | 一种时间敏感网络中信用值流量调度系统及方法 | |
| CN114501654A (zh) | 一种有线无线融合的航天器内部无线时间敏感网络时隙分配方法 | |
| WO2022057462A1 (zh) | 拥塞控制方法和装置 | |
| CN112822268A (zh) | 一种工业物联网中面向多服务共存以及确定性网络需求的ip层数据包调度算法 | |
| CN102752192A (zh) | 基于SCTP的ForCES传输映射层的带宽分配方法 | |
| CN117880195A (zh) | 一种车内软件定义时间敏感网络的流量调度系统及方法 | |
| WO2016008399A1 (en) | Flow control | |
| CN117675606A (zh) | 一种基于数字化仿真的时间敏感网络资源调度算法组合方法 | |
| WO2012106905A1 (zh) | 报文处理方法及装置 | |
| CN101771603A (zh) | 数据交换方法及设备 | |
| WO2023207628A1 (zh) | 一种报文传输方法以及报文转发设备 | |
| CN116708871A (zh) | 一种基于tsn视频业务传输方法、设备及介质 | |
| CN118611708A (zh) | 一种电力线载波通讯方法 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07846061 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2007846061 Country of ref document: EP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2675930 Country of ref document: CA |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
