System and method for identifying candidate flows in a data packet network
阅读说明:本技术 用于识别数据分组网络中的候选流的系统和方法 (System and method for identifying candidate flows in a data packet network ) 是由 米歇尔·米勒 约翰·M·伯内特 本·哈多恩 戴夫·吉本斯 肖恩·布莱恩 于 2019-01-25 设计创作,主要内容包括:一种计算机实现的方法和传输管理器系统用于通过以下方式来减少网络拥塞:检测网络中的一个或多个数据流;使用候选流检测阈值来确定一个或多个数据流中的数据流是否是候选流,候选流检测阈值基于一个或多个数据流的一个或多个特性;以及响应于确定数据流是候选流,管理数据流。一个或多个数据流的消耗速率、持续时间、传递的字节的数量、吞吐量、或者聚合特性可以用于确定候选流检测阈值。(A computer-implemented method and transmission manager system for reducing network congestion by: detecting one or more data flows in a network; determining whether a data stream of the one or more data streams is a candidate stream using a candidate stream detection threshold, the candidate stream detection threshold based on one or more characteristics of the one or more data streams; and managing the data flow in response to determining that the data flow is a candidate flow. The consumption rate, duration, number of bytes delivered, throughput, or aggregate characteristics of one or more data streams may be used to determine the candidate stream detection threshold.)
1. A computer-implemented method for reducing network congestion, comprising:
detecting one or more data flows in a network;
determining whether a data flow of the one or more data flows is a candidate flow using a candidate flow detection threshold, the candidate flow detection threshold based on one or more characteristics of the one or more data flows; and
managing the data flow in response to determining that the data flow is the candidate flow.
2. The method of claim 1, further comprising:
determining a consumption rate of data carried by the data flow, the consumption rate of data being used to determine the candidate flow detection threshold.
3. The method of claim 2, wherein the consumption rate is related to a coding rate.
4. The method of claim 2, further comprising:
determining a throughput of the data flow, the throughput being used to determine the candidate flow detection threshold as a function of a transmission time required to buffer an amount of data corresponding to the consumption rate multiplied by a predetermined duration,
wherein the duration of the data stream is compared to the candidate stream detection threshold to determine whether the data stream is the candidate stream.
5. The method of claim 2, wherein a predetermined buffer growth rate value is added to the consumption rate to determine the candidate flow detection threshold, the method further comprising:
determining the throughput of said data stream, an
Comparing the throughput of the data flow to the candidate flow detection threshold to determine whether the data flow is a candidate flow.
6. The method of claim 1, further comprising:
determining a percentage of the plurality of data streams determined to be candidate streams;
comparing the determined percentage to a predetermined target percentage;
in response to the determined percentage being less than the predetermined target percentage, adjusting the candidate flow detection threshold to increase the number of the plurality of data flows determined to be candidate flows; and
in response to the determined percentage being greater than the predetermined target percentage, adjusting the candidate flow detection threshold to reduce the number of the plurality of data flows determined to be candidate flows,
wherein the candidate flow detection threshold is adjusted to determine the candidate flow detection threshold.
7. The method of claim 1, further comprising:
determining a distribution of the one or more characteristics or a distribution of a combination of the one or more characteristics;
determining a value corresponding to a predetermined percentile of the determined distribution; and
determining the candidate flow detection threshold according to the determined value.
8. The method of claim 1, wherein the method is performed by a first network node, the method further comprising:
receiving information about the plurality of data flows from a second network node; and
determining the candidate flow detection threshold using the received information.
9. The method of claim 1, wherein the method is performed by a first network node, the method further comprising:
receiving a network metric from a second network node; and
determining the candidate flow detection threshold using the received network metric,
wherein the first network node is a network node that receives the plurality of data streams.
10. The method of claim 1, further comprising:
determining a current time of day; and
determining the candidate flow detection threshold according to a current time of day.
11. The method of claim 1, further comprising:
determining a duration of time that the data flow is actively transmitting packets; and
comparing the duration to the candidate flow detection threshold to determine whether the data flow is the candidate flow.
12. The method of claim 1, further comprising:
determining a number of bytes communicated by the data stream; and
comparing the number of bytes to the candidate stream detection threshold to determine whether the data stream is the candidate stream.
13. A transport manager system, comprising:
one or more processors, a network interface, a queue, and a storage device communicatively coupled to one another, the storage device storing computer-executable instructions that, when executed by the one or more processors, cause the transmission manager system to:
detecting one or more data flows in a network;
determining whether a data stream is a candidate stream using a candidate stream detection threshold, the candidate stream detection threshold based on one or more characteristics of the one or more data streams; and
managing the data flow in response to determining that the data flow is the candidate flow.
14. The transport manager system of claim 13, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a consumption rate of data carried by the data flow, the consumption rate being used to determine the candidate flow detection threshold.
15. The transport manager system of claim 14, wherein the consumption rate is related to an encoding rate.
16. The transport manager system of claim 14, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a throughput of the data flow, the throughput being used to determine the candidate flow detection threshold as a function of a transmission time required to buffer an amount of data corresponding to the consumption rate multiplied by a predetermined duration,
wherein the duration of the data stream is compared to the candidate stream detection threshold to determine whether the data stream is the candidate stream.
17. The transport manager system of claim 14, wherein a predetermined buffer growth rate value is added to the consumption rate to determine the candidate flow detection threshold, and wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a throughput of the data stream; and
comparing the throughput of the data flow to the candidate flow detection threshold to determine whether the data flow is a candidate flow.
18. The transport manager system of claim 13, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a percentage of the plurality of data streams determined to be candidate streams;
comparing the determined percentage to a predetermined target percentage;
in response to the determined percentage being less than the predetermined target percentage, adjusting the candidate flow detection threshold to increase the number of the plurality of data flows determined to be candidate flows; and
in response to the determined percentage being greater than the predetermined target percentage, adjusting the candidate flow detection threshold to reduce the number of the plurality of data flows determined to be candidate flows.
19. The transport manager system of claim 13, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a distribution of the one or more characteristics or a distribution of a combination of the one or more characteristics;
determining a value corresponding to a predetermined percentile of the determined distribution; and
determining the candidate flow detection threshold according to the determined value.
20. The transport manager system of claim 13, wherein the transport manager system is included in a first network node, and wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
receiving information about the plurality of data flows from a second network node; and
determining the candidate flow detection threshold using the received information.
21. The transport manager system of claim 13, wherein the transport manager system is included in a first network node, and wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
receiving a network metric from a second network node; and
determining the candidate flow detection threshold using the received network metric.
22. The transport manager system of claim 13, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a current time of day; and
determining the candidate flow detection threshold according to a current time of day.
23. The transport manager system of claim 13, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a duration of time that the data flow is actively transmitting packets; and
comparing the duration to the candidate flow detection threshold to determine whether the data flow is the candidate flow.
24. The transport manager system of claim 13, wherein the computer-executable instructions, when executed by the one or more processors, cause the transport manager system to:
determining a number of bytes communicated by the data stream; and
comparing the number of bytes to the candidate stream detection threshold to determine whether the data stream is the candidate stream.
Background
One of the increasing problems faced by today's data networks, which may include wireless, wired and/or fiber optic networks, is the burden placed on these data networks by a single data flow ("stream") that consumes a disproportionate amount of network resources over an extended period of time during transmission to or from end user devices, which may be referred to as "elephant flow". Streaming media content (audio, video, and/or other types of content data), large downloads, and the like typically produce such elephant streams.
Consumer access networks are typically designed for the transmission of short bursts of data and short periodic use of network resources by client devices. As a result, large flows can be a major challenge for network traffic engineers attempting to meet peak usage demands of many users using limited network resources. The presence of a large number of unmanaged elephant flows in the network may create network congestion that results in a slow network response for all users and their applications.
Accordingly, it is desirable to be able to identify candidate flows (e.g., elephant flows) that may be managed (e.g., by per-flow bandwidth allocation) to preserve network performance.
Disclosure of Invention
Systems and methods are described herein for determining candidate flows (e.g., elephant flows) that may be processed and/or managed to prevent performance degradation of a network. Candidate flows may be determined using candidate flow detection thresholds that may be dynamically adjusted.
According to various embodiments, the candidate flow detection threshold may vary based on the network environment and many other factors. Embodiments may adjust the candidate flow detection threshold by: identifying a stream that is a video stream, and identifying or inferring the video encoding rate being delivered to the device by directly observing the encoding rate or by observing the throughput of the stream. The embodiment may calculate the amount of buffer that has been transferred and/or the rate at which the buffer is growing and use one or more of these values to adjust one or more candidate flow detection thresholds for that flow up or down.
Another embodiment may identify a target percentage of expected candidate flows for an entire data transmission network or a subset of the network (e.g., a single eNodeB, destination, source, subscriber type, etc.), and dynamically adjust the candidate flow detection threshold(s) until the target percentage of candidate flows is met.
Another embodiment utilizes percentiles of data transfer sessions across the entire data transfer network or a subset of the data transfer network (i.e., nodes within the data network, or content destinations, or content sources, content types, etc.) to determine candidate flow detection thresholds. For example, within a network node, an embodiment may find the value of the 90 th percentile of all data transfer session sizes and determine a candidate flow detection threshold for that network node based on that value. Other embodiments may similarly operate using other data transfer session indicators, including but not limited to data transfer session throughput or duration of a data transfer session, as inputs to determine an appropriate candidate flow detection threshold.
Another embodiment may measure the frequency of candidate flow identifications based on the source or destination of the data transmission session. This information may be used to increase or decrease the candidate flow detection threshold for that source or destination.
In another embodiment, an embodiment may receive information from another network node and utilize the information to adjust the candidate flow detection threshold upward or downward based on current data transmission network conditions reported by the network node. For example, if network node resource usage increases or decreases, embodiments may increase or decrease the number of candidate flows it detects for that network node.
In an embodiment, the candidate flow detection threshold may be adjusted based on the time of day, such that the threshold may be adapted to situations where traffic increases during a particular time of day, and then decreases during different time periods. Embodiments may allow for identification of multiple time periods for which different respective candidate flow detection thresholds are defined.
Drawings
FIG. 1A illustrates an example network environment.
FIG. 1B illustrates another example network environment.
Fig. 2A is a block diagram of a transmission manager system according to an embodiment.
Fig. 2B is a block diagram of a transmission manager system according to another embodiment.
Fig. 2C is a block diagram of a transmission manager system according to another embodiment.
Fig. 2D is a block diagram of a transmission manager system according to another embodiment.
Fig. 3 is a block diagram of a user equipment according to an embodiment.
Fig. 4 illustrates decision logic in accordance with an embodiment in which a candidate stream detector identifies a stream that may be video, and adjusts a candidate stream detection threshold based on the amount that has been delivered to the device.
Fig. 5 illustrates decision logic in accordance with an embodiment, wherein the candidate stream detector identifies a data transfer session, which may be video, or a data transfer session that is informed by another network node that the data transfer session is video, and adjusts the candidate stream detection threshold based on the rate of growth of the video playback buffer.
Fig. 6 illustrates decision logic in accordance with an embodiment in which a flow detector adjusts a candidate flow detection threshold to achieve a target percentage of candidate flows in a set of flows.
Fig. 7 illustrates decision logic in accordance with an embodiment in which a stream detector adjusts candidate stream detection thresholds to achieve a target percentile value among a set of streams.
Fig. 8 illustrates decision logic in accordance with an embodiment in which a candidate flow detector uses information collected by the candidate flow detector or reported from a separate network node and uses that information to adjust the candidate flow detection threshold for those flows.
Fig. 9 illustrates decision logic according to an embodiment, wherein the candidate flow detector adjusts the candidate flow threshold based on information received from another network node.
Fig. 10 illustrates decision logic in accordance with an embodiment in which a candidate flow detector adjusts a candidate flow detection threshold based on time of day.
FIG. 11 illustrates a computer system, according to an embodiment.
Detailed Description
Systems and methods are described herein for determining candidate flows (e.g., elephant flows) that may be processed and/or managed to prevent performance degradation of a network. Candidate flows may be determined using a candidate flow detection threshold.
When a data network is congested, the rate at which data packets (e.g., data flows or streams) traverse the network will typically decrease, resulting in less than optimal data throughput. One of the reasons for network congestion is the presence or existence of "elephant flows" or other types of flows that are relatively heavy when using network resources including shared throughput capacity.
Candidate flow detection may be performed at any point within the packet data network to identify heavy flows that may be managed to improve network performance. Candidate flow detection may be performed directly on the data traffic or on a duplicate of the traffic. There are many reasons for wanting to identify candidate flows in a network versus other data flows, including, but not limited to, applying quality of service (QoS) policies to candidate flows, creating security policies based on identified candidate flow behaviors, statistical analysis of user behaviors, and the like.
A flow may be defined as a flow of datagram packets from one or more source devices to one or more destination devices, regardless of the transport protocol. As an example, a flow may include one or more data transfer sessions. For example, if an end user device, such as a smartphone, initiates a video, it may create one data transfer session for the video stream and another data transfer session for the audio stream, both to the same destination server. According to various embodiments, this may be viewed as a single stream from the perspective of the candidate stream detection system.
Examples of candidate streams include, for example, packet data streams associated with media content (e.g., video and/or audio files) that use a large portion of the network bandwidth. In some cases, a candidate flow may be defined as a data flow that consumes a portion of the total network bandwidth that is greater than some threshold level. In other cases, a candidate flow may be defined as a data flow having a data rate that exceeds some threshold amount. In other cases, a candidate stream may be defined as a data stream having a duration greater than a threshold duration. Of course, the threshold level and the threshold amount values may be a design choice based on a number of factors including, for example, the type of data network involved, the number of end users, the total network bandwidth, and the like. In an embodiment, the threshold level may be dynamically adjusted.
The identification of candidate flows in the network may be accomplished based on one or more criteria. In one embodiment, flows exceeding a size threshold (e.g., transmitting at least 3MB) are classified as candidate flows. In an embodiment, a flow that transmits packets within a minimum amount of time is classified as a candidate flow (e.g., a flow that actively transmits packets within 45 seconds or longer). In another example, a candidate stream may be defined as a stream having a data rate that exceeds a certain threshold (e.g., 2 megabits/second). U.S. patent application No.15/060,486, filed 3.3.2016 and published as U.S. patent application publication No.2016/0261510, specifies example conditions and thresholds for identifying candidate streams (e.g., elephant streams), and is incorporated herein by reference in its entirety.
FIG. 1A illustrates an example network environment 100 according to an embodiment. As shown, the network environment 100 includes a transmission manager system 102a, a
In embodiments, the
In an embodiment, the
The two
In some embodiments, the
In some embodiments, the
In some embodiments, it may be assumed that at least a selected portion of the packet data traffic between the
As will be described with reference to fig. 2A and 2B, the transmission manager system 102A includes a stream detector to monitor multiple data streams and select one or more data streams for further processing and/or management.
FIG. 1B illustrates another
The
In an embodiment, selective packet data flows may be identified based on configuration policies or templates characterizing data flows traversing the
The particular flow type (e.g., elephant flow) of packets flowing through
In some embodiments, data flows are identified as candidate flows by sampling packets of an aggregate combination of one or more flows and selecting flows that exceed a threshold data rate measured over a defined sampling duration. In other embodiments, data streams are identified as candidate streams by sampling and selecting streams that exceed a continuous activity duration threshold, where the continuous activity duration threshold may be defined by measuring multiple continuous data rates or a series of data rates, where each data rate exceeds a threshold data rate. In other embodiments, data flows are identified as candidate flows by randomly sampling only some packets of an aggregate combination of one or more flows and selecting flows that exceed a relative detection probability that indicates a relatively disproportionate use of aggregate traffic bandwidth. In other embodiments, these methods may be used in combination or with other similar methods.
In some cases, when the network or transport layer packet data payload is encrypted or obscured, the payload header may be processed/examined to identify packets belonging to the same packet data flow. In other cases, where the packet data payload is readable, information in the network or transport packet payload may be processed/examined to further assist in identifying packets associated with a particular data stream or data stream type (e.g., streaming video).
In some embodiments, once
In some cases, flows may be unidirectional (e.g., uplink or downlink flows) or may be bidirectional by pairing with flows in the opposite direction (e.g., packets with interchanged destination and source network addresses, interchanged port addresses, common flow labels, etc.) that belong to a communication pair connecting endpoints. In some embodiments, both directions of a bi-directional flow pair may be directed to the
In some embodiments, the
Fig. 2A illustrates a
As further shown in fig. 2A, the
FIG. 2B illustrates another transport manager system 200B that may be used to implement the transport manager system 102a of FIG. 1A. The transport manager system 200b includes some of the same components as the
In other embodiments, the stream manager 212A of fig. 2A or the stream manager 212B of fig. 2B may be implemented using a combination of custom circuitry and software executed by programmable circuitry (e.g., by the processor 204).
Fig. 2C illustrates a transport manager system 200C that may be used to implement the transport manager system 102B of fig. 1B. The transport manager system 200c includes many of the same components as the
FIG. 2D illustrates another transport manager system 200D that may be used to implement the transport manager system 102B of FIG. 1B. The transport manager system 200d includes many of the same components as the transport manager system 200B of fig. 2B, but does not include the
In an embodiment, a stream manager, such as
Fig. 3 is a block diagram of a
- 上一篇:一种医用注射器针头装配设备
- 下一篇:片上网络中的端到端服务质量