Techniques for reordering network packets on egress
阅读说明:本技术 用于对出口上的网络分组进行重排序的技术 (Techniques for reordering network packets on egress ) 是由 B.理查森 A.坎宁安 A.J.莱基 B.赖安 P.弗勒明 P.康诺尔 D.亨特 A. 于 2019-05-29 设计创作,主要内容包括:用于对出口上的网络分组进行重排序的技术包括网络接口控制器(NIC),其被配置成使所接收的网络分组与描述符相关联,生成所接收的网络分组的序列标识符,并且将所生成的序列标识符插入到相关联的描述符中。NIC还被配置成确定所接收的网络分组是否要被从与NIC相关联的计算设备传输至另一计算设备,并且响应于所接收的网络分组要被传输至另一计算设备的确定而将描述符插入到描述符传输队列中。此外,NIC被配置成基于描述符在描述符传输队列中的位置来传输网络分组,所述位置基于所生成的序列标识符。本文中描述了其他实施例。(Techniques for reordering network packets on an egress include a Network Interface Controller (NIC) configured to associate a received network packet with a descriptor, generate a sequence identifier for the received network packet, and insert the generated sequence identifier into the associated descriptor. The NIC is further configured to determine whether the received network packet is to be transmitted from the computing device associated with the NIC to another computing device, and insert the descriptor into the descriptor transmission queue in response to a determination that the received network packet is to be transmitted to the other computing device. Further, the NIC is configured to transmit the network packet based on a location of the descriptor in the descriptor transmission queue, the location based on the generated sequence identifier. Other embodiments are described herein.)
1. A Network Interface Controller (NIC) for reordering network packets on an egress, the NIC comprising:
network traffic entry management circuitry to:
network packets received by the NIC are associated with descriptors,
generates a sequence identifier of the received network packet, and
the generated sequence identifier is inserted into the associated descriptor,
network traffic egress management circuitry to:
determining whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device,
inserting the descriptor into a descriptor transmission queue in response to a determination that the received network packet is to be transmitted to another computing device, and
the network packet is transmitted based on a location of the descriptor in the descriptor transmission queue, the location based on the generated sequence identifier.
2. The network interface controller of claim 1, wherein the network traffic ingress management circuitry is further to identify a flow associated with the received network packet, wherein generating the sequence identifier of the received network packet comprises generating the sequence identifier of the received network packet based on the identified flow.
3. The network interface controller of claim 1, wherein inserting the descriptors into the descriptor transfer queue comprises inserting the descriptors into the descriptor transfer queue based on the generated sequence identifier of the descriptors.
4. The network interface controller of claim 1, wherein inserting the descriptor into the descriptor transfer queue comprises inserting the descriptor at a tail of the descriptor transfer queue.
5. The network interface controller of claim 4, wherein the network traffic egress management circuitry is further to reorder the descriptor transfer queue based on a sequence identifier for each descriptor in the descriptor transfer queue.
6. The network interface controller of claim 1, further comprising processing operation management circuitry to orchestrate execution of one or more processing operations on at least a portion of the received network packet, and wherein determining whether the received network packet is to be transmitted from the computing device to another computing device comprises determining that the one or more processing operations have been executed.
7. The network interface controller of claim 1, wherein inserting the generated sequence identifier into the associated descriptor comprises inserting the generated sequence identifier into a predefined sequence identifier field of the associated descriptor.
8. The network interface controller of claim 1, wherein generating a sequence identifier comprises generating one of a timestamp, a sequence number, or a value of an incrementing counter.
9. A Network Interface Controller (NIC) for reordering network packets on an egress, the NIC comprising:
means for associating network packets received by the NIC with descriptors;
means for generating a sequence identifier for the received network packet;
means for inserting the generated sequence identifier into an associated descriptor;
means for determining whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device;
means for inserting a descriptor into a descriptor transmission queue in response to a determination that the received network packet is to be transmitted to another computing device; and
means for transmitting the network packet based on a location of the descriptor in a descriptor transmission queue, the location based on the generated sequence identifier.
10. The network interface controller of claim 9, further comprising means for identifying a flow associated with the received network packet, wherein means for generating a sequence identifier of the received network packet comprises means for generating a sequence identifier of the received network packet based on the identified flow.
11. The network interface controller of claim 9, wherein the means for inserting descriptors into the descriptor transfer queue comprises means for inserting descriptors into the descriptor transfer queue based on the generated sequence identifier of the descriptor.
12. The network interface controller of claim 9, wherein means for inserting descriptors into a descriptor transfer queue comprises means for inserting descriptors at the tail of a descriptor transfer queue.
13. The network interface controller of claim 12, further comprising means for reordering a descriptor transfer queue based on a sequence identifier for each descriptor in the descriptor transfer queue.
14. The network interface controller of claim 9, further comprising means for orchestrating execution of one or more processing operations on at least a portion of the received network packet, and wherein means for determining whether the received network packet is to be transmitted from the computing device to another computing device comprises means for determining that the one or more processing operations have been performed.
15. The network interface controller of claim 9, wherein means for inserting the generated sequence identifier into an associated descriptor comprises means for inserting the generated sequence identifier into a predefined sequence identifier field of the associated descriptor.
16. The network interface controller of claim 9, wherein the means for generating a sequence identifier comprises means for generating one of a timestamp, a sequence number, or a value of an incrementing counter.
17. A method for reordering network packets on an egress, the method comprising:
associating, by a Network Interface Controller (NIC), a network packet received by the NIC with a descriptor;
generating, by the NIC, a sequence identifier of the received network packet;
inserting, by the NIC, the generated sequence identifier into the associated descriptor;
determining, by the NIC, whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device;
inserting, by the NIC and in response to a determination that the received network packet is to be transmitted to another computing device, a descriptor into a descriptor transmission queue; and
transmitting, by the NIC, the network packet based on a location of the descriptor in the descriptor transmission queue, the location based on the generated sequence identifier.
18. The method of claim 17, further comprising identifying, by the NIC, a flow associated with the received network packet, wherein generating the sequence identifier of the received network packet comprises generating the sequence identifier of the received network packet based on the identified flow.
19. The method of claim 17, wherein inserting the descriptors into the descriptor transmit queue comprises inserting the descriptors into the descriptor transmit queue based on a generated sequence identifier of the descriptors.
20. The method of claim 17, wherein inserting the descriptor into the descriptor transfer queue comprises inserting the descriptor at a tail of the descriptor transfer queue.
21. The method of claim 20, further comprising reordering, by the NIC, the descriptor transfer queue based on the sequence identifier of each descriptor in the descriptor transfer queue.
22. The method of claim 17, further comprising orchestrating, by the NIC, execution of one or more processing operations on at least a portion of the received network packet, wherein determining whether the received network packet is to be transmitted from the computing device to another computing device comprises determining that the one or more processing operations have been performed.
23. The method of claim 17, wherein inserting the generated sequence identifier into the associated descriptor comprises inserting the generated sequence identifier into a predefined sequence identifier field of the associated descriptor.
24. The method of claim 17, wherein generating a sequence identifier comprises generating one of a timestamp, a sequence number, or a value of an incrementing counter.
25. A Network Interface Controller (NIC) for reordering network packets on an egress, the NIC comprising:
circuitry for associating network packets received by the NIC with descriptors;
means for generating a sequence identifier for the received network packet;
means for inserting the generated sequence identifier into an associated descriptor;
circuitry for determining whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device;
means for inserting a descriptor into a descriptor transmission queue in response to a determination that the received network packet is to be transmitted to another computing device; and
circuitry for transmitting a network packet based on a location of a descriptor in a descriptor transmission queue, the location based on the generated sequence identifier.
Background
Modern computing devices have become ubiquitous tools for personal, business, and social uses. As such, many modern computing devices are capable of connecting to various data networks, including the internet, to transmit and receive data communications at different rates over the various data networks. To facilitate communication between computing devices, data networks typically include one or more network computing devices (e.g., computing servers, storage servers, etc.) to route communications (e.g., north-south network traffic (traffic)) entering/leaving the network and communications (e.g., east-west network traffic) between network computing devices in the network (e.g., via switches, routers, etc.). In current packet-switched network architectures, data is transmitted in the form of network packets between networked computing devices. At a high level, data is packetized into network packets at one computing device, and the resulting packets are transmitted over a network to another computing device via a transmitting device (e.g., a Network Interface Controller (NIC) of the computing device).
However, certain network infrastructures, such as data centers, may include thousands or more of such computing nodes, making transmission and reception of network packets in sequential order difficult at best. As such, various out-of-order end-to-end protocols have been implemented to manage the transmission and reception of potentially thousands of network packets that may be in flight (in flight) at any given time. For example, to handle such large out-of-order network traffic, microarchitectural techniques have been implemented to provide a hardware level end-to-end retry protocol to take advantage of simply accepting the next packets in a sequence of network packets, regardless of the order in which they are received. In some cases, software has been utilized to impose reliability by reordering the headers of network packets to restore sequential order without hardware checking; however, such software-based reordering is generally not applicable to direct data placement operations or atomic operations. Similarly, the Transmission Control Protocol (TCP) supports selective rejection capability; however, such use cases typically depend on reassembly of received out-of-order network packets in host memory, which typically introduces overhead and latency.
Drawings
In the accompanying drawings, the concepts described herein are illustrated by way of example and not by way of limitation. For simplicity and clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. Where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements.
Fig. 1 is a simplified block diagram of at least one embodiment of a system for reordering network packets on an egress (egress), the system including a source computing device, a destination computing device, and an intermediary computing device communicatively coupled via a network;
FIG. 2 is a simplified block diagram of at least one embodiment of an intermediary computing device of the system of FIG. 1;
FIG. 3 is a simplified block diagram of at least one embodiment of an environment of the intermediary computing device of FIGS. 1 and 2;
FIG. 4 is a simplified flow diagram of at least one embodiment of a method for processing network packets on an egress port that may be performed by the intermediary computing device of FIGS. 1-3; and
fig. 5 is a simplified flow diagram of at least one embodiment of a method for managing transmission of network packets on an egress that may be performed by the intermediary computing device of fig. 1-3.
Detailed Description
While the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intention to limit the concepts of the present disclosure to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with the present disclosure and the appended claims.
References in the specification to "one embodiment," "an illustrative embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may or may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is claimed that: it is within the knowledge of one skilled in the art to effect such features, structures, or characteristics in connection with other embodiments, whether or not explicitly described. Further, it should be appreciated that an item included in the list in the form of "at least one of A, B and C" may mean (a); (B) (ii) a (C) (ii) a (A and B); (A and C); (B and C); or (A, B and C). Similarly, an item listed in the form of "at least one of A, B or C" may mean (a); (B) (ii) a (C) (ii) a (A and B); (A and C); (B and C); or (A, B and C).
In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a form readable by a machine (e.g., a volatile or non-volatile memory, a media disk, or other media device).
In the drawings, some structural or methodical features may be shown in a particular arrangement and/or ordering. However, it should be appreciated that such a particular arrangement and/or ordering may not be required. More specifically, in some embodiments, such features may be arranged in a manner and/or order different from that shown in the illustrative figures. Moreover, the inclusion of structural or methodical features in a particular figure is not intended to imply that such features are required in all embodiments and may not be included or may be combined with other features in some embodiments.
Referring now to fig. 1, in an illustrative embodiment, a
In use, the
Upon receiving the network packet, the
The NIC212 may then perform one or more processing operations on at least a portion of the data associated with the received network packet. Upon completion of the processing operation(s), the NIC212 is also configured to schedule the processed network packet for transmission (e.g., to the destination computing device 108) and enqueue the corresponding descriptor in the transmit descriptor queue. Accordingly, unlike current techniques, the NIC212 is configured to reorder the enqueued network packet descriptors based on their corresponding sequence identifiers and transmit the network packets based on the order of the descriptors in the transmit descriptor queue.
In other words, the sequence identifier may be retrieved again by the NIC212 at transmission time in the same manner that the NIC212 may provide sequencing metadata at reception time, so that the NIC212 may perform reordering of network packets in the transmission queue prior to transmission of the network packets. It should be appreciated that by including the sequence identifier in the descriptor, reordering of the transmission can be made on the transmit descriptor queue (i.e., without forcing the NIC212 to first fetch the network packet). Accordingly, because reordering is performed on the descriptors, rather than the network packets themselves, communication bandwidth (e.g., Peripheral Component Interconnect (PCI) bandwidth) and any additional storage, as may be required to store the retrieved network packets on the NIC212, may be saved. Although illustratively described herein as being performed by the NIC212, it should be appreciated that reordering may be performed during offload operations by any other device, such as an I/O device, accelerator, or the like.
The
Referring now to FIG. 2, an illustrative
The
Processor(s) 202 may be embodied as any type of processor(s) capable of performing the functions described herein. For example, processor(s) 202 may be embodied as one or more single-core processors, multi-core processors, Digital Signal Processors (DSPs), microcontrollers, or other processor(s) or processing/control circuit(s). In some embodiments, processor(s) 202 may be embodied as, include or otherwise be coupled to an FPGA, an ASIC, reconfigurable hardware or hardware circuits, or other special purpose hardware that facilitates the performance of the functions described herein.
The
The one or more
The
It should be appreciated that in some embodiments, the
In some embodiments, execution of one or more of the functions of the
The
The one or more
The
The
Referring now to FIG. 3, in use,
It should be appreciated that in such embodiments, one or more of the circuits (e.g., network traffic
For example, as schematically shown, the network traffic
Further, in some embodiments, one or more of the illustrative components may form a part of another component, and/or one or more of the illustrative components may be independent of each other. Additionally, in some embodiments, one or more of the components of the
In the
The network
The network traffic ingress manager is further configured to generate a sequence identifier for each received network packet and insert the generated sequence identifier into the associated descriptor. As previously described, the sequence identifier may be any type of data that may be used to identify the order of a received network packet relative to other previously received and/or future received network packets, such as a timestamp, a sequence number, a value of an incremental counter, and so forth. In some embodiments, the network
In some embodiments, at least a portion of the network packet (e.g., at least a portion of a header of the network packet, at least a portion of a payload of the network packet, at least a portion of a trailer (footer) of the network packet, etc.) may be stored (e.g., via a DMA operation) in a buffer local to NIC212 and/or a host buffer external to
The network
Further, the network
The
The
Referring now to fig. 4, a
In
In
Referring now to fig. 5, a
In
If the NIC212 determines not to reorder the transmit queue (e.g., the NIC212 inserts the inserted descriptor into the transmit queue based on its sequence identifier), the method skips to block 516, described below. It should be appreciated that in some embodiments, flow indicators may be used such that out-of-order processing and reordering is performed only within a given flow or only on certain flows. Otherwise, if the NIC212 determines to reorder the transmit queue in
In
Examples of the invention
Illustrative examples of the techniques disclosed herein are provided below. Embodiments of the technology may include any one or more of the examples described below, as well as any combination thereof.
Example 1 includes a Network Interface Controller (NIC) to reorder network packets on an egress, the NIC comprising: network traffic ingress management circuitry to associate network packets received by the NIC with descriptors, generate sequence identifiers for the received network packets, and insert the generated sequence identifiers into the associated descriptors; network traffic egress management circuitry to determine whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device, insert a descriptor into a descriptor transmission queue in response to a determination that the received network packet is to be transmitted to another computing device, and transmit the network packet based on a position of the descriptor in the descriptor transmission queue, the position based on the generated sequence identifier.
Example 2 includes the subject matter of example 1, and wherein the network traffic ingress management circuitry is further to identify a flow associated with the received network packet, wherein generating the sequence identifier of the received network packet comprises generating the sequence identifier of the received network packet based on the identified flow.
Example 3 includes the subject matter of any of examples 1 and 2, and wherein inserting the descriptor into the descriptor transmission queue comprises inserting the descriptor into the descriptor transmission queue based on the generated sequence identifier of the descriptor.
Example 4 includes the subject matter of any of examples 1-3, and wherein inserting the descriptor into the descriptor transfer queue comprises inserting the descriptor at a tail of the descriptor transfer queue.
Example 5 includes the subject matter of any of examples 1-4, and wherein the network traffic egress management circuitry is further to reorder the descriptor transfer queue based on a sequence identifier of each descriptor in the descriptor transfer queue.
Example 6 includes the subject matter of any of examples 1-5, and further comprising processing operation management circuitry to orchestrate execution of one or more processing operations on at least a portion of the received network packet, and wherein determining whether the received network packet is to be transmitted from the computing device to another computing device comprises determining that the one or more processing operations have been performed.
Example 7 includes the subject matter of any of examples 1-6, and wherein inserting the generated sequence identifier into the associated descriptor comprises inserting the generated sequence identifier into a predefined sequence identifier field of the associated descriptor.
Example 8 includes the subject matter of any of examples 1-7, and wherein generating the sequence identifier comprises generating one of a timestamp, a sequence number, or a value of an incrementing counter.
Example 9 includes one or more machine-readable storage media comprising a plurality of instructions stored thereon that in response to being executed result in a Network Interface Controller (NIC): associating network packets received by the NIC with descriptors; generating a sequence identifier for the received network packet; inserting the generated sequence identifier into an associated descriptor; determining whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device; inserting a descriptor into a descriptor transmission queue in response to a determination that the received network packet is to be transmitted to another computing device; and transmitting the network packet based on a location of the descriptor in the descriptor transmission queue, the location based on the generated sequence identifier.
Example 10 includes the subject matter of example 9, and wherein the plurality of instructions further cause the NIC to identify a flow associated with the received network packet, wherein generating the sequence identifier of the received network packet comprises generating the sequence identifier of the received network packet based on the identified flow.
Example 11 includes the subject matter of any of examples 9 and 10, and wherein inserting the descriptor into the descriptor transmission queue comprises inserting the descriptor into the descriptor transmission queue based on the generated sequence identifier of the descriptor.
Example 12 includes the subject matter of any of examples 9-11, and wherein inserting the descriptor into the descriptor transfer queue comprises inserting the descriptor at a tail of the descriptor transfer queue.
Example 13 includes the subject matter of any of examples 9-12, and wherein the plurality of instructions further cause the NIC to reorder the descriptor transfer queue based on a sequence identifier of each descriptor in the descriptor transfer queue.
Example 14 includes the subject matter of any of examples 9-13, and wherein the plurality of instructions further cause the NIC to orchestrate execution of one or more processing operations on at least a portion of the received network packet, and wherein determining whether the received network packet is to be transmitted from the computing device to another computing device comprises determining that the one or more processing operations have been performed.
Example 15 includes the subject matter of any of examples 9-14, and wherein inserting the generated sequence identifier into the associated descriptor comprises inserting the generated sequence identifier into a predefined sequence identifier field of the associated descriptor.
Example 16 includes the subject matter of any of examples 9-15, and wherein generating the sequence identifier includes generating one of a timestamp, a sequence number, or a value of an incrementing counter.
Example 17 includes a method for reordering network packets on an egress, the method comprising: associating, by a Network Interface Controller (NIC), a network packet received by the NIC with a descriptor; generating, by the NIC, a sequence identifier of the received network packet; inserting, by the NIC, the generated sequence identifier into the associated descriptor; determining, by the NIC, whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device; inserting, by the NIC and in response to a determination that the received network packet is to be transmitted to another computing device, a descriptor into a descriptor transmission queue; and transmitting, by the NIC, the network packet based on a location of the descriptor in the descriptor transmission queue, the location based on the generated sequence identifier.
Example 18 includes the subject matter of example 17, and further comprising identifying, by the NIC, a flow associated with the received network packet, wherein generating the sequence identifier of the received network packet comprises generating the sequence identifier of the received network packet based on the identified flow.
Example 19 includes the subject matter of any of examples 17 and 18, and wherein inserting the descriptor into the descriptor transmission queue comprises inserting the descriptor into the descriptor transmission queue based on the generated sequence identifier of the descriptor.
Example 20 includes the subject matter of any of examples 17-19, and wherein inserting the descriptor into the descriptor transfer queue comprises inserting the descriptor at a tail of the descriptor transfer queue.
Example 21 includes the subject matter of any of examples 17-20, and further comprising reordering, by the NIC, the descriptor transfer queue based on the sequence identifier of each descriptor in the descriptor transfer queue.
Example 22 includes the subject matter of any of examples 17-21, and further comprising orchestrating, by the NIC, execution of one or more processing operations on at least a portion of the received network packet, wherein determining whether the received network packet is to be transmitted from the computing device to another computing device comprises determining that the one or more processing operations have been performed.
Example 23 includes the subject matter of any of examples 17-22, and wherein inserting the generated sequence identifier into the associated descriptor comprises inserting the generated sequence identifier into a predefined sequence identifier field of the associated descriptor.
Example 24 includes the subject matter of any of examples 17-23, and wherein generating the sequence identifier includes generating one of a timestamp, a sequence number, or a value of an incrementing counter.
Example 25 includes a Network Interface Controller (NIC) to reorder network packets on an egress, the NIC comprising circuitry to associate network packets received by the NIC with descriptors; means for generating a sequence identifier for the received network packet; means for inserting the generated sequence identifier into an associated descriptor; circuitry for determining whether the received network packet is to be transmitted from a computing device associated with the NIC to another computing device; means for inserting a descriptor into a descriptor transmission queue in response to a determination that the received network packet is to be transmitted to another computing device; and circuitry for transmitting the network packet based on a location of the descriptor in the descriptor transmission queue, the location based on the generated sequence identifier.
- 上一篇:一种医用注射器针头装配设备
- 下一篇:一种基于多链路聚合的数据发送方法和装置