阅读说明：本技术 一种PCIe Switch的系统扩展管理方法 (PCIe Switch system expansion management method ) 是由 苏海亮 宁佐林 宁丽霞 马乐 于 2021-08-03 设计创作，主要内容包括：本发明提供了一种PCIe Switch的系统扩展管理方法,包括：确定待扩展的PCIe Switch设备下行端口DP的数量,如果DP的数量超过预设阈值,采用第一扩展模式连接第一PCIe Switch设备和第二PCIe Switch设备；如果DP的数量未超过预设阈值,采用第二扩展模式连接第一PCIe Switch设备和第二PCIe Switch设备,第二扩展模式与第一扩展模式不同。本发明灵活支持多个模式的PCIe Switch扩展配置,可以根据不同的应用场景,通过更新软件固件实现不同模式的适配。特别在DP端口数量较少时,使用内部端口的方式互联,节省Bus编号,适用于Bus编号资源有限的PCIe系统场景。(The invention provides a system expansion management method of PCIe Switch, which comprises the following steps: determining the number of the downlink ports DP of the PCIe Switch equipment to be expanded, and if the number of the DP exceeds a preset threshold value, connecting the first PCIe Switch equipment and the second PCIe Switch equipment by adopting a first expansion mode; and if the number of the DPs does not exceed the preset threshold, connecting the first PCIe Switch equipment and the second PCIe Switch equipment by adopting a second expansion mode, wherein the second expansion mode is different from the first expansion mode. The invention flexibly supports PCIe Switch expansion configuration of a plurality of modes, and can realize adaptation of different modes by updating software firmware according to different application scenes. Particularly, when the number of the DP ports is small, the internal ports are used for interconnection, Bus numbers are saved, and the method is suitable for PCIe system scenes with limited Bus number resources.)

1. A system expansion management method of PCIe Switch equipment is characterized by comprising the following steps:

determining the number of downlink ports DP of PCIe Switch equipment to be expanded;

if the number of the downlink ports DP exceeds a preset threshold value, connecting a first PCIe Switch device and a second PCIe Switch device by adopting a first expansion mode;

and if the number of the downlink ports DP does not exceed the preset threshold, connecting the first PCIe Switch equipment and the second PCIe Switch equipment by adopting a second expansion mode, wherein the second expansion mode is different from the first expansion mode.

2. The system expansion management method for PCIe Switch devices according to claim 1, wherein said connecting the first PCIe Switch device and the second PCIe Switch device in the first expansion mode further comprises:

connecting a downstream port DP of the first PCIe Switch device with an upstream port UP of the second PCIe Switch device;

and configuring a downstream port DP of the first PCIe Switch device as a virtual downstream port VDP, and configuring an upstream port UP of the second PCIe Switch device as a virtual upstream port VUP.

3. The system expansion management method for PCIe Switch devices as recited in claim 2, wherein:

in the first extended mode, the Bus numbers of the first PCIe Switch device and the second PCIe Switch device are different.

4. The system expansion management method for PCIe Switch devices according to claim 1, wherein said connecting the first PCIe Switch device and the second PCIe Switch device in the second expansion mode further comprises:

connecting the first internal port of the first PCIe Switch device with the second internal port of the second PCIe Switch device.

5. The system expansion management method for PCIe Switch devices according to claim 4, wherein in the second expansion mode, all the downstream ports DP in the first PCIe Switch device and the second PCIe Switch device share the same Bus number.

6. The system expansion management method for PCIe Switch devices of claim 5, wherein when the first PCIe Switch device is connected with a host device, the internal port is transparent to the host device.

7. The method of claim 1, wherein the predetermined threshold is a maximum number of devices that can be hooked by a single Bus number in the PCIe Switch device.

8. The system expansion management method for PCIe Switch devices according to claim 1, wherein the first PCIe Switch device and the second PCIe Switch device are connected through an interconnection bus, and the interconnection bus comprises a high-speed I/O pin interconnection interface or a high-speed Serdes interconnection interface.

9. The system expansion management method for the PCIe Switch devices according to claim 1, wherein the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch chip and a second PCIe Switch chip, respectively.

10. The system expansion management method for PCIe Switch devices according to claim 1, wherein the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch Die and a second PCIe Switch Die, respectively.

Technical Field

The invention belongs to the field of bus design, and particularly relates to a system expansion management method for PCIe Switch.

Background

The PCIe Switch is an expansion device in the PCIe topology, and may be used to expand the number of lanes (lanes) for PCIe, so as to enhance the expansion capability of the PCIe device of the main CPU. PCIe Switch is widely applied to a board card device of a large storage system or a server platform. A conventional PCIe expansion mode is as shown in fig. 1, one PCIe Switch chip is connected to another PCIe Switch chip, each device is an independent PCIe Switch chip, and no matter how many devices can be connected to Downstream Ports (DP) of a lower PCIe Switch chip, how many functions of the devices are, an UP port of the lower PCIe Switch always needs to occupy one Bus number in the whole PCIe topology. As shown in fig. 1, there are three PCIe switches, PCIe Switch 1, PCIe Switch 2, and PCIe Switch 3. Each Switch's Upstream Port (UP) occupies a Bus number in the PCIe topology. It can be seen that the prior art has the following problems in the practical application of PCIe extension: the extension of PCIe Switch chips is fixed and lacks flexibility. When the device is extended through the PCIe Switch chip, each PCIe Switch is an individual chip, and occupies a larger space on an actual product board, and device extension is performed through the PCIe Switch chip, which is performed by adding the PCIe Switch individual chip, so that the BOM cost of the board is inevitably increased.

Disclosure of Invention

The invention aims to provide a system expansion management method of PCIe Switch equipment, which comprises the following steps:

determining the number of downlink ports DP of PCIe Switch equipment to be expanded;

if the number of the DPs exceeds a preset threshold value, connecting a first PCIe Switch device and a second PCIe Switch device by adopting a first expansion mode;

and if the number of the DPs does not exceed the preset threshold, connecting the first PCIe Switch equipment and the second PCIe Switch equipment by adopting a second expansion mode, wherein the second expansion mode is different from the first expansion mode.

Preferably, the connecting the first PCIe Switch device and the second PCIe Switch device in the first expansion mode further includes:

connecting a downstream port DP of the first PCIe Switch device with an upstream port UP of the second PCIe Switch device; and configuring a downstream port DP of the first PCIe Switch device as a virtual downstream port VDP, and configuring an upstream port UP of the second PCIe Switch device as a virtual upstream port VUP.

Preferably, in the first extended mode, the Bus numbers of the first PCIe Switch device and the second PCIe Switch device are different.

Preferably, the connecting the first PCIe Switch device and the second PCIe Switch device in the second expansion mode further includes:

connecting the first internal port of the first PCIe Switch device with the second internal port of the second PCIe Switch device.

Preferably, in the second extended mode, all the downstream ports DP in the first PCIe Switch device and the second PCIe Switch device share the same Bus number. When the first PCIe Switch device is connected with a host device, the internal port is transparent to the host device.

Preferably, the preset threshold is the maximum number of devices that can be hooked by a single Bus number in the PCIe device.

Preferably, the first PCIe Switch device and the second PCIe Switch device are connected through an interconnection bus, where the interconnection bus includes a high speed I/O pin interconnection interface or a high speed Serdes interconnection interface.

Preferably, the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch chip and a second PCIe Switch chip, respectively.

Preferably, the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch Die and a second PCIe Switch Die, respectively.

Compared with the prior art, the scheme of the invention selects different modes according to the number of DP ports of Die or PCIe Switch extension. For the Die or PCIe Switch chip of the same PCIe Switch, configuration of multiple modes can be flexibly supported, so that Die or PCIe Switch chip extensions of multiple forms are supported. The configuration is carried out in the initialization stage of PCIe Switch or Die, and the adaptation of different modes can be realized by updating software and firmware according to different application scenes. When the number of the DP ports is small, the internal ports are used for interconnection, Bus numbers are saved, and the method is particularly suitable for PCIe system scenes with limited Bus number resources.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic diagram of the structure of an extended mode of PCIe Switch according to the prior art.

Fig. 2 is a schematic diagram of a first extended mode of PCIe Switch according to the present invention.

Fig. 3 is a schematic diagram of a second extended mode of PCIe Switch according to the present invention.

Fig. 4 is a schematic diagram of an interconnect structure of more than two PCIe switches according to the second expansion mode of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention discloses an expansion scheme of PCIe Switch, which comprises the steps of firstly judging the number of expanded PCIe downlink ports DP, determining an expansion mode to be used according to the difference of the number of the expanded PCIe downlink ports DP, and if the number of the DPs exceeds a preset threshold value, selecting the same mode as the general PCIe Switch expansion as a first expansion mode. If the number of DPs does not exceed the preset threshold, the way in which multiple PCIe switches share the Bus number may be selected as the second expansion mode. The preset threshold is the maximum number of devices that a single Bus number can be hooked in the PCIe Switch.

The PCIe Switch is connected with the PCIe Switch through an interconnection bus, the interconnection bus is a self-defined interconnection interface, and the interconnection bus can be configured into ports such as VUP (virtual private protocol), VDP (virtual private protocol) and the like according to application scenes. However, for the Host, the Host end does not need to distinguish the interconnect interface from the common PCIe port. For example, the interconnect interface may be a high speed I/O pin interconnect or a high speed Serdes interconnect.

In one embodiment, the preset threshold is 32. In the first expansion mode, namely when the number of the downstream ports DP exceeds 32, the mode of the common PCIe Switch expansion is utilized. As shown in fig. 2, since a Bus number can be attached with 32 devices at most in the PCIe protocol, when the number of PCIe DPs exceeds 32, it is necessary to connect the UP port of the lower PCIe Switch to one of the DP ports of the upper PCIe Switch through a new Bus number by expanding the Bus number, and attach another device under the new Bus number. And by analogy, hanging 32 devices on the new Bus number at most, and if the devices which are not hung still exist, continuing to expand the new Bus number.

The port of the interconnection interface between PCIe switches is also a port of the Bus internal switching network, and the ports may be configured in multiple roles, for example, three roles, such as a VUP (virtual UP port), a VDP (virtual DP port), and an internal port. Referring to fig. 2, in a preferred embodiment, if two switches PCIe Switch 1 and PCIe Switch 2 are interconnected, and the number of extended DP ports exceeds 32, the ports of the interconnect interface of PCIe Switch 1 are configured as VDP ports, fig. 2 exemplarily shows DP1_0 as VDP, and the ports of the interconnect interface of PCIe Switch 2 are configured as VUP ports, and fig. 2 exemplarily shows UP2 as VUP.

For example, if the VDP port is number DP1_0 in PCIe Switch 1, the VUP port is UP2 throughout the PCIe topology. The Host detects the presence of a DP1_0 device on Bus64 during the enumeration phase, but the DP1_0 appears to the Host to be identical to the other DPs. And Host will detect that a UP2, i.e. a UP under Bus65, is hooked under the DP1_0 device, and the UP is VUP. But from the Host's perspective, the UP2 is identical to the ordinary PCIe UP port.

The VUPs and VDPs comprise at least PCIe 4KB configuration space registers, other PCIe functions can be extended as needed, and all the VUPs and VDPs conform to PCIe protocol specifications.

In one embodiment, the preset threshold is 32. In the second expansion mode, that is, when the downstream port DP does not exceed the maximum number of devices that can be attached by a single Bus number, two PCIe Switch expansion devices may be used, but all the downstream ports DP in the two PCIe switches share one Bus number, as shown in fig. 3.

If the two switches of the PCIe Switch 1 and the PCIe Switch 2 are interconnected, and the number of the expanded DP ports is less than 32, the interconnection is performed in the manner shown in fig. 3, at this time, the ports of the two switches of the PCIe Switch 1 and the PCIe Switch 2 are both configured as internal ports, and the interconnection port at this time is only one port of the Switch internal switching network. One Bus number can be saved. Because of the internal port, the Host end can not sense the existence of the connection of PCIe Switch 2, and can not sense the port of VUP/VDP in the PCIe Switch 2, and can only detect that a maximum of 32 DPs are connected under Bus 64.

Those skilled in the art will appreciate that the topology of the elements and the number of functional blocks described in the above embodiments are by way of example only. Those skilled in the art can apply the method of the present invention to the interconnection of two PCIe Switch devices or to the interconnection of a plurality of PCIe Switch devices as needed. Fig. 4 shows a case where PCIe Switch 1 and PCIe Switch 2 and PCIe Switch 3 are respectively interconnected.

As shown in fig. 4, three PCIe switches share one Bus number, and ports of two switches of PCIe Switch 1, PCIe Switch 2, and PCIe Switch 3 are configured as internal ports, and the interconnect port at this time is only one port of the Switch internal switching network. Two Bus numbers can be saved. The Host end can not sense the existence of the connection of PCIe switches 2 and 3, and can only detect that a maximum of 32 DPs are connected under Bus 64.

According to an alternative embodiment, the PCIe Switch 1 and PCIe Switch 2 may be two PCIe Switch Die, or may be two independent Switch chips.

When the PCIe Switch Die 1 and Die 2 need to be connected in an expansion mode, the expansion mode used is determined according to the number of the DP of the downstream ports, and if the number of the DP exceeds a preset threshold, the same mode as that of the general PCIe expansion can be selected. If the number of DPs does not exceed the preset threshold, a way of sharing the Bus number by multiple Die may be selected. The preset threshold is the maximum number of devices which can be hooked by a single Bus number in the Die. The above ports can be configured in various roles, for example, three roles of VUP (virtual UP port), VDP (virtual DP port), internal port, and the like.

In one embodiment, the preset threshold is 32. Namely, when the number of the downstream ports DP exceeds 32, the mode of the common PCIe extension is utilized. Since a Bus number can be hooked with 32 devices at most, when the number of the DPs exceeds 32, the Bus number needs to be expanded. Connecting the UP port of the lower-level Die with one of the DP ports of the upper-level Die through a new Bus number, and additionally hanging other equipment under the new Bus number. And by analogy, hanging 32 devices on the new Bus number at most, and if the devices which are not hung still exist, continuing to expand the new Bus number.

When the number of the downlink ports DP does not exceed the maximum equipment number which can be hooked by a single Bus number, two Dies can share one Bus number. At this time, the ports of both switches of Die 1 and Die 2 are configured as internal ports, and the interconnect port at this time is only one port of the Die internal switching network. Thus saving one Bus number. Because of the internal ports, the Host end cannot sense the existence of the connection of Die 2, and cannot sense the port of VUP/VDP in Die 2, and only up to 32 DPs connected under Bus64 can be detected.

When the PCIe Switch 1 and the PCIe Switch 2 are both Die, a chip can be sealed for use, and the BOM cost of a board can be saved.

Therefore, by adopting the technical scheme of the invention, the Die or PCIe Switch chip of the same PCIe Switch can flexibly support the configuration of a plurality of modes, thereby supporting the expansion of the Die or PCIe Switch chip of a plurality of forms. The invention can select different modes according to the number of DP ports of Die or PCIe Switch expansion. The configuration is carried out in the initialization stage of PCIe Switch or Die, and the adaptation of different modes can be realized by updating software and firmware according to different application scenes. When the number of the DP ports is excessive, the data transmission method is interconnected by using a VUP (virtual private protocol) and VDP (virtual private protocol) mode, and is compatible with the expansion mode of the current PCIe system. When the number of the DP ports is small, the internal ports are used for interconnection, Bus numbers are saved, and the method is particularly suitable for PCIe system scenes with limited Bus number resources.

Although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.