阅读说明：本技术 基于PCIe的高速实时传输系统 (PCIe-based high-speed real-time transmission system ) 是由 晏佳 白静 唐俊勇 刘远漠 于 2021-11-15 设计创作，主要内容包括：本发明提供一种基于PCIe的高速实时传输系统,所述基于PCIe的高速实时传输系统包括基于FPGA的PCIe板卡和计算机,所述基于FPGA的PCIe板卡插在所述计算机的PCI-E X8插槽；所述基于FPGA的PCIe板卡的通道类型为PCI-E X8,以FPGA芯片为核心,FPGA芯片端外扩高速接口和并行接口,且外接内存,其中并行接口通过隔离芯片与FPGA相连。通过本发明,充分利用FPGA的高速传输接口和并行接口,兼顾高速串行传输和并行传输的优点,提供了两种不同的对外传输方式,确保了传输的高速率、高实时性。(The invention provides a PCIe-based high-speed real-time transmission system, which comprises a PCIe board card based on an FPGA and a computer, wherein the PCIe board card based on the FPGA is inserted into a PCI-E X8 slot of the computer; the PCIe board card based on the FPGA is characterized in that the channel type of the PCIe board card based on the FPGA is PCI-E X8, an FPGA chip is used as a core, a high-speed interface and a parallel interface are externally expanded at the end of the FPGA chip, and the FPGA chip is externally connected with a memory, wherein the parallel interface is connected with the FPGA through an isolation chip. According to the invention, the high-speed transmission interface and the parallel interface of the FPGA are fully utilized, the advantages of high-speed serial transmission and parallel transmission are taken into consideration, two different external transmission modes are provided, and the high speed and the high real-time performance of transmission are ensured.)

1. The PCIe-based high-speed real-time transmission system is characterized by comprising an FPGA-based PCIe board card and a computer, wherein the FPGA-based PCIe board card is inserted into a PCI-E X8 slot of the computer; the PCIe board card based on the FPGA is characterized in that the channel type of the PCIe board card based on the FPGA is PCI-E X8, an FPGA chip is used as a core, a high-speed interface and a parallel interface are externally expanded at the end of the FPGA chip, and the FPGA chip is externally connected with a memory, wherein the parallel interface is connected with the FPGA through an isolation chip.

2. The PCIe-based high-speed real-time transport system of claim 1, wherein the FPGA chip implements PCIe protocol and DMA transport through an XDMA core, the XDMA core comprising an M _ AXI interface and an M-AXI-Lite interface, wherein the M _ AXI interface is used to transport DMA data and the M-AXI-Lite interface is used to access register data.

3. The PCIe-based high-speed real-time transport system of claim 2, wherein the FPGA chip is connected to the high-speed interface through a register-switched communication link to implement high-speed serial transport; the FPGA chip is connected with the parallel interface through the register switching communication link to realize parallel transmission.

4. The PCIe-based high-speed real-time transport system of claim 3, wherein when the high-speed interface is connected, the DMA data enters the memory cache first, the Register data is cached in the Register BRAM, and the DMA data or the Register read-write data is selected by the high-speed transport control mechanism for data processing.

5. The PCIe-based high speed real time transport system of claim 3, wherein when the parallel interface is connected, DMA data or register read and write data is selected for data processing by a parallel transport control mechanism.

6. The PCIe-based high-speed real-time transmission system of claim 1, wherein the computer, during a boot process, through the PCIe driver, iteratively reads a configuration space of all PCIe boards in a running processor system according to a PCIe bus number, a device name, and a function number, the configuration space being located in an XDMA core of the PCIe boards; comparing the Vendor ID field and the Device ID field in the configuration space with the Vendor ID and the Device ID of the PCIe board card to be searched, and if the two fields are equal, successfully searching; initializing the base address register space of the searched PCIe board card after the search is successful; and after the initialization is completed, data transmission is carried out.

7. The PCIe-based high speed real time transport system of claim 6, wherein the PCIe driver is designed based on an RTX-based real time system.

8. The PCIe-based high speed real time transport system of claim 6, wherein the step of initializing the base address register space for the located PCIe board comprises:

and allocating the address space of the basic address register of the PCIe board card in the PCIe bus domain.

9. The high speed real time transmission system of claim 6 wherein said step of performing data transmission comprises:

the computer accesses the M-AXI-Lite interface user register of the XDMA core of the PCIe board card, and then accesses the next level board card through the parallel interface.

10. The high speed real time transmission system of claim 6 wherein said step of performing data transmission comprises:

the computer selects a transmission interface of the PCIe board card as a high-speed interface, applies for a section of continuous memory space for DMA data transmission; creating a section of continuous memory space according to the size of the descriptor, and writing a descriptor base address; the DMA transfer is turned on.

Technical Field

The invention relates to the technical field of communication, in particular to a high-speed real-time transmission system based on PCIe.

Background

With the development of communication technology, the transmission and processing of big data put forward higher and higher requirements on high-speed communication, and especially when the application software at the PC end needs to exchange data with hardware, the common high-speed interconnection interfaces closest to the application software at the PC end have PCIe, ethernet, USB, HDMI, etc., and these interfaces have their own characteristics, wherein because the PCIe bus has a high transmission rate, a mature technology and a flexible implementation method, it is one of the most common high-speed transmission interfaces, and is also applied to various fields more and more widely, especially in occasions with large data throughput.

PCIe cards are often used in Windows and Linux system environments. Because Windows and Linux are non-real-time systems, the requirements can be well met under the condition that the real-time performance of data transmission is not required. However, in the field of automation industry, in particular in the field of measurement, testing and control technology, it is indispensable to ensure real-time data transmission. For the use scene with higher real-time requirement, although the Linux system can solve the problem by making real-time patches, the actual use environment of the client software is more that when the client software runs under the environment of the Windows system, the real-time requirement cannot be met.

Disclosure of Invention

The invention mainly aims to provide a high-speed real-time transmission system based on PCIe (peripheral component interface express), aiming at the technical problem of low transmission real-time performance in the prior art.

In a first aspect, the invention provides a PCIe-based high-speed real-time transmission system, which includes a PCIe board based on an FPGA and a computer, where the PCIe board based on the FPGA is inserted into a PCI-E X8 slot of the computer; the PCIe board card based on the FPGA is characterized in that the channel type of the PCIe board card based on the FPGA is PCI-E X8, an FPGA chip is used as a core, a high-speed interface and a parallel interface are externally expanded at the end of the FPGA chip, and the FPGA chip is externally connected with a memory, wherein the parallel interface is connected with the FPGA through an isolation chip.

Optionally, the FPGA chip implements PCIe protocol and DMA transfer through the XDMA core, where the XDMA core includes an M _ AXI interface and an M-AXI-Lite interface, where the M _ AXI interface is used to transfer DMA data, and the M-AXI-Lite interface is used to access register data.

Optionally, the FPGA chip is connected to the high-speed interface through a register switching communication link to implement high-speed serial transmission; the FPGA chip is connected with the parallel interface through the register switching communication link to realize parallel transmission.

Optionally, when the high-speed interface is connected, the DMA data first enters the memory to be cached, the Register data is cached in the Register BRAM, and the DMA data or the Register read-write data is selected to perform data processing through the high-speed transmission control mechanism.

Optionally, when the parallel interface is connected, the DMA data or the register read-write data is selected by the parallel transmission control mechanism to perform data processing.

Optionally, in the process of starting up the computer, the computer iteratively reads configuration spaces of all PCIe boards in an operating processor system according to the PCIe bus number, the device name, and the function number through a PCIe driver, where the configuration spaces are located in an XDMA core of the PCIe board; comparing the Vendor ID field and the Device ID field in the configuration space with the Vendor ID and the Device ID of the PCIe board card to be searched, and if the two fields are equal, successfully searching; initializing the base address register space of the searched PCIe board card after the search is successful; and after the initialization is completed, data transmission is carried out.

Optionally, the PCIe driver is designed based on an RTX real-time system.

Optionally, the step of initializing the searched basic address register space of the PCIe board includes:

and allocating the address space of the basic address register of the PCIe board card in the PCIe bus domain.

Optionally, the step of performing data transmission includes:

the computer accesses the M-AXI-Lite interface user register of the XDMA core of the PCIe board card, and then accesses the next level board card through the parallel interface.

Optionally, the step of performing data transmission includes:

the computer selects a transmission interface of the PCIe board card as a high-speed interface, applies for a section of continuous memory space for DMA data transmission; creating a section of continuous memory space according to the size of the descriptor, and writing a descriptor base address; the DMA transfer is turned on.

The PCIe-based high-speed real-time transmission system comprises a PCIe board card based on an FPGA and a computer, wherein the PCIe board card based on the FPGA is inserted into a PCI-E X8 slot of the computer; the PCIe board card based on the FPGA is characterized in that the channel type of the PCIe board card based on the FPGA is PCI-E X8, an FPGA chip is used as a core, a high-speed interface and a parallel interface are externally expanded at the end of the FPGA chip, and the FPGA chip is externally connected with a memory, wherein the parallel interface is connected with the FPGA through an isolation chip. According to the invention, the high-speed transmission interface and the parallel interface of the FPGA are fully utilized, the advantages of high-speed serial transmission and parallel transmission are taken into consideration, two different external transmission modes are provided, and the high speed and the high real-time performance of transmission are ensured.

Drawings

FIG. 1 is a schematic diagram of an architecture of an FPGA-based PCIe board;

FIG. 2 is a logic schematic block diagram of an FPGA chip;

FIG. 3 is a block diagram of a RTX real-time system based PCIe driven architecture;

FIG. 4 is a flowchart illustrating DMA read and write operations performed by a computer according to an embodiment.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In a first aspect, an embodiment of the present invention provides a PCIe-based high-speed real-time transmission system.

In one embodiment, the PCIe-based high-speed real-time transmission system comprises an FPGA-based PCIe board and a computer, wherein the FPGA-based PCIe board is inserted into a PCI-E X8 slot of the computer; the PCIe board card based on the FPGA is characterized in that the channel type of the PCIe board card based on the FPGA is PCI-E X8, an FPGA chip is used as a core, a high-speed interface and a parallel interface are externally expanded at the end of the FPGA chip, and the FPGA chip is externally connected with a memory, wherein the parallel interface is connected with the FPGA through an isolation chip.

Referring to fig. 1, fig. 1 is a schematic diagram of an architecture of an FPGA-based PCIe board. As shown in fig. 1, the PCIe board channel type based on the FPGA is PCI-E X8, which can support high-speed transmission of PCIe GEN3x8, and the FPGA chip is used as a core, and the FPGA chip end external expansion high-speed interface (i.e., the optical fiber interface QSFP +) in fig. 1) and the parallel interface (i.e., the Local Bus CN in fig. 1) are connected to an external memory (i.e., the DDR4 in fig. 1) for data caching. The Local Bus is connected with the Program Logic part of the FPGA through the isolation chip, so that the FPGA chip is well protected, and the level conversion is completed.

Further, in an embodiment, the FPGA chip implements PCIe protocol and DMA transfer through an XDMA core, and the XDMA core includes an M _ AXI interface and an M-AXI-Lite interface, where the M _ AXI interface is used to transfer DMA data, and the M-AXI-Lite interface is used to access register data.

In this embodiment, a PCIe hard core is integrated in the FPGA chip, and development software thereof has a plurality of PCIe IP implementation methods, and here, an XDMA core is used to implement PCIe protocol and DMA transmission.

Referring to fig. 2, fig. 2 is a logic schematic block diagram of an FPGA chip. As shown in fig. 2, the user interface of the XDMA core is divided into an M _ AXI interface, which is a transfer DMA data, and an M-AXI-Lite interface, which is an access register data. The Register data is cached by a Register BRAM and is exchanged with the subsequent logic, and in addition, some control instructions of the subsequent logic are also generated by the Register data.

Further, in an embodiment, the FPGA chip is connected to the high-speed interface through a register switching communication link to implement high-speed serial transmission; the FPGA chip is connected with the parallel interface through the register switching communication link to realize parallel transmission.

In the embodiment, in order to fully utilize the advantages of high rate of serial data transmission and low time delay of parallel data transmission, the communication link is switched to connect with the optical fiber interface QSFP + through the register inside the FPGA chip, so that high-speed serial transmission is realized; the parallel interface Local Bus CN is connected through a register switching communication link to realize parallel transmission.

Further, in an embodiment, when the high-speed interface is connected, the DMA data first enters the memory buffer, the Register data is buffered in the Register BRAM, and the DMA data or the Register read-write data is selected by the high-speed transmission control mechanism to perform data processing.

In this embodiment, when the optical fiber interface QSFP + is connected, in order to ensure that the Data rate of PCIe DMA Data and the optical fiber interface QSFP + is matched, the DMA Data may first enter the DDR4 for buffering, the Register Data may be buffered in the Register BRAM, and the DMA Data or the Register read-write Data is selected to be processed with the Data Process through a control mechanism of a High Speed Transmission Controller, where the processing of the Data Process is to perform packaging and unpacking processing according to a self-defined Data structure, so as to use an Aurora Transmission protocol, and finally, the Data is output from the optical fiber interface QSFP +.

Further, in an embodiment, when the parallel interface is connected, the DMA data or the register read/write data is selected by the parallel transfer control mechanism to perform data processing.

In this embodiment, when the parallel interface Local Bus CN is connected, DMA Data or register read/write Data is selected to perform Data processing by the control mechanism of the Local Bus Transmission Controller. The Data Process is a parallel transmission interface which converts the transmitted Data and control signals into a parallel interface with Local Bus. The Local Bus interface is composed of 32-bit data lines, 24-bit address lines, and necessary control lines.

Further, in an embodiment, in the boot process of the computer, through a PCIe driver, iteratively reading configuration spaces of all PCIe boards in a running processor system according to a PCIe bus number, a device name, and a function number, where the configuration spaces are located in an XDMA core of the PCIe board; comparing the Vendor ID field and the Device ID field in the configuration space with the Vendor ID and the Device ID of the PCIe board card to be searched, and if the two fields are equal, successfully searching; initializing the base address register space of the searched PCIe board card after the search is successful; and after the initialization is completed, data transmission is carried out.

Further, in an embodiment, the PCIe driver is designed based on an RTX real-time system.

In this embodiment, the computer controls the PCIe board based on the FPGA through the PCIe driver based on the RTX real-time system. The computer system is Windows, and since Windows is a non-real-time system and is realized based on PCIe drive of RTX real-time system, firstly, Windows can be reformed into a hard real-time operating system with definite response. The Windows-based RTX real-time extension subsystem developed by Ardence corporation is the first choice, mainly because the RTX is seamlessly compatible with the Windows operating system, and can utilize various advantages of the Windows system, including a large number of standard API functions, an efficient memory management mechanism, and various universal resources under Windows. As a complete Windows extension system, RTX does not make any encapsulation and modification to the Windows system architecture. Compared with the PCIe driver based on Windows, the implementation process of the PCIe driver based on RTX is simple and clear due to the API function provided by RTX. A schematic diagram of a PCIe driver based on the RTX real-time system is shown in fig. 3. The method comprises the steps of firstly searching a PCIe board card, initializing a basic address register space of the searched PCIe board card after the PCIe board card is successfully searched, and transmitting data after the initialization is completed. If the PCIe board card is searched for failure, exiting; and if the initialization fails, exiting.

The step of finding the PCIe board card specifically comprises the following steps:

iteratively reading configuration spaces of all PCIe boards in a running processor system according to the serial number, the device name and the function number of the PCIe bus, wherein the configuration spaces are positioned in an XDMA core of the PCIe boards; and comparing the Vendor ID field and the Device ID field in the configuration space with the Vendor ID and the Device ID of the PCIe board card to be searched, and if the two fields are equal, successfully searching.

In this embodiment, in a processor system, each Host bridge pushes out a PCIe bus number. How many PCIe bridges, including the HOST HOST bridge, are in a PCI bus count, and how many PCIe buses are in the count. When the system searches for the appointed PCIe devices in a traversal mode, the corresponding PCIe bus is found firstly, various PCIe devices are hooked under one PCIe bus, and each PCIe device has a unique device number under the PCIe bus. The system software locates a PCIe device by bus number and device number. Each PCI device may have only one function or may have up to 8 functions, i.e., multi-function devices. Regardless of how many functions the PCIe device has, each function has a unique and independent configuration space corresponding to it.

The specific implementation of finding the PCIe board card is that the configuration space of all PCIe board cards is read iteratively in a running processor system according to the PCIe bus number, the Device name and the function number, the Vendor ID field and the Device ID field in the configuration space are compared with the Vendor ID and the Device ID of the PCIe board card to be found, if the two fields are equal, the two fields exist, and if the two fields are not equal, the finding is failed, and the program is exited.

Because the FPGA of the PCIe board card can change the Vendor ID field and the Device ID field of the PCIe Device when the XDMA IP core is configured, when the Vendor ID fields and the Device ID fields of a plurality of PCI devices are consistent, the search fails.

Further, in an embodiment, the step of initializing the searched base address register space of the PCIe board includes:

and allocating the address space of the basic address register of the PCIe board card in the PCIe bus domain.

In this embodiment, before the PCIe board performs data transmission, the system software needs to initialize the BAR0-5 register of the PCIe board. The system software completes the initialization of the registers, namely the address space of the devices in the PCIe bus domain is distributed when the PCIe bus passes through. After the registers are initialized, the PCIe board card can use the PCIe bus address to perform data transmission.

Addresses in the BAR (base Address register) 0-5 in the PCIe boards are PCIe bus addresses that have an image in the memory domain of the processor system, and if the BAR space of one PCIe board does not have an image in the memory domain, the processor will not be able to access the BAR space of the PCIe board.

The processor isolates the PCIe bus domain from the memory domain through the HOST bridge. When the processor accesses the address control of the PCIe board, the address space of the device in the memory domain needs to be accessed first, and after the address space of the memory domain is converted into the address space of the PCIe bus domain through the HOST HOST bridge, the data is sent to the appointed PCIe board by using PCIe bus transaction.

Firstly, the address of the BAR space is converted into a physical address which can be directly accessed by the CPU, then the converted physical address is mapped to a virtual address which can be accessed by an application program, and after the converted physical address is mapped to the virtual address space, a user can normally read and write the memory space or the I/O space of the PCIe board card.

Further, in an embodiment, the step of performing data transmission includes:

the computer accesses the M-AXI-Lite interface user register of the XDMA core of the PCIe board card, and then accesses the next level board card through the parallel interface.

In this embodiment, an application program in the computer accesses an M-AXI-Lite interface user register of the XDMA in the board card through a PCIe driver based on an RTX, and then accesses the next-level board card through a Local Bus parallel interface, and thus, low latency is achieved in hardware, thereby ensuring real-time performance of the entire transmission link.

Further, in an embodiment, the step of performing data transmission includes:

the computer selects a transmission interface of the PCIe board card as a high-speed interface, applies for a section of continuous memory space for DMA data transmission; creating a section of continuous memory space according to the size of the descriptor, and writing a descriptor base address; the DMA transfer is turned on.

In this embodiment, referring to fig. 4, fig. 4 is a flowchart illustrating DMA read/write performed by a computer in an embodiment. As shown in fig. 4, a section of continuous memory space is applied for DMA data transmission; then, a section of continuous memory space is created according to the size of the descriptor; writing a descriptor base address, and starting DMA transmission; detecting whether the transmission is finished or not, and if not, continuing the transmission; if the DMA operation is finished, the DMA operation is stopped.

In the embodiment, the PCIe-based high-speed real-time transmission system comprises an FPGA-based PCIe board card and a computer, wherein the FPGA-based PCIe board card is inserted into a PCI-E X8 slot of the computer; the PCIe board card based on the FPGA is characterized in that the channel type of the PCIe board card based on the FPGA is PCI-E X8, an FPGA chip is used as a core, a high-speed interface and a parallel interface are externally expanded at the end of the FPGA chip, and the FPGA chip is externally connected with a memory, wherein the parallel interface is connected with the FPGA through an isolation chip. Through this embodiment, make full use of FPGA's high-speed transmission interface and parallel interface, compromise the advantage of high-speed serial transmission and parallel transmission, provide two kinds of different external transmission modes, ensured high speed, the high real-time nature of transmission.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for causing a terminal device to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.