Direct data transfer between FPGAs Virtex-7 via PCI Express bus

• Main Author: Yu. A. Rumyantsev
• Format: Article
• Language: English
• Published: Ivannikov Institute for System Programming of the Russian Academy of Sciences 2018-10-01
• Series: Труды Института системного программирования РАН
• Subjects: плис; fpga; pci express
• Online Access: https://ispranproceedings.elpub.ru/jour/article/view/953

This article describes two types of data transfers via PCI Express bus involving several FPGA. The first one is a simultaneous DMA data transfer between the system memory and different FPGA chips. The second one is a simultaneous direct data transfer between different FPGA. The data transfer speed was measured for both cases with results being about 99% from maximum speed for PCIe x4 Gen 2.0 link for the direct transfer between FPGAs (1603 MB/s for 128 bytes payload and 1740 MB/s for 256 bytes payload). The direct data transfer latency was also measured to be 0,7 us for one intermediate PCIe switch and 1 us for three intermediate switches. Also the effect of simultaneous transfers on data transfer speed was studied with the result that, as long as the aggregate transfer speed does not overcome the shared link bandwidth, each transfer is performed on its maximum speed; after that the shared link utilization reaches 100% with its bandwidth being distributed equally between individual transfers.