FPGA Gigabit UDP network video transmission, implemented based on 88E1518 PHY, providing engineering and QT host computer source code plus technical support

1 Introduction

Even those who have never played with UDP protocol stack are embarrassed to say that they have played with FPGA. This is a sentence said by a CSDN boss, and I firmly believe it. . .
The UDP protocol stack is widely used in actual projects, especially in the medical and military industries. The image splicing solutions currently on the market mainly include the Video Mixer solution officially launched by Xilinx and the custom solution of hand-shredding the code; the Video Mixer officially launched by Xilinx The solution directly calls IP and can be implemented through SDK configuration. However, it is difficult to enable and requires high FPGA resources. It is not suitable for small-scale FPGA. It is very used on zynq and K7 and above platforms. If you are interested in Video If you are interested in the Mixer solution, you can refer to my previous blog. Blog address:
Click to go directly

This article uses Xilinx's Kintex7 FPGA based on the 88E1518 network PHY chip to implement Gigabit UDP video transmission. There are two video sources, corresponding to whether the developer has a camera in hand. One is to use a cheap OV5640 camera module; the other is The first is that if you do not have a camera in your hand, or your development board does not have a camera input interface, you can use the dynamic color bar generated inside the code to simulate the camera video. The video source is selected through the `define macro definition at the top level of the code. Power on. The OV5640 camera module is selected as the video input source by default; after the FPGA collects the video, it uses FDMA to cache the video into DDR3, then reads the video out, packages the video with UDP data according to the communication protocol with the QT host computer, and then uses our The UDP protocol stack encapsulates the UDP data of the video, then sends the data to the Tri Mode Ethernet MAC IP, outputs it to the 88E1518 network PHY on the development board, and then transmits the UDP video to the computer through the RJ45 network port on the development board through the network cable. The host and computer use the QT host computer we provide to collect images and display them; provide the FPGA engineering source code of the vivado2019.1 version and the QT host computer and its source code;

This blog describes in detail the design of FPGA based on the 88E1518 network PHY chip to implement Gigabit UDP video transmission. The engineering code can be comprehensively compiled and debugged on the board, and can be directly transplanted into the project. It is suitable for school students and graduate student project development, and is also suitable for in-service applications. Engineers learn and improve, which can be applied to high-speed interfaces or image processing fields in medical, military and other industries;
complete and run-through engineering source code and technical support are provided;
the method of obtaining engineering source code and technical support is placed at the end of the article, please be patient See the end;

Version update instructions

This version is the 2nd version. Based on readers' suggestions, the following improvements and updates have been made to the 1st version of the project:
1: Added the option of inputting video dynamic color bars. Some readers said that they do not have an OV5640 camera in their hands, or the camera principle The picture is inconsistent with mine, which makes the transplantation process very difficult. Based on this, a dynamic color bar is added. It is generated internally by the FPGA and can be used without an external camera. The usage method is explained later; 2:
Optimized FDMA, the data read and write burst length of AXI4 in the previous FDMA was 256, which resulted in insufficient bandwidth on low-end FPGAs, resulting in poor image quality. Based on this, the data read and write burst length of AXI4 in FDMA was changed to 128;
3: Optimized the code of the UDP protocol stack and its data buffer FIFO group, and added the code description of this part in the blog post; 4: Added instructions for the use, update, and modification of the
Tri Mode Ethernet MAC IP core to a separate document The form is placed in the information package;
5: The overall code structure is optimized, making the code that seemed messy before become fresh and concise;

Disclaimer

This project and its source code include both parts written by myself and parts obtained from public channels on the Internet (including CSDN, Xilinx official website, Altera official website, etc.). If you feel offended, please send a private message to criticize and educate; based on this, this project The project and its source code are limited to readers or fans for personal study and research, and are prohibited from being used for commercial purposes. If legal issues arise due to commercial use by readers or fans themselves, this blog and the blogger have nothing to do with it, so please use it with caution. . .

2. The Ethernet solution I already have here

At present, I have a large number of UDP protocol project source codes here, including UDP data loopback, video transmission, AD acquisition and transmission, etc. There are also TCP protocol projects, as well as RDMA NIC 10G 25G 100G network card project source code. Brothers who have needs for network communication can Go and have a look: Click directly to go.
The engineering blog of the Gigabit TCP protocol is as follows:
Click directly to go.

3. Design idea framework

The FPGA engineering design block diagram is as follows:

Video source selection

There are two types of video sources, corresponding to whether the developer has a camera in hand. One is to use the cheap OV5640 camera module; the other is if you don’t have a camera in hand, or your development board does not have a camera input interface. , you can use the dynamic color bar generated inside the code to simulate camera video. The video source is selected through the macro definition at the top level of the code. The OV5640 camera module is selected as the video input source by default after power-on; the video source is selected through the `define at the top level of the code
. The macro definition is carried out; as follows:

The selection logic code part is as follows:

The selection logic is as follows:
when (comment) define COLOR_IN, the input source video is a dynamic color bar;
when (no comment) define COLOR_IN, the input source video is the ov5640 camera;

OV5640 camera configuration and collection

The OV5640 camera requires i2c configuration before it can be used. The video data from the DVP interface needs to be collected into RGB565 or RGB888 format video data. Both parts are implemented using the verilog code module. The code location is as follows: The camera is configured with a resolution of 1280x720, as follows

:

Camera The acquisition module supports video output in RGB565 and RGB888 formats, which can be configured by parameters, as follows:

RGB_TYPE=0 outputs the RGB565 format;
RGB_TYPE=1 outputs the RGB888 format;
select the RGB565 format for design;

Dynamic color bar

The dynamic color bar can be configured for videos of different resolutions. The border width of the video, the size of the dynamic moving block, the moving speed, etc. can all be parameterized. Here I configure the resolution as 1280x720, the code location of the dynamic color bar module and the top-level interface. An example is as follows:

UDP protocol stack

This UDP protocol stack solution needs to be used with Xilinx's Tri Mode Ethernet MAC three-speed network IP. The UDP protocol stack netlist file is used. Although the source code cannot be seen, UDP communication can be realized normally. The protocol stack is not currently open source and only provides network table file, but does not affect use. The protocol stack has a user interface, so that users do not need to care about the complex UDP protocol but only need to care about the simple user interface timing to operate UDP transceiver. It is very simple; the protocol stack architecture is as follows: Protocol
stack

performance The performance is as follows:
1: Supports UDP receive checksum verification function, but does not support UDP send checksum generation;
2: Supports IP header checksum generation and verification, and supports the PING function in the ICMP protocol, which can receive and Respond to PING requests from devices within the same subnet;
3: Can automatically initiate or respond to ARP requests from devices within the same subnet, and ARP transmission and reception are fully adaptive. The ARP table can save 256 IP and MAC address pairs within the same subnet;
4: Supports ARP timeout mechanism, which can detect whether the destination IP address of the required data packet is reachable;
5: The protocol stack transmission bandwidth utilization can reach 93% , under high sending bandwidth, the internal arbitration mechanism ensures that the PING and ARP functions are not affected in any way;
6: The sending process will not cause packet loss;
7: Provides a 64-bit wide AXI4-Stream MAC interface, which can be used with Xilinx official Gigabit The Ethernet IP core Tri Mode Ethernet MAC is used together with the 10 Gigabit Ethernet IP core 10 Gigabit Ethernet Subsystem and 10 Gigabit Ethernet MAC;
with this protocol stack, we do not need to care about the implementation of the complex UDP protocol and can directly call the interface use. . .
The sending timing of this UDP protocol stack user interface is as follows:

The receiving timing of this UDP protocol stack user interface is as follows:

UDP video data packet

To realize the packaging of UDP video data, UDP data sending must be consistent with the receiving program of the QT host computer. The UDP frame format defined by the host computer includes the frame header UDP data. The frame header is defined as follows: The UDP data packaging code on the FPGA side must be consistent with the above

. The data frame format of the figure corresponds, otherwise QT cannot parse it. The data packet state machine and data frame are defined in the code, as follows: In addition, since UDP

transmission is 64-bit data width, and image pixel data is 24-bit width, it must Reassembling UDP data to ensure alignment of pixel data is the difficulty of the entire project, and is also the difficulty of UDP data transmission for all FPGAs;

UDP protocol stack data sending

The UDP protocol stack has sending and receiving functions, but only sending is used here. The code structure of this part is as follows:

I have already prepared the UDP protocol stack code group, and users can use it directly;

UDP protocol stack data buffer

Here is the following explanation of the data buffer FIFO group used in the code:
Since the AXI-Stream data interface bit width of the UDP IP protocol stack is 64 bits, the AXI-Stream data interface bit width of the Tri Mode Ethernet MAC is 8 bits. Therefore, to interconnect the UDP IP protocol stack and Tri Mode Ethernet MAC through the AXI-Stream interface, clock domain and data bit width conversion is required. The implementation plan is shown in the figure below:

The transceiver path (this design only uses transmission) uses two AXI-Stream DATA FIFOs. One FIFO realizes the conversion of asynchronous clock domain, and the other FIFO
realizes mode function; since the AXI-Stream data interface synchronization clock signal of the Tri Mode Ethernet MAC at Gigabit rate is 125MHz, at this time, the AXI-Stream data interface synchronization clock signal of the UDP protocol stack 64bit should be 125MHz/(64/8)= 15.625MHz, therefore,
the clocks at both ends of the asynchronous AXI-Stream DATA FIFO are 125MHz (8bit) and 15.625MHz (64bit) respectively; after the AXI-Stream interface of the UDP IP protocol stack undergoes FIFO clock domain conversion, the data bit width needs to be Conversion, the conversion of data bit width is completed through AXI4-Stream Data Width Converter. In the receiving path, the conversion is performed from 8bit to 64bit; in the transmitting path, the conversion from 64bit to 8bit is performed;

Modification of IP address and port number

The UDP protocol stack sets aside IP address and port number modification ports for users to freely modify. The locations are as follows:

Introduction to Tri Mode Ethernet MAC and porting considerations

This design calls the official IP of Xilinx: Tri Mode Ethernet MAC. Its position in the code is as follows: You

can see that the Tri Mode Ethernet MAC IP core is in a locked state. This is intentional. The purpose is to extend the IP according to different PHY Modify its internal code and internal timing constraint code based on time parameters. Since the network PHY used in this design is 88E1518, here we focus on the modification and transplantation of Tri Mode Ethernet MAC when using 88E1518. When you need to transplant the project, or your vivado When the version is inconsistent with mine, the Tri Mode Ethernet MAC needs to be upgraded in vivado. However, since the IP has been artificially locked by us, upgrading and modification require some high-end operations. Regarding the operation method, I wrote a special document. It is attached to the information package as follows:

88E1518 PHY

The network PHY used in this design development board is 88E1518, which works in delay mode. The schematic diagram leads to MDIO, but there is no need for MDIO configuration in the code. 88E1518 can be made to work in delay mode by pulling up and down resistors. The PHY supports up to Gigabit, and can automatically negotiate between 10M/100M/1000M, but this design is fixed at 1000M on the Tri Mode Ethernet MAC side; in the data package, we provide the schematic diagram of 88E1518;

QT host computer and source code

We provide QT screenshot display host computer and its source code that match the UDP communication protocol. The directory is as follows:

Our QT currently only supports video snapshot display with 1280x720 resolution, but also reserves a 1080P interface. For QT development, Interested friends can try to modify the code to adapt to 1080P, because QT is just a verification tool here and is not the focus of this project, so I won’t go into details. For details, please refer to the QT source code of the reference package, the location is as follows:

4. Detailed explanation of vivado project

Development board FPGA model: Xilinx–Kintex7–xc7k325tffg676-2;
Development environment: Vivado2019.1;
Input: OV5640 camera or dynamic color bar, resolution 1280x720;
Output: Gigabit UDP protocol stack, 88E1518 PHY, RJ45 network port;
engineering role : Gigabit UDP network video transmission;
the project BD is as follows:

the project code structure is as follows:

the resource consumption and power consumption of the project are as follows:

5. Project transplantation instructions

Vivado version inconsistency handling

1: If your vivado version is consistent with the vivado version of this project, open the project directly;
2: If your vivado version is lower than the vivado version of this project, you need to open the project and click File –> Save As; but this method does not It is not safe. The safest way is to upgrade your vivado version to the vivado version of this project or a higher version;

3: If your vivado version is higher than the vivado version of this project, the solution is as follows:

After opening the project, you will find that the IP has been It is locked, as follows:

At this time, the IP needs to be upgraded, and the operation is as follows:

FPGA model inconsistency handling

If your FPGA model is inconsistent with mine, you need to change the FPGA model. The operation is as follows:



After changing the FPGA model, you also need to upgrade the IP. The method of upgrading the IP has been described previously;

Other things to note

1: Since the DDR of each board is not necessarily exactly the same, the MIG IP needs to be configured according to your own schematic. You can even directly delete the MIG of my original project and re-add the IP and reconfigure it; 2: According to your
own To modify the pin constraints of the schematic diagram, just modify it in the xdc file;
3: When transplanting pure FPGA to Zynq, you need to add the zynq soft core to the project;

6. Board debugging, verification and demonstration

Preparation

First connect the development board and the computer. After the development board is connected, it is as shown below:

Then change the IP address of your computer to be consistent with the IP specified in the code. Of course, the IP in the code can be set arbitrarily, but the IP in the code can be modified. Finally, the IP on the computer must also be changed accordingly. My settings are as follows:

ping

Before starting the test, we first ping to test whether UDP is connected, as follows:

static presentation

The ov5640 camera 1280x720 input UDP network transmission QT host computer displays as follows:

Dynamic color bar 1280x720 input UDP network transmission QT host computer displays as follows:

Dynamic presentation

The dynamic video demonstration is as follows:

FPGA-UDP-video transmission-K7

7. Benefits: Obtain project source code

Bonus: Acquisition of engineering code.
The code is too large to be sent by email. It will be sent via a certain network disk link.
The information acquisition method is: private, or the V business card at the end of the article.
The network disk information is as follows: