Xilinx Zynq-7000 series FPGA implements video splicing display, providing two sets of engineering source code and technical support

1 Introduction
- Disclaimer
2. Recommendation of relevant solutions
- FPGA image processing solution
- Recommended FPGA video splicing and overlay fusion solution
3. Detailed explanation of design ideas
- Introduction to Video Mixer
4. Project code 1: 2-channel video splicing HDMI output
- PL side FPGA logic design
- PS side SDK software design
5. Project code 2: 4-channel video splicing HDMI output
- PL side FPGA logic design
- PS side SDK software design
6. Project transplantation instructions
7. Board debugging, verification and demonstration
8. Benefits: Obtain project source code

1 Introduction

I am embarrassed to say that I have played with FPGA if I have never played video splicing. This is what a CSDN boss said, and I firmly believe it. . .
The mainstream FPGA video splicing solutions currently on the market are as follows:
1: Xilinx’s HLS solution, which is simple and easy to implement, but can only be used by Xilinx itself FPGA;
2: Non-pure Verilog solution, most of the code is implemented using Verilog, but the intermediate fifo or ram uses IP, resulting in poor portability and difficulty in Xilinx, Altera and domestic Free migration between FPGAs;
3: Pure Verilog solution;

This article uses Xilinx Zynq7000 series FPGA Zynq7020 to implement HLS image scaling, and the input video source uses the OV5640 camera module; the FPGA collects the OV5640 camera video DVP and converts it to RGB888, and calls the on-chip i2c controller of the Zynq soft core to configure the OV5640 to 960x540@30Hz resolution; The project uses a copy of one camera data to simulate two camera inputs; then calls two Xilinx official Video In to AXI4-Stream IP cores to convert the RGB video stream into an AXI4-Stream video stream; then calls two Xilinx official VDMA The IP performs the video cache operation of DDR3 on the PS side. VDMA requires Zynq software to be configured as a three-frame cache, which is essentially register configuration through AXI_Lite; then the official Xilinx Video Mixer IP core is called to perform a two-way video splicing operation. Video Mixer requires Zynq software is configured as a three-frame cache, which essentially uses AXI_Lite to configure registers; then calls Xilinx's official Video Timing Controller IP and AXI4-Stream to Video Out IP to convert the AXI4-Stream video stream into an RGB video stream; and then adds customization The HDMI sending IP converts RGB video into TMDS differential video and sends it to the monitor for display;

Provides two sets of project source code and technical support for vivado2019.1 version. The difference between the two sets of projects lies in the number of video channels to be spliced. The details are as follows:
The first set of vivado Project source code: input, OV5640 camera; 2-way splicing output, HDMI display;
The second set of vivado project source code: input, OV5640 camera; 4-way splicing output, HDMI display;

Xilinx Zynq-7000 series FPGA implements a design solution for video splicing display. The engineering code can be comprehensively compiled and debugged on the board, and the project can be directly transplanted. It is suitable for school students and graduate project development, and is also suitable for on-the-job engineers to learn and improve. It can be applied to High-speed interface or image processing fields in medical, military and other industries; the entire project calls the Zynq soft core for IP configuration, and the Zynq configuration is run in the SDK in the form of C language software code, so the entire project includes FPGA logic design and SDK software design The two parts require comprehensive capabilities in FPGA and embedded C language, and are not suitable for beginners or novices;

Provide complete, run-through engineering source code and technical support;
How to obtain the engineering source code and technical support is placed at the end of the article, please be patient until the end;

Disclaimer

This project and its source code include both parts written by myself and parts obtained from public channels on the Internet (including CSDN, Xilinx official website, Altera official website, etc.). If you feel offended, please send a private message to criticize and educate; based on this, this project The project and its source code are limited to readers or fans for personal study and research, and are prohibited from being used for commercial purposes. If legal issues arise due to commercial use by readers or fans themselves, this blog and the blogger have nothing to do with it, so please use it with caution. . .

2. Recommendation of relevant solutions

FPGA image processing solution

My homepage currently has an FPGA image processing column. The new column includes the FPGA image processing solutions I currently have, including image scaling, image recognition, image splicing, image fusion, image defogging, and image overlay. , image rotation, image enhancement, image character overlay, etc.; the following is the column address:
Click to go directly

3. Detailed explanation of design ideas

The first set of vivado project source code design block diagram is as follows:
Insert image description here
The second set of vivado project source code design block diagram is as follows:

Block diagram explanation: arrows indicate the data flow direction , the text inside the arrow indicates the data format, and the numbers outside the arrow indicate the steps of data flow;

Introduction to Video Mixer

Since the IPs used in the project are all commonly used IPs, here we focus on the Video Mixer IP;
Supports the maximum resolution: 8K, which means it can process videos up to 8K;< /span> Video Mixer logic resources are as follows, please carefully evaluate your FPGA resources; The module occupies smaller FPGA logic resources. Compared with the HLS video splicing written by yourself, the official Video Mixer resource occupies approximately less About 30% smaller and more efficient: provides a custom configuration API, by calling the The library functions can be easily used, please refer to the SDK code for details; requires SDK software configuration, which is essentially register configuration through AXI_Lite; Output video format: AXI4-Stream; Input video format: AXI4-Stream;
Supports up to 16 layers of video splicing and overlay, that is, up to 16 channels of video can be spliced;

Insert image description here

4. Project code 1: 2-channel video splicing HDMI output

PL side FPGA logic design

Development board FPGA model: Xilinx–Zynq7020–xc7z020clg400-2;
Development environment: Vivado2019.1;
Input: OV5640 camera, resolution 960x540;
Output: HDMI, 1920x1080 splicing 2-channel video output on black background;
Engineering role: Xilinx Zynq-7000 series FPGA realizes video splicing display, HDMI output ;
The project BD is as follows:
Insert image description here
The project code structure is as follows:

The resource consumption and power consumption of the project are as follows:

PS side SDK software design

The code structure of the PS side SDK software project is as follows:

The main function of the PS project is as follows:

#include "I2C_16bit.h"
#include "xiicps.h"
#include "xil_io.h"
#include "xparameters.h"
#include "helai_vdma.h"
#include "helai_mixer.h"
#include "uart.h"

XIicPs	Iic;
XScuGic Intc;              //中断控制器驱动程序实例
XUartPs Uart_Ps;           //串口驱动程序实例
void main()
{
    
    
	uart_init(&Uart_Ps);    //串口初始化
	uart_intr_init(&Intc, &Uart_Ps); //串口中断初始化
	I2C_config_init();	//ov5640 配置完成
	helai_vdma();	// VDMA 配置完成
	helai_mixer();	// video mixer 配置完成
	print("zynq7020 两路视频拼接:\n\r");
	print("输入视频分辨率:960x540\n\r");
	print("1920x1080黑色背景下拼接两路视频输出\n\r");
	while (1) ;
}

5. Project code 2: 4-channel video splicing HDMI output

PL side FPGA logic design

Development board FPGA model: Xilinx–Zynq7020–xc7z020clg400-2;
Development environment: Vivado2019.1;
Input: OV5640 camera, resolution 960x540;
Output: HDMI, 1920x1080 splicing 4-channel video output on black background;
Engineering role: Xilinx Zynq-7000 series FPGA realizes video splicing display, HDMI output ;
The project BD is as follows:
Insert image description here
The project code structure is as follows:

The resource consumption and power consumption of the project are as follows:

PS side SDK software design

The PS side SDK software engineering code structure is as follows:
Insert image description here

#include "I2C_16bit.h"
#include "xiicps.h"
#include "xil_io.h"
#include "xparameters.h"
#include "helai_vdma.h"
#include "helai_mixer.h"
#include "uart.h"

XIicPs	Iic;
XScuGic Intc;              //中断控制器驱动程序实例
XUartPs Uart_Ps;           //串口驱动程序实例
void main()
{
    
    
	uart_init(&Uart_Ps);    //串口初始化
	uart_intr_init(&Intc, &Uart_Ps); //串口中断初始化
	I2C_config_init();	//ov5640 配置完成
	helai_vdma();	// VDMA 配置完成
	helai_mixer();	// video mixer 配置完成
	print("zynq7020 四路视频拼接:\n\r");
	print("输入视频分辨率:960x540\n\r");
	print("1920x1080黑色背景下拼接四路视频输出\n\r");
	while (1) ;
}

6. Project transplantation instructions

Vivado version inconsistency handling

1: If your vivado version is consistent with the vivado version of this project, open the project directly;
2: If your vivado version is lower than the vivado version of this project, you need to open it After the project, click File –> Save As; however, this method is not safe. The safest way is to upgrade your vivado version to the vivado version of this project or a higher version;
Insert image description here
3 : If your vivado version is higher than the vivado version of this project, the solution is as follows:

After opening the project, you will find that the IPs are locked, as follows:

At this time It is necessary to upgrade the IP, please do as follows:

FPGA model inconsistency handling

If your FPGA model is inconsistent with mine, you need to change the FPGA model. The operation is as follows:
Insert image description here

After changing the FPGA model, you also need to upgrade the IP. The method of upgrading the IP has been described previously. ;

Other things to note

1: Since the DDR of each board is not necessarily exactly the same, the MIG IP needs to be configured according to your own schematic diagram. You can even directly delete the MIG of my original project here and re-add the IP and reconfigure it;
2: Modify the pin constraints according to your own schematic diagram, just modify it in the xdc file;
3: Transplanting pure FPGA to Zynq needs to be done in the project Add zynq soft core;

7. Board debugging, verification and demonstration

Preparation

Zynq7000 series development board;
OV5640 camera;
HDMI display, supporting 1080P;

Output static presentation

Project 1: ov5640 input resolution is 960x540, 2-channel video splicing output is as follows:
Insert image description here
Project 2: ov5640 input resolution is 960x540, 4-channel video splicing output is as follows:

Output dynamic demonstration

A short video was recorded, and the output dynamic demonstration is as follows:

video-mixer-hdmi

8. Benefits: Obtain project source code

Benefit: Acquisition of engineering code
The code is too large to be sent by email. It will be sent via a certain network disk link.
How to obtain data: Private, or the V business card at the end of the article.
The network disk information is as follows:
Insert image description here