Table of contents
1 Introduction
The most complex and technically difficult protocol in the field of FPGA image acquisition is probably the MIPI protocol. MIPI decoding is so difficult that it has defeated countless heroes, so much so that Xilinx officially had to launch a dedicated IP core for developers to use, otherwise it would be too high-end. This operation directly scares away a large number of FPGA developers, and no one will play with it.
This design is based on Xilinx's Zynq UltraScale+ To output the Bayer video, call the Xilinx official Sensor Demosaic IP core to perform Bayer conversion to RGB and then output the RGB video, call the Xilinx official Gamma LUT IP core to do the gamma correction module to enhance the image quality, and then call the Xilinx official VDMA to send the image to the DDR3 on the PS side. Make three frames buffered and read out; call Xilinx official VTC module to generate output video timing, zynq dynamically configures the output timing of VTC through SDK software, supports three timing outputs of 1080P, 720P and 640P, calls Xilinx official AXI4-Stream to Video Out Perform data stream conversion, output VGA timing RGB data, and finally output the video to the display through the onboard DP interface;
Zynq UltraScale+ It is used in the fields of digital imaging and image transmission in medical, military and other industries;
provides complete, run-through engineering source code and technical support;
the method of obtaining engineering source code and technical support is placed at the end of the article, please be patient and read to the end;
About MIPI Please search for the protocol by yourself. There are many big guys in csdn who explain it in detail, so I won’t write more about it;
Disclaimer
This project and its source code include both parts written by myself and parts obtained from public channels on the Internet (including CSDN, Xilinx official website, Altera official website, etc.). If you feel offended, please send a private message to criticize and educate; based on this, this project The project and its source code are limited to readers or fans for personal study and research, and are prohibited from being used for commercial purposes. If legal issues arise due to commercial use by readers or fans themselves, this blog and the blogger have nothing to do with it, so please use it with caution. . .
2. The MIPI codec solution I already have here
I currently have a wealth of MIPI encoding and decoding solutions based on FPGA, mainly MIPI decoding. There are both MIPI decoding implemented by pure vhdl, and MIPI decoding implemented by calling Xilinx official IP. There are both 2line MIPI decoding and 4line. MIPI decoding, including MIPI decoding with 4K resolution and MIPI decoding with resolution as small as 720P, MIPI decoding based on Xilinx platform FPGA, MIPI decoding based on Altera platform FPGA, and MIPI decoding based on Lattice platform FPGA. In the future, we will continue to launch MIPI decoding solutions that are better than domestic FPGAs. After all, domestic solutions are currently the mainstream in the future. We will also launch more MIPI encoding DSI solutions in the future, and strive to make MIPI encoding and decoding solutions for FPGAs affordable. . .
Based on this, I created a special column for MIPI encoding and decoding, and put all MIPI encoding and decoding blogs into the column. Brothers who have project needs or learning interests in FPGA encoding and decoding MIPI can go to my column to have a look. The address is as follows:
Click to go directly to the column
3. Performance and advantages of this MIPI CSI2 module
In one word: awesome, the performance is as follows:
1: Using Xilinx’s official MIPI CSI-2 RX Subsystem decoding solution, which is stable and reliable;
2: The portability is okay and can be freely transplanted on Xilinx Zynq UltraScale+ series FPGA;
3: The algorithm has reached the ceiling , the standard CSI2 receiving protocol implements decoding;
4: The practicality reaches the ceiling, using the OV5640 camera as the input (mainly because it is cheap). Different from the verification and experimental projects on the market, this design is directly oriented to practical projects and is close to real projects. Brothers who are working on similar projects can use it directly and get one month’s salary directly. . .
5: Support MIPI video decoding up to 4K resolution;
4. Detailed design plan
Design schematic diagram
The design principle block diagram is as follows:
OV5640 camera and its configuration
The OV5640 camera I use outputs 2 Line MIPI format. The output resolutions are 720p@60Hz and 1080p@30Hz. You can choose which one to configure through the SDK software; OV5640 outputs RAW10 data; zynq is configured through i2c on-chip peripherals. OV5640, this operation is completed in the SDK software, and the PL side hardware does not need to be involved;
MIPI CSI-2 RX Subsystem
Call Xilinx's official MIPI CSI-2 RX Subsystem IP core to do MIPI decoding and then output Bayer video. The MIPI CSI-2 RX Subsystem configuration is as follows:
Sensor Demosaic
Call the Xilinx official Sensor Demosaic IP core to perform Bayer conversion to RGB and then output RGB video. The Sensor Demosaic configuration is as follows:
Gamma LUT
Call Xilinx's official Gamma LUT IP core to do the gamma correction module to enhance image quality. The Gamma LUT configuration is as follows:
MIPI D-PHY hardware solution
This design uses the resistor network solution officially recommended by xilinx, which requires 800M Hz or less and the wiring within 30mm; the OV5640 camera pin is connected to the GT resource dedicated BANK of Zynq UltraScale+ XCZU3EG;
5. Detailed explanation of vivado project
PL side FPGA hardware design
Development board FPGA model: Xilinx–Zynq UltraScale±-xczu3eg-sfvc784-1-e;
development environment: Vivado2019.1;
input: OV5640 MIPI 2 Line RAW10;
output: DP display, 720P;
application: Zynq UltraScale+ XCZU3EG decoding MIPI video DP Output;
the project Block Design is as follows:
the project code structure is as follows:
FPGA resource consumption and power consumption estimates after comprehensive compilation are as follows:
PS SDK software design
The SDK C language software code structure is as follows:
6. Project transplantation instructions
Vivado version inconsistency handling
1: If your vivado version is consistent with the vivado version of this project, open the project directly;
2: If your vivado version is lower than the vivado version of this project, you need to open the project and click File –> Save As; but this method does not It is not safe. The safest way is to upgrade your vivado version to the vivado version of this project or a higher version;
3: If your vivado version is higher than the vivado version of this project, the solution is as follows:
After opening the project, you will find that the IP has been It is locked, as follows:
At this time, the IP needs to be upgraded, and the operation is as follows:
FPGA model inconsistency handling
If your FPGA model is inconsistent with mine, you need to change the FPGA model. The operation is as follows:
After changing the FPGA model, you also need to upgrade the IP. The method of upgrading the IP has been described previously;
Other things to note
1: Since the DDR of each board is not necessarily exactly the same, the MIG IP needs to be configured according to your own schematic. You can even directly delete the MIG of my original project and re-add the IP and reconfigure it; 2: According to your
own To modify the pin constraints of the schematic diagram, just modify it in the xdc file;
3: When transplanting pure FPGA to Zynq, you need to add the zynq soft core to the project;
7. Board debugging and verification
The output is as follows:
8. Benefits: Obtaining engineering codes
Bonus: Acquisition of engineering code.
The code is too large and cannot be sent by email. It will be sent via a certain network disk link.
The information acquisition method is: private, or the V business card at the end of the article.
The network disk information is as follows:
