Table of contents
1 Introduction
The most complex and technically difficult protocol in the field of FPGA image acquisition is probably the MIPI protocol. MIPI decoding is so difficult that it has defeated countless heroes, so much so that Xilinx officially had to launch a dedicated IP core for developers to use, otherwise it would be too high-end. This operation directly scares away a large number of FPGA developers, and no one will play with it.
This design is based on Xilinx's Kintex7 development board. It collects 4Line MIPI videos from 2 channels of IMX214 cameras. The IMX214 camera pins are connected to the LVDS BANK of Kintex7. The MC20901 chip converts the IMX214 MIPI signals into LVDS signals and outputs them to the FPGA, and then the CSI2 RX module outputs Bayer video, and then output the RGB video through the Bayer to RGB module, and then use the gamma correction module to enhance the image quality, and then call 2 Xilinx official Video Scalers for image scaling to reduce the input 1920x1080 video to 960x1080; then call 3 Xilinx The official VDMA sends the image to DDR3 on the PS side for three-frame buffering and then reads it out; it calls the Xilinx official VTC module to generate the output video timing, and Kintex7 calls MicroBlaze to dynamically configure the VTC output timing through the SDK software, supporting 1080P, 720P and 640P. Timing output, call Xilinx official AXI4-Stream to Video Out for data stream conversion, output VGA timing RGB data, and finally output the video to the display through the RGB to HDMI module implemented by the onboard silicon9134;
This article describes in detail the design scheme of FPGA pure VHDL decoding IMX214 MIPI video and 2-channel video splicing output. The engineering code is compiled and debugged and verified on the board. It can be directly transplanted to the project. It is suitable for school students to do graduation projects and graduate project development. It is suitable for on-the-job engineers to do project development, and can be used in the fields of digital imaging and image transmission in medical, military and other industries; it
provides complete, run-through engineering source code and technical support;
the method of obtaining engineering source code and technical support is placed at the end of the article. , please read to the end patiently;
about the MIPI protocol, please search it yourself, there are many big guys in csdn who talk about it in detail, so I won’t write more about it;
Disclaimer
This project and its source code include both parts written by myself and parts obtained from public channels on the Internet (including CSDN, Xilinx official website, Altera official website, etc.). If you feel offended, please send a private message to criticize and educate; based on this, this project The project and its source code are limited to readers or fans for personal study and research, and are prohibited from being used for commercial purposes. If legal issues arise due to commercial use by readers or fans themselves, this blog and the blogger have nothing to do with it, so please use it with caution. . .
2. The MIPI codec solution I already have here
I currently have a wealth of MIPI encoding and decoding solutions based on FPGA, mainly MIPI decoding. There are both MIPI decoding implemented by pure vhdl, and MIPI decoding implemented by calling Xilinx official IP. There are both 2line MIPI decoding and 4line. MIPI decoding, including MIPI decoding with 4K resolution and MIPI decoding with resolution as small as 720P, MIPI decoding based on Xilinx platform FPGA, MIPI decoding based on Altera platform FPGA, and MIPI decoding based on Lattice platform FPGA. In the future, we will continue to launch MIPI decoding solutions that are better than domestic FPGAs. After all, domestic solutions are currently the mainstream in the future. We will also launch more MIPI encoding DSI solutions in the future, and strive to make MIPI encoding and decoding solutions for FPGAs affordable. . .
Based on this, I created a special column for MIPI encoding and decoding, and put all MIPI encoding and decoding blogs into the column. Brothers who have project needs or learning interests in FPGA encoding and decoding MIPI can go to my column to have a look. The address is as follows:
Click to go directly to the column
3. Performance and advantages of this MIPI CSI2 module
In one word: awesome, the performance is as follows:
1: Pure VHDL code implementation, the learning and readability have reached the ceiling;
2: The portability is okay, as long as the FPGA is compatible with the Xilinx deserialization source language, it can be transplanted;
3: The algorithm has reached the ceiling, The standard CSI2 receiving protocol implements decoding;
4: The practicality reaches the ceiling, using the IMX214 camera as input (mainly because it is cheap). Different from the verification and experimental projects on the market, this design is directly oriented to practical projects and is close to real projects. Brothers with similar projects can use it directly and get one month’s salary directly. . . 5: Supports MIPI video decoding with resolutions up to
1920
It will be lower, but users only need to know the user interface and do not need to look at the complex internal code;
7: Easy to use, although it is implemented in VHDL code, it has been encapsulated as a custom IP, and users do not need to care about the code implementation For complex logic, you only need to call the IP and configure it through the UI interface to use it. Of course, if you want to see the source code inside, you can still open it directly and watch it; 8
: Collect 2 MIPI cameras with non-source clocks at the same time, and decode 2 channels. Video splicing display is highly consistent with realistic high-end projects;
4. Detailed design plan
Design schematic diagram
The design principle block diagram is as follows:
IMX214 camera and its configuration
The output of the IMX214 camera I use is 4 Line MIPI format, and the output resolution is 1080p@60Hz; it outputs RAW10 data; the FPGA configures the IMX214 by calling AXI-IIC, and this operation is completed in the SDK software;
D-PHY module
D-PHY is implemented using a hardware solution, using the MC20901 chip (mainly cheap, with average performance) to convert the MIPI signal of IMX214 into an LVDS signal and output it to the FPGA. The schematic block diagram is as follows:
CSI-2-RX module
Regarding MIPI CSI-2-RX, there are a lot of articles on the Internet that introduce the principles and concepts. I will not repeat them here. Here I focus on the implementation using FPGA;
we use pure VHDL code to implement the MIPI CSI-2-RX function. In order to take care of everyone To get used to the habit of reading VHDL code, we have encapsulated some of the code into custom IP. Users do not need to care about the complex logic of code implementation. They only need to call the IP and configure it through the UI interface to use it. Of course, if you want to see it The source code can still be opened directly for viewing; this MIPI CSI-2-RX can only support 4 lines of MIPI video, and the data format supports RAW10; the
CSI-2-RX custom IP call in this project is as follows:
MIPI CSI-2-RX pure The VHDL source code is as follows:
Bayer to RGB module
Regarding MIPI Bayer to RGB conversion, there are a lot of articles on the Internet that introduce the principles and concepts. I will not repeat them here. Here we focus on the implementation using FPGA;
we use pure VHDL code to implement the MIPI Bayer to RGB conversion function, in order to take care of everyone who is not used to reading VHDL code. According to the custom, we have encapsulated some of the code into custom IP. Users do not need to care about the complex logic of code implementation. They only need to call the IP and configure it through the UI interface to use it. Of course, if you want to see the source code inside, you can still directly Open and watch;
the Bayer to RGB custom IP call in this project is as follows:
MIPI Bayer to RGB pure VHDL source code is as follows:
Gamma correction module
Regarding MIPI gamma correction, there are a lot of articles on the Internet introducing the principles and concepts. I will not repeat them here. Here I focus on the implementation using FPGA;
we use pure VHDL code to implement the MIPI gamma correction function, in order to take care of everyone who is not used to reading VHDL code. According to the custom, we have encapsulated some of the code into custom IP. Users do not need to care about the complex logic of code implementation. They only need to call the IP and configure it through the UI interface to use it. Of course, if you want to see the source code inside, you can still directly Open and watch;
the gamma correction custom IP call in this project is as follows:
MIPI gamma correction pure VHDL source code is as follows:
VDMA image cache
Call 3-channel VDMA, 2 of which are only configured in write mode, and only need to write video to DDE3, and the other 1 is configured for read operation. The purpose of this design is to do 2-channel video splicing, and 2-channel VDMA writes the DDR address of the image. Different, it can ensure that the image address in the memory does not conflict, but when reading the image, only one VDMA channel can be used to perform the read operation. This is not only a conventional design idea, but also a requirement to reduce the load of the HP interface AXI4 bus; it is configured in write-only
mode The VDMA is as follows:
The VDMA configured in read mode is as follows:
Video Scaler image cache
Call two Xilinx official Video Scalers for image scaling, and reduce the input 1920x1080 video to 960x1080; the purpose of this is to splice the 2-channel video to a display with an output resolution of 1920x1080; the Video Scaler scaling operation is completed in the SDK , the IP is as follows:
HDMI output
HDMI output is completed using silcom9134. Silcom9134 requires i2c configuration before it can be used. Kintex7 calls MicroBlaze to configure through SDK software. Regarding the configuration of the silcom9134 chip, please refer to my previous blog. Blog address: Click to go directly
5. Detailed explanation of vivado project
PL side FPGA hardware design
Development board FPGA model: Xilinx–Kintex7–xc7k325tffg900-2;
Development environment: Vivado2019.1;
Input: 2 channels IMX214 MIPI 4 Line RAW10;
Output: HDMI 2-way splicing display, 1080P;
Application: FPGA pure VHDL decoding IMX214 MIPI video , 2-channel video splicing output;
the project Block Design is as follows:
the project code structure is as follows:
the FPGA resource consumption and power consumption estimate after comprehensive compilation is as follows:
PS SDK software design
The SDK C++ language software code structure is as follows:
6. Project transplantation instructions
Vivado version inconsistency handling
1: If your vivado version is consistent with the vivado version of this project, open the project directly;
2: If your vivado version is lower than the vivado version of this project, you need to open the project and click File –> Save As; but this method does not It is not safe. The safest way is to upgrade your vivado version to the vivado version of this project or a higher version;
3: If your vivado version is higher than the vivado version of this project, the solution is as follows:
After opening the project, you will find that the IP has been It is locked, as follows:
At this time, the IP needs to be upgraded, and the operation is as follows:
FPGA model inconsistency handling
If your FPGA model is inconsistent with mine, you need to change the FPGA model. The operation is as follows:
After changing the FPGA model, you also need to upgrade the IP. The method of upgrading the IP has been described previously;
Other things to note
1: Since the DDR of each board is not necessarily exactly the same, the MIG IP needs to be configured according to your own schematic. You can even directly delete the MIG of my original project and re-add the IP and reconfigure it; 2: According to your
own To modify the pin constraints of the schematic diagram, just modify it in the xdc file;
3: When transplanting pure FPGA to Zynq, you need to add the zynq soft core to the project;
7. Board debugging and verification
The output is as follows:
8. Benefits: Obtaining engineering codes
Bonus: Acquisition of engineering code.
The code is too large and cannot be sent by email. It will be sent via a certain network disk link.
The information acquisition method is: private, or the V business card at the end of the article.
The network disk information is as follows:
