FPGA decoding MIPI video OV5647 2line CSI2 720P resolution acquisition provides engineering source code and technical support

1 Introduction

In the field of FPGA image acquisition, the most complicated protocol and the most technically difficult protocol should be the MIPI protocol. The difficulty of MIPI decoding has made countless heroes compete, so that Xilinx officially has to release a dedicated IP core for developers to use, otherwise it is too high-end The operation directly scares away a large number of FPGA developers, and no one will play.

This design is based on Xilinx's Kintex7 development board, which captures 720P 2Line MIPI video from the OV5647 camera. The pins of the OV5647 camera are connected to the BANK16 LVDS_25 differential pins of Kintex7. The CSI2 RX module implemented by pure vhdl outputs the AXIS video stream, and then converts the bayer data to RGB , and then send the image to DDR3 through FDMA to do three-frame buffer and read it out, and finally output it to the display through the HDMI sending module;

This article describes the design scheme in detail. After the engineering code is compiled and passed, it can be debugged and verified on the board, and the project can be directly transplanted. It is suitable for students in school to do graduation design, graduate project development, and also for in-service engineers to do project development. It can be applied to medical and military industries. Digital imaging and image transmission fields in other industries;
provide complete and smooth engineering source code and technical support;
the method of obtaining engineering source code and technical support is at the end of the article, please be patient to the end;
about the MIPI protocol, please search by yourself , there are many big guys on csdn who talk about it in detail, so I won’t write more about it;

2. The official MIPI decoding scheme promoted by Xilinx

The MIPI decoding solution officially promoted by Xilinx is a dedicated IP core . Enter MIPI in the ip warehouse of vivado and you will see a bunch of IPs as follows: The data format is AXIS, which is convenient for docking with other IPs such as VDMA. You don’t need to care about the complicated MIPI protocol to play MIPI, but the problem comes, what if the FPGA you use is not from Xilinx?
For Xilinx's own MIPI solution, please refer to my previous article; Xilinx's MIPI solution

3. The performance and advantages of this MIPI CSI2 module

The video timing after MIPI decoding in this solution is VGA timing, that is, line synchronization, field synchronization, valid data, and data signal; it is convenient for direct processing at the back end; the deserialization part uses Xilinx source language, and this project uses 7 series source language, more For high-level FPGA applications, it is necessary to change the source language parameters, but it is not a big problem;
one word: awesome, the performance is as follows:
1: Pure Vhdl code implementation, learning and readability reach the ceiling;
2: Portability is okay, as long as it is compatible FPGA of Xilinx deserialization source language can be transplanted;
3: The algorithm reaches the ceiling, and the standard CSI2 receiving protocol realizes decoding;
4: The practicability reaches the ceiling, using the OV4689 camera as input, which is different from the verification and experimental projects on the market. This design is directly oriented to practical projects, close to real projects, MIPI input, 2-channel video output, brothers who do similar projects can use it directly, and get a month's salary directly. . .
5: Support MIPI video decoding up to 4K resolution;
6: The timing convergence is in place, considering the complexity of the MIPI protocol and the high timing requirements, so VHDL is not used, which is not strong in timing convergence, but VHDL, although the readability is possible It will be lower, but users only need to know the user interface, and do not need to look at the internal complex code;
7: Custom IP package, convenient for Xilinx series FPGA users, CSI-2 custom IP supports 2line or 4line input, It can be manually configured on the IP configuration interface;

4. The MIPI codec scheme I have here

I currently have a wealth of FPGA-based MIPI encoding and decoding solutions, mainly MIPI decoding, including MIPI decoding implemented by pure vhdl, and MIPI decoding implemented by calling Xilinx official IP, including 2-line MIPI decoding and 4-line decoding. MIPI decoding, including MIPI decoding with 4K resolution and MIPI decoding with resolution as small as 720P, MIPI decoding based on Xilinx platform FPGA, MIPI decoding based on Altera platform FPGA, and MIPI decoding based on Lattice platform FPGA, In the future, we will continue to introduce MIPI decoding solutions that are more advanced than domestic FPGAs. After all, the current localized solutions are the mainstream in the future. More MIPI-encoded DSI solutions will be launched in the future, and efforts will be made to make FPGA MIPI encoding and decoding solutions cheap. . .
Based on this, I specially built a MIPI codec column, and put all the MIPI codec blogs in the column to sort out. Brothers who have project needs or learning interests in FPGA codec MIPI can go to my column. Column The address is as follows:
Click to go directly to the column

5. Detailed design plan

The detailed design scheme is as follows:

Video input:
OV5647 camera input, MIPI 2line, 720P resolution;
MIPI decoding:
CSI-2 decoder written in pure vhdl code, which can decode MIPI video input from 2line or 4line, and output as AXIS data stream, Output VGA format data through AXIS to VGA, and output VGA format RGB888 video data through Bayer to RGB;
image cache:
classic, my commonly used FDMA image cache architecture, friends who often read my blog should be familiar with it, want to know about FDMA For the image cache architecture, please refer to my previous article FDMA image cache architecture.
Image output:
After 3 frame buffers of the FDMA image, it is equivalent to MIPI video entering DDR3 for a while and then coming back, and sending the drive output through VGA timing generator VTC and HDMI The display and HDMI sending driver are written in pure verilog code, without any IP, and the output is stable and reliable. My project has been using it all the time. If you want to know about HDMI sending driver, you can refer to my previous article HDMI sending driver

6. Introduction to vivado project

Development board: Xilinx Kintex7 development board;
development environment: vivado2020.2;
input: OV5647 camera input, MIPI 2line, 720P resolution;
output: HDMI, 720P resolution;
the project BD is as follows:

The IP of the MIPI decoding part is built as follows:

CSI- 2 The IP configuration interface is as follows:

The CSI-2 to AXIS

FDMA configuration is as follows:

The integrated engineering code structure is as follows:

FPGA resource consumption and power consumption are estimated as follows:

7. Board debugging and verification

Because the camera was burnt out by the handicapped party, I can't demonstrate it. It's really bad luck for TND. The last picture is shocking.

8. Benefits: Acquisition of engineering codes

Benefits: Obtaining the engineering code
The code is too large to be sent by email, and it is sent by a link to a certain network disk.
Data acquisition method 1: private, or the V business card at the end of the article.
Data acquisition method 2: XX number at the end of the article, reply 002004
network disk data is as follows: