Domestic Gowin FPGA: pure verilog realizes video image scaling, providing 6 sets of Gowin engineering source code and technical support

Domestic Gowin FPGA: pure verilog realizes video image scaling, providing 6 sets of Gowin engineering source code and technical support

1 Introduction

"If you live and die for the benefit of the country, how can you avoid it because of misfortunes and blessings!" I am an outstanding underground clerk on the other side of the ocean. I am keenly aware that the motherland's shortcomings lie in the manufacturing field of high-precision semiconductors, so in the spirit of the great rejuvenation of the Chinese nation With the original intention of making a modest contribution to the Chinese Dream, Mr. Understanding planned from a high strategic height and a long-term perspective, and would rather bear the infamy of the contemporary generation than to serve the motherland forever. In July 2018, Mr. Understanding officially started the sweater war, and then The use of Chabo paper tactics is aimed at making independent and controllable strategic promotion for the motherland's advanced process semiconductor field; here, please accept my thank you! ! ! ! ! !

When I first debuted in early 2019, it was still an era when Xilinx was far ahead (it seems to be the same now). At that time, domestic FPGAs were still in the hot chicken category. Domestic FPGAs looked up to Xilinx and couldn’t help but say: You think you can’t find it if you hide here. To you? It's useless. A man like you, no matter where you are, is as bright and outstanding as a firefly in the dark night. Your melancholy eyes, thin beard, miraculous sword skills, and... I was deeply fascinated by that cup of Dry martine. . . However, in just 4 years, today's domestic FPGA belongs to a situation where a hundred schools of thought contend, a hundred flowers bloom, the Eight Immortals cross the sea, the gods fight, the ascendant, Longwangshu, and friends and businessmen all cry with joy. Faced with this situation, I have to sing the old man's poem: Wei Wu With a whip, there is a posthumous chapter in Jieshi in the east. The bleak autumn wind is now again, and the human world has changed. . .
Closer to home, the current consensus on domestic FPGAs is as follows:
1: High cost performance, compared with chips from foreign major manufacturers of the same level, the price difference is several times Even more than ten times;
2: Independent and controllable, domestic FPGA has a complete industrial chain of independent intellectual property rights, from chips to related EDA tools
3: Rapid response , FAE technical support is in place and problems encountered during the development process are solved in a timely manner. After all, the Chinese data manual is in Chinese. .
4: Convenient procurement, independent and controllable industrial chain, convenient procurement

This article uses the domestic Gowin GW2A-LV18PG484C7/I6 model FPGA to implement pure verilog to achieve video image scaling. There are two video sources, corresponding to whether the developer has a camera. One is to use a cheap OV5640 camera; if you If you do not have an OV5640 camera or the FPGA development board does not have a camera input interface, you can use the dynamic color bar generated inside the code to simulate the camera video. The video source is selected through the `define macro definition at the top level of the code. The OV5640 camera is selected by default when powering on. Input source; FPGA first uses the i2c controller implemented in pure verilog to configure the ov5640 camera, and configures its resolution to 640x480@60Hz. At the same time, it generates dynamic color bars implemented in pure verilog, with a resolution of 640x480@60Hz; FPGA collects the input video Finally, the image is first sent to the buffer FIFO for cross-clock domain processing, that is, the screen background resolution of 1280x720@60Hz is used as the clock for reading the FIFO. This ensures that the image is sent during image scaling operations of various resolutions. The clock of the scaling module is large enough, and it is also an operation with sufficient bandwidth; then the video is sent to the image scaling module implemented in pure verilog to perform the image scaling operation. This module is very simple, and only needs to give the input resolution and output resolution. The rate is enough, and there is no need to worry about issues such as scaling. Generally speaking, the input resolution is unchanged, and only the output resolution needs to be changed; the original timing of the scaled video has been completely destroyed. Therefore, it is necessary to call Gowin's official Video Frame Buffer IP core to send the video to the external DDR3 for three-frame buffering; call Gowin's official DDR3 Memory Interface IP core to implement the transfer of image data to DDR3 particles, similar to Xilinx's MIG; then read out the video and send it to VGA to output timing synchronized pixel data. The VGA output resolution is 1280x720@60Hz; the scaled video is equivalent to being superimposed on a 1280x720 background; finally, the official DVI TX IP core of Gowin is called to implement RGB Video to HDMI video conversion, output monitor display;

Provides 5 sets of Gowin-V1.9 version project source code; the difference between the 5 sets of projects lies in the different output resolutions after scaling, as follows:
The first set of Gowin projects: The input resolution is 640x480; the output resolution is 640x480; no scaling operation is performed;
The second set of Gowin projects: the input resolution is 640x480; the output resolution is 300x300; the scaling operation is performed;
The third set of Gowin projects: input resolution 640x480; output resolution 100x100; do the reduction operation;
The fourth set of Gowin projects: input resolution 640x480; output resolution 300x720; cannot be done Regular scaling operation;
The fifth set of Gowin projects: input resolution 640x480; output resolution 1280x360; perform irregular scaling operations;
The sixth set of Gowin Project: input resolution 640x480; output resolution 1280x720; perform zoom operation;

This article describes in detail the design scheme of the domestic Gowin FPGA image and video acquisition system. The engineering code can be comprehensively compiled and debugged on the board, and can be directly transplanted into the project. It is suitable for school students and graduate student project development, and is also suitable for in-service engineers. For learning and improvement, it can be applied to high-speed interfaces or image processing fields in medical, military and other industries;
Provide complete, run-through engineering source code and technical support;
How to obtain the project source code and technical support is at the end of the article, please be patient and read to the end;

Disclaimer

This project and its source code include both parts written by myself and parts obtained from public channels on the Internet (including CSDN, Xilinx official website, Altera official website, domestic Gowin FPGA official website, Unisoc FPGA official website, etc.). If the bosses feel that there is If you are offended, please send a private message to criticize and educate; based on this, this project and its source code are limited to readers or fans for personal study and research, and are prohibited from being used for commercial purposes. If there are legal issues arising from commercial use due to readers or fans themselves, we will not This blog has nothing to do with the blogger, please use it with caution. . .

2. Recommendation of relevant solutions

Recommended solutions related to domestic Gowin FPGA

In view of the excellent performance and market demand of domestic Gowin FPGA, I have specially opened a column for domestic Gowin FPGA, which includes image processing, UDP network communication, GT high-speed interface, PCIE based on domestic Gowin FPGA Wait for the blog. If you are interested, you can check it out. Blog address: Click to go directly

Domestic Gowin FPGA basic tutorial

Gowin FPGA development software Gowin’s downloading, installation, license sharing, project construction, code addition, synthesis, compilation, downloading, various IP calling, configuration, use and other basic operations are for Gowin FPGA development Of course, if you are already an experienced engineer, you can omit this step. To this end, I have specially opened a column to describe in detail the basic tutorials of domestic Gowin FPGAs, which can even be said to be nanny-level tutorials. The column address is as follows ：
Click to go directly

3. Design idea framework

This article uses the domestic Gowin GW2A-LV18PG484C7/I6 model FPGA to implement pure verilog to achieve video image scaling. There are two video sources, corresponding to whether the developer has a camera. One is to use a cheap OV5640 camera; if you If you do not have an OV5640 camera or the FPGA development board does not have a camera input interface, you can use the dynamic color bar generated inside the code to simulate the camera video. The video source is selected through the `define macro definition at the top level of the code. The OV5640 camera is selected by default when powering on. Input source; FPGA first uses the i2c controller implemented in pure verilog to configure the ov5640 camera, and configures its resolution to 640x480@60Hz. At the same time, it generates dynamic color bars implemented in pure verilog, with a resolution of 640x480@60Hz; FPGA collects the input video Finally, the image is first sent to the buffer FIFO for cross-clock domain processing, that is, the screen background resolution of 1280x720@60Hz is used as the clock for reading the FIFO. This ensures that the image is sent during image scaling operations of various resolutions. The clock of the scaling module is large enough, and it is also an operation with sufficient bandwidth; then the video is sent to the image scaling module implemented in pure verilog to perform the image scaling operation. This module is very simple, and only needs to give the input resolution and output resolution. The rate is enough, and there is no need to worry about issues such as scaling. Generally speaking, the input resolution is unchanged, and only the output resolution needs to be changed; the original timing of the scaled video has been completely destroyed. Therefore, it is necessary to call Gowin's official Video Frame Buffer IP core to send the video to the external DDR3 for three-frame buffering; call Gowin's official DDR3 Memory Interface IP core to implement the transfer of image data to DDR3 particles, similar to Xilinx's MIG; then read out the video and send it to VGA to output timing synchronized pixel data. The VGA output resolution is 1280x720@60Hz; the scaled video is equivalent to being superimposed on a 1280x720 background; finally, the official DVI TX IP core of Gowin is called to implement RGB Video to HDMI video conversion, output monitor display;

Provides 5 sets of Gowin-V1.9 version project source code; the difference between the 5 sets of projects lies in the different output resolutions after scaling, as follows:
The first set of Gowin projects: The input resolution is 640x480; the output resolution is 640x480; no scaling operation is performed;
The second set of Gowin projects: the input resolution is 640x480; the output resolution is 300x300; the scaling operation is performed;
The third set of Gowin projects: input resolution 640x480; output resolution 100x100; do the reduction operation;
The fourth set of Gowin projects: input resolution 640x480; output resolution 300x720; cannot be done Regular scaling operation;
The fifth set of Gowin projects: input resolution 640x480; output resolution 1280x360; perform irregular scaling operations;
The sixth set of Gowin Project: input resolution 640x480; output resolution 1280x720; perform zoom operation;
The design block diagram is as follows:

Note: The numbers in the block diagram indicate the order of data flow;

Video source selection

There are two video sources, which correspond to whether the developer has a camera in hand. One is to use the cheap OV5640 camera; if you do not have an OV5640 camera or the FPGA development board does not have a camera input interface, you can Use the dynamic color bar generated inside the code to simulate camera video. The video source is selected through the `define macro definition at the top level of the code. The OV5640 camera is selected as the input source by default after power-on. The video source selection is as follows:

The video source selection logic code part is as follows:

The selection logic is as follows:
When (note) define COLOR_IN, the input source video is OV5640; < a i=4> When (without comment) define COLOR_IN, the input source video is a dynamic color bar;

OV5640 camera configuration and collection

The OV5640 camera requires i2c configuration before it can be used. The video data from the DVP interface needs to be collected into RGB565 or RGB888 format video data. Both parts are implemented using the verilog code module. The code location is as follows:

The camera is configured with a resolution of 1280x720, as follows:

The camera acquisition module supports video output in RGB565 and RGB888 formats, which can be configured by parameters, as follows:

RGB_TYPE =0 Output this RGB565 format;
RGB_TYPE=1 Output this RGB888 format;
Select RGB888 format for design;

Dynamic color bar

Dynamic color bars can be configured for videos of different resolutions. The border width of the video, the size of the dynamic moving square, the moving speed, etc. can all be parameterized. Taking Project 1 as an example, it is configured with a resolution of 1280x720 and the code location of the dynamic color bar module. And the top-level interface and instantiation are as follows:

Cross-clock FIFO

Call Gowin’s official FIFO; the purpose of the cross-clock FIFO is to solve the cross-clock domain problem. When the video is not scaled, there is no video cross-clock domain problem, but this problem exists when the video is reduced or enlarged. Question, using FIFO buffering can make the image scaling module read valid input data every time. Note that the input timing of the original video has been disrupted here;
About Gao Yun For more detailed explanations of the official FIFO IP, please refer to my column: Gowin FPGA Development Basic Tutorial. The column address is as follows:
Click to go directly

Detailed explanation of image scaling module

Because our QT host computer currently only supports 1280x720, we need to scale, that is, reduce the input resolution from 1920x1080 to 1280x720; use a laptop to simulate the HDMI video input source;

Design block diagram

This design integrates commonly used bilinear interpolation and neighborhood interpolation algorithms into one code, and selects a certain algorithm by inputting parameters; the code is implemented using pure verilog, without any IP, and can be used on Xilinx, Intel, Any migration between domestic FPGAs; the code uses ram and fifo as the core for data caching and interpolation implementation. The design architecture is as follows:

The video input timing requirements are as follows:

Input pixels The data can be changed when dInValid and nextDin are high at the same time;
The video output timing requirements are as follows:

The output pixel data can be output only when dOutValid and nextdOut are high at the same time. ;

Code block diagram

The code is implemented using pure verilog, without any IP, and can be transplanted arbitrarily between Xilinx, Intel, and domestic FPGAs;
There are many ways to implement image scaling, the simplest is Xilinx's HLS method can be implemented using the opencv library and a few lines of code in C++ language. Regarding HLS implementation of image scaling, please refer to my previous articleHLS implementation of image scaling
There are other image scaling routine codes on the Internet, but most of them use IP, which makes transplantation to other FPGA devices difficult and has poor versatility. In contrast, this design code is universal; the code structure is as follows Figure;

The top-level interface part is as follows:

Integration and selection of 2 interpolation algorithms

This design integrates commonly used bilinear interpolation and neighborhood interpolation algorithms into one code, and selects a certain algorithm by inputting parameters;
The specific selection parameters are as follows:

input  wire i_scaler_type //0-->bilinear;1-->neighbor

You can select by entering the value of i_scaler_type;

Enter 0 to select the bilinear interpolation algorithm;
Enter 1 to select the neighborhood interpolation algorithm;

For the mathematical differences between these two algorithms, please refer to my previous articleHLS implements image scaling

Video Frame Buffer image cache

Call Gowin’s official Video Frame Buffer IP core to send the video to the external DDR3 for three-frame buffering; this part is the key core difficulty of the image acquisition and display system. If it is other FPGA, you need to write a lot The code needs to be debugged before it can be implemented, which takes a lot of time and energy. However, Gowin FPGA can easily realize the modification function because it directly made the IP, namely Video Frame Buffer; here is a brief introduction to the IP, because Gowin has detailed The Chinese manual explains the use of this IP. I have also included the manual in the information package;
My configuration of this IP only applies to my design. If you want to modify the IP configuration, You can modify it as follows and then regenerate the IP:

Video Frame Buffer IP configuration is as follows:

For more detailed explanation of Video Frame Buffer IP, please refer to me Column: Basic Tutorial on Gowin FPGA Development, the column address is as follows:
Click to go directly

DDR3 Memory Interface

Call Gowin's official DDR3 Memory Interface IP core to implement the transfer of image data to DDR3 particles, similar to Xilinx's MIG; the DDR3 Memory Interface IP configuration is as follows:

About DDR3 Memory For more detailed explanations of Interface IP, please refer to my column: Gowin FPGA Development Basic Tutorial. The column address is as follows:
Click to go directly

4. Gowin Project 1: 640x480 non-scaling operation

Development board FPGA model: Domestic Gowin-GW2A-LV18PG484C7/I6;
Development environment: Gowin-V1.9
Input: OV5640 camera Or dynamic color bar, resolution 640x480;
Video resolution before scaling: 640x480;
Video resolution after scaling: 640x480; < a i=5> Output: HDMI, superimposed on the scaled 640x480 video in the background of 1280x720; Engineering function: The video passes through the scaling module, but no scaling operation is performed, that is, one-to-one scaling; The project code structure is as follows: The resource consumption and power consumption of the project are as follows: All the projects I released have been compiled and passed. As follows:

5. Gowin Project 2: 640x480 reduced to 300x300

Development board FPGA model: Domestic Gowin-GW2A-LV18PG484C7/I6;
Development environment: Gowin-V1.9
Input: OV5640 camera Or dynamic color bar, resolution 640x480;
Video resolution before scaling: 640x480;
Video resolution after scaling: 300x300; < a i=5> Output: HDMI, superimposed on the scaled 300x300 video in the background of 1280x720; Engineering function: video reduction operation; Project code structure, The resource consumption and power consumption of the project are basically consistent with the content in Chapter 4;

6. Gowin Project 3: 640x480 reduced to 100x100

Development board FPGA model: Domestic Gowin-GW2A-LV18PG484C7/I6;
Development environment: Gowin-V1.9
Input: OV5640 camera Or dynamic color bar, resolution 640x480;
Video resolution before scaling: 640x480;
Video resolution after scaling: 100x100; < a i=5> Output: HDMI, superimposed on the scaled 100x100 video in the background of 1280x720; Engineering function: video reduction operation; Project code structure, The resource consumption and power consumption of the project are basically consistent with the content in Chapter 4;

7. Gowin Project 4: 640x480 reduced to 300x720

Development board FPGA model: Domestic Gowin-GW2A-LV18PG484C7/I6;
Development environment: Gowin-V1.9
Input: OV5640 camera Or dynamic color bar, resolution 640x480;
Video resolution before scaling: 640x480;
Video resolution after scaling: 300x720; < a i=5> Output: HDMI, superimposed on the scaled 300x720 video in the background of 1280x720; Engineering function: irregular scaling operation of the video; The project code architecture, project resource consumption and power consumption are basically consistent with the content in Chapter 4;

8. Gowin Project 5: 640x480 reduced to 1280*360

Development board FPGA model: Domestic Gowin-GW2A-LV18PG484C7/I6;
Development environment: Gowin-V1.9
Input: OV5640 camera Or dynamic color bar, resolution 640x480;
Video resolution before scaling: 640x480;
Video resolution after scaling: 1280360;
Output: HDMI, scaled 1280360 video overlaid on 1280x720 background;
Engineering role: irregular zooming operations on videos;
The engineering code structure, resource consumption and power consumption of the project are basically consistent with the content in Chapter 4;

9. Gowin Project 6: 640x480 reduced to 1280x720

Development board FPGA model: Domestic Gowin-GW2A-LV18PG484C7/I6;
Development environment: Gowin-V1.9
Input: OV5640 camera Or dynamic color bar, resolution 640x480;
Video resolution before scaling: 640x480;
Video resolution after scaling: 1280x720; < a i=5> Output: HDMI, superimposed on the scaled 1280x720 video in the 1280x720 background; Engineering function: video amplification operation; Project code structure , the resource consumption and power consumption of the project are basically consistent with the content in Chapter 4;

10. Board debugging, verification and demonstration

Preparation

You need the following equipment to transplant and test the project code:
1: FPGA development board;
2: OV5640 camera, if not available Yes, just choose dynamic color bars;
3: HDMI transmission line;
4: HDMI display, the resolution is required to support 1280x720;
The connection is as follows:

static presentation

The first set of Gowin projects: input resolution 640x480; output resolution 640x480; no scaling operation; the output is as follows:

The second set of Gowin projects: input resolution 640x480; output Resolution 300x300; perform reduction operation;

The third set of Gowin project: input resolution 640x480; output resolution 100x100; perform reduction operation;

The fourth set of Gowin Project: input resolution 640x480; output resolution 300x720; perform irregular scaling operations;

The fifth set of Gowin projects: input resolution 640x480; output resolution 1280x360; perform irregular scaling operations ;

The sixth Gowin project: input resolution 640x480; output resolution 1280x720; zoom operation;

11. Benefits: Obtain project source code

Benefit: Acquisition of engineering code
The code is too large to be sent by email. It will be sent via a certain network disk link.
How to obtain data: Private, or the V business card at the end of the article.