1. Brief description
In the previous part, use Matlab to convert the picture into a txt data file, and now use these data to simulate Modelsim that is close to the real situation; at the same time briefly explain the not easy to understand places in the RTL code;
2.vivado block diagram
The red box is the algorithm module. I used to encapsulate the camera acquisition module (that is, the module in the blue box) and the algorithm module into an IP. Later, I found that the entire module needs to be modified if a different camera is changed, which is more troublesome. So this time separate the two parts. At the same time, in order to facilitate the adjustment of the threshold, the algorithm module is integrated into a module with an AXI slave interface, and the coordinate value of the identified target is transmitted to the ARM to calculate the distance value.
3. Algorithm module structure
The red box is the code file level, and the black box is the module of the AXI save interface (you can use vivado to generate it directly), and all algorithm modules are instantiated under this module. The orange box is the algorithm module, in which the camera_sync module regenerates the 2 Pclk and 1 data caused by the data splicing in the camera acquisition module to regenerate a new 1 Pclk and 1 data camera sequence, which is convenient for the algorithm processing later; the Bin_filter module includes RGB2YCbCR processing, threshold binarization and corrosion filtering; the bounding_box module obtains the upper, lower, left, and right coordinates of the target object and
frame selection of the target bounding box; directly instantiates the top layer of the algorithm module in tb_sim for simulation, and imports the txt data file generated by Matlab into tb , Simulate a real camera timing from the camera acquisition module to improve the reliability of simulation;
4. Doubts about the algorithm module code
1) camera_sync module
The vsync_i signal is the field synchronization signal of the camera. As shown in the figure below, there will be a period of high level at the beginning of a frame of image. To ensure the complete timing of the new camera generated, this signal is inverted and the reset signal is ANDed to obtain a The new reset signal resets all the signals in this module when starting a new frame of image to ensure that the data will not go wrong for a long time.
2) Bin_filter module
The picture below is RGB2YCbCr. Because FPGA floating point calculation is more difficult, all coefficients are multiplied by 128 to integer.
5. Simulation tb file
1) txt data reading
Read the txt file data through the readmemh function and store it in the cam_L_data_mem and cam_R_data_mem arrays;
2) Generate camera timing
(1) Since the camera acquisition module is 2 Pclk and 1 data, col_num is 640x2, plus 120 idle clock; row_num 480x1400, plus 8000 idle; 
(2) Note that the new timing is generated here and the simulation simulation above The timing is different, as shown in the figure below, FRAME_VALID and LINE_VALID are the field synchronization and line synchronization signals respectively;
(3) Generate a new line synchronization signal;
(4) Generate a new field synchronization signal;
(5) Get image data from the array;
6. Simulation waveform
(1) The camera timing generated by tb is basically the same as the above datasheet timing diagram.
(2) New image sequence generated
(3) The target coordinate value obtained by the whole algorithm in the red box is not much different from the number simulated by Matlab. Integerization with the previous floating point affects the calculation accuracy.
7.SDK software
The orange box is the threshold of AXI slave configuration binarization; the red box reads the target coordinates; the
blue box is used for mean filtering; the green box is for calculating the distance, and the denominator parameter is calculated from the formula simulated by Matlab;
8. Recognition effect
The recognition effect is shown in the figure. Color recognition is more affected by light, so it is difficult to recognize darker areas. The second picture is the data printed by the serial port. The distance is between 48 and 49. Compared with the actual physical distance, the error is not large. Of course, this accuracy is easily affected by the environment. If you want to improve the accuracy, the easiest way is to change the camera lens. To be distortion-free, try to fix the two cameras on a line at the same time.
9. Use of simulation project
My project uses vivado2019.1 and Modelsim10.6d. I don’t know if the simulation can be compatible with other versions; of course, you need to compile the vivado library and complete the simulation settings in vivado;
enter the simulation project as shown in the figure below In the directory shown, double-click simulate.bat to start modelsim for automated simulation; if you don’t understand this part, you can read my article:
modelsim automated independent simulation vivado ip nuclear engineering
10. Use of AXI slave interface
I have generated an AXI slave with 16 registers. Currently, I only use 0-7 for writing, and 0 and 1 for reading. Others are convenient for future expansion. Note that in the second picture, the 0 and 1 registers read are the coordinates of the connection. Data, used for arm reading, other default write registers;
11. Summary
At this point, the FPGA binocular ranging open source project is finished. If you have any questions or suggestions, you can chat with me privately through the public account; there are two FPGA projects in this part, one is for algorithm simulation, and the other is a complete project;
two projects focus on my public number: core Kingdom, backstage reply eyes ranging complete engineering and binocular ranging engineering simulation available.
