This article is a sharing of the national first prize works of the Anmou Technology Cup in the 2023 Seventh National College Student Integrated Circuit Innovation and Entrepreneurship Competition ("Jichuang Competition"). Participate in the [Prize-winning Collection] of Jishu Community to share your 2023 Jichuang Competition works . , Show off your works , share your works in the 2023 JiChuang Competition to expand your influence, and more rich electronic gifts are waiting for you to claim!
team introduction
Participating unit: Beijing Institute of Technology
Team name: Maplehara Manyo single promoter
Instructor: Wu Qiongzhi
Participating team member: Lu Haobo
National finals award: First Prize (Enterprise Award)
Anmou Technology Cup Enterprise Proposition
Cup title: Intelligent game console design based on Arm processor.
Content: This contest requires contestants to build a demonstrable game console on-chip based on Arm® Cortex®-M0 or Cortex-M3 processor on a limited FPGA platform. system to complete a game content. On this basis, appropriate functions are added to demonstrate the application of display technology, artificial intelligence, the Internet of Things, and smart sensors in game consoles. The smart game console is required to be practical, demonstrable, and innovative.
1. Project introduction
This project develops an all-in-one portable game console. The work integrates ARM M3 DesignStart Eval IP on Anlu Technology PH1A60 FPGA to implement a processor system based on the Cortex-M3 core. Through self-developed three-dimensional graphics processing IP core and digital signal processing-neural network speech recognition processor, three-dimensional graphics rendering and speech recognition functions are realized. The project uses Solidworks to draw the game console casing, manufactures the casing through 3D printing, and assembles it with self-developed circuit boards to form an integrated structure.
Figure 1 The physical picture of the work
2. Work engineering design
Board level interconnection design
This competition is limited to two devices: Anlu EG4S20 and PH1A60, of which the PH1A60 chip has more abundant resources. PH1A60 is an FPGA chip produced by Anlu Technology. It has just been launched this year, and there are few boards on the market. At present, the common Anlu PH1A60 development board on the market is the "Thumb" series development board produced by Hardwood Classroom. The board has HDMI, LCD video output interface and PWM audio output interface, with TF card, SDRAM storage function. This board can meet the design needs of most projects. However, in order to realize the three-dimensional graphics display function in this project, the circuit structure has been specially designed, resulting in the inability to realize all functions using this board.
The game machine designed in this project has the capability of displaying three-dimensional graphics. According to the drawing method of three-dimensional graphics, the graphics data needs to be written into the frame buffer out of order first, and then the graphics data must be read out from the frame buffer sequentially when the screen is refreshed. Therefore, not only the texture image needs to be saved, but also all the data of a frame of image need to be saved at the same time. If the FPGA on-chip memory is used to save, only very low-resolution graphics rendering can be achieved. In order to save more graphics data and improve the gaming experience, external memory can be used to expand the FPGA storage capacity. Since the PH1A60 chip does not support DDR, SDR SDRAM can be used to expand the video memory capacity. However, external SDRAM has limited storage bandwidth. When using an LCD screen, each PCLK must send color data. At this time, if the SDRAM bandwidth is insufficient and data cannot be obtained, a display error will appear on the screen. This directly restricts the maximum resolution that the game console can achieve.
In order to cope with the high memory bandwidth brought by three-dimensional graphics rendering, this project independently designed the main control circuit board of the game console, equipped with Anlu Technology PH1A60 FPGA chip, and integrated dual-channel SDRAM on the board. The circuit system is designed for discharging working conditions. The game console has built-in dual-cell lithium batteries and a 15W (1.5 Lu) charging circuit. In addition, the circuit board also contains digital microphones, high-fidelity digital audio amplifiers and other functions.
In this design, the FPGA is directly welded to the PCB without the need for transfer through the core board. Circuit board traces are impedance matched and equal length to ensure signal integrity.
Figure 2 Use Altium Designer to draw the main control circuit board
Figure 3 Main control circuit board design diagram
Figure 4 Physical picture of the main control circuit board
SoC system design
The project implements an SoC system based on FPGA. The SoC includes the Cortex-M3 core, a self-developed digital signal processing-neural network speech recognition processor, and a self-developed graphics processing unit "Lan Paddle Graphics Engine+". The SoC system bus is also equipped with hardware peripheral interfaces such as UART and GPIO.
Figure 5 SoC system block diagram
Speech recognition processor design
The speech recognition processor gives the game console the function of recognizing the player's voice commands. Players can interact with the game through voice during the game and experience the fun of smart games.
The speech recognition module consists of three parts: I2S digital microphone audio input module, fast Fourier transform module and neural network calculation module. The sampling frequency of the digital microphone is 46875Hz. After the digital microphone obtains the voice signal, it will first save the data in the buffer. The digital signal processing module takes 256 points of audio data from the buffer each time and performs fast Fourier transform. Direct sampling is equivalent to adding a rectangular window to the signal, causing spectrum leakage. Therefore, the digital signal processing module will first add a Hanning window to the signal to suppress the occurrence of spectrum leakage. After windowing the signal, the digital signal processing module performs fast Fourier transform on a frame of signal. In order to avoid losing the data change information of the 256-point data edge of each frame, the 256-point data taken out by the digital signal processing module from the input buffer each time should overlap. In this design, the sampling stride is 128 points, that is, the sampling overlap rate is 50%. The digital signal processing module completes about 366 256-point FFT operations per second. After the speech signal is processed by FFT, the audio frequency spectrogram is obtained. A spectrogram shows how the frequency of a speech signal changes over time. The spectrogram can be used as a feature of the speech signal to distinguish different types of feature words. The spectrogram is input into the neural network, and the predicted probability of the feature words is obtained through the forward propagation of the neural network.
Graphics processing unit design
Implementing three-dimensional graphics rendering on FPGA is the difficulty of this project and is also the feature of the game console. This project self-developed a three-dimensional graphics drawing engine, named "Lan Paddle Graphics Engine+". The word "lan oar" is taken from Su Shi's "Red Cliff Ode": "The orchid oar floats on the osmanthus, hitting the sky and tracing the light." The Blue Paddle graphics engine uses mathematics as a brush to draw a colorful and gorgeous world. The Lan Paddle graphics engine adopts hardware-SoC-algorithm co-design, making FPGA-based three-dimensional graphics rendering possible.
The three-dimensional graphics calculation is completed by the relay of Cortex-M3 core and FPGA. The project uses C++ to develop a computer graphics computing library, including a matrix computing library, an MVP transformation library, and a geometry shader library. The pixel shader implemented by FPGA is based on the deep pipeline, which improves the degree of parallelization of calculations and enables real-time operation of 3D graphics.
After the divisional round, the team significantly rewrote the rendering pipeline of the graphics engine, reconstructed the SDRAM controller IP core, and reached more than 100 stages in the pixel shader pipeline. Using the deep pipeline design, the memory access efficiency is greatly improved, and the graphics drawing speed is improved. The graphics unit of the FPGA consists of a pixel shader controller, a rasterizer, a texture SDRAM controller, and a video memory controller. The graphics unit structure is shown in the figure below:
Figure 6 Graphics acceleration processor structure
The functions of each part are as follows:
Pixel shader controller:
Given the screen coordinates and texture coordinates of the triangle, let the rasterizer traverse each pixel in the triangle bounding box in order to find the color of the pixel.
Rasterizer:
Given information such as current pixel coordinates, triangle vertex coordinates, texture UV coordinates, etc., find the address of the current pixel in SDRAM.
Mapped SDRAM controller:
Handle the write request of the rasterizer, read and write the depth buffer, and fetch the texture color data from SDRAM.
Memory controller:
The image drawn by the graphics processing accelerator is cached to solve the problem of inconsistency between the drawing speed and the screen output speed.
In order to speed up the design process of the 3D model in the game, this project uses Python to develop a 3D model conversion tool. After using the 3D modeling software Blender to draw the model, it can be directly converted by the tool to generate the data format supported by the blue paddle graphics engine.
Figure 7 Model conversion tool
software design
This project transplants the FreeRTOS embedded real-time operating system on the Cortex-M3 core. The FreeRTOS operating system supports a preemptive kernel, enabling real-time multitasking.
In order to improve development efficiency and enhance code portability, this project develops software API functions for calling peripheral interfaces and system functions. The API function structure is similar to the STM32 HAL library, and the control and initialization of the peripheral interface can be realized by calling the function.
structural design
This project uses Solidworks software to model and design the casing of the game console, and makes the casing by 3D printing. The shape and structure of the game console circuit board fit together, which improves the integrity of the system and brings players a better grip experience.
Figure 8 Using Solidworks software to design the structure
Figure 9 Production of game console structure through 3D printing
game design
The game console implements a parkour game. Qi Liangliang is a courier in Inazuma City. During the delivery process, he will turn into a "cat box" and run on the road. There are many obstacles on the way to deliver the goods, and obstacles appear on the road from time to time: there are flower beds, billboards, and sometimes small pine trees blocking the way. However, sometimes gold coins can be found along the way. Let's go on an adventure with Qi Liangliang!
Players can use three methods: button control, motion control, and voice recognition to operate the characters in the game, avoid obstacles on the road, and pick up randomly dropped gold coins to obtain higher game scores.
Figure 10 Game console running game
Behind the scenes of the competition
This content was supplemented with the permission of Mr. Lu, and quoted from the article of the Arm Technology School. The seventh collective competition has come to an end, and the works of the Arm Technology Track are unlimited in creativity .
As the only team composed of individuals in the Anmou Technology Cup in the finals of this competition, Lu Haobo aroused questions from the judges and experts as soon as he appeared at the defense scene: "Are you the only one?"
This one-person team won the ARM Technology Cup Enterprise Award in this year's Jichuang Competition. However, getting this honor was not easy.
One day at the end of February, classmate Lu Haobo saw this year's competition questions on the Chichuang Competition website. Among them, the Arm Technology Cup competition question - "Smart Game Console Design Based on Arm Processor" immediately caught his eye. Unlike the fixed proposition form, the open question design of the ARM Technology Cup leaves more room for imagination. What kind of form does a smart game console take? Is it a novel way of interaction or the ability to achieve unique game content? After thinking excitedly for several days and nights, Lu Haobo focused his creativity on the computer graphics he had taught himself. Computer graphics is a discipline that uses mathematical algorithms to convert two- or three-dimensional graphics into images. It is the basic technology for 3D games. At present, the drawing algorithms of 3D games are mostly implemented by GPU. Running three-dimensional graphics calculations through FPGA is a new attempt. After the excitement passed, Lu quickly devoted himself to the verification of the algorithm. At that time, the registration for the Jichuang Competition was drawing to a close. After a week of intensive development, Lu Haobo finally verified the feasibility of FPGA to implement three-dimensional graphics algorithms.
As the registration deadline approached, student Lu faced a difficult problem: who should I find as a teammate? In the Innovation Competition system, each team can consist of up to three students. The three students cooperated with each other to complete the competition works. However, computer graphics algorithms and FPGA development involve different subject directions. Few students who have studied FPGA have studied computer graphics, and it is not easy to learn computer graphics from scratch. This prevents the largest workload of the project from being shared. Until the registration deadline, he still couldn't find a suitable teammate, so he decided to participate in the Jichuang competition alone.
The consequence of one person participating is that the workload increases exponentially. Computer graphics algorithms involve a large number of mathematical operations, but there are few reference cases for implementing three-dimensional graphics rendering on FPGA. As the development progress progresses, problems in the project are constantly exposed: due to insufficient storage bandwidth of the FPGA development board, the picture cannot be drawn smoothly; the graphics algorithm running on the computer cannot be run directly on the Cortex-M3 core and requires a large amount of data. Extensive rewriting... The submission date for the preliminary works is just around the corner. Student Lu is working day and night to develop. When a method doesn't work, he will design the system at a lower level. Finally, the console was completed on schedule before submission date.
In this way, after preliminary rounds and regional rounds, the work successfully advanced to the national finals. Lu Haobo gave up weekends and summer vacations and spent day and night in the laboratory debugging circuits and programs. Late at night, in a person's laboratory, there is only the sound of rapid typing on the keyboard. It was faith and courage that brought the work to the national finals.
On August 22, at the National Finals Award Ceremony of the 7th National College Student Integrated Circuit Innovation and Entrepreneurship Competition, the one-man team of "Maplehara Manyo" won the ARM Technology Cup Enterprise Award. The host said: "He single-handedly withstood thousands of troops." Looking back on the experience of the competition, classmate Lu said: "Maybe this is the courage and enterprising spirit of youth."
Participate in the Jishu community’s [Prize-based Collection] and share your 2023 original competition works to show off your work . Share your 2023 original competition works to expand your influence. There are also rich electronic gifts waiting for you!